2024-04-27T03:23:23.386 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-27T03:23:23.390 DEBUG:teuthology.run:Teuthology command: teuthology --archive /home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672 --description rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_latest}} --name teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi --verbose --owner scheduled_teuthology@teuthology -- /home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672/orig.config.yaml 2024-04-27T03:23:23.422 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T03:23:23.512 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672 branch: squid description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_latest}} email: ceph-qa@ceph.com first_in_suite: false job_id: '7675672' kernel: kdb: 1 sha1: distro last_in_suite: false machine_type: smithi name: teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 3 size: 10 os_type: centos os_version: 9.stream overrides: admin_socket: branch: squid ceph: conf: client: debug rgw: 20 setgroup: ceph setuser: ceph mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(PG_AVAILABILITY\) - \(PG_DEGRADED\) - \(POOL_APP_NOT_ENABLED\) - not have an application enabled sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a rgw: ec-data-pool: false frontend: beast selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: squid sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a owner: scheduled_teuthology@teuthology priority: 100 repo: https://git.ceph.com/ceph.git roles: - - mon.a - osd.0 - - mgr.x - client.0 - osd.1 - osd.2 - osd.3 seed: 7340 sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a sleep_before_teardown: 0 subset: 0/1 suite: rgw suite_branch: squid suite_path: /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph.git suite_sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a targets: smithi089.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDzYiMSBF4aqIe5Qm+cDeIt3txT+9fS5+hAXpK/EXm3W+qL7WXrCZfJJPTqR2VaxGfXa4aPjPJzZFh2uJ6OY6qU= smithi165.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC7Z6J+tAFj4YXchxvYjTYnocjQ/CfPgtt0ScdIakZyVglcgpY1HYvs2afnrXKPnPc9TImTaFRFe8RvhQVpI0OA= tasks: - install: null - ceph: conf: client: debug ms: 1 rgw gc obj min wait: 15 osd: debug ms: 1 debug objclass: 20 - rgw: client.0: null - radosgw-admin: null - radosgw-admin-rest: null teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-26_21:40:02 tube: smithi user: teuthology verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-27T03:23:23.512 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa; will attempt to use it 2024-04-27T03:23:23.513 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa/tasks 2024-04-27T03:23:23.513 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-27T03:23:23.514 INFO:teuthology.task.internal:Checking packages... 2024-04-27T03:23:23.538 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a' 2024-04-27T03:23:23.539 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-27T03:23:23.539 INFO:teuthology.packaging:ref: None 2024-04-27T03:23:23.539 INFO:teuthology.packaging:tag: None 2024-04-27T03:23:23.539 INFO:teuthology.packaging:branch: squid 2024-04-27T03:23:23.539 INFO:teuthology.packaging:sha1: b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-27T03:23:23.539 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=squid 2024-04-27T03:23:23.755 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-2447.gb22e2ebd 2024-04-27T03:23:23.756 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-27T03:23:23.764 INFO:teuthology.task.internal:no buildpackages task found 2024-04-27T03:23:23.764 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-27T03:23:23.810 INFO:teuthology.task.internal:Saving configuration 2024-04-27T03:23:23.822 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-27T03:23:23.836 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-27T03:23:23.860 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi089.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 03:18:06.404657', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDzYiMSBF4aqIe5Qm+cDeIt3txT+9fS5+hAXpK/EXm3W+qL7WXrCZfJJPTqR2VaxGfXa4aPjPJzZFh2uJ6OY6qU='} 2024-04-27T03:23:23.881 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi165.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-27 03:18:06.405651', 'locked_by': 'scheduled_teuthology@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC7Z6J+tAFj4YXchxvYjTYnocjQ/CfPgtt0ScdIakZyVglcgpY1HYvs2afnrXKPnPc9TImTaFRFe8RvhQVpI0OA='} 2024-04-27T03:23:23.881 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-27T03:23:23.889 INFO:teuthology.task.internal:roles: ubuntu@smithi089.front.sepia.ceph.com - ['mon.a', 'osd.0'] 2024-04-27T03:23:23.889 INFO:teuthology.task.internal:roles: ubuntu@smithi165.front.sepia.ceph.com - ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3'] 2024-04-27T03:23:23.889 INFO:teuthology.run_tasks:Running task console_log... 2024-04-27T03:23:23.978 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f127ee30e50>, signals=[15]) 2024-04-27T03:23:23.979 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-27T03:23:23.985 INFO:teuthology.task.internal:Opening connections... 2024-04-27T03:23:23.985 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi089.front.sepia.ceph.com 2024-04-27T03:23:23.987 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T03:23:24.065 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi165.front.sepia.ceph.com 2024-04-27T03:23:24.066 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T03:23:24.139 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-27T03:23:24.146 DEBUG:teuthology.orchestra.run.smithi089:> uname -m 2024-04-27T03:23:24.164 INFO:teuthology.orchestra.run.smithi089.stdout:x86_64 2024-04-27T03:23:24.164 DEBUG:teuthology.orchestra.run.smithi089:> cat /etc/os-release 2024-04-27T03:23:24.220 INFO:teuthology.orchestra.run.smithi089.stdout:NAME="CentOS Stream" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION="9" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:ID="centos" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:ID_LIKE="rhel fedora" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:VERSION_ID="9" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:PLATFORM_ID="platform:el9" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:ANSI_COLOR="0;31" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:LOGO="fedora-logo-icon" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:HOME_URL="https://centos.org/" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-27T03:23:24.221 INFO:teuthology.orchestra.run.smithi089.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-27T03:23:24.222 INFO:teuthology.lock.ops:Updating smithi089.front.sepia.ceph.com on lock server 2024-04-27T03:23:24.246 DEBUG:teuthology.orchestra.run.smithi165:> uname -m 2024-04-27T03:23:24.264 INFO:teuthology.orchestra.run.smithi165.stdout:x86_64 2024-04-27T03:23:24.264 DEBUG:teuthology.orchestra.run.smithi165:> cat /etc/os-release 2024-04-27T03:23:24.322 INFO:teuthology.orchestra.run.smithi165.stdout:NAME="CentOS Stream" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION="9" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:ID="centos" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:ID_LIKE="rhel fedora" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:VERSION_ID="9" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:PLATFORM_ID="platform:el9" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:ANSI_COLOR="0;31" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:LOGO="fedora-logo-icon" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:HOME_URL="https://centos.org/" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-27T03:23:24.323 INFO:teuthology.orchestra.run.smithi165.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-27T03:23:24.324 INFO:teuthology.lock.ops:Updating smithi165.front.sepia.ceph.com on lock server 2024-04-27T03:23:24.346 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-27T03:23:24.354 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-27T03:23:24.378 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-27T03:23:24.378 DEBUG:teuthology.orchestra.run.smithi089:> test '!' -e /home/ubuntu/cephtest 2024-04-27T03:23:24.381 DEBUG:teuthology.orchestra.run.smithi165:> test '!' -e /home/ubuntu/cephtest 2024-04-27T03:23:24.397 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-27T03:23:24.405 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-27T03:23:24.405 DEBUG:teuthology.orchestra.run.smithi089:> test -z $(ls -A /var/lib/ceph) 2024-04-27T03:23:24.438 DEBUG:teuthology.orchestra.run.smithi165:> test -z $(ls -A /var/lib/ceph) 2024-04-27T03:23:24.532 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-27T03:23:24.592 INFO:teuthology.run_tasks:Running task kernel... 2024-04-27T03:23:24.608 INFO:teuthology.task.kernel:normalize config orig: {'kdb': 1, 'sha1': 'distro'} 2024-04-27T03:23:24.608 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'kdb': 1, 'sha1': 'distro'}}, timeout 300 2024-04-27T03:23:24.609 DEBUG:teuthology.orchestra.run.smithi089:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-27T03:23:24.609 DEBUG:teuthology.orchestra.run.smithi165:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-27T03:23:24.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T03:23:24.626 DEBUG:teuthology.orchestra.run.smithi089:> uname -r 2024-04-27T03:23:24.626 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T03:23:24.627 DEBUG:teuthology.orchestra.run.smithi165:> uname -r 2024-04-27T03:23:24.682 INFO:teuthology.orchestra.run.smithi089.stdout:5.14.0-437.el9.x86_64 2024-04-27T03:23:24.683 INFO:teuthology.task.kernel:Running kernel on smithi089: 5.14.0-437.el9.x86_64 2024-04-27T03:23:24.683 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum install -y kernel 2024-04-27T03:23:24.684 INFO:teuthology.orchestra.run.smithi165.stdout:5.14.0-437.el9.x86_64 2024-04-27T03:23:24.684 INFO:teuthology.task.kernel:Running kernel on smithi165: 5.14.0-437.el9.x86_64 2024-04-27T03:23:24.685 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum install -y kernel 2024-04-27T03:23:27.208 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - BaseOS 9.3 MB/s | 8.0 MB 00:00 2024-04-27T03:23:28.147 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 4.3 MB/s | 8.0 MB 00:01 2024-04-27T03:23:29.148 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - AppStream 22 MB/s | 19 MB 00:00 2024-04-27T03:23:30.339 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 17 MB/s | 19 MB 00:01 2024-04-27T03:23:33.257 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-27T03:23:34.373 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 13 MB/s | 6.2 MB 00:00 2024-04-27T03:23:34.882 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - Extras packages 57 kB/s | 16 kB 00:00 2024-04-27T03:23:36.023 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 52 kB/s | 16 kB 00:00 2024-04-27T03:23:36.445 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 15 MB/s | 21 MB 00:01 2024-04-27T03:23:37.183 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 20 MB/s | 21 MB 00:01 2024-04-27T03:23:41.567 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-27T03:23:42.216 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-27T03:23:43.415 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-27T03:23:43.415 INFO:teuthology.orchestra.run.smithi089.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-27T03:23:43.510 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:23:43.511 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2024-04-27T03:23:43.511 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:23:43.635 DEBUG:teuthology.orchestra.run.smithi089:> echo no | sudo yum reinstall kernel || true 2024-04-27T03:23:44.080 INFO:teuthology.orchestra.run.smithi165.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-27T03:23:44.080 INFO:teuthology.orchestra.run.smithi165.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-27T03:23:44.130 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:03 ago on Sat 27 Apr 2024 03:23:41 AM UTC. 2024-04-27T03:23:44.174 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:23:44.174 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-27T03:23:44.175 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:23:44.254 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:23:44.265 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:23:44.265 INFO:teuthology.orchestra.run.smithi089.stdout: Package Architecture Version Repository Size 2024-04-27T03:23:44.265 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:23:44.265 INFO:teuthology.orchestra.run.smithi089.stdout:Reinstalling: 2024-04-27T03:23:44.265 INFO:teuthology.orchestra.run.smithi089.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-27T03:23:44.265 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:23:44.265 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:23:44.265 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:23:44.265 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:23:44.266 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 255 k 2024-04-27T03:23:44.266 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 0 2024-04-27T03:23:44.266 INFO:teuthology.orchestra.run.smithi089.stderr:Operation aborted. 2024-04-27T03:23:44.288 DEBUG:teuthology.orchestra.run.smithi165:> echo no | sudo yum reinstall kernel || true 2024-04-27T03:23:44.327 INFO:teuthology.orchestra.run.smithi089.stdout:Is this ok [y/N]: 2024-04-27T03:23:44.327 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum reinstall -y kernel || true 2024-04-27T03:23:44.787 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:02 ago on Sat 27 Apr 2024 03:23:42 AM UTC. 2024-04-27T03:23:44.813 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:03 ago on Sat 27 Apr 2024 03:23:41 AM UTC. 2024-04-27T03:23:44.919 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:23:44.928 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:23:44.929 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:23:44.929 INFO:teuthology.orchestra.run.smithi089.stdout: Package Architecture Version Repository Size 2024-04-27T03:23:44.929 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:23:44.929 INFO:teuthology.orchestra.run.smithi089.stdout:Reinstalling: 2024-04-27T03:23:44.930 INFO:teuthology.orchestra.run.smithi089.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-27T03:23:44.930 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:23:44.930 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:23:44.930 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:23:44.930 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:23:44.930 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 255 k 2024-04-27T03:23:44.930 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 0 2024-04-27T03:23:44.930 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:23:44.933 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:23:44.934 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2024-04-27T03:23:44.934 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:23:44.934 INFO:teuthology.orchestra.run.smithi165.stdout:Reinstalling: 2024-04-27T03:23:44.934 INFO:teuthology.orchestra.run.smithi165.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-27T03:23:44.934 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:23:44.934 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:23:44.934 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:23:44.934 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:23:44.935 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 255 k 2024-04-27T03:23:44.935 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 0 2024-04-27T03:23:44.935 INFO:teuthology.orchestra.run.smithi165.stderr:Operation aborted. 2024-04-27T03:23:44.992 INFO:teuthology.orchestra.run.smithi165.stdout:Is this ok [y/N]: 2024-04-27T03:23:44.992 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum reinstall -y kernel || true 2024-04-27T03:23:45.202 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.6 MB/s | 255 kB 00:00 2024-04-27T03:23:45.202 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:23:45.203 INFO:teuthology.orchestra.run.smithi089.stdout:Total 935 kB/s | 255 kB 00:00 2024-04-27T03:23:45.222 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:23:45.397 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:23:45.398 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:23:45.402 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:23:45.403 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:23:45.491 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:03 ago on Sat 27 Apr 2024 03:23:42 AM UTC. 2024-04-27T03:23:45.574 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:23:45.607 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:23:45.608 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:23:45.608 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2024-04-27T03:23:45.608 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:23:45.608 INFO:teuthology.orchestra.run.smithi165.stdout:Reinstalling: 2024-04-27T03:23:45.608 INFO:teuthology.orchestra.run.smithi165.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-27T03:23:45.608 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:23:45.608 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:23:45.608 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:23:45.608 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:23:45.609 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 255 k 2024-04-27T03:23:45.609 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 0 2024-04-27T03:23:45.609 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:23:45.678 INFO:teuthology.orchestra.run.smithi089.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-27T03:23:45.679 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T03:23:46.043 INFO:teuthology.orchestra.run.smithi165.stdout:kernel-5.14.0-437.el9.x86_64.rpm 1.5 MB/s | 255 kB 00:00 2024-04-27T03:23:46.043 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:23:46.043 INFO:teuthology.orchestra.run.smithi165.stdout:Total 587 kB/s | 255 kB 00:00 2024-04-27T03:23:46.058 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:23:46.242 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:23:46.242 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:23:46.247 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:23:46.248 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:23:46.358 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T03:23:46.358 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-27T03:23:46.393 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:23:46.460 INFO:teuthology.orchestra.run.smithi165.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-27T03:23:46.461 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T03:23:47.079 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T03:23:47.079 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:23:47.079 INFO:teuthology.orchestra.run.smithi089.stdout:Reinstalled: 2024-04-27T03:23:47.080 INFO:teuthology.orchestra.run.smithi089.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-27T03:23:47.080 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:23:47.080 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:23:47.104 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T03:23:47.104 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-27T03:23:47.259 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q kernel | sort -rV | head -n 1 2024-04-27T03:23:47.305 INFO:teuthology.orchestra.run.smithi089.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-27T03:23:47.305 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-27T03:23:47.305 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-27T03:23:47.306 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2024-04-27T03:23:47.306 DEBUG:teuthology.orchestra.run.smithi089:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-27T03:23:47.335 INFO:teuthology.orchestra.run.smithi089.stdout:ttyS1 2024-04-27T03:23:47.356 DEBUG:teuthology.parallel:result is None 2024-04-27T03:23:47.757 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-27T03:23:47.757 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:23:47.757 INFO:teuthology.orchestra.run.smithi165.stdout:Reinstalled: 2024-04-27T03:23:47.757 INFO:teuthology.orchestra.run.smithi165.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-27T03:23:47.757 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:23:47.758 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:23:47.937 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q kernel | sort -rV | head -n 1 2024-04-27T03:23:48.024 INFO:teuthology.orchestra.run.smithi165.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-27T03:23:48.024 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-27T03:23:48.024 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-27T03:23:48.024 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2024-04-27T03:23:48.024 DEBUG:teuthology.orchestra.run.smithi165:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2024-04-27T03:23:48.060 INFO:teuthology.orchestra.run.smithi165.stdout:ttyS1 2024-04-27T03:23:48.084 DEBUG:teuthology.parallel:result is None 2024-04-27T03:23:48.085 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-27T03:23:48.092 INFO:teuthology.task.internal:Creating test directory... 2024-04-27T03:23:48.092 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-27T03:23:48.095 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-27T03:23:48.139 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-27T03:23:48.148 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-27T03:23:48.174 INFO:teuthology.task.internal:Creating archive directory... 2024-04-27T03:23:48.174 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-27T03:23:48.176 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-27T03:23:48.211 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-27T03:23:48.219 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-27T03:23:48.219 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-27T03:23:48.249 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-27T03:23:48.282 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T03:23:48.284 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T03:23:48.291 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T03:23:48.294 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-27T03:23:48.296 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-27T03:23:48.304 INFO:teuthology.task.internal:Configuring sudo... 2024-04-27T03:23:48.304 DEBUG:teuthology.orchestra.run.smithi089:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-27T03:23:48.339 DEBUG:teuthology.orchestra.run.smithi165:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-27T03:23:48.362 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-27T03:23:48.372 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-27T03:23:48.372 DEBUG:teuthology.orchestra.run.smithi089:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-27T03:23:48.405 DEBUG:teuthology.orchestra.run.smithi165:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-27T03:23:48.419 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T03:23:48.499 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T03:23:48.552 DEBUG:teuthology.orchestra.run.smithi089:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T03:23:48.629 DEBUG:teuthology.orchestra.run.smithi089:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T03:23:48.673 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:23:48.673 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-27T03:23:48.736 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T03:23:48.771 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-27T03:23:48.821 DEBUG:teuthology.orchestra.run.smithi165:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T03:23:48.896 DEBUG:teuthology.orchestra.run.smithi165:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-27T03:23:48.943 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:23:48.943 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-27T03:23:49.006 DEBUG:teuthology.orchestra.run.smithi089:> sudo service rsyslog restart 2024-04-27T03:23:49.008 DEBUG:teuthology.orchestra.run.smithi165:> sudo service rsyslog restart 2024-04-27T03:23:49.039 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T03:23:49.079 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T03:23:49.442 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-27T03:23:49.450 INFO:teuthology.task.internal:Starting timer... 2024-04-27T03:23:49.450 INFO:teuthology.run_tasks:Running task pcp... 2024-04-27T03:23:49.506 INFO:teuthology.run_tasks:Running task selinux... 2024-04-27T03:23:49.537 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-27T03:23:49.538 DEBUG:teuthology.orchestra.run.smithi089:> sudo service auditd rotate 2024-04-27T03:23:49.601 INFO:teuthology.orchestra.run.smithi089.stdout:Rotating logs: 2024-04-27T03:23:49.603 DEBUG:teuthology.orchestra.run.smithi165:> sudo service auditd rotate 2024-04-27T03:23:49.658 INFO:teuthology.orchestra.run.smithi165.stdout:Rotating logs: 2024-04-27T03:23:49.660 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-27T03:23:49.661 DEBUG:teuthology.orchestra.run.smithi089:> /usr/sbin/getenforce 2024-04-27T03:23:49.704 INFO:teuthology.orchestra.run.smithi089.stdout:Permissive 2024-04-27T03:23:49.705 DEBUG:teuthology.orchestra.run.smithi165:> /usr/sbin/getenforce 2024-04-27T03:23:49.726 INFO:teuthology.orchestra.run.smithi165.stdout:Permissive 2024-04-27T03:23:49.726 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi089.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi165.front.sepia.ceph.com': 'permissive'} 2024-04-27T03:23:49.727 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T03:23:49.751 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T03:23:49.751 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T03:23:49.790 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T03:23:49.791 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-27T03:23:49.791 DEBUG:teuthology.orchestra.run.smithi089:> sudo /usr/sbin/setenforce permissive 2024-04-27T03:23:49.823 DEBUG:teuthology.orchestra.run.smithi165:> sudo /usr/sbin/setenforce permissive 2024-04-27T03:23:49.862 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-27T03:23:49.873 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-27T03:23:49.888 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-27T03:23:49.890 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi089.front.sepia.ceph.com,smithi165.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-27T03:27:16.716 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi089.front.sepia.ceph.com'), Remote(name='ubuntu@smithi165.front.sepia.ceph.com')] 2024-04-27T03:27:16.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-04-27T03:27:16.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi089.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T03:27:16.803 DEBUG:teuthology.orchestra.run.smithi089:> true 2024-04-27T03:27:16.897 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi089.front.sepia.ceph.com' 2024-04-27T03:27:16.897 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-27T03:27:16.898 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi165.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-27T03:27:16.972 DEBUG:teuthology.orchestra.run.smithi165:> true 2024-04-27T03:27:17.050 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi165.front.sepia.ceph.com' 2024-04-27T03:27:17.050 INFO:teuthology.run_tasks:Running task clock... 2024-04-27T03:27:17.062 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-27T03:27:17.062 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-27T03:27:17.062 DEBUG:teuthology.orchestra.run.smithi089:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T03:27:17.065 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-27T03:27:17.065 DEBUG:teuthology.orchestra.run.smithi165:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T03:27:17.090 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-27T03:27:17.105 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-27T03:27:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-27T03:27:17.128 INFO:teuthology.orchestra.run.smithi089.stderr:sudo: ntpd: command not found 2024-04-27T03:27:17.132 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-27T03:27:17.138 INFO:teuthology.orchestra.run.smithi089.stdout:506 Cannot talk to daemon 2024-04-27T03:27:17.151 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-27T03:27:17.154 INFO:teuthology.orchestra.run.smithi165.stderr:sudo: ntpd: command not found 2024-04-27T03:27:17.164 INFO:teuthology.orchestra.run.smithi089.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-27T03:27:17.165 INFO:teuthology.orchestra.run.smithi165.stdout:506 Cannot talk to daemon 2024-04-27T03:27:17.177 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-27T03:27:17.190 INFO:teuthology.orchestra.run.smithi165.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-27T03:27:17.215 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-04-27T03:27:17.219 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T03:27:17.219 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-04-27T03:27:17.219 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:27:17.219 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:27:17.219 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:27:17.219 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:27:17.247 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-04-27T03:27:17.252 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T03:27:17.252 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-04-27T03:27:17.252 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:27:17.252 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:27:17.252 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:27:17.253 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:27:17.253 INFO:teuthology.run_tasks:Running task install... 2024-04-27T03:27:17.264 DEBUG:teuthology.task.install:project ceph 2024-04-27T03:27:17.264 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'}} 2024-04-27T03:27:17.264 DEBUG:teuthology.task.install:config {'flavor': 'default', 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'} 2024-04-27T03:27:17.264 INFO:teuthology.task.install:Using flavor: default 2024-04-27T03:27:17.272 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-27T03:27:17.272 INFO:teuthology.task.install:extra packages: [] 2024-04-27T03:27:17.272 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a', 'tag': None, 'wait_for_package': False} 2024-04-27T03:27:17.273 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-27T03:27:17.275 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a', 'tag': None, 'wait_for_package': False} 2024-04-27T03:27:17.275 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-27T03:27:17.499 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/b22e2ebdeb24376882b7bda2a7329c8cccc2276a/centos/9/flavors/default/ 2024-04-27T03:27:17.499 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2447.gb22e2ebd 2024-04-27T03:27:17.503 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/squid/b22e2ebdeb24376882b7bda2a7329c8cccc2276a/centos/9/flavors/default/ 2024-04-27T03:27:17.503 INFO:teuthology.task.install.rpm:Package version is 19.0.0-2447.gb22e2ebd 2024-04-27T03:27:17.614 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/b22e2ebdeb24376882b7bda2a7329c8cccc2276a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/b22e2ebdeb24376882b7bda2a7329c8cccc2276a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/b22e2ebdeb24376882b7bda2a7329c8cccc2276a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-27T03:27:17.614 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:27:17.614 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-27T03:27:17.639 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-27T03:27:17.640 DEBUG:teuthology.orchestra.run.smithi089:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b22e2ebdeb24376882b7bda2a7329c8cccc2276a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-27T03:27:17.672 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://4.chacra.ceph.com/r/ceph/squid/b22e2ebdeb24376882b7bda2a7329c8cccc2276a/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/b22e2ebdeb24376882b7bda2a7329c8cccc2276a/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://4.chacra.ceph.com/r/ceph/squid/b22e2ebdeb24376882b7bda2a7329c8cccc2276a/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-27T03:27:17.672 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:27:17.672 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-27T03:27:17.698 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2024-04-27T03:27:17.698 DEBUG:teuthology.orchestra.run.smithi165:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b22e2ebdeb24376882b7bda2a7329c8cccc2276a/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-27T03:27:17.704 DEBUG:teuthology.orchestra.run.smithi089:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-27T03:27:17.762 DEBUG:teuthology.orchestra.run.smithi165:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-27T03:27:17.776 DEBUG:teuthology.orchestra.run.smithi089:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-27T03:27:17.835 DEBUG:teuthology.orchestra.run.smithi165:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-27T03:27:17.839 INFO:teuthology.orchestra.run.smithi089.stdout:check_obsoletes = 1 2024-04-27T03:27:17.840 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum clean all 2024-04-27T03:27:17.898 INFO:teuthology.orchestra.run.smithi165.stdout:check_obsoletes = 1 2024-04-27T03:27:17.899 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum clean all 2024-04-27T03:27:18.062 INFO:teuthology.orchestra.run.smithi089.stdout:48 files removed 2024-04-27T03:27:18.089 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-radosgw 2024-04-27T03:27:18.125 INFO:teuthology.orchestra.run.smithi165.stdout:48 files removed 2024-04-27T03:27:18.153 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-radosgw 2024-04-27T03:27:18.818 INFO:teuthology.orchestra.run.smithi165.stdout:ceph packages for x86_64 204 kB/s | 83 kB 00:00 2024-04-27T03:27:19.176 INFO:teuthology.orchestra.run.smithi165.stdout:ceph noarch packages 37 kB/s | 11 kB 00:00 2024-04-27T03:27:19.552 INFO:teuthology.orchestra.run.smithi165.stdout:ceph source packages 5.6 kB/s | 1.9 kB 00:00 2024-04-27T03:27:20.127 INFO:teuthology.orchestra.run.smithi089.stdout:ceph packages for x86_64 47 kB/s | 83 kB 00:01 2024-04-27T03:27:20.553 INFO:teuthology.orchestra.run.smithi089.stdout:ceph noarch packages 30 kB/s | 11 kB 00:00 2024-04-27T03:27:20.970 INFO:teuthology.orchestra.run.smithi089.stdout:ceph source packages 5.0 kB/s | 1.9 kB 00:00 2024-04-27T03:27:21.437 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - BaseOS 19 MB/s | 8.0 MB 00:00 2024-04-27T03:27:26.629 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - AppStream 4.6 MB/s | 19 MB 00:04 2024-04-27T03:27:28.493 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - BaseOS 925 kB/s | 8.0 MB 00:08 2024-04-27T03:27:31.794 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - AppStream 8.6 MB/s | 19 MB 00:02 2024-04-27T03:27:32.931 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - CRB 2.2 MB/s | 6.2 MB 00:02 2024-04-27T03:27:34.714 INFO:teuthology.orchestra.run.smithi089.stdout:CentOS Stream 9 - Extras packages 27 kB/s | 16 kB 00:00 2024-04-27T03:27:36.536 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - CRB 5.5 MB/s | 6.2 MB 00:01 2024-04-27T03:27:37.244 INFO:teuthology.orchestra.run.smithi089.stdout:Extra Packages for Enterprise Linux 8.7 MB/s | 21 MB 00:02 2024-04-27T03:27:38.153 INFO:teuthology.orchestra.run.smithi165.stdout:CentOS Stream 9 - Extras packages 35 kB/s | 16 kB 00:00 2024-04-27T03:27:42.424 INFO:teuthology.orchestra.run.smithi089.stdout:lab-extras 12 kB/s | 1.7 kB 00:00 2024-04-27T03:27:43.074 INFO:teuthology.orchestra.run.smithi165.stdout:Extra Packages for Enterprise Linux 4.4 MB/s | 21 MB 00:04 2024-04-27T03:27:44.607 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:27:44.609 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:27:44.609 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-04-27T03:27:44.609 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:27:44.610 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:27:44.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 10 M 2024-04-27T03:27:44.610 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrading: 2024-04-27T03:27:44.610 INFO:teuthology.orchestra.run.smithi089.stdout: librados2 x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 3.4 M 2024-04-27T03:27:44.610 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1 x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 3.1 M 2024-04-27T03:27:44.610 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-04-27T03:27:44.610 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-27T03:27:44.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 5.4 M 2024-04-27T03:27:44.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 21 M 2024-04-27T03:27:44.610 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 25 k 2024-04-27T03:27:44.610 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-27T03:27:44.610 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-27T03:27:44.611 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-27T03:27:44.611 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2 x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 745 k 2024-04-27T03:27:44.611 INFO:teuthology.orchestra.run.smithi089.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-27T03:27:44.611 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-27T03:27:44.611 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-27T03:27:44.611 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1 x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 501 k 2024-04-27T03:27:44.611 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-27T03:27:44.611 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2 x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 5.3 M 2024-04-27T03:27:44.611 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-27T03:27:44.611 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-27T03:27:44.611 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-27T03:27:44.611 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 45 k 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 133 k 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 161 k 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 321 k 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 298 k 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 99 k 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout:Installing weak dependencies: 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:27:44.612 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:27:44.613 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:27:44.613 INFO:teuthology.orchestra.run.smithi089.stdout:Install 30 Packages 2024-04-27T03:27:44.613 INFO:teuthology.orchestra.run.smithi089.stdout:Upgrade 2 Packages 2024-04-27T03:27:44.613 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:27:44.613 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 60 M 2024-04-27T03:27:44.613 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:27:46.808 INFO:teuthology.orchestra.run.smithi089.stdout:(1/32): ceph-base-19.0.0-2447.gb22e2ebd.el9.x86 3.6 MB/s | 5.4 MB 00:01 2024-04-27T03:27:46.892 INFO:teuthology.orchestra.run.smithi089.stdout:(2/32): ceph-selinux-19.0.0-2447.gb22e2ebd.el9. 303 kB/s | 25 kB 00:00 2024-04-27T03:27:47.101 INFO:teuthology.orchestra.run.smithi089.stdout:(3/32): ceph-radosgw-19.0.0-2447.gb22e2ebd.el9. 5.7 MB/s | 10 MB 00:01 2024-04-27T03:27:47.142 INFO:teuthology.orchestra.run.smithi089.stdout:(4/32): libcephfs2-19.0.0-2447.gb22e2ebd.el9.x8 2.9 MB/s | 745 kB 00:00 2024-04-27T03:27:47.201 INFO:teuthology.orchestra.run.smithi089.stdout:(5/32): libradosstriper1-19.0.0-2447.gb22e2ebd. 4.9 MB/s | 501 kB 00:00 2024-04-27T03:27:47.276 INFO:teuthology.orchestra.run.smithi089.stdout:(6/32): python3-ceph-argparse-19.0.0-2447.gb22e 602 kB/s | 45 kB 00:00 2024-04-27T03:27:47.343 INFO:teuthology.orchestra.run.smithi089.stdout:(7/32): python3-ceph-common-19.0.0-2447.gb22e2e 1.9 MB/s | 133 kB 00:00 2024-04-27T03:27:47.418 INFO:teuthology.orchestra.run.smithi089.stdout:(8/32): python3-cephfs-19.0.0-2447.gb22e2ebd.el 2.1 MB/s | 161 kB 00:00 2024-04-27T03:27:47.494 INFO:teuthology.orchestra.run.smithi089.stdout:(9/32): python3-rados-19.0.0-2447.gb22e2ebd.el9 4.2 MB/s | 321 kB 00:00 2024-04-27T03:27:47.602 INFO:teuthology.orchestra.run.smithi089.stdout:(10/32): python3-rbd-19.0.0-2447.gb22e2ebd.el9. 2.7 MB/s | 298 kB 00:00 2024-04-27T03:27:47.686 INFO:teuthology.orchestra.run.smithi089.stdout:(11/32): python3-rgw-19.0.0-2447.gb22e2ebd.el9. 1.2 MB/s | 99 kB 00:00 2024-04-27T03:27:47.870 INFO:teuthology.orchestra.run.smithi089.stdout:(12/32): librgw2-19.0.0-2447.gb22e2ebd.el9.x86_ 7.3 MB/s | 5.3 MB 00:00 2024-04-27T03:27:47.895 INFO:teuthology.orchestra.run.smithi089.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 513 kB/s | 107 kB 00:00 2024-04-27T03:27:48.020 INFO:teuthology.orchestra.run.smithi089.stdout:(14/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 361 kB/s | 45 kB 00:00 2024-04-27T03:27:48.129 INFO:teuthology.orchestra.run.smithi089.stdout:(15/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 619 kB/s | 160 kB 00:00 2024-04-27T03:27:48.137 INFO:teuthology.orchestra.run.smithi165.stdout:lab-extras 24 kB/s | 1.7 kB 00:00 2024-04-27T03:27:48.287 INFO:teuthology.orchestra.run.smithi089.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2024-04-27T03:27:48.321 INFO:teuthology.orchestra.run.smithi089.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 2.2 MB/s | 662 kB 00:00 2024-04-27T03:27:48.346 INFO:teuthology.orchestra.run.smithi089.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 719 kB/s | 42 kB 00:00 2024-04-27T03:27:48.397 INFO:teuthology.orchestra.run.smithi089.stdout:(19/32): qatlib-service-24.02.0-1.el9.x86_64.rp 716 kB/s | 36 kB 00:00 2024-04-27T03:27:48.422 INFO:teuthology.orchestra.run.smithi089.stdout:(20/32): qatlib-24.02.0-1.el9.x86_64.rpm 2.1 MB/s | 221 kB 00:00 2024-04-27T03:27:48.455 INFO:teuthology.orchestra.run.smithi089.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 809 kB/s | 47 kB 00:00 2024-04-27T03:27:48.923 INFO:teuthology.orchestra.run.smithi089.stdout:(22/32): ceph-common-19.0.0-2447.gb22e2ebd.el9. 5.8 MB/s | 21 MB 00:03 2024-04-27T03:27:49.414 INFO:teuthology.orchestra.run.smithi089.stdout:(23/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 311 kB/s | 309 kB 00:00 2024-04-27T03:27:49.464 INFO:teuthology.orchestra.run.smithi089.stdout:(24/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 48 kB/s | 26 kB 00:00 2024-04-27T03:27:49.557 INFO:teuthology.orchestra.run.smithi089.stdout:(25/32): liboath-2.6.7-2.el9.x86_64.rpm 343 kB/s | 49 kB 00:00 2024-04-27T03:27:49.798 INFO:teuthology.orchestra.run.smithi089.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 202 kB/s | 67 kB 00:00 2024-04-27T03:27:49.957 INFO:teuthology.orchestra.run.smithi089.stdout:(27/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 2.0 MB/s | 839 kB 00:00 2024-04-27T03:27:50.116 INFO:teuthology.orchestra.run.smithi089.stdout:(28/32): libarrow-9.0.0-11.el9.x86_64.rpm 2.7 MB/s | 4.4 MB 00:01 2024-04-27T03:27:50.149 INFO:teuthology.orchestra.run.smithi089.stdout:(29/32): re2-20211101-3.el9.x86_64.rpm 546 kB/s | 192 kB 00:00 2024-04-27T03:27:50.290 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:27:50.292 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:27:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-27T03:27:50.292 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:27:50.292 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:27:50.292 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 10 M 2024-04-27T03:27:50.292 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrading: 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: librados2 x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 3.4 M 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1 x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 3.1 M 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 5.4 M 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 21 M 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 25 k 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow x86_64 9.0.0-11.el9 epel 4.4 M 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc noarch 9.0.0-11.el9 epel 26 k 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2 x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 745 k 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-27T03:27:50.293 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1 x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 501 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2 x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 5.3 M 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs x86_64 9.0.0-11.el9 epel 839 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 45 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 133 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 161 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 321 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 298 k 2024-04-27T03:27:50.294 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 99 k 2024-04-27T03:27:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-27T03:27:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-27T03:27:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-27T03:27:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-27T03:27:50.295 INFO:teuthology.orchestra.run.smithi165.stdout:Installing weak dependencies: 2024-04-27T03:27:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-27T03:27:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:27:50.295 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:27:50.295 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:27:50.295 INFO:teuthology.orchestra.run.smithi165.stdout:Install 30 Packages 2024-04-27T03:27:50.295 INFO:teuthology.orchestra.run.smithi165.stdout:Upgrade 2 Packages 2024-04-27T03:27:50.295 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:27:50.296 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 60 M 2024-04-27T03:27:50.296 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:27:50.308 INFO:teuthology.orchestra.run.smithi089.stdout:(30/32): thrift-0.15.0-2.el9.x86_64.rpm 4.5 MB/s | 1.6 MB 00:00 2024-04-27T03:27:50.642 INFO:teuthology.orchestra.run.smithi089.stdout:(31/32): librados2-19.0.0-2447.gb22e2ebd.el9.x8 6.4 MB/s | 3.4 MB 00:00 2024-04-27T03:27:50.709 INFO:teuthology.orchestra.run.smithi089.stdout:(32/32): librbd1-19.0.0-2447.gb22e2ebd.el9.x86_ 5.6 MB/s | 3.1 MB 00:00 2024-04-27T03:27:50.714 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:27:50.714 INFO:teuthology.orchestra.run.smithi089.stdout:Total 9.8 MB/s | 60 MB 00:06 2024-04-27T03:27:50.816 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:27:50.853 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:27:50.853 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:27:51.317 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:27:51.318 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:27:51.573 INFO:teuthology.orchestra.run.smithi165.stdout:(1/32): ceph-base-19.0.0-2447.gb22e2ebd.el9.x86 8.2 MB/s | 5.4 MB 00:00 2024-04-27T03:27:51.632 INFO:teuthology.orchestra.run.smithi165.stdout:(2/32): ceph-selinux-19.0.0-2447.gb22e2ebd.el9. 433 kB/s | 25 kB 00:00 2024-04-27T03:27:51.732 INFO:teuthology.orchestra.run.smithi165.stdout:(3/32): libcephfs2-19.0.0-2447.gb22e2ebd.el9.x8 7.3 MB/s | 745 kB 00:00 2024-04-27T03:27:51.816 INFO:teuthology.orchestra.run.smithi165.stdout:(4/32): libradosstriper1-19.0.0-2447.gb22e2ebd. 5.9 MB/s | 501 kB 00:00 2024-04-27T03:27:51.941 INFO:teuthology.orchestra.run.smithi165.stdout:(5/32): ceph-radosgw-19.0.0-2447.gb22e2ebd.el9. 10 MB/s | 10 MB 00:01 2024-04-27T03:27:51.999 INFO:teuthology.orchestra.run.smithi165.stdout:(6/32): python3-ceph-argparse-19.0.0-2447.gb22e 777 kB/s | 45 kB 00:00 2024-04-27T03:27:52.066 INFO:teuthology.orchestra.run.smithi165.stdout:(7/32): python3-ceph-common-19.0.0-2447.gb22e2e 2.0 MB/s | 133 kB 00:00 2024-04-27T03:27:52.133 INFO:teuthology.orchestra.run.smithi165.stdout:(8/32): python3-cephfs-19.0.0-2447.gb22e2ebd.el 2.4 MB/s | 161 kB 00:00 2024-04-27T03:27:52.208 INFO:teuthology.orchestra.run.smithi165.stdout:(9/32): librgw2-19.0.0-2447.gb22e2ebd.el9.x86_6 14 MB/s | 5.3 MB 00:00 2024-04-27T03:27:52.233 INFO:teuthology.orchestra.run.smithi165.stdout:(10/32): python3-rados-19.0.0-2447.gb22e2ebd.el 3.1 MB/s | 321 kB 00:00 2024-04-27T03:27:52.259 INFO:teuthology.orchestra.run.smithi165.stdout:(11/32): python3-rbd-19.0.0-2447.gb22e2ebd.el9. 5.8 MB/s | 298 kB 00:00 2024-04-27T03:27:52.284 INFO:teuthology.orchestra.run.smithi165.stdout:(12/32): python3-rgw-19.0.0-2447.gb22e2ebd.el9. 1.9 MB/s | 99 kB 00:00 2024-04-27T03:27:52.418 INFO:teuthology.orchestra.run.smithi165.stdout:(13/32): boost-program-options-1.75.0-8.el9.x86 675 kB/s | 107 kB 00:00 2024-04-27T03:27:52.443 INFO:teuthology.orchestra.run.smithi165.stdout:(14/32): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2024-04-27T03:27:52.459 INFO:teuthology.orchestra.run.smithi165.stdout:(15/32): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2024-04-27T03:27:52.526 INFO:teuthology.orchestra.run.smithi165.stdout:(16/32): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-27T03:27:52.560 INFO:teuthology.orchestra.run.smithi165.stdout:(17/32): librdkafka-1.6.1-102.el9.x86_64.rpm 5.5 MB/s | 662 kB 00:00 2024-04-27T03:27:52.577 INFO:teuthology.orchestra.run.smithi165.stdout:(18/32): python3-prettytable-0.7.2-27.el9.noarc 838 kB/s | 42 kB 00:00 2024-04-27T03:27:52.627 INFO:teuthology.orchestra.run.smithi165.stdout:(19/32): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-27T03:27:52.736 INFO:teuthology.orchestra.run.smithi165.stdout:(20/32): qatlib-service-24.02.0-1.el9.x86_64.rp 537 kB/s | 36 kB 00:00 2024-04-27T03:27:52.736 INFO:teuthology.orchestra.run.smithi165.stdout:(21/32): qatzip-libs-1.2.0-1.el9.x86_64.rpm 945 kB/s | 47 kB 00:00 2024-04-27T03:27:52.781 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:27:52.870 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-27T03:27:52.899 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-27T03:27:52.924 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-27T03:27:52.947 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-27T03:27:52.966 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-27T03:27:53.103 INFO:teuthology.orchestra.run.smithi165.stdout:(22/32): gperftools-libs-2.9.1-2.el9.x86_64.rpm 673 kB/s | 309 kB 00:00 2024-04-27T03:27:53.189 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-27T03:27:53.195 INFO:teuthology.orchestra.run.smithi165.stdout:(23/32): libarrow-doc-9.0.0-11.el9.noarch.rpm 283 kB/s | 26 kB 00:00 2024-04-27T03:27:53.243 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-27T03:27:53.287 INFO:teuthology.orchestra.run.smithi165.stdout:(24/32): liboath-2.6.7-2.el9.x86_64.rpm 531 kB/s | 49 kB 00:00 2024-04-27T03:27:53.496 INFO:teuthology.orchestra.run.smithi165.stdout:(25/32): libarrow-9.0.0-11.el9.x86_64.rpm 5.4 MB/s | 4.4 MB 00:00 2024-04-27T03:27:53.520 INFO:teuthology.orchestra.run.smithi165.stdout:(26/32): libunwind-1.6.2-1.el9.x86_64.rpm 288 kB/s | 67 kB 00:00 2024-04-27T03:27:53.580 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-27T03:27:53.709 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librados2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 7/34 2024-04-27T03:27:53.793 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 7/34 2024-04-27T03:27:53.882 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rados-2:19.0.0-2447.gb22e2ebd.el9.x86_64 8/34 2024-04-27T03:27:53.911 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 9/34 2024-04-27T03:27:53.957 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 9/34 2024-04-27T03:27:53.986 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-27T03:27:54.007 INFO:teuthology.orchestra.run.smithi165.stdout:(27/32): ceph-common-19.0.0-2447.gb22e2ebd.el9. 6.8 MB/s | 21 MB 00:03 2024-04-27T03:27:54.029 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-argparse-2:19.0.0-2447.gb22e2ebd.el9. 11/34 2024-04-27T03:27:54.099 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cephfs-2:19.0.0-2447.gb22e2ebd.el9.x86_64 12/34 2024-04-27T03:27:54.121 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libradosstriper1-2:19.0.0-2447.gb22e2ebd.el9.x86_6 13/34 2024-04-27T03:27:54.185 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2447.gb22e2ebd.el9.x86_6 13/34 2024-04-27T03:27:54.211 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-27T03:27:54.246 INFO:teuthology.orchestra.run.smithi165.stdout:(28/32): re2-20211101-3.el9.x86_64.rpm 264 kB/s | 192 kB 00:00 2024-04-27T03:27:54.321 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-27T03:27:54.353 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-04-27T03:27:54.394 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-27T03:27:54.419 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-27T03:27:54.448 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-04-27T03:27:54.480 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-27T03:27:54.481 INFO:teuthology.orchestra.run.smithi165.stdout:(29/32): parquet-libs-9.0.0-11.el9.x86_64.rpm 852 kB/s | 839 kB 00:00 2024-04-27T03:27:54.564 INFO:teuthology.orchestra.run.smithi165.stdout:(30/32): librados2-19.0.0-2447.gb22e2ebd.el9.x8 11 MB/s | 3.4 MB 00:00 2024-04-27T03:27:54.689 INFO:teuthology.orchestra.run.smithi165.stdout:(31/32): thrift-0.15.0-2.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-04-27T03:27:54.795 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-27T03:27:54.806 INFO:teuthology.orchestra.run.smithi165.stdout:(32/32): librbd1-19.0.0-2447.gb22e2ebd.el9.x86_ 9.7 MB/s | 3.1 MB 00:00 2024-04-27T03:27:54.809 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:27:54.809 INFO:teuthology.orchestra.run.smithi165.stdout:Total 13 MB/s | 60 MB 00:04 2024-04-27T03:27:54.843 INFO:teuthology.orchestra.run.smithi089.stdout: Upgrading : librbd1-2:19.0.0-2447.gb22e2ebd.el9.x86_64 22/34 2024-04-27T03:27:54.912 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:27:54.941 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:19.0.0-2447.gb22e2ebd.el9.x86_64 22/34 2024-04-27T03:27:54.948 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:27:54.948 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:27:54.973 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 23/34 2024-04-27T03:27:55.009 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-27T03:27:55.413 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:27:55.413 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:27:55.471 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-04-27T03:27:55.505 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librgw2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 26/34 2024-04-27T03:27:55.576 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librgw2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 26/34 2024-04-27T03:27:55.654 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 27/34 2024-04-27T03:27:55.682 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x8 28/34 2024-04-27T03:27:56.905 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:27:56.990 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/34 2024-04-27T03:27:57.019 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 2/34 2024-04-27T03:27:57.044 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 3/34 2024-04-27T03:27:57.067 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-27T03:27:57.089 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 4/34 2024-04-27T03:27:57.251 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-27T03:27:57.304 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 5/34 2024-04-27T03:27:57.679 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : thrift-0.15.0-2.el9.x86_64 6/34 2024-04-27T03:27:57.745 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librados2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 7/34 2024-04-27T03:27:57.837 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 7/34 2024-04-27T03:27:57.924 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rados-2:19.0.0-2447.gb22e2ebd.el9.x86_64 8/34 2024-04-27T03:27:57.964 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 9/34 2024-04-27T03:27:58.018 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 9/34 2024-04-27T03:27:58.029 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x86_64 29/34 2024-04-27T03:27:58.046 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : liboath-2.6.7-2.el9.x86_64 10/34 2024-04-27T03:27:58.071 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x86_64 29/34 2024-04-27T03:27:58.089 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-argparse-2:19.0.0-2447.gb22e2ebd.el9. 11/34 2024-04-27T03:27:58.158 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cephfs-2:19.0.0-2447.gb22e2ebd.el9.x86_64 12/34 2024-04-27T03:27:58.182 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libradosstriper1-2:19.0.0-2447.gb22e2ebd.el9.x86_6 13/34 2024-04-27T03:27:58.246 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libradosstriper1-2:19.0.0-2447.gb22e2ebd.el9.x86_6 13/34 2024-04-27T03:27:58.280 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 14/34 2024-04-27T03:27:58.387 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : re2-1:20211101-3.el9.x86_64 15/34 2024-04-27T03:27:58.423 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-9.0.0-11.el9.x86_64 16/34 2024-04-27T03:27:58.463 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/34 2024-04-27T03:27:58.488 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 18/34 2024-04-27T03:27:58.517 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libarrow-doc-9.0.0-11.el9.noarch 19/34 2024-04-27T03:27:58.550 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 20/34 2024-04-27T03:27:58.614 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x86_64 29/34 2024-04-27T03:27:58.649 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-base-2:19.0.0-2447.gb22e2ebd.el9.x86_64 30/34 2024-04-27T03:27:58.746 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-base-2:19.0.0-2447.gb22e2ebd.el9.x86_64 30/34 2024-04-27T03:27:58.746 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-27T03:27:58.747 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-27T03:27:58.747 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:27:58.775 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-selinux-2:19.0.0-2447.gb22e2ebd.el9.x86_64 31/34 2024-04-27T03:27:58.852 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 21/34 2024-04-27T03:27:58.887 INFO:teuthology.orchestra.run.smithi165.stdout: Upgrading : librbd1-2:19.0.0-2447.gb22e2ebd.el9.x86_64 22/34 2024-04-27T03:27:58.960 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:19.0.0-2447.gb22e2ebd.el9.x86_64 22/34 2024-04-27T03:27:58.992 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 23/34 2024-04-27T03:27:59.028 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 24/34 2024-04-27T03:27:59.497 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : parquet-libs-9.0.0-11.el9.x86_64 25/34 2024-04-27T03:27:59.523 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librgw2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 26/34 2024-04-27T03:27:59.594 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librgw2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 26/34 2024-04-27T03:27:59.662 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 27/34 2024-04-27T03:27:59.693 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x8 28/34 2024-04-27T03:28:01.990 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x86_64 29/34 2024-04-27T03:28:02.071 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x86_64 29/34 2024-04-27T03:28:02.610 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x86_64 29/34 2024-04-27T03:28:02.641 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-base-2:19.0.0-2447.gb22e2ebd.el9.x86_64 30/34 2024-04-27T03:28:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-base-2:19.0.0-2447.gb22e2ebd.el9.x86_64 30/34 2024-04-27T03:28:02.725 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-27T03:28:02.725 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-27T03:28:02.725 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:02.759 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-selinux-2:19.0.0-2447.gb22e2ebd.el9.x86_64 31/34 2024-04-27T03:28:08.202 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2447.gb22e2ebd.el9.x86_64 31/34 2024-04-27T03:28:08.203 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /sys 2024-04-27T03:28:08.203 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /proc 2024-04-27T03:28:08.203 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /mnt 2024-04-27T03:28:08.203 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /var/tmp 2024-04-27T03:28:08.203 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /home 2024-04-27T03:28:08.203 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /root 2024-04-27T03:28:08.203 INFO:teuthology.orchestra.run.smithi089.stdout:skipping the directory /tmp 2024-04-27T03:28:08.203 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:09.167 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-radosgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 32/34 2024-04-27T03:28:09.191 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 32/34 2024-04-27T03:28:09.191 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:28:09.191 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-27T03:28:09.191 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-27T03:28:09.191 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-27T03:28:09.191 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:09.193 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-27T03:28:09.278 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-27T03:28:09.278 INFO:teuthology.orchestra.run.smithi089.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-27T03:28:10.426 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-27T03:28:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-base-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/34 2024-04-27T03:28:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2/34 2024-04-27T03:28:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-radosgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 3/34 2024-04-27T03:28:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-selinux-2:19.0.0-2447.gb22e2ebd.el9.x86_64 4/34 2024-04-27T03:28:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 5/34 2024-04-27T03:28:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libradosstriper1-2:19.0.0-2447.gb22e2ebd.el9.x86_6 6/34 2024-04-27T03:28:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librgw2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 7/34 2024-04-27T03:28:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2447.gb22e2ebd.el9. 8/34 2024-04-27T03:28:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x8 9/34 2024-04-27T03:28:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cephfs-2:19.0.0-2447.gb22e2ebd.el9.x86_64 10/34 2024-04-27T03:28:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rados-2:19.0.0-2447.gb22e2ebd.el9.x86_64 11/34 2024-04-27T03:28:10.427 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 12/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 13/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-27T03:28:10.428 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-27T03:28:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-04-27T03:28:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-27T03:28:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-27T03:28:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 31/34 2024-04-27T03:28:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-27T03:28:10.429 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:19.0.0-2447.gb22e2ebd.el9.x86_64 33/34 2024-04-27T03:28:11.283 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-27T03:28:11.283 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:11.283 INFO:teuthology.orchestra.run.smithi089.stdout:Upgraded: 2024-04-27T03:28:11.283 INFO:teuthology.orchestra.run.smithi089.stdout: librados2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.283 INFO:teuthology.orchestra.run.smithi089.stdout: librbd1-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.283 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:28:11.283 INFO:teuthology.orchestra.run.smithi089.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-27T03:28:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-base-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-radosgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-selinux-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-27T03:28:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-27T03:28:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-27T03:28:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-27T03:28:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-27T03:28:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-27T03:28:11.284 INFO:teuthology.orchestra.run.smithi089.stdout: libradosstriper1-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: librgw2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-argparse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cephfs-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rados-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-27T03:28:11.285 INFO:teuthology.orchestra.run.smithi089.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-27T03:28:11.286 INFO:teuthology.orchestra.run.smithi089.stdout: re2-1:20211101-3.el9.x86_64 2024-04-27T03:28:11.286 INFO:teuthology.orchestra.run.smithi089.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-27T03:28:11.286 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:11.286 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:28:11.582 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-test 2024-04-27T03:28:12.005 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-selinux-2:19.0.0-2447.gb22e2ebd.el9.x86_64 31/34 2024-04-27T03:28:12.005 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /sys 2024-04-27T03:28:12.005 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /proc 2024-04-27T03:28:12.005 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /mnt 2024-04-27T03:28:12.005 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /var/tmp 2024-04-27T03:28:12.005 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /home 2024-04-27T03:28:12.005 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /root 2024-04-27T03:28:12.005 INFO:teuthology.orchestra.run.smithi165.stdout:skipping the directory /tmp 2024-04-27T03:28:12.005 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:12.113 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:30 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:28:12.234 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:28:12.234 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:12.234 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-04-27T03:28:12.234 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:12.235 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:28:12.235 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 48 M 2024-04-27T03:28:12.235 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-04-27T03:28:12.235 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 164 k 2024-04-27T03:28:12.235 INFO:teuthology.orchestra.run.smithi089.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-27T03:28:12.235 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-27T03:28:12.235 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:12.235 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:28:12.235 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:12.235 INFO:teuthology.orchestra.run.smithi089.stdout:Install 4 Packages 2024-04-27T03:28:12.235 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:12.236 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 49 M 2024-04-27T03:28:12.236 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 207 M 2024-04-27T03:28:12.236 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:28:12.923 INFO:teuthology.orchestra.run.smithi089.stdout:(1/4): libcephsqlite-19.0.0-2447.gb22e2ebd.el9. 314 kB/s | 164 kB 00:00 2024-04-27T03:28:12.982 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-radosgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 32/34 2024-04-27T03:28:13.006 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 32/34 2024-04-27T03:28:13.006 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:28:13.006 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-27T03:28:13.006 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-27T03:28:13.007 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-27T03:28:13.007 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:13.008 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-27T03:28:13.093 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 33/34 2024-04-27T03:28:13.093 INFO:teuthology.orchestra.run.smithi165.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-27T03:28:13.663 INFO:teuthology.orchestra.run.smithi089.stdout:(2/4): socat-1.7.4.1-5.el9.x86_64.rpm 242 kB/s | 305 kB 00:01 2024-04-27T03:28:13.756 INFO:teuthology.orchestra.run.smithi089.stdout:(3/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 76 kB/s | 64 kB 00:00 2024-04-27T03:28:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 34/34 2024-04-27T03:28:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-base-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/34 2024-04-27T03:28:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2/34 2024-04-27T03:28:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-radosgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 3/34 2024-04-27T03:28:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-selinux-2:19.0.0-2447.gb22e2ebd.el9.x86_64 4/34 2024-04-27T03:28:14.231 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 5/34 2024-04-27T03:28:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libradosstriper1-2:19.0.0-2447.gb22e2ebd.el9.x86_6 6/34 2024-04-27T03:28:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librgw2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 7/34 2024-04-27T03:28:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-argparse-2:19.0.0-2447.gb22e2ebd.el9. 8/34 2024-04-27T03:28:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x8 9/34 2024-04-27T03:28:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cephfs-2:19.0.0-2447.gb22e2ebd.el9.x86_64 10/34 2024-04-27T03:28:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rados-2:19.0.0-2447.gb22e2ebd.el9.x86_64 11/34 2024-04-27T03:28:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 12/34 2024-04-27T03:28:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 13/34 2024-04-27T03:28:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 14/34 2024-04-27T03:28:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 15/34 2024-04-27T03:28:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 16/34 2024-04-27T03:28:14.232 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 17/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 18/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 19/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 20/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 21/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 22/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 23/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-9.0.0-11.el9.x86_64 24/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libarrow-doc-9.0.0-11.el9.noarch 25/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 26/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 27/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : parquet-libs-9.0.0-11.el9.x86_64 28/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : re2-1:20211101-3.el9.x86_64 29/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 30/34 2024-04-27T03:28:14.233 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 31/34 2024-04-27T03:28:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 32/34 2024-04-27T03:28:14.234 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:19.0.0-2447.gb22e2ebd.el9.x86_64 33/34 2024-04-27T03:28:14.965 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 34/34 2024-04-27T03:28:14.966 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:14.966 INFO:teuthology.orchestra.run.smithi165.stdout:Upgraded: 2024-04-27T03:28:14.966 INFO:teuthology.orchestra.run.smithi165.stdout: librados2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.966 INFO:teuthology.orchestra.run.smithi165.stdout: librbd1-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.966 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:28:14.966 INFO:teuthology.orchestra.run.smithi165.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-27T03:28:14.966 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-base-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.966 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.966 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-radosgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.966 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-selinux-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.967 INFO:teuthology.orchestra.run.smithi165.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-27T03:28:14.967 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-9.0.0-11.el9.x86_64 2024-04-27T03:28:14.967 INFO:teuthology.orchestra.run.smithi165.stdout: libarrow-doc-9.0.0-11.el9.noarch 2024-04-27T03:28:14.967 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.967 INFO:teuthology.orchestra.run.smithi165.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-27T03:28:14.967 INFO:teuthology.orchestra.run.smithi165.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-27T03:28:14.967 INFO:teuthology.orchestra.run.smithi165.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-27T03:28:14.967 INFO:teuthology.orchestra.run.smithi165.stdout: libradosstriper1-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.967 INFO:teuthology.orchestra.run.smithi165.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-27T03:28:14.967 INFO:teuthology.orchestra.run.smithi165.stdout: librgw2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.967 INFO:teuthology.orchestra.run.smithi165.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-27T03:28:14.967 INFO:teuthology.orchestra.run.smithi165.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: parquet-libs-9.0.0-11.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-argparse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ceph-common-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cephfs-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rados-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: re2-1:20211101-3.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-27T03:28:14.968 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:14.969 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:28:15.199 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-test 2024-04-27T03:28:15.707 INFO:teuthology.orchestra.run.smithi089.stdout:(4/4): ceph-test-19.0.0-2447.gb22e2ebd.el9.x86_ 15 MB/s | 48 MB 00:03 2024-04-27T03:28:15.709 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:28:15.709 INFO:teuthology.orchestra.run.smithi089.stdout:Total 14 MB/s | 49 MB 00:03 2024-04-27T03:28:15.735 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:28:15.738 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:27 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:28:15.755 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:28:15.755 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:28:15.861 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:28:15.862 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:15.862 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-27T03:28:15.862 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:15.862 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:28:15.862 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 48 M 2024-04-27T03:28:15.862 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-27T03:28:15.862 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 164 k 2024-04-27T03:28:15.862 INFO:teuthology.orchestra.run.smithi165.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-27T03:28:15.862 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-27T03:28:15.863 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:15.863 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:28:15.863 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:15.863 INFO:teuthology.orchestra.run.smithi165.stdout:Install 4 Packages 2024-04-27T03:28:15.863 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:15.863 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 49 M 2024-04-27T03:28:15.864 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 207 M 2024-04-27T03:28:15.864 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:28:16.117 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:28:16.117 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:28:16.140 INFO:teuthology.orchestra.run.smithi165.stdout:(1/4): socat-1.7.4.1-5.el9.x86_64.rpm 1.4 MB/s | 305 kB 00:00 2024-04-27T03:28:16.190 INFO:teuthology.orchestra.run.smithi165.stdout:(2/4): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.3 MB/s | 64 kB 00:00 2024-04-27T03:28:16.366 INFO:teuthology.orchestra.run.smithi165.stdout:(3/4): libcephsqlite-19.0.0-2447.gb22e2ebd.el9. 377 kB/s | 164 kB 00:00 2024-04-27T03:28:16.640 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:28:16.715 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-27T03:28:16.755 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-27T03:28:16.784 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephsqlite-2:19.0.0-2447.gb22e2ebd.el9.x86_64 3/4 2024-04-27T03:28:18.261 INFO:teuthology.orchestra.run.smithi165.stdout:(4/4): ceph-test-19.0.0-2447.gb22e2ebd.el9.x86_ 21 MB/s | 48 MB 00:02 2024-04-27T03:28:18.263 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:28:18.263 INFO:teuthology.orchestra.run.smithi165.stdout:Total 20 MB/s | 49 MB 00:02 2024-04-27T03:28:18.288 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:28:18.307 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:28:18.308 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:28:18.672 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:28:18.672 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:28:19.183 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:28:19.252 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 1/4 2024-04-27T03:28:19.293 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 2/4 2024-04-27T03:28:19.321 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephsqlite-2:19.0.0-2447.gb22e2ebd.el9.x86_64 3/4 2024-04-27T03:28:21.286 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2447.gb22e2ebd.el9.x86_64 3/4 2024-04-27T03:28:21.615 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-test-2:19.0.0-2447.gb22e2ebd.el9.x86_64 4/4 2024-04-27T03:28:23.882 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephsqlite-2:19.0.0-2447.gb22e2ebd.el9.x86_64 3/4 2024-04-27T03:28:23.983 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-test-2:19.0.0-2447.gb22e2ebd.el9.x86_64 4/4 2024-04-27T03:28:24.115 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-test-2:19.0.0-2447.gb22e2ebd.el9.x86_64 4/4 2024-04-27T03:28:24.115 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-test-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/4 2024-04-27T03:28:24.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephsqlite-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2/4 2024-04-27T03:28:24.116 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-27T03:28:24.404 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-27T03:28:24.405 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:24.405 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:28:24.405 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-test-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:24.405 INFO:teuthology.orchestra.run.smithi089.stdout: libcephsqlite-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:24.405 INFO:teuthology.orchestra.run.smithi089.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-27T03:28:24.405 INFO:teuthology.orchestra.run.smithi089.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-27T03:28:24.405 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:24.405 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:28:24.683 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph 2024-04-27T03:28:25.189 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:43 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:28:25.318 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:28:25.321 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:25.321 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-04-27T03:28:25.321 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:25.321 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:28:25.321 INFO:teuthology.orchestra.run.smithi089.stdout: ceph x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 6.6 k 2024-04-27T03:28:25.321 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-04-27T03:28:25.321 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 2.4 M 2024-04-27T03:28:25.321 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 1.5 M 2024-04-27T03:28:25.321 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 249 k 2024-04-27T03:28:25.321 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 4.7 M 2024-04-27T03:28:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 17 M 2024-04-27T03:28:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-27T03:28:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-27T03:28:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-27T03:28:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-27T03:28:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-27T03:28:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-27T03:28:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-27T03:28:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-27T03:28:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-27T03:28:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-27T03:28:25.322 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-27T03:28:25.323 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-27T03:28:25.324 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-27T03:28:25.324 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-27T03:28:25.324 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-27T03:28:25.324 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-27T03:28:25.324 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-27T03:28:25.324 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-27T03:28:25.324 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:25.324 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:28:25.324 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:25.324 INFO:teuthology.orchestra.run.smithi089.stdout:Install 36 Packages 2024-04-27T03:28:25.325 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:25.325 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 30 M 2024-04-27T03:28:25.325 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 107 M 2024-04-27T03:28:25.325 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:28:26.325 INFO:teuthology.orchestra.run.smithi089.stdout:(1/36): ceph-19.0.0-2447.gb22e2ebd.el9.x86_64.r 38 kB/s | 6.6 kB 00:00 2024-04-27T03:28:26.402 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-test-2:19.0.0-2447.gb22e2ebd.el9.x86_64 4/4 2024-04-27T03:28:26.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-test-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/4 2024-04-27T03:28:26.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephsqlite-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2/4 2024-04-27T03:28:26.402 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 3/4 2024-04-27T03:28:26.664 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 4/4 2024-04-27T03:28:26.664 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:26.664 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:28:26.664 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-test-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:26.664 INFO:teuthology.orchestra.run.smithi165.stdout: libcephsqlite-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:26.664 INFO:teuthology.orchestra.run.smithi165.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-27T03:28:26.664 INFO:teuthology.orchestra.run.smithi165.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-27T03:28:26.664 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:26.664 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:28:26.675 INFO:teuthology.orchestra.run.smithi089.stdout:(2/36): ceph-mon-19.0.0-2447.gb22e2ebd.el9.x86_ 13 MB/s | 4.7 MB 00:00 2024-04-27T03:28:26.950 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph 2024-04-27T03:28:27.484 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:39 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:28:27.518 INFO:teuthology.orchestra.run.smithi089.stdout:(3/36): ceph-osd-19.0.0-2447.gb22e2ebd.el9.x86_ 20 MB/s | 17 MB 00:00 2024-04-27T03:28:27.577 INFO:teuthology.orchestra.run.smithi089.stdout:(4/36): ceph-mds-19.0.0-2447.gb22e2ebd.el9.x86_ 1.7 MB/s | 2.4 MB 00:01 2024-04-27T03:28:27.602 INFO:teuthology.orchestra.run.smithi089.stdout:(5/36): ceph-mgr-modules-core-19.0.0-2447.gb22e 2.9 MB/s | 249 kB 00:00 2024-04-27T03:28:27.612 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:28:27.614 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:27.615 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-27T03:28:27.615 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:27.615 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:28:27.615 INFO:teuthology.orchestra.run.smithi165.stdout: ceph x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 6.6 k 2024-04-27T03:28:27.615 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-27T03:28:27.615 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 2.4 M 2024-04-27T03:28:27.615 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 1.5 M 2024-04-27T03:28:27.615 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 249 k 2024-04-27T03:28:27.615 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 4.7 M 2024-04-27T03:28:27.615 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 17 M 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: noarch 3.0.0-8.el9 epel 23 k 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-27T03:28:27.616 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-27T03:28:27.617 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-27T03:28:27.618 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-27T03:28:27.618 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-27T03:28:27.618 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-27T03:28:27.618 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:27.618 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:28:27.618 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:27.618 INFO:teuthology.orchestra.run.smithi165.stdout:Install 36 Packages 2024-04-27T03:28:27.618 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:27.619 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 30 M 2024-04-27T03:28:27.619 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 107 M 2024-04-27T03:28:27.619 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:28:27.744 INFO:teuthology.orchestra.run.smithi089.stdout:(6/36): ceph-mgr-19.0.0-2447.gb22e2ebd.el9.x86_ 960 kB/s | 1.5 MB 00:01 2024-04-27T03:28:27.802 INFO:teuthology.orchestra.run.smithi089.stdout:(7/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2024-04-27T03:28:27.852 INFO:teuthology.orchestra.run.smithi089.stdout:(8/36): python3-pycparser-2.20-6.el9.noarch.rpm 2.6 MB/s | 135 kB 00:00 2024-04-27T03:28:27.911 INFO:teuthology.orchestra.run.smithi089.stdout:(9/36): python3-requests-2.25.1-8.el9.noarch.rp 2.1 MB/s | 125 kB 00:00 2024-04-27T03:28:27.936 INFO:teuthology.orchestra.run.smithi089.stdout:(10/36): python3-ply-3.11-14.el9.noarch.rpm 553 kB/s | 106 kB 00:00 2024-04-27T03:28:27.978 INFO:teuthology.orchestra.run.smithi089.stdout:(11/36): python3-cryptography-36.0.1-4.el9.x86_ 3.3 MB/s | 1.2 MB 00:00 2024-04-27T03:28:28.004 INFO:teuthology.orchestra.run.smithi089.stdout:(12/36): python3-urllib3-1.26.5-5.el9.noarch.rp 2.3 MB/s | 215 kB 00:00 2024-04-27T03:28:28.029 INFO:teuthology.orchestra.run.smithi089.stdout:(13/36): lua-5.4.4-4.el9.x86_64.rpm 2.0 MB/s | 188 kB 00:00 2024-04-27T03:28:28.054 INFO:teuthology.orchestra.run.smithi089.stdout:(14/36): python3-mako-1.1.4-6.el9.noarch.rpm 2.2 MB/s | 172 kB 00:00 2024-04-27T03:28:28.079 INFO:teuthology.orchestra.run.smithi089.stdout:(15/36): python3-markupsafe-1.1.1-12.el9.x86_64 460 kB/s | 35 kB 00:00 2024-04-27T03:28:28.105 INFO:teuthology.orchestra.run.smithi089.stdout:(16/36): python3-pytz-2021.1-5.el9.noarch.rpm 674 kB/s | 51 kB 00:00 2024-04-27T03:28:28.130 INFO:teuthology.orchestra.run.smithi089.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 552 kB/s | 42 kB 00:00 2024-04-27T03:28:28.155 INFO:teuthology.orchestra.run.smithi089.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 294 kB/s | 22 kB 00:00 2024-04-27T03:28:28.322 INFO:teuthology.orchestra.run.smithi089.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 226 kB/s | 43 kB 00:00 2024-04-27T03:28:28.389 INFO:teuthology.orchestra.run.smithi089.stdout:(20/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 736 kB/s | 172 kB 00:00 2024-04-27T03:28:28.414 INFO:teuthology.orchestra.run.smithi089.stdout:(21/36): luarocks-3.9.2-1.el9.noarch.rpm 488 kB/s | 151 kB 00:00 2024-04-27T03:28:28.456 INFO:teuthology.orchestra.run.smithi089.stdout:(22/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 160 kB/s | 11 kB 00:00 2024-04-27T03:28:28.482 INFO:teuthology.orchestra.run.smithi089.stdout:(23/36): python3-jaraco-classes-3.2.1-5.el9.noa 264 kB/s | 18 kB 00:00 2024-04-27T03:28:28.511 INFO:teuthology.orchestra.run.smithi165.stdout:(1/36): ceph-19.0.0-2447.gb22e2ebd.el9.x86_64.r 30 kB/s | 6.6 kB 00:00 2024-04-27T03:28:28.523 INFO:teuthology.orchestra.run.smithi089.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2024-04-27T03:28:28.549 INFO:teuthology.orchestra.run.smithi089.stdout:(25/36): python3-jaraco-collections-3.0.0-8.el9 252 kB/s | 23 kB 00:00 2024-04-27T03:28:28.574 INFO:teuthology.orchestra.run.smithi089.stdout:(26/36): python3-jaraco-functools-3.5.0-2.el9.n 211 kB/s | 19 kB 00:00 2024-04-27T03:28:28.616 INFO:teuthology.orchestra.run.smithi089.stdout:(27/36): python3-jaraco-text-3.2.0-6.el9.noarch 294 kB/s | 20 kB 00:00 2024-04-27T03:28:28.628 INFO:teuthology.orchestra.run.smithi165.stdout:(2/36): ceph-mgr-19.0.0-2447.gb22e2ebd.el9.x86_ 4.4 MB/s | 1.5 MB 00:00 2024-04-27T03:28:28.641 INFO:teuthology.orchestra.run.smithi089.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 502 kB/s | 46 kB 00:00 2024-04-27T03:28:28.666 INFO:teuthology.orchestra.run.smithi089.stdout:(29/36): python3-more-itertools-8.12.0-2.el9.no 855 kB/s | 79 kB 00:00 2024-04-27T03:28:28.692 INFO:teuthology.orchestra.run.smithi089.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-27T03:28:28.717 INFO:teuthology.orchestra.run.smithi089.stdout:(31/36): python3-portend-3.1.0-2.el9.noarch.rpm 218 kB/s | 16 kB 00:00 2024-04-27T03:28:28.742 INFO:teuthology.orchestra.run.smithi089.stdout:(32/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2024-04-27T03:28:28.767 INFO:teuthology.orchestra.run.smithi089.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 473 kB/s | 36 kB 00:00 2024-04-27T03:28:28.826 INFO:teuthology.orchestra.run.smithi089.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 2.1 MB/s | 230 kB 00:00 2024-04-27T03:28:28.852 INFO:teuthology.orchestra.run.smithi089.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 238 kB/s | 20 kB 00:00 2024-04-27T03:28:28.877 INFO:teuthology.orchestra.run.smithi089.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 3.1 MB/s | 427 kB 00:00 2024-04-27T03:28:28.878 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:28:28.878 INFO:teuthology.orchestra.run.smithi089.stdout:Total 8.5 MB/s | 30 MB 00:03 2024-04-27T03:28:29.017 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:28:29.057 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:28:29.057 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:28:29.219 INFO:teuthology.orchestra.run.smithi165.stdout:(3/36): ceph-mds-19.0.0-2447.gb22e2ebd.el9.x86_ 2.5 MB/s | 2.4 MB 00:00 2024-04-27T03:28:29.336 INFO:teuthology.orchestra.run.smithi165.stdout:(4/36): ceph-mgr-modules-core-19.0.0-2447.gb22e 2.1 MB/s | 249 kB 00:00 2024-04-27T03:28:29.350 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:28:29.350 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:28:29.503 INFO:teuthology.orchestra.run.smithi165.stdout:(5/36): ceph-osd-19.0.0-2447.gb22e2ebd.el9.x86_ 19 MB/s | 17 MB 00:00 2024-04-27T03:28:29.678 INFO:teuthology.orchestra.run.smithi165.stdout:(6/36): python3-cffi-1.14.5-5.el9.x86_64.rpm 741 kB/s | 253 kB 00:00 2024-04-27T03:28:29.854 INFO:teuthology.orchestra.run.smithi165.stdout:(7/36): python3-cryptography-36.0.1-4.el9.x86_6 3.5 MB/s | 1.2 MB 00:00 2024-04-27T03:28:29.904 INFO:teuthology.orchestra.run.smithi165.stdout:(8/36): python3-ply-3.11-14.el9.noarch.rpm 473 kB/s | 106 kB 00:00 2024-04-27T03:28:29.929 INFO:teuthology.orchestra.run.smithi165.stdout:(9/36): python3-pycparser-2.20-6.el9.noarch.rpm 1.8 MB/s | 135 kB 00:00 2024-04-27T03:28:29.988 INFO:teuthology.orchestra.run.smithi165.stdout:(10/36): python3-requests-2.25.1-8.el9.noarch.r 1.5 MB/s | 125 kB 00:00 2024-04-27T03:28:30.013 INFO:teuthology.orchestra.run.smithi165.stdout:(11/36): python3-urllib3-1.26.5-5.el9.noarch.rp 2.5 MB/s | 215 kB 00:00 2024-04-27T03:28:30.088 INFO:teuthology.orchestra.run.smithi165.stdout:(12/36): ceph-mon-19.0.0-2447.gb22e2ebd.el9.x86 3.0 MB/s | 4.7 MB 00:01 2024-04-27T03:28:30.180 INFO:teuthology.orchestra.run.smithi165.stdout:(13/36): python3-markupsafe-1.1.1-12.el9.x86_64 380 kB/s | 35 kB 00:00 2024-04-27T03:28:30.206 INFO:teuthology.orchestra.run.smithi165.stdout:(14/36): python3-mako-1.1.4-6.el9.noarch.rpm 897 kB/s | 172 kB 00:00 2024-04-27T03:28:30.230 INFO:teuthology.orchestra.run.smithi165.stdout:(15/36): lua-5.4.4-4.el9.x86_64.rpm 777 kB/s | 188 kB 00:00 2024-04-27T03:28:30.256 INFO:teuthology.orchestra.run.smithi165.stdout:(16/36): python3-pytz-2021.1-5.el9.noarch.rpm 675 kB/s | 51 kB 00:00 2024-04-27T03:28:30.281 INFO:teuthology.orchestra.run.smithi165.stdout:(17/36): python3-toml-0.10.2-6.el9.noarch.rpm 553 kB/s | 42 kB 00:00 2024-04-27T03:28:30.306 INFO:teuthology.orchestra.run.smithi165.stdout:(18/36): lua-devel-5.4.4-4.el9.x86_64.rpm 295 kB/s | 22 kB 00:00 2024-04-27T03:28:30.398 INFO:teuthology.orchestra.run.smithi165.stdout:(19/36): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 373 kB/s | 43 kB 00:00 2024-04-27T03:28:30.448 INFO:teuthology.orchestra.run.smithi165.stdout:(20/36): luarocks-3.9.2-1.el9.noarch.rpm 905 kB/s | 151 kB 00:00 2024-04-27T03:28:30.474 INFO:teuthology.orchestra.run.smithi165.stdout:(21/36): python3-cheroot-8.6.0-4.el9.noarch.rpm 1.0 MB/s | 172 kB 00:00 2024-04-27T03:28:30.594 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:28:30.599 INFO:teuthology.orchestra.run.smithi165.stdout:(22/36): python3-jaraco-classes-3.2.1-5.el9.noa 142 kB/s | 18 kB 00:00 2024-04-27T03:28:30.649 INFO:teuthology.orchestra.run.smithi165.stdout:(23/36): python3-jaraco-collections-3.0.0-8.el9 461 kB/s | 23 kB 00:00 2024-04-27T03:28:30.662 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-27T03:28:30.674 INFO:teuthology.orchestra.run.smithi165.stdout:(24/36): python3-cherrypy-18.6.1-2.el9.noarch.r 1.3 MB/s | 358 kB 00:00 2024-04-27T03:28:30.695 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-27T03:28:30.699 INFO:teuthology.orchestra.run.smithi165.stdout:(25/36): python3-jaraco-functools-3.5.0-2.el9.n 387 kB/s | 19 kB 00:00 2024-04-27T03:28:30.716 INFO:teuthology.orchestra.run.smithi165.stdout:(26/36): python3-jaraco-text-3.2.0-6.el9.noarch 471 kB/s | 20 kB 00:00 2024-04-27T03:28:30.732 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-27T03:28:30.792 INFO:teuthology.orchestra.run.smithi165.stdout:(27/36): python3-more-itertools-8.12.0-2.el9.no 1.0 MB/s | 79 kB 00:00 2024-04-27T03:28:30.802 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-27T03:28:30.808 INFO:teuthology.orchestra.run.smithi165.stdout:(28/36): python3-logutils-0.3.5-21.el9.noarch.r 425 kB/s | 46 kB 00:00 2024-04-27T03:28:30.825 INFO:teuthology.orchestra.run.smithi165.stdout:(29/36): python3-jaraco-8.2.1-3.el9.noarch.rpm 28 kB/s | 11 kB 00:00 2024-04-27T03:28:30.859 INFO:teuthology.orchestra.run.smithi165.stdout:(30/36): python3-pecan-1.4.2-3.el9.noarch.rpm 4.0 MB/s | 272 kB 00:00 2024-04-27T03:28:30.872 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-27T03:28:30.893 INFO:teuthology.orchestra.run.smithi165.stdout:(31/36): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.3 MB/s | 90 kB 00:00 2024-04-27T03:28:30.905 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-27T03:28:30.918 INFO:teuthology.orchestra.run.smithi165.stdout:(32/36): python3-portend-3.1.0-2.el9.noarch.rpm 151 kB/s | 16 kB 00:00 2024-04-27T03:28:30.985 INFO:teuthology.orchestra.run.smithi165.stdout:(33/36): python3-tempora-5.0.0-2.el9.noarch.rpm 285 kB/s | 36 kB 00:00 2024-04-27T03:28:30.995 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-27T03:28:31.019 INFO:teuthology.orchestra.run.smithi165.stdout:(34/36): python3-webob-1.8.7-6.el9.noarch.rpm 1.8 MB/s | 230 kB 00:00 2024-04-27T03:28:31.044 INFO:teuthology.orchestra.run.smithi165.stdout:(35/36): python3-zc-lockfile-2.0-10.el9.noarch. 341 kB/s | 20 kB 00:00 2024-04-27T03:28:31.074 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-27T03:28:31.116 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-27T03:28:31.127 INFO:teuthology.orchestra.run.smithi165.stdout:(36/36): python3-werkzeug-2.0.3-3.el9.1.noarch. 2.0 MB/s | 427 kB 00:00 2024-04-27T03:28:31.129 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:28:31.129 INFO:teuthology.orchestra.run.smithi165.stdout:Total 8.6 MB/s | 30 MB 00:03 2024-04-27T03:28:31.151 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-27T03:28:31.181 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-27T03:28:31.226 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-27T03:28:31.257 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-27T03:28:31.267 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:28:31.298 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-27T03:28:31.306 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:28:31.306 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:28:31.332 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-27T03:28:31.367 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-27T03:28:31.405 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-27T03:28:31.486 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-27T03:28:31.599 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:28:31.601 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:28:31.624 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-27T03:28:31.704 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-27T03:28:31.766 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-27T03:28:31.805 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-27T03:28:31.862 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-27T03:28:31.917 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-27T03:28:32.104 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-27T03:28:32.151 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-27T03:28:32.229 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-27T03:28:32.317 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-27T03:28:32.358 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-27T03:28:32.474 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-27T03:28:32.639 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2447.gb22e2ebd.el9. 31/36 2024-04-27T03:28:32.669 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-2:19.0.0-2447.gb22e2ebd.el9.x86_64 32/36 2024-04-27T03:28:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2447.gb22e2ebd.el9.x86_64 32/36 2024-04-27T03:28:32.695 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:28:32.695 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-27T03:28:32.695 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-27T03:28:32.695 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-27T03:28:32.695 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:32.849 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:28:32.922 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/36 2024-04-27T03:28:32.954 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/36 2024-04-27T03:28:32.982 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/36 2024-04-27T03:28:33.045 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 4/36 2024-04-27T03:28:33.098 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : luarocks-3.9.2-1.el9.noarch 5/36 2024-04-27T03:28:33.131 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 6/36 2024-04-27T03:28:33.223 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 7/36 2024-04-27T03:28:33.301 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 8/36 2024-04-27T03:28:33.341 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 9/36 2024-04-27T03:28:33.385 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 10/36 2024-04-27T03:28:33.415 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 11/36 2024-04-27T03:28:33.449 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 12/36 2024-04-27T03:28:33.482 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 13/36 2024-04-27T03:28:33.523 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 14/36 2024-04-27T03:28:33.575 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 15/36 2024-04-27T03:28:33.609 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 16/36 2024-04-27T03:28:33.647 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 17/36 2024-04-27T03:28:33.720 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 18/36 2024-04-27T03:28:33.856 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 19/36 2024-04-27T03:28:33.938 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 20/36 2024-04-27T03:28:33.991 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 21/36 2024-04-27T03:28:34.029 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 22/36 2024-04-27T03:28:34.078 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-ply-3.11-14.el9.noarch 23/36 2024-04-27T03:28:34.128 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-osd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 33/36 2024-04-27T03:28:34.132 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 24/36 2024-04-27T03:28:34.154 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-osd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 33/36 2024-04-27T03:28:34.154 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:28:34.154 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-27T03:28:34.154 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-27T03:28:34.155 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-27T03:28:34.155 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:34.329 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 25/36 2024-04-27T03:28:34.375 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 26/36 2024-04-27T03:28:34.454 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 27/36 2024-04-27T03:28:34.543 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 28/36 2024-04-27T03:28:34.582 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 29/36 2024-04-27T03:28:34.604 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mon-2:19.0.0-2447.gb22e2ebd.el9.x86_64 34/36 2024-04-27T03:28:34.627 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mon-2:19.0.0-2447.gb22e2ebd.el9.x86_64 34/36 2024-04-27T03:28:34.627 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:28:34.627 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-27T03:28:34.627 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-27T03:28:34.627 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-27T03:28:34.627 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:34.698 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 30/36 2024-04-27T03:28:34.854 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mds-2:19.0.0-2447.gb22e2ebd.el9.x86_64 35/36 2024-04-27T03:28:34.856 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-2447.gb22e2ebd.el9. 31/36 2024-04-27T03:28:34.877 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mds-2:19.0.0-2447.gb22e2ebd.el9.x86_64 35/36 2024-04-27T03:28:34.877 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:28:34.877 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-27T03:28:34.877 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-27T03:28:34.877 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-27T03:28:34.878 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:34.884 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-2:19.0.0-2447.gb22e2ebd.el9.x86_64 32/36 2024-04-27T03:28:34.909 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-2:19.0.0-2447.gb22e2ebd.el9.x86_64 32/36 2024-04-27T03:28:34.909 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:28:34.909 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-27T03:28:34.909 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-27T03:28:34.910 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-27T03:28:34.910 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:34.941 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-2:19.0.0-2447.gb22e2ebd.el9.x86_64 36/36 2024-04-27T03:28:36.343 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-osd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 33/36 2024-04-27T03:28:36.370 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-osd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 33/36 2024-04-27T03:28:36.370 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:28:36.370 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-27T03:28:36.370 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-27T03:28:36.370 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-27T03:28:36.370 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:36.733 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-2:19.0.0-2447.gb22e2ebd.el9.x86_64 36/36 2024-04-27T03:28:36.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/36 2024-04-27T03:28:36.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mds-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2/36 2024-04-27T03:28:36.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-2:19.0.0-2447.gb22e2ebd.el9.x86_64 3/36 2024-04-27T03:28:36.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mon-2:19.0.0-2447.gb22e2ebd.el9.x86_64 4/36 2024-04-27T03:28:36.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-osd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 5/36 2024-04-27T03:28:36.733 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2447.gb22e2ebd.el9. 6/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-27T03:28:36.734 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-27T03:28:36.735 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-27T03:28:36.811 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mon-2:19.0.0-2447.gb22e2ebd.el9.x86_64 34/36 2024-04-27T03:28:36.834 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mon-2:19.0.0-2447.gb22e2ebd.el9.x86_64 34/36 2024-04-27T03:28:36.834 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:28:36.834 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-27T03:28:36.834 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-27T03:28:36.834 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-27T03:28:36.834 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:37.053 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mds-2:19.0.0-2447.gb22e2ebd.el9.x86_64 35/36 2024-04-27T03:28:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mds-2:19.0.0-2447.gb22e2ebd.el9.x86_64 35/36 2024-04-27T03:28:37.076 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:28:37.076 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-27T03:28:37.076 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-27T03:28:37.076 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-27T03:28:37.076 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:37.139 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-2:19.0.0-2447.gb22e2ebd.el9.x86_64 36/36 2024-04-27T03:28:37.722 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-27T03:28:37.723 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:37.723 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:28:37.723 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:37.723 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mds-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:37.723 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:37.723 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-modules-core-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:28:37.723 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mon-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:37.723 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-osd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:37.723 INFO:teuthology.orchestra.run.smithi089.stdout: lua-5.4.4-4.el9.x86_64 2024-04-27T03:28:37.723 INFO:teuthology.orchestra.run.smithi089.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-27T03:28:37.723 INFO:teuthology.orchestra.run.smithi089.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-27T03:28:37.724 INFO:teuthology.orchestra.run.smithi089.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-ply-3.11-14.el9.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-27T03:28:37.725 INFO:teuthology.orchestra.run.smithi089.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-27T03:28:37.726 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:37.726 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:28:38.109 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-base 2024-04-27T03:28:38.623 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:56 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:28:38.701 INFO:teuthology.orchestra.run.smithi089.stdout:Package ceph-base-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:28:38.741 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:28:38.742 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2024-04-27T03:28:38.742 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:28:38.806 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install cephadm 2024-04-27T03:28:38.881 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-2:19.0.0-2447.gb22e2ebd.el9.x86_64 36/36 2024-04-27T03:28:38.881 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/36 2024-04-27T03:28:38.881 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mds-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2/36 2024-04-27T03:28:38.881 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-2:19.0.0-2447.gb22e2ebd.el9.x86_64 3/36 2024-04-27T03:28:38.881 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mon-2:19.0.0-2447.gb22e2ebd.el9.x86_64 4/36 2024-04-27T03:28:38.881 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-osd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 5/36 2024-04-27T03:28:38.881 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-2447.gb22e2ebd.el9. 6/36 2024-04-27T03:28:38.881 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 7/36 2024-04-27T03:28:38.881 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 8/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-ply-3.11-14.el9.noarch 9/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 10/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 11/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 12/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-5.4.4-4.el9.x86_64 13/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 14/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 15/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 16/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 17/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 18/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 19/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 20/36 2024-04-27T03:28:38.882 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 21/36 2024-04-27T03:28:38.883 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 22/36 2024-04-27T03:28:38.883 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 23/36 2024-04-27T03:28:38.883 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 24/36 2024-04-27T03:28:38.883 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 25/36 2024-04-27T03:28:38.883 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 26/36 2024-04-27T03:28:38.883 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 27/36 2024-04-27T03:28:38.883 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 28/36 2024-04-27T03:28:38.883 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 29/36 2024-04-27T03:28:38.883 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 30/36 2024-04-27T03:28:38.884 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 31/36 2024-04-27T03:28:38.884 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 32/36 2024-04-27T03:28:38.884 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 33/36 2024-04-27T03:28:38.884 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 34/36 2024-04-27T03:28:38.884 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 35/36 2024-04-27T03:28:39.310 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:57 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:28:39.430 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:28:39.431 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-04-27T03:28:39.431 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:39.431 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:28:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 762 k 2024-04-27T03:28:39.431 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:39.431 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:28:39.431 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:39.431 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2024-04-27T03:28:39.432 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:39.432 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 762 k 2024-04-27T03:28:39.432 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 768 k 2024-04-27T03:28:39.432 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:28:39.738 INFO:teuthology.orchestra.run.smithi089.stdout:cephadm-19.0.0-2447.gb22e2ebd.el9.noarch.rpm 2.4 MB/s | 762 kB 00:00 2024-04-27T03:28:39.738 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:28:39.738 INFO:teuthology.orchestra.run.smithi089.stdout:Total 2.4 MB/s | 762 kB 00:00 2024-04-27T03:28:39.739 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:28:39.744 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:28:39.744 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:28:39.750 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:28:39.750 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:28:39.819 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:28:39.944 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 36/36 2024-04-27T03:28:39.944 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:39.944 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mds-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-modules-core-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mon-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-osd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: lua-5.4.4-4.el9.x86_64 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-27T03:28:39.945 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-ply-3.11-14.el9.noarch 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-27T03:28:39.946 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-27T03:28:39.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-27T03:28:39.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-27T03:28:39.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-27T03:28:39.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-27T03:28:39.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-27T03:28:39.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-27T03:28:39.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-27T03:28:39.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-27T03:28:39.947 INFO:teuthology.orchestra.run.smithi165.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-27T03:28:39.947 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:39.947 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:28:40.209 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/1 2024-04-27T03:28:40.289 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/1 2024-04-27T03:28:40.296 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-base 2024-04-27T03:28:40.602 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/1 2024-04-27T03:28:40.817 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:52 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:28:40.830 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/1 2024-04-27T03:28:40.831 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:40.831 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:28:40.831 INFO:teuthology.orchestra.run.smithi089.stdout: cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:28:40.831 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:40.831 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:28:40.896 INFO:teuthology.orchestra.run.smithi165.stdout:Package ceph-base-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:28:40.938 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:28:40.939 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-27T03:28:40.939 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:28:41.014 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install cephadm 2024-04-27T03:28:41.060 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-immutable-object-cache 2024-04-27T03:28:41.523 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:53 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:28:41.570 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:00:59 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:28:41.644 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:28:41.644 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:41.645 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-27T03:28:41.645 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:41.645 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:28:41.645 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 762 k 2024-04-27T03:28:41.645 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:41.645 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:28:41.645 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:41.645 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-27T03:28:41.645 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:41.645 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 762 k 2024-04-27T03:28:41.645 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 768 k 2024-04-27T03:28:41.645 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:28:41.691 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:28:41.691 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:41.693 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repo Size 2024-04-27T03:28:41.693 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:41.693 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:28:41.693 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 145 k 2024-04-27T03:28:41.693 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:41.693 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:28:41.693 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:41.693 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2024-04-27T03:28:41.694 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:41.694 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 145 k 2024-04-27T03:28:41.694 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 436 k 2024-04-27T03:28:41.694 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:28:42.036 INFO:teuthology.orchestra.run.smithi089.stdout:ceph-immutable-object-cache-19.0.0-2447.gb22e2e 422 kB/s | 145 kB 00:00 2024-04-27T03:28:42.036 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:28:42.037 INFO:teuthology.orchestra.run.smithi089.stdout:Total 420 kB/s | 145 kB 00:00 2024-04-27T03:28:42.037 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:28:42.044 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:28:42.044 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:28:42.101 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:28:42.102 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:28:42.109 INFO:teuthology.orchestra.run.smithi165.stdout:cephadm-19.0.0-2447.gb22e2ebd.el9.noarch.rpm 1.6 MB/s | 762 kB 00:00 2024-04-27T03:28:42.109 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:28:42.110 INFO:teuthology.orchestra.run.smithi165.stdout:Total 1.6 MB/s | 762 kB 00:00 2024-04-27T03:28:42.110 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:28:42.115 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:28:42.115 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:28:42.121 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:28:42.121 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:28:42.189 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:28:42.244 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:28:42.353 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2447.gb22e2ebd. 1/1 2024-04-27T03:28:42.377 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2447.gb22e2ebd. 1/1 2024-04-27T03:28:42.377 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:28:42.377 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-27T03:28:42.377 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:42.664 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/1 2024-04-27T03:28:42.754 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/1 2024-04-27T03:28:43.082 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/1 2024-04-27T03:28:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2447.gb22e2ebd. 1/1 2024-04-27T03:28:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:43.222 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:28:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-immutable-object-cache-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:43.222 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:43.222 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:28:43.323 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/1 2024-04-27T03:28:43.323 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:43.323 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:28:43.323 INFO:teuthology.orchestra.run.smithi165.stdout: cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:28:43.323 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:43.323 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:28:43.446 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-mgr 2024-04-27T03:28:43.536 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-immutable-object-cache 2024-04-27T03:28:43.962 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:01 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:28:44.043 INFO:teuthology.orchestra.run.smithi089.stdout:Package ceph-mgr-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:28:44.047 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:56 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:28:44.084 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:28:44.085 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2024-04-27T03:28:44.086 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:28:44.167 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:28:44.168 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:44.168 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2024-04-27T03:28:44.168 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:44.168 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:28:44.168 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 145 k 2024-04-27T03:28:44.168 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:44.168 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:28:44.168 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:44.168 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-27T03:28:44.168 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:44.168 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 145 k 2024-04-27T03:28:44.169 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 436 k 2024-04-27T03:28:44.169 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:28:44.173 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-mgr-dashboard 2024-04-27T03:28:44.472 INFO:teuthology.orchestra.run.smithi165.stdout:ceph-immutable-object-cache-19.0.0-2447.gb22e2e 477 kB/s | 145 kB 00:00 2024-04-27T03:28:44.473 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:28:44.473 INFO:teuthology.orchestra.run.smithi165.stdout:Total 475 kB/s | 145 kB 00:00 2024-04-27T03:28:44.473 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:28:44.481 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:28:44.481 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:28:44.536 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:28:44.537 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:28:44.670 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:02 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:28:44.681 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:28:44.773 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-2447.gb22e2ebd. 1/1 2024-04-27T03:28:44.789 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:28:44.790 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:44.791 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-04-27T03:28:44.791 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:44.791 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:28:44.791 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 3.5 M 2024-04-27T03:28:44.791 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-04-27T03:28:44.791 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-27T03:28:44.791 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 32 k 2024-04-27T03:28:44.791 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 15 k 2024-04-27T03:28:44.791 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-27T03:28:44.791 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-27T03:28:44.791 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-27T03:28:44.792 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-27T03:28:44.792 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-27T03:28:44.792 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-27T03:28:44.792 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-27T03:28:44.792 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-27T03:28:44.792 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:44.792 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:28:44.792 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:28:44.792 INFO:teuthology.orchestra.run.smithi089.stdout:Install 12 Packages 2024-04-27T03:28:44.792 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:44.793 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 8.6 M 2024-04-27T03:28:44.793 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 103 M 2024-04-27T03:28:44.793 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:28:44.797 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-2447.gb22e2ebd. 1/1 2024-04-27T03:28:44.797 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:28:44.797 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-27T03:28:44.797 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:45.355 INFO:teuthology.orchestra.run.smithi089.stdout:(1/12): ceph-grafana-dashboards-19.0.0-2447.gb2 161 kB/s | 32 kB 00:00 2024-04-27T03:28:45.498 INFO:teuthology.orchestra.run.smithi089.stdout:(2/12): ceph-prometheus-alerts-19.0.0-2447.gb22 44 kB/s | 15 kB 00:00 2024-04-27T03:28:45.640 INFO:teuthology.orchestra.run.smithi089.stdout:(3/12): protobuf-3.14.0-13.el9.x86_64.rpm 3.5 MB/s | 1.0 MB 00:00 2024-04-27T03:28:45.682 INFO:teuthology.orchestra.run.smithi089.stdout:(4/12): python3-protobuf-3.14.0-13.el9.noarch.r 1.4 MB/s | 269 kB 00:00 2024-04-27T03:28:45.731 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-2447.gb22e2ebd. 1/1 2024-04-27T03:28:45.731 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:45.731 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:28:45.731 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-immutable-object-cache-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:28:45.732 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:45.732 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:28:45.765 INFO:teuthology.orchestra.run.smithi089.stdout:(5/12): ceph-mgr-dashboard-19.0.0-2447.gb22e2eb 5.7 MB/s | 3.5 MB 00:00 2024-04-27T03:28:45.924 INFO:teuthology.orchestra.run.smithi089.stdout:(6/12): protobuf-compiler-3.14.0-13.el9.x86_64. 3.0 MB/s | 863 kB 00:00 2024-04-27T03:28:45.986 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr 2024-04-27T03:28:46.149 INFO:teuthology.orchestra.run.smithi089.stdout:(7/12): grpc-data-1.46.7-10.el9.noarch.rpm 54 kB/s | 19 kB 00:00 2024-04-27T03:28:46.400 INFO:teuthology.orchestra.run.smithi089.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 764 kB/s | 548 kB 00:00 2024-04-27T03:28:46.425 INFO:teuthology.orchestra.run.smithi089.stdout:(9/12): python3-grpcio-tools-1.46.7-10.el9.x86_ 525 kB/s | 144 kB 00:00 2024-04-27T03:28:46.500 INFO:teuthology.orchestra.run.smithi089.stdout:(10/12): python3-repoze-lru-0.7-16.el9.noarch.r 307 kB/s | 31 kB 00:00 2024-04-27T03:28:46.512 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:58 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:28:46.592 INFO:teuthology.orchestra.run.smithi165.stdout:Package ceph-mgr-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:28:46.600 INFO:teuthology.orchestra.run.smithi089.stdout:(11/12): python3-routes-2.5.1-5.el9.noarch.rpm 1.1 MB/s | 188 kB 00:00 2024-04-27T03:28:46.634 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:28:46.635 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-27T03:28:46.635 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:28:46.706 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr-dashboard 2024-04-27T03:28:46.775 INFO:teuthology.orchestra.run.smithi089.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 2.4 MB/s | 2.0 MB 00:00 2024-04-27T03:28:46.779 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:28:46.779 INFO:teuthology.orchestra.run.smithi089.stdout:Total 4.3 MB/s | 8.6 MB 00:01 2024-04-27T03:28:46.827 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:28:46.843 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:28:46.843 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:28:46.996 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:28:46.997 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:28:47.212 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:00:59 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:28:47.332 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:28:47.333 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:47.333 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-27T03:28:47.334 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:47.334 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:28:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 3.5 M 2024-04-27T03:28:47.334 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-27T03:28:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-27T03:28:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 32 k 2024-04-27T03:28:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 15 k 2024-04-27T03:28:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-27T03:28:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-27T03:28:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-27T03:28:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-27T03:28:47.334 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-27T03:28:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-27T03:28:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-27T03:28:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-27T03:28:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:47.335 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:28:47.335 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:28:47.335 INFO:teuthology.orchestra.run.smithi165.stdout:Install 12 Packages 2024-04-27T03:28:47.335 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:47.336 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 8.6 M 2024-04-27T03:28:47.336 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 103 M 2024-04-27T03:28:47.336 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:28:47.545 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:28:47.618 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-27T03:28:47.653 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-27T03:28:47.699 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-27T03:28:47.747 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-27T03:28:47.822 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-27T03:28:47.963 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-27T03:28:48.048 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-27T03:28:48.091 INFO:teuthology.orchestra.run.smithi165.stdout:(1/12): ceph-prometheus-alerts-19.0.0-2447.gb22 76 kB/s | 15 kB 00:00 2024-04-27T03:28:48.093 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-27T03:28:48.122 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-27T03:28:48.153 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2447.gb22e2ebd.el9 10/12 2024-04-27T03:28:48.200 INFO:teuthology.orchestra.run.smithi165.stdout:(2/12): protobuf-3.14.0-13.el9.x86_64.rpm 9.3 MB/s | 1.0 MB 00:00 2024-04-27T03:28:48.275 INFO:teuthology.orchestra.run.smithi165.stdout:(3/12): python3-protobuf-3.14.0-13.el9.noarch.r 3.5 MB/s | 269 kB 00:00 2024-04-27T03:28:48.317 INFO:teuthology.orchestra.run.smithi165.stdout:(4/12): ceph-grafana-dashboards-19.0.0-2447.gb2 74 kB/s | 32 kB 00:00 2024-04-27T03:28:48.442 INFO:teuthology.orchestra.run.smithi165.stdout:(5/12): ceph-mgr-dashboard-19.0.0-2447.gb22e2eb 6.3 MB/s | 3.5 MB 00:00 2024-04-27T03:28:48.576 INFO:teuthology.orchestra.run.smithi165.stdout:(6/12): grpc-data-1.46.7-10.el9.noarch.rpm 145 kB/s | 19 kB 00:00 2024-04-27T03:28:48.601 INFO:teuthology.orchestra.run.smithi165.stdout:(7/12): protobuf-compiler-3.14.0-13.el9.x86_64. 2.6 MB/s | 863 kB 00:00 2024-04-27T03:28:48.635 INFO:teuthology.orchestra.run.smithi165.stdout:(8/12): abseil-cpp-20211102.0-3.el9.x86_64.rpm 1.7 MB/s | 548 kB 00:00 2024-04-27T03:28:48.685 INFO:teuthology.orchestra.run.smithi165.stdout:(9/12): python3-repoze-lru-0.7-16.el9.noarch.rp 617 kB/s | 31 kB 00:00 2024-04-27T03:28:48.744 INFO:teuthology.orchestra.run.smithi165.stdout:(10/12): python3-routes-2.5.1-5.el9.noarch.rpm 3.1 MB/s | 188 kB 00:00 2024-04-27T03:28:48.819 INFO:teuthology.orchestra.run.smithi165.stdout:(11/12): python3-grpcio-tools-1.46.7-10.el9.x86 664 kB/s | 144 kB 00:00 2024-04-27T03:28:48.902 INFO:teuthology.orchestra.run.smithi165.stdout:(12/12): python3-grpcio-1.46.7-10.el9.x86_64.rp 6.3 MB/s | 2.0 MB 00:00 2024-04-27T03:28:48.904 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:28:48.905 INFO:teuthology.orchestra.run.smithi165.stdout:Total 5.5 MB/s | 8.6 MB 00:01 2024-04-27T03:28:48.954 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:28:48.972 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:28:48.972 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:28:49.125 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:28:49.126 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:28:49.186 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2447.gb22e2ebd.el 11/12 2024-04-27T03:28:49.221 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2447.gb22e2ebd.el9.noa 12/12 2024-04-27T03:28:49.663 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:28:49.736 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 1/12 2024-04-27T03:28:49.780 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 2/12 2024-04-27T03:28:49.826 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 3/12 2024-04-27T03:28:49.865 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 4/12 2024-04-27T03:28:49.942 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 5/12 2024-04-27T03:28:50.066 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 6/12 2024-04-27T03:28:50.159 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 7/12 2024-04-27T03:28:50.203 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 8/12 2024-04-27T03:28:50.231 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/12 2024-04-27T03:28:50.263 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-2447.gb22e2ebd.el9 10/12 2024-04-27T03:28:50.572 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2447.gb22e2ebd.el9.noa 12/12 2024-04-27T03:28:50.572 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2447.gb22e2ebd.el 1/12 2024-04-27T03:28:50.572 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2447.gb22e2ebd.el9.noa 2/12 2024-04-27T03:28:50.572 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2447.gb22e2ebd.el9 3/12 2024-04-27T03:28:50.572 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-27T03:28:50.572 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-27T03:28:50.572 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-27T03:28:50.573 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-27T03:28:50.573 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-27T03:28:50.573 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-27T03:28:50.573 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-27T03:28:50.573 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-27T03:28:50.869 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-27T03:28:50.869 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:50.869 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:28:50.869 INFO:teuthology.orchestra.run.smithi089.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-27T03:28:50.869 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-grafana-dashboards-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:28:50.869 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-dashboard-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:28:50.870 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-prometheus-alerts-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:28:50.870 INFO:teuthology.orchestra.run.smithi089.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-27T03:28:50.870 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-27T03:28:50.870 INFO:teuthology.orchestra.run.smithi089.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-27T03:28:50.870 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-27T03:28:50.870 INFO:teuthology.orchestra.run.smithi089.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-27T03:28:50.870 INFO:teuthology.orchestra.run.smithi089.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-27T03:28:50.870 INFO:teuthology.orchestra.run.smithi089.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-27T03:28:50.870 INFO:teuthology.orchestra.run.smithi089.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-27T03:28:50.870 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:50.870 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:28:51.109 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-27T03:28:51.327 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-2447.gb22e2ebd.el 11/12 2024-04-27T03:28:51.363 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-2447.gb22e2ebd.el9.noa 12/12 2024-04-27T03:28:51.610 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:09 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:28:51.730 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:28:51.731 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-04-27T03:28:51.731 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 7.4 M 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-27T03:28:51.732 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-27T03:28:51.733 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-27T03:28:51.733 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-27T03:28:51.733 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:51.733 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:28:51.733 INFO:teuthology.orchestra.run.smithi089.stdout:======================================================================================= 2024-04-27T03:28:51.733 INFO:teuthology.orchestra.run.smithi089.stdout:Install 12 Packages 2024-04-27T03:28:51.733 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:28:51.733 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 38 M 2024-04-27T03:28:51.734 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 209 M 2024-04-27T03:28:51.734 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:28:52.283 INFO:teuthology.orchestra.run.smithi089.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 933 kB/s | 195 kB 00:00 2024-04-27T03:28:52.375 INFO:teuthology.orchestra.run.smithi089.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 2.6 MB/s | 803 kB 00:00 2024-04-27T03:28:52.409 INFO:teuthology.orchestra.run.smithi089.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 261 kB/s | 33 kB 00:00 2024-04-27T03:28:52.459 INFO:teuthology.orchestra.run.smithi089.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 364 kB/s | 18 kB 00:00 2024-04-27T03:28:52.510 INFO:teuthology.orchestra.run.smithi089.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-04-27T03:28:52.566 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-2447.gb22e2ebd.el9.noa 12/12 2024-04-27T03:28:52.566 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-2447.gb22e2ebd.el 1/12 2024-04-27T03:28:52.567 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-2447.gb22e2ebd.el9.noa 2/12 2024-04-27T03:28:52.567 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-2447.gb22e2ebd.el9 3/12 2024-04-27T03:28:52.567 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 4/12 2024-04-27T03:28:52.567 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 5/12 2024-04-27T03:28:52.567 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 6/12 2024-04-27T03:28:52.567 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 7/12 2024-04-27T03:28:52.568 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/12 2024-04-27T03:28:52.568 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/12 2024-04-27T03:28:52.568 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/12 2024-04-27T03:28:52.568 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/12 2024-04-27T03:28:52.660 INFO:teuthology.orchestra.run.smithi089.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 11 MB/s | 3.0 MB 00:00 2024-04-27T03:28:52.710 INFO:teuthology.orchestra.run.smithi089.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 4.8 MB/s | 244 kB 00:00 2024-04-27T03:28:52.802 INFO:teuthology.orchestra.run.smithi089.stdout:(8/12): ceph-mgr-diskprediction-local-19.0.0-24 10 MB/s | 7.4 MB 00:00 2024-04-27T03:28:52.844 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/12 2024-04-27T03:28:52.844 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:52.844 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:28:52.844 INFO:teuthology.orchestra.run.smithi165.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-27T03:28:52.844 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-grafana-dashboards-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:28:52.844 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-dashboard-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:28:52.844 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-prometheus-alerts-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:28:52.844 INFO:teuthology.orchestra.run.smithi165.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-27T03:28:52.844 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-27T03:28:52.844 INFO:teuthology.orchestra.run.smithi165.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-27T03:28:52.844 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-27T03:28:52.844 INFO:teuthology.orchestra.run.smithi165.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-27T03:28:52.845 INFO:teuthology.orchestra.run.smithi165.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-27T03:28:52.845 INFO:teuthology.orchestra.run.smithi165.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-27T03:28:52.845 INFO:teuthology.orchestra.run.smithi165.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-27T03:28:52.845 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:52.845 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:28:52.886 INFO:teuthology.orchestra.run.smithi089.stdout:(9/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 13 MB/s | 4.7 MB 00:00 2024-04-27T03:28:52.961 INFO:teuthology.orchestra.run.smithi089.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 20 MB/s | 5.1 MB 00:00 2024-04-27T03:28:53.091 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr-diskprediction-local 2024-04-27T03:28:53.094 INFO:teuthology.orchestra.run.smithi089.stdout:(11/12): python3-numpy-f2py-1.20.1-5.el9.x86_64 858 kB/s | 250 kB 00:00 2024-04-27T03:28:53.253 INFO:teuthology.orchestra.run.smithi089.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 44 MB/s | 16 MB 00:00 2024-04-27T03:28:53.254 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:28:53.254 INFO:teuthology.orchestra.run.smithi089.stdout:Total 25 MB/s | 38 MB 00:01 2024-04-27T03:28:53.602 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:28:53.603 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:05 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:28:53.628 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:28:53.628 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:28:53.724 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:28:53.725 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-27T03:28:53.726 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-27T03:28:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-27T03:28:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:28:53.726 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 7.4 M 2024-04-27T03:28:53.726 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-27T03:28:53.726 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-27T03:28:53.726 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-27T03:28:53.726 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-27T03:28:53.726 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-27T03:28:53.727 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-27T03:28:53.727 INFO:teuthology.orchestra.run.smithi165.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-27T03:28:53.727 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-27T03:28:53.727 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-27T03:28:53.727 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-27T03:28:53.727 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-27T03:28:53.727 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-27T03:28:53.727 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:28:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:======================================================================================= 2024-04-27T03:28:53.727 INFO:teuthology.orchestra.run.smithi165.stdout:Install 12 Packages 2024-04-27T03:28:53.727 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:28:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 38 M 2024-04-27T03:28:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 209 M 2024-04-27T03:28:53.728 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:28:54.032 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:28:54.032 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:28:54.310 INFO:teuthology.orchestra.run.smithi165.stdout:(1/12): libquadmath-11.4.1-3.el9.x86_64.rpm 1.3 MB/s | 195 kB 00:00 2024-04-27T03:28:54.384 INFO:teuthology.orchestra.run.smithi165.stdout:(2/12): libgfortran-11.4.1-3.el9.x86_64.rpm 3.5 MB/s | 803 kB 00:00 2024-04-27T03:28:54.410 INFO:teuthology.orchestra.run.smithi165.stdout:(3/12): flexiblas-3.0.4-8.el9.x86_64.rpm 327 kB/s | 33 kB 00:00 2024-04-27T03:28:54.518 INFO:teuthology.orchestra.run.smithi165.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-8.el9.x 168 kB/s | 18 kB 00:00 2024-04-27T03:28:54.568 INFO:teuthology.orchestra.run.smithi165.stdout:(5/12): openblas-0.3.21-2.el9.x86_64.rpm 707 kB/s | 35 kB 00:00 2024-04-27T03:28:54.702 INFO:teuthology.orchestra.run.smithi165.stdout:(6/12): flexiblas-netlib-3.0.4-8.el9.x86_64.rpm 9.4 MB/s | 3.0 MB 00:00 2024-04-27T03:28:54.761 INFO:teuthology.orchestra.run.smithi165.stdout:(7/12): python3-devel-3.9.18-3.el9.x86_64.rpm 4.1 MB/s | 244 kB 00:00 2024-04-27T03:28:54.853 INFO:teuthology.orchestra.run.smithi165.stdout:(8/12): openblas-openmp-0.3.21-2.el9.x86_64.rpm 17 MB/s | 4.7 MB 00:00 2024-04-27T03:28:54.888 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:28:54.941 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-27T03:28:54.953 INFO:teuthology.orchestra.run.smithi165.stdout:(9/12): python3-numpy-f2py-1.20.1-5.el9.x86_64. 2.5 MB/s | 250 kB 00:00 2024-04-27T03:28:54.976 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-27T03:28:55.036 INFO:teuthology.orchestra.run.smithi165.stdout:(10/12): python3-numpy-1.20.1-5.el9.x86_64.rpm 19 MB/s | 5.1 MB 00:00 2024-04-27T03:28:55.076 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-27T03:28:55.127 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-27T03:28:55.195 INFO:teuthology.orchestra.run.smithi165.stdout:(11/12): ceph-mgr-diskprediction-local-19.0.0-2 7.2 MB/s | 7.4 MB 00:01 2024-04-27T03:28:55.305 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-27T03:28:55.335 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-27T03:28:55.403 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-27T03:28:55.478 INFO:teuthology.orchestra.run.smithi165.stdout:(12/12): python3-scipy-1.6.2-8.el9.x86_64.rpm 31 MB/s | 16 MB 00:00 2024-04-27T03:28:55.479 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:28:55.480 INFO:teuthology.orchestra.run.smithi165.stdout:Total 22 MB/s | 38 MB 00:01 2024-04-27T03:28:55.827 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:28:55.853 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:28:55.854 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:28:56.043 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-27T03:28:56.156 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-27T03:28:56.262 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:28:56.263 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:28:57.105 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:28:57.158 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 1/12 2024-04-27T03:28:57.202 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 2/12 2024-04-27T03:28:57.303 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 3/12 2024-04-27T03:28:57.344 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 4/12 2024-04-27T03:28:57.378 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-27T03:28:57.524 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-0.3.21-2.el9.x86_64 5/12 2024-04-27T03:28:57.551 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 6/12 2024-04-27T03:28:57.628 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 7/12 2024-04-27T03:28:58.279 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 8/12 2024-04-27T03:28:58.390 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 9/12 2024-04-27T03:28:58.470 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-27T03:28:58.494 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2447.gb22e2 12/12 2024-04-27T03:28:59.628 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 10/12 2024-04-27T03:29:00.681 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 11/12 2024-04-27T03:29:00.708 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-2447.gb22e2 12/12 2024-04-27T03:29:01.197 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2447.gb22e2 12/12 2024-04-27T03:29:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2447.gb22e2 1/12 2024-04-27T03:29:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-27T03:29:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-27T03:29:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-27T03:29:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-27T03:29:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-27T03:29:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-27T03:29:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-27T03:29:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-27T03:29:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-27T03:29:01.198 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-27T03:29:01.604 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-27T03:29:01.604 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:01.604 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:29:01.604 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:01.605 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:01.977 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-mgr-rook 2024-04-27T03:29:02.499 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:20 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:02.620 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:02.621 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-04-27T03:29:02.621 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:02.621 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:29:02.621 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 49 k 2024-04-27T03:29:02.622 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-04-27T03:29:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-27T03:29:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-27T03:29:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-27T03:29:02.622 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-27T03:29:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-27T03:29:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-27T03:29:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-27T03:29:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-27T03:29:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-27T03:29:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-27T03:29:02.623 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-27T03:29:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-27T03:29:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:02.624 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:29:02.624 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:02.624 INFO:teuthology.orchestra.run.smithi089.stdout:Install 13 Packages 2024-04-27T03:29:02.624 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:02.625 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 2.2 M 2024-04-27T03:29:02.626 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 26 M 2024-04-27T03:29:02.626 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:29:03.103 INFO:teuthology.orchestra.run.smithi089.stdout:(1/13): python3-jsonpointer-2.0-4.el9.noarch.rp 133 kB/s | 19 kB 00:00 2024-04-27T03:29:03.128 INFO:teuthology.orchestra.run.smithi089.stdout:(2/13): python3-jsonpatch-1.21-16.el9.noarch.rp 152 kB/s | 26 kB 00:00 2024-04-27T03:29:03.153 INFO:teuthology.orchestra.run.smithi089.stdout:(3/13): ceph-mgr-rook-19.0.0-2447.gb22e2ebd.el9 254 kB/s | 49 kB 00:00 2024-04-27T03:29:03.271 INFO:teuthology.orchestra.run.smithi089.stdout:(4/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.1 MB/s | 159 kB 00:00 2024-04-27T03:29:03.296 INFO:teuthology.orchestra.run.smithi089.stdout:(5/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.1 MB/s | 222 kB 00:00 2024-04-27T03:29:03.338 INFO:teuthology.orchestra.run.smithi089.stdout:(6/13): python3-requests-oauthlib-1.3.0-12.el9. 804 kB/s | 54 kB 00:00 2024-04-27T03:29:03.419 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-2447.gb22e2 12/12 2024-04-27T03:29:03.419 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-2447.gb22e2 1/12 2024-04-27T03:29:03.419 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 2/12 2024-04-27T03:29:03.419 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 3/12 2024-04-27T03:29:03.419 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 4/12 2024-04-27T03:29:03.419 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 5/12 2024-04-27T03:29:03.419 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 6/12 2024-04-27T03:29:03.419 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 7/12 2024-04-27T03:29:03.420 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 8/12 2024-04-27T03:29:03.420 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 9/12 2024-04-27T03:29:03.420 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 10/12 2024-04-27T03:29:03.420 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 11/12 2024-04-27T03:29:03.446 INFO:teuthology.orchestra.run.smithi089.stdout:(7/13): python3-pyasn1-modules-0.4.8-6.el9.noar 954 kB/s | 279 kB 00:00 2024-04-27T03:29:03.580 INFO:teuthology.orchestra.run.smithi089.stdout:(8/13): python3-cachetools-4.2.4-1.el9.noarch.r 113 kB/s | 32 kB 00:00 2024-04-27T03:29:03.630 INFO:teuthology.orchestra.run.smithi089.stdout:(9/13): python3-certifi-2021.10.8-2.el9.noarch. 56 kB/s | 15 kB 00:00 2024-04-27T03:29:03.681 INFO:teuthology.orchestra.run.smithi089.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 882 kB/s | 206 kB 00:00 2024-04-27T03:29:03.756 INFO:teuthology.orchestra.run.smithi089.stdout:(11/13): python3-kubernetes-26.1.0-2.el9.noarch 5.8 MB/s | 1.0 MB 00:00 2024-04-27T03:29:03.777 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 12/12 2024-04-27T03:29:03.777 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:03.777 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:29:03.777 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-diskprediction-local-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:29:03.777 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-27T03:29:03.777 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-27T03:29:03.777 INFO:teuthology.orchestra.run.smithi165.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-27T03:29:03.777 INFO:teuthology.orchestra.run.smithi165.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-27T03:29:03.777 INFO:teuthology.orchestra.run.smithi165.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-27T03:29:03.777 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-27T03:29:03.777 INFO:teuthology.orchestra.run.smithi165.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-27T03:29:03.778 INFO:teuthology.orchestra.run.smithi165.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-27T03:29:03.778 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-27T03:29:03.778 INFO:teuthology.orchestra.run.smithi165.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-27T03:29:03.778 INFO:teuthology.orchestra.run.smithi165.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-27T03:29:03.778 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:03.778 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:03.789 INFO:teuthology.orchestra.run.smithi089.stdout:(12/13): python3-rsa-4.9-2.el9.noarch.rpm 373 kB/s | 59 kB 00:00 2024-04-27T03:29:03.814 INFO:teuthology.orchestra.run.smithi089.stdout:(13/13): python3-websocket-client-1.2.3-2.el9.n 673 kB/s | 90 kB 00:00 2024-04-27T03:29:03.816 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:03.816 INFO:teuthology.orchestra.run.smithi089.stdout:Total 1.8 MB/s | 2.2 MB 00:01 2024-04-27T03:29:03.879 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:29:03.893 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:29:03.893 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:29:03.976 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:29:03.976 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:29:04.219 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr-rook 2024-04-27T03:29:04.476 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:29:04.644 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-27T03:29:04.698 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-27T03:29:04.742 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:16 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:04.763 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-27T03:29:04.800 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-27T03:29:04.841 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-27T03:29:04.864 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:04.865 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:04.865 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-27T03:29:04.865 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:04.865 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:29:04.865 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 49 k 2024-04-27T03:29:04.865 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-27T03:29:04.865 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-27T03:29:04.865 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-27T03:29:04.865 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth noarch 1:2.29.0-1.el9 epel 206 k 2024-04-27T03:29:04.865 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout:Install 13 Packages 2024-04-27T03:29:04.866 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:04.867 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 2.2 M 2024-04-27T03:29:04.867 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 26 M 2024-04-27T03:29:04.867 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:29:04.943 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-27T03:29:05.045 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-27T03:29:05.102 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-27T03:29:05.427 INFO:teuthology.orchestra.run.smithi165.stdout:(1/13): python3-jsonpatch-1.21-16.el9.noarch.rp 256 kB/s | 26 kB 00:00 2024-04-27T03:29:05.452 INFO:teuthology.orchestra.run.smithi165.stdout:(2/13): python3-jsonpointer-2.0-4.el9.noarch.rp 153 kB/s | 19 kB 00:00 2024-04-27T03:29:05.577 INFO:teuthology.orchestra.run.smithi165.stdout:(3/13): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.2 MB/s | 159 kB 00:00 2024-04-27T03:29:05.602 INFO:teuthology.orchestra.run.smithi165.stdout:(4/13): python3-oauthlib-3.1.1-5.el9.noarch.rpm 1.5 MB/s | 222 kB 00:00 2024-04-27T03:29:05.627 INFO:teuthology.orchestra.run.smithi165.stdout:(5/13): ceph-mgr-rook-19.0.0-2447.gb22e2ebd.el9 164 kB/s | 49 kB 00:00 2024-04-27T03:29:05.661 INFO:teuthology.orchestra.run.smithi165.stdout:(6/13): python3-pyasn1-modules-0.4.8-6.el9.noar 3.2 MB/s | 279 kB 00:00 2024-04-27T03:29:05.686 INFO:teuthology.orchestra.run.smithi165.stdout:(7/13): python3-requests-oauthlib-1.3.0-12.el9. 640 kB/s | 54 kB 00:00 2024-04-27T03:29:05.811 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-27T03:29:05.857 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-27T03:29:05.891 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-27T03:29:05.939 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-27T03:29:05.945 INFO:teuthology.orchestra.run.smithi165.stdout:(8/13): python3-certifi-2021.10.8-2.el9.noarch. 53 kB/s | 15 kB 00:00 2024-04-27T03:29:05.961 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-rook-2:19.0.0-2447.gb22e2ebd.el9.noarch 13/13 2024-04-27T03:29:05.970 INFO:teuthology.orchestra.run.smithi165.stdout:(9/13): python3-cachetools-4.2.4-1.el9.noarch.r 94 kB/s | 32 kB 00:00 2024-04-27T03:29:06.162 INFO:teuthology.orchestra.run.smithi165.stdout:(10/13): python3-google-auth-2.29.0-1.el9.noarc 432 kB/s | 206 kB 00:00 2024-04-27T03:29:06.271 INFO:teuthology.orchestra.run.smithi165.stdout:(11/13): python3-rsa-4.9-2.el9.noarch.rpm 197 kB/s | 59 kB 00:00 2024-04-27T03:29:06.296 INFO:teuthology.orchestra.run.smithi165.stdout:(12/13): python3-websocket-client-1.2.3-2.el9.n 672 kB/s | 90 kB 00:00 2024-04-27T03:29:06.555 INFO:teuthology.orchestra.run.smithi165.stdout:(13/13): python3-kubernetes-26.1.0-2.el9.noarch 1.7 MB/s | 1.0 MB 00:00 2024-04-27T03:29:06.555 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:06.555 INFO:teuthology.orchestra.run.smithi165.stdout:Total 1.3 MB/s | 2.2 MB 00:01 2024-04-27T03:29:06.621 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:29:06.636 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:29:06.636 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:29:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2447.gb22e2ebd.el9.noarch 13/13 2024-04-27T03:29:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/13 2024-04-27T03:29:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-27T03:29:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-27T03:29:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-27T03:29:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-27T03:29:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-27T03:29:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-27T03:29:06.652 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-27T03:29:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-27T03:29:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-27T03:29:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-27T03:29:06.653 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-27T03:29:06.720 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:29:06.722 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:29:06.976 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-27T03:29:06.976 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:06.976 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:29:06.976 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-rook-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:29:06.976 INFO:teuthology.orchestra.run.smithi089.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-27T03:29:06.976 INFO:teuthology.orchestra.run.smithi089.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-27T03:29:06.976 INFO:teuthology.orchestra.run.smithi089.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-27T03:29:06.976 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-27T03:29:06.977 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-27T03:29:06.977 INFO:teuthology.orchestra.run.smithi089.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-27T03:29:06.977 INFO:teuthology.orchestra.run.smithi089.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-27T03:29:06.977 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-27T03:29:06.977 INFO:teuthology.orchestra.run.smithi089.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-27T03:29:06.977 INFO:teuthology.orchestra.run.smithi089.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-27T03:29:06.977 INFO:teuthology.orchestra.run.smithi089.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-27T03:29:06.977 INFO:teuthology.orchestra.run.smithi089.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-27T03:29:06.977 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:06.977 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:07.214 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-mgr-cephadm 2024-04-27T03:29:07.227 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:29:07.385 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/13 2024-04-27T03:29:07.438 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/13 2024-04-27T03:29:07.494 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/13 2024-04-27T03:29:07.531 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/13 2024-04-27T03:29:07.573 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 5/13 2024-04-27T03:29:07.667 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/13 2024-04-27T03:29:07.721 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:25 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:07.769 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-google-auth-1:2.29.0-1.el9.noarch 7/13 2024-04-27T03:29:07.816 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 8/13 2024-04-27T03:29:07.841 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:07.842 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:07.842 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-04-27T03:29:07.842 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:07.842 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:29:07.842 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 141 k 2024-04-27T03:29:07.842 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-04-27T03:29:07.843 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-27T03:29:07.843 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-27T03:29:07.843 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-27T03:29:07.843 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-27T03:29:07.843 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-27T03:29:07.843 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:07.843 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:29:07.843 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:07.843 INFO:teuthology.orchestra.run.smithi089.stdout:Install 6 Packages 2024-04-27T03:29:07.843 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:07.844 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 7.0 M 2024-04-27T03:29:07.844 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 33 M 2024-04-27T03:29:07.844 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:29:08.483 INFO:teuthology.orchestra.run.smithi089.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 1.4 MB/s | 248 kB 00:00 2024-04-27T03:29:08.531 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 9/13 2024-04-27T03:29:08.596 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 10/13 2024-04-27T03:29:08.621 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 11/13 2024-04-27T03:29:08.670 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 12/13 2024-04-27T03:29:08.699 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-rook-2:19.0.0-2447.gb22e2ebd.el9.noarch 13/13 2024-04-27T03:29:08.711 INFO:teuthology.orchestra.run.smithi089.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 15 MB/s | 6.0 MB 00:00 2024-04-27T03:29:08.736 INFO:teuthology.orchestra.run.smithi089.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2447.gb22e2ebd.e 335 kB/s | 141 kB 00:00 2024-04-27T03:29:09.253 INFO:teuthology.orchestra.run.smithi089.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 107 kB/s | 58 kB 00:00 2024-04-27T03:29:09.278 INFO:teuthology.orchestra.run.smithi089.stdout:(5/6): python3-typing-extensions-4.4.0-2.el9.no 94 kB/s | 51 kB 00:00 2024-04-27T03:29:09.439 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-2447.gb22e2ebd.el9.noarch 13/13 2024-04-27T03:29:09.439 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-rook-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/13 2024-04-27T03:29:09.439 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 2/13 2024-04-27T03:29:09.439 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 3/13 2024-04-27T03:29:09.439 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 4/13 2024-04-27T03:29:09.439 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 5/13 2024-04-27T03:29:09.439 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 6/13 2024-04-27T03:29:09.439 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 7/13 2024-04-27T03:29:09.439 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 8/13 2024-04-27T03:29:09.439 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 9/13 2024-04-27T03:29:09.439 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-google-auth-1:2.29.0-1.el9.noarch 10/13 2024-04-27T03:29:09.440 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 11/13 2024-04-27T03:29:09.440 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 12/13 2024-04-27T03:29:09.620 INFO:teuthology.orchestra.run.smithi089.stdout:(6/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 482 kB/s | 548 kB 00:01 2024-04-27T03:29:09.623 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:09.623 INFO:teuthology.orchestra.run.smithi089.stdout:Total 3.9 MB/s | 7.0 MB 00:01 2024-04-27T03:29:09.694 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:29:09.705 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:29:09.706 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:29:09.752 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 13/13 2024-04-27T03:29:09.752 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:09.752 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:29:09.752 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-rook-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:29:09.752 INFO:teuthology.orchestra.run.smithi165.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-27T03:29:09.752 INFO:teuthology.orchestra.run.smithi165.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-27T03:29:09.753 INFO:teuthology.orchestra.run.smithi165.stdout: python3-google-auth-1:2.29.0-1.el9.noarch 2024-04-27T03:29:09.753 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-27T03:29:09.753 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-27T03:29:09.753 INFO:teuthology.orchestra.run.smithi165.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-27T03:29:09.753 INFO:teuthology.orchestra.run.smithi165.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-27T03:29:09.753 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-27T03:29:09.753 INFO:teuthology.orchestra.run.smithi165.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-27T03:29:09.753 INFO:teuthology.orchestra.run.smithi165.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-27T03:29:09.753 INFO:teuthology.orchestra.run.smithi165.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-27T03:29:09.753 INFO:teuthology.orchestra.run.smithi165.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-27T03:29:09.753 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:09.753 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:09.802 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:29:09.803 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:29:10.005 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-mgr-cephadm 2024-04-27T03:29:10.099 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:29:10.278 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-27T03:29:10.330 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-27T03:29:10.512 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:22 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:10.633 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:10.633 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:10.633 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-27T03:29:10.633 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:10.633 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:29:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 141 k 2024-04-27T03:29:10.634 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-27T03:29:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-27T03:29:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-27T03:29:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-27T03:29:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-27T03:29:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-27T03:29:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:10.634 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:29:10.634 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:10.634 INFO:teuthology.orchestra.run.smithi165.stdout:Install 6 Packages 2024-04-27T03:29:10.634 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:10.635 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 7.0 M 2024-04-27T03:29:10.635 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 33 M 2024-04-27T03:29:10.635 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:29:10.849 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-27T03:29:10.880 INFO:teuthology.orchestra.run.smithi165.stdout:(1/6): python3-jinja2-2.11.3-5.el9.noarch.rpm 2.5 MB/s | 248 kB 00:00 2024-04-27T03:29:10.949 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-27T03:29:11.012 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-27T03:29:11.042 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 6/6 2024-04-27T03:29:11.055 INFO:teuthology.orchestra.run.smithi165.stdout:(2/6): python3-babel-2.9.1-2.el9.noarch.rpm 22 MB/s | 6.0 MB 00:00 2024-04-27T03:29:11.164 INFO:teuthology.orchestra.run.smithi165.stdout:(3/6): ceph-mgr-cephadm-19.0.0-2447.gb22e2ebd.e 372 kB/s | 141 kB 00:00 2024-04-27T03:29:11.197 INFO:teuthology.orchestra.run.smithi165.stdout:(4/6): python3-natsort-7.1.1-5.el9.noarch.rpm 407 kB/s | 58 kB 00:00 2024-04-27T03:29:11.281 INFO:teuthology.orchestra.run.smithi165.stdout:(5/6): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.3 MB/s | 548 kB 00:00 2024-04-27T03:29:11.348 INFO:teuthology.orchestra.run.smithi165.stdout:(6/6): python3-typing-extensions-4.4.0-2.el9.no 278 kB/s | 51 kB 00:00 2024-04-27T03:29:11.350 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:11.350 INFO:teuthology.orchestra.run.smithi165.stdout:Total 9.8 MB/s | 7.0 MB 00:00 2024-04-27T03:29:11.420 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:29:11.431 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:29:11.431 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:29:11.528 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:29:11.528 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:29:11.762 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 6/6 2024-04-27T03:29:11.762 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/6 2024-04-27T03:29:11.762 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-27T03:29:11.762 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-27T03:29:11.762 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-27T03:29:11.762 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-27T03:29:11.830 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:29:11.998 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 1/6 2024-04-27T03:29:12.024 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-27T03:29:12.025 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:12.025 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:29:12.025 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-mgr-cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:29:12.025 INFO:teuthology.orchestra.run.smithi089.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-27T03:29:12.025 INFO:teuthology.orchestra.run.smithi089.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-27T03:29:12.025 INFO:teuthology.orchestra.run.smithi089.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-27T03:29:12.025 INFO:teuthology.orchestra.run.smithi089.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-27T03:29:12.025 INFO:teuthology.orchestra.run.smithi089.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-27T03:29:12.025 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:12.025 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:12.074 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/6 2024-04-27T03:29:12.256 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-fuse 2024-04-27T03:29:12.597 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/6 2024-04-27T03:29:12.676 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 4/6 2024-04-27T03:29:12.739 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 5/6 2024-04-27T03:29:12.762 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:30 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:12.770 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 6/6 2024-04-27T03:29:12.880 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:12.881 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:12.881 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-04-27T03:29:12.881 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:12.881 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:29:12.881 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 889 k 2024-04-27T03:29:12.881 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-04-27T03:29:12.881 INFO:teuthology.orchestra.run.smithi089.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-27T03:29:12.881 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:12.881 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:29:12.882 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:12.882 INFO:teuthology.orchestra.run.smithi089.stdout:Install 2 Packages 2024-04-27T03:29:12.882 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:12.882 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 969 k 2024-04-27T03:29:12.882 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 2.8 M 2024-04-27T03:29:12.882 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:29:13.238 INFO:teuthology.orchestra.run.smithi089.stdout:(1/2): ceph-fuse-19.0.0-2447.gb22e2ebd.el9.x86_ 2.9 MB/s | 889 kB 00:00 2024-04-27T03:29:13.510 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 6/6 2024-04-27T03:29:13.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/6 2024-04-27T03:29:13.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/6 2024-04-27T03:29:13.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 3/6 2024-04-27T03:29:13.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 4/6 2024-04-27T03:29:13.510 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 5/6 2024-04-27T03:29:13.630 INFO:teuthology.orchestra.run.smithi089.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 117 kB/s | 80 kB 00:00 2024-04-27T03:29:13.630 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:13.631 INFO:teuthology.orchestra.run.smithi089.stdout:Total 1.3 MB/s | 969 kB 00:00 2024-04-27T03:29:13.643 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:29:13.665 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:29:13.665 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:29:13.742 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:29:13.742 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:29:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 6/6 2024-04-27T03:29:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:13.756 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:29:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-mgr-cephadm-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:29:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-27T03:29:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-27T03:29:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-27T03:29:13.756 INFO:teuthology.orchestra.run.smithi165.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-27T03:29:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-27T03:29:13.757 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:13.757 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:13.918 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:29:13.990 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-fuse 2024-04-27T03:29:14.056 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-27T03:29:14.136 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2/2 2024-04-27T03:29:14.507 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:26 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:14.626 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:14.626 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:14.626 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-27T03:29:14.626 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:14.626 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:29:14.626 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 889 k 2024-04-27T03:29:14.627 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-27T03:29:14.627 INFO:teuthology.orchestra.run.smithi165.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-27T03:29:14.627 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:14.627 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:29:14.627 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:14.627 INFO:teuthology.orchestra.run.smithi165.stdout:Install 2 Packages 2024-04-27T03:29:14.627 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:14.627 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 969 k 2024-04-27T03:29:14.628 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 2.8 M 2024-04-27T03:29:14.628 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:29:14.747 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2/2 2024-04-27T03:29:14.747 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/2 2024-04-27T03:29:15.000 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-27T03:29:15.000 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:15.000 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:29:15.000 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-27T03:29:15.000 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:15.000 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:15.057 INFO:teuthology.orchestra.run.smithi165.stdout:(1/2): ceph-fuse-19.0.0-2447.gb22e2ebd.el9.x86_ 3.1 MB/s | 889 kB 00:00 2024-04-27T03:29:15.257 INFO:teuthology.orchestra.run.smithi165.stdout:(2/2): fuse-2.9.9-15.el9.x86_64.rpm 168 kB/s | 80 kB 00:00 2024-04-27T03:29:15.257 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:15.258 INFO:teuthology.orchestra.run.smithi165.stdout:Total 1.5 MB/s | 969 kB 00:00 2024-04-27T03:29:15.266 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install ceph-volume 2024-04-27T03:29:15.269 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:29:15.292 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:29:15.292 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:29:15.369 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:29:15.369 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:29:15.548 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:29:15.683 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : fuse-2.9.9-15.el9.x86_64 1/2 2024-04-27T03:29:15.755 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2/2 2024-04-27T03:29:15.775 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:33 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:15.895 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:15.896 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:15.896 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repository Size 2024-04-27T03:29:15.896 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:15.896 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:29:15.896 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 264 k 2024-04-27T03:29:15.896 INFO:teuthology.orchestra.run.smithi089.stdout:Installing dependencies: 2024-04-27T03:29:15.896 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-27T03:29:15.896 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:15.896 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:29:15.896 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:15.896 INFO:teuthology.orchestra.run.smithi089.stdout:Install 2 Packages 2024-04-27T03:29:15.896 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:15.897 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 341 k 2024-04-27T03:29:15.897 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 1.5 M 2024-04-27T03:29:15.897 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:29:16.235 INFO:teuthology.orchestra.run.smithi089.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 404 kB/s | 77 kB 00:00 2024-04-27T03:29:16.302 INFO:teuthology.orchestra.run.smithi089.stdout:(2/2): ceph-volume-19.0.0-2447.gb22e2ebd.el9.no 1.0 MB/s | 264 kB 00:00 2024-04-27T03:29:16.303 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:16.303 INFO:teuthology.orchestra.run.smithi089.stdout:Total 837 kB/s | 341 kB 00:00 2024-04-27T03:29:16.315 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:29:16.324 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:29:16.324 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:29:16.349 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:29:16.350 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:29:16.483 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2/2 2024-04-27T03:29:16.484 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/2 2024-04-27T03:29:16.496 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:29:16.676 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-27T03:29:16.713 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : ceph-volume-2:19.0.0-2447.gb22e2ebd.el9.noarch 2/2 2024-04-27T03:29:16.727 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: ceph-volume-2:19.0.0-2447.gb22e2ebd.el9.noarch 2/2 2024-04-27T03:29:16.727 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:29:16.728 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-27T03:29:16.728 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:16.742 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 2/2 2024-04-27T03:29:16.742 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:16.742 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:29:16.742 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 fuse-2.9.9-15.el9.x86_64 2024-04-27T03:29:16.742 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:16.743 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:17.003 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install ceph-volume 2024-04-27T03:29:17.388 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : ceph-volume-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/2 2024-04-27T03:29:17.527 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:29 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:17.650 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:17.651 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:17.651 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repository Size 2024-04-27T03:29:17.651 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:17.651 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:29:17.651 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume noarch 2:19.0.0-2447.gb22e2ebd.el9 ceph-noarch 264 k 2024-04-27T03:29:17.651 INFO:teuthology.orchestra.run.smithi165.stdout:Installing dependencies: 2024-04-27T03:29:17.651 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-27T03:29:17.652 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:17.652 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:29:17.652 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:17.652 INFO:teuthology.orchestra.run.smithi165.stdout:Install 2 Packages 2024-04-27T03:29:17.652 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:17.652 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 341 k 2024-04-27T03:29:17.652 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 1.5 M 2024-04-27T03:29:17.652 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:29:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-27T03:29:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:17.675 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:29:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: ceph-volume-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:29:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-27T03:29:17.675 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:17.675 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:17.886 INFO:teuthology.orchestra.run.smithi165.stdout:(1/2): python3-packaging-20.9-5.el9.noarch.rpm 474 kB/s | 77 kB 00:00 2024-04-27T03:29:17.961 INFO:teuthology.orchestra.run.smithi165.stdout:(2/2): ceph-volume-19.0.0-2447.gb22e2ebd.el9.no 1.1 MB/s | 264 kB 00:00 2024-04-27T03:29:17.961 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:17.962 INFO:teuthology.orchestra.run.smithi165.stdout:Total 1.1 MB/s | 341 kB 00:00 2024-04-27T03:29:17.966 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install librados-devel 2024-04-27T03:29:17.974 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:29:17.983 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:29:17.983 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:29:18.009 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:29:18.010 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:29:18.149 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:29:18.320 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : python3-packaging-20.9-5.el9.noarch 1/2 2024-04-27T03:29:18.363 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : ceph-volume-2:19.0.0-2447.gb22e2ebd.el9.noarch 2/2 2024-04-27T03:29:18.378 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: ceph-volume-2:19.0.0-2447.gb22e2ebd.el9.noarch 2/2 2024-04-27T03:29:18.379 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:29:18.379 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-27T03:29:18.379 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:18.481 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:36 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:18.605 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:18.606 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:18.606 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repo Size 2024-04-27T03:29:18.606 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:18.606 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:29:18.606 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 127 k 2024-04-27T03:29:18.606 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:18.606 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:29:18.606 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:18.606 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2024-04-27T03:29:18.606 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:18.606 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 127 k 2024-04-27T03:29:18.606 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 456 k 2024-04-27T03:29:18.607 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:29:18.825 INFO:teuthology.orchestra.run.smithi089.stdout:librados-devel-19.0.0-2447.gb22e2ebd.el9.x86_64 581 kB/s | 127 kB 00:00 2024-04-27T03:29:18.825 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:18.826 INFO:teuthology.orchestra.run.smithi089.stdout:Total 577 kB/s | 127 kB 00:00 2024-04-27T03:29:18.826 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:29:18.831 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:29:18.831 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:29:18.894 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:29:18.894 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:29:19.025 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : ceph-volume-2:19.0.0-2447.gb22e2ebd.el9.noarch 1/2 2024-04-27T03:29:19.046 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:29:19.145 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : librados-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:19.300 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 2/2 2024-04-27T03:29:19.301 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:19.301 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:29:19.301 INFO:teuthology.orchestra.run.smithi165.stdout: ceph-volume-2:19.0.0-2447.gb22e2ebd.el9.noarch 2024-04-27T03:29:19.301 INFO:teuthology.orchestra.run.smithi165.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-27T03:29:19.301 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:19.301 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:19.568 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: librados-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:19.609 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librados-devel 2024-04-27T03:29:19.815 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : librados-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:19.816 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:19.816 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:29:19.816 INFO:teuthology.orchestra.run.smithi089.stdout: librados-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:29:19.816 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:19.816 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:20.050 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install libcephfs2 2024-04-27T03:29:20.133 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:32 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:20.258 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:20.258 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:20.259 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2024-04-27T03:29:20.259 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:20.259 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:29:20.259 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 127 k 2024-04-27T03:29:20.259 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:20.259 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:29:20.259 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:20.259 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-27T03:29:20.259 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:20.259 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 127 k 2024-04-27T03:29:20.259 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 456 k 2024-04-27T03:29:20.259 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:29:20.491 INFO:teuthology.orchestra.run.smithi165.stdout:librados-devel-19.0.0-2447.gb22e2ebd.el9.x86_64 547 kB/s | 127 kB 00:00 2024-04-27T03:29:20.492 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:20.492 INFO:teuthology.orchestra.run.smithi165.stdout:Total 544 kB/s | 127 kB 00:00 2024-04-27T03:29:20.492 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:29:20.497 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:29:20.497 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:29:20.561 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:29:20.561 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:29:20.568 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:38 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:20.648 INFO:teuthology.orchestra.run.smithi089.stdout:Package libcephfs2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:20.690 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:20.691 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2024-04-27T03:29:20.691 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:20.707 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:29:20.767 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install libcephfs-devel 2024-04-27T03:29:20.812 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : librados-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:21.235 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: librados-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:21.289 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:39 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:21.410 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:21.411 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:21.411 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repo Size 2024-04-27T03:29:21.411 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:21.411 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:29:21.411 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 32 k 2024-04-27T03:29:21.411 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:21.411 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:29:21.411 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:21.411 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2024-04-27T03:29:21.411 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:21.411 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 32 k 2024-04-27T03:29:21.411 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 142 k 2024-04-27T03:29:21.412 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:29:21.463 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : librados-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:21.463 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:21.463 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:29:21.463 INFO:teuthology.orchestra.run.smithi165.stdout: librados-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:29:21.464 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:21.464 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:21.678 INFO:teuthology.orchestra.run.smithi089.stdout:libcephfs-devel-19.0.0-2447.gb22e2ebd.el9.x86_6 119 kB/s | 32 kB 00:00 2024-04-27T03:29:21.679 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:21.679 INFO:teuthology.orchestra.run.smithi089.stdout:Total 118 kB/s | 32 kB 00:00 2024-04-27T03:29:21.679 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:29:21.683 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:29:21.683 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:29:21.685 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:29:21.685 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:29:21.691 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install libcephfs2 2024-04-27T03:29:21.768 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:29:21.873 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : libcephfs-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:22.216 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:34 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:22.218 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:22.297 INFO:teuthology.orchestra.run.smithi165.stdout:Package libcephfs2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:22.340 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:22.341 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-27T03:29:22.341 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : libcephfs-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:22.433 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:29:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: libcephfs-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:29:22.433 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:22.433 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:22.437 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install libcephfs-devel 2024-04-27T03:29:22.608 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install librados2 2024-04-27T03:29:22.958 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:34 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:23.079 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:23.080 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2024-04-27T03:29:23.080 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:23.080 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:29:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 32 k 2024-04-27T03:29:23.080 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:23.080 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:29:23.080 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:23.081 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-27T03:29:23.081 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:23.081 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 32 k 2024-04-27T03:29:23.081 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 142 k 2024-04-27T03:29:23.081 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:29:23.118 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:41 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:23.196 INFO:teuthology.orchestra.run.smithi089.stdout:Package librados2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:23.237 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:23.238 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2024-04-27T03:29:23.238 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:23.305 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install librbd1 2024-04-27T03:29:23.335 INFO:teuthology.orchestra.run.smithi165.stdout:libcephfs-devel-19.0.0-2447.gb22e2ebd.el9.x86_6 125 kB/s | 32 kB 00:00 2024-04-27T03:29:23.336 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:23.336 INFO:teuthology.orchestra.run.smithi165.stdout:Total 124 kB/s | 32 kB 00:00 2024-04-27T03:29:23.336 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:29:23.339 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:29:23.340 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:29:23.342 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:29:23.342 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:29:23.430 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:29:23.530 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : libcephfs-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:23.811 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:41 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:23.889 INFO:teuthology.orchestra.run.smithi089.stdout:Package librbd1-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:23.910 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: libcephfs-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:23.930 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:23.931 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2024-04-27T03:29:23.931 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:23.998 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install python3-rados 2024-04-27T03:29:24.112 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : libcephfs-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:24.112 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:24.112 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:29:24.112 INFO:teuthology.orchestra.run.smithi165.stdout: libcephfs-devel-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:29:24.112 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:24.112 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:24.292 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librados2 2024-04-27T03:29:24.511 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:42 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:24.591 INFO:teuthology.orchestra.run.smithi089.stdout:Package python3-rados-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:24.632 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2024-04-27T03:29:24.633 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:24.702 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install python3-rgw 2024-04-27T03:29:24.800 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:36 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:24.878 INFO:teuthology.orchestra.run.smithi165.stdout:Package librados2-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:24.919 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:24.920 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-27T03:29:24.920 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:24.992 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install librbd1 2024-04-27T03:29:25.202 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:43 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:25.280 INFO:teuthology.orchestra.run.smithi089.stdout:Package python3-rgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:25.321 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:25.322 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2024-04-27T03:29:25.322 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:25.385 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install python3-cephfs 2024-04-27T03:29:25.500 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:37 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:25.579 INFO:teuthology.orchestra.run.smithi165.stdout:Package librbd1-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:25.619 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:25.620 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-27T03:29:25.621 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:25.689 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-rados 2024-04-27T03:29:25.895 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:43 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:25.974 INFO:teuthology.orchestra.run.smithi089.stdout:Package python3-cephfs-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:26.015 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:26.016 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2024-04-27T03:29:26.016 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:26.089 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install python3-rbd 2024-04-27T03:29:26.198 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:38 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:26.277 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-rados-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:26.320 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:26.321 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-27T03:29:26.321 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:26.397 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-rgw 2024-04-27T03:29:26.596 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:44 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:26.674 INFO:teuthology.orchestra.run.smithi089.stdout:Package python3-rbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:26.715 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:26.716 INFO:teuthology.orchestra.run.smithi089.stdout:Nothing to do. 2024-04-27T03:29:26.716 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:26.785 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install rbd-fuse 2024-04-27T03:29:26.912 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:38 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:26.992 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-rgw-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:27.032 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:27.033 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-27T03:29:27.033 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:27.100 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-cephfs 2024-04-27T03:29:27.302 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:45 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:27.430 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:27.431 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:27.431 INFO:teuthology.orchestra.run.smithi089.stdout: Package Architecture Version Repository Size 2024-04-27T03:29:27.431 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:27.431 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:29:27.431 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 85 k 2024-04-27T03:29:27.431 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:27.431 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:29:27.431 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:27.432 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2024-04-27T03:29:27.432 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:27.432 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 85 k 2024-04-27T03:29:27.432 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 231 k 2024-04-27T03:29:27.432 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:29:27.611 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:39 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:27.648 INFO:teuthology.orchestra.run.smithi089.stdout:rbd-fuse-19.0.0-2447.gb22e2ebd.el9.x86_64.rpm 393 kB/s | 85 kB 00:00 2024-04-27T03:29:27.649 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:27.649 INFO:teuthology.orchestra.run.smithi089.stdout:Total 390 kB/s | 85 kB 00:00 2024-04-27T03:29:27.650 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:29:27.657 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:29:27.657 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:29:27.690 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-cephfs-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:27.720 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:29:27.720 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:29:27.730 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:27.732 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-27T03:29:27.732 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:27.799 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install python3-rbd 2024-04-27T03:29:27.865 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:29:27.969 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:28.307 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:40 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:28.326 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:28.387 INFO:teuthology.orchestra.run.smithi165.stdout:Package python3-rbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 is already installed. 2024-04-27T03:29:28.427 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:28.428 INFO:teuthology.orchestra.run.smithi165.stdout:Nothing to do. 2024-04-27T03:29:28.428 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:28.499 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-fuse 2024-04-27T03:29:28.537 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:28.537 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:28.538 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:29:28.538 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:29:28.538 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:28.538 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:28.785 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install rbd-mirror 2024-04-27T03:29:29.010 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:41 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:29.130 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:29.131 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:29.131 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2024-04-27T03:29:29.131 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:29.131 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:29:29.131 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 85 k 2024-04-27T03:29:29.131 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:29.131 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:29:29.131 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:29.132 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-27T03:29:29.132 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:29.132 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 85 k 2024-04-27T03:29:29.132 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 231 k 2024-04-27T03:29:29.132 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:29:29.296 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:47 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:29.416 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:29.417 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:29.417 INFO:teuthology.orchestra.run.smithi089.stdout: Package Arch Version Repo Size 2024-04-27T03:29:29.417 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:29.417 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:29:29.417 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 3.1 M 2024-04-27T03:29:29.417 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:29.417 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:29:29.417 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:29.417 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2024-04-27T03:29:29.417 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:29.417 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 3.1 M 2024-04-27T03:29:29.418 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 13 M 2024-04-27T03:29:29.418 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:29:29.446 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-fuse-19.0.0-2447.gb22e2ebd.el9.x86_64.rpm 271 kB/s | 85 kB 00:00 2024-04-27T03:29:29.447 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:29.447 INFO:teuthology.orchestra.run.smithi165.stdout:Total 270 kB/s | 85 kB 00:00 2024-04-27T03:29:29.447 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:29:29.454 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:29:29.454 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:29:29.517 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:29:29.517 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:29:29.658 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:29:29.759 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:29.802 INFO:teuthology.orchestra.run.smithi089.stdout:rbd-mirror-19.0.0-2447.gb22e2ebd.el9.x86_64.rpm 8.0 MB/s | 3.1 MB 00:00 2024-04-27T03:29:29.803 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:29.803 INFO:teuthology.orchestra.run.smithi089.stdout:Total 8.0 MB/s | 3.1 MB 00:00 2024-04-27T03:29:29.803 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:29:29.812 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:29:29.813 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:29:29.893 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:29:29.893 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:29:30.136 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:30.323 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:29:30.378 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-mirror-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:30.379 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:30.379 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:30.379 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:29:30.380 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-fuse-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:29:30.380 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:30.380 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:30.403 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:30.403 INFO:teuthology.orchestra.run.smithi089.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:29:30.403 INFO:teuthology.orchestra.run.smithi089.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-27T03:29:30.404 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-27T03:29:30.404 INFO:teuthology.orchestra.run.smithi089.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-27T03:29:30.404 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:30.651 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-mirror 2024-04-27T03:29:31.172 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:43 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:31.295 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:31.296 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:31.296 INFO:teuthology.orchestra.run.smithi165.stdout: Package Arch Version Repo Size 2024-04-27T03:29:31.296 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:31.297 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:29:31.297 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 3.1 M 2024-04-27T03:29:31.297 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:31.297 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:29:31.297 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:31.298 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-27T03:29:31.298 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:31.298 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 3.1 M 2024-04-27T03:29:31.298 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 13 M 2024-04-27T03:29:31.298 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:29:31.422 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-mirror-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:31.422 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:31.422 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:29:31.422 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-mirror-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:29:31.422 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:31.423 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:31.637 DEBUG:teuthology.orchestra.run.smithi089:> sudo yum -y install rbd-nbd 2024-04-27T03:29:31.858 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-mirror-19.0.0-2447.gb22e2ebd.el9.x86_64.rpm 5.5 MB/s | 3.1 MB 00:00 2024-04-27T03:29:31.859 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:31.859 INFO:teuthology.orchestra.run.smithi165.stdout:Total 5.5 MB/s | 3.1 MB 00:00 2024-04-27T03:29:31.859 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:29:31.869 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:29:31.869 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:29:31.952 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:29:31.953 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:29:32.151 INFO:teuthology.orchestra.run.smithi089.stdout:Last metadata expiration check: 0:01:50 ago on Sat 27 Apr 2024 03:27:42 AM UTC. 2024-04-27T03:29:32.274 INFO:teuthology.orchestra.run.smithi089.stdout:Dependencies resolved. 2024-04-27T03:29:32.275 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:32.275 INFO:teuthology.orchestra.run.smithi089.stdout: Package Architecture Version Repository Size 2024-04-27T03:29:32.275 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:32.275 INFO:teuthology.orchestra.run.smithi089.stdout:Installing: 2024-04-27T03:29:32.275 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 170 k 2024-04-27T03:29:32.275 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:32.275 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction Summary 2024-04-27T03:29:32.275 INFO:teuthology.orchestra.run.smithi089.stdout:================================================================================ 2024-04-27T03:29:32.276 INFO:teuthology.orchestra.run.smithi089.stdout:Install 1 Package 2024-04-27T03:29:32.276 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:32.276 INFO:teuthology.orchestra.run.smithi089.stdout:Total download size: 170 k 2024-04-27T03:29:32.276 INFO:teuthology.orchestra.run.smithi089.stdout:Installed size: 494 k 2024-04-27T03:29:32.276 INFO:teuthology.orchestra.run.smithi089.stdout:Downloading Packages: 2024-04-27T03:29:32.403 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:29:32.467 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-mirror-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-mirror-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:32.493 INFO:teuthology.orchestra.run.smithi165.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-27T03:29:32.493 INFO:teuthology.orchestra.run.smithi165.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-27T03:29:32.493 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-27T03:29:32.493 INFO:teuthology.orchestra.run.smithi165.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-27T03:29:32.493 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:32.505 INFO:teuthology.orchestra.run.smithi089.stdout:rbd-nbd-19.0.0-2447.gb22e2ebd.el9.x86_64.rpm 743 kB/s | 170 kB 00:00 2024-04-27T03:29:32.506 INFO:teuthology.orchestra.run.smithi089.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:32.506 INFO:teuthology.orchestra.run.smithi089.stdout:Total 737 kB/s | 170 kB 00:00 2024-04-27T03:29:32.506 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction check 2024-04-27T03:29:32.514 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction check succeeded. 2024-04-27T03:29:32.514 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction test 2024-04-27T03:29:32.577 INFO:teuthology.orchestra.run.smithi089.stdout:Transaction test succeeded. 2024-04-27T03:29:32.577 INFO:teuthology.orchestra.run.smithi089.stdout:Running transaction 2024-04-27T03:29:32.730 INFO:teuthology.orchestra.run.smithi089.stdout: Preparing : 1/1 2024-04-27T03:29:32.844 INFO:teuthology.orchestra.run.smithi089.stdout: Installing : rbd-nbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:33.222 INFO:teuthology.orchestra.run.smithi089.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:33.463 INFO:teuthology.orchestra.run.smithi089.stdout: Verifying : rbd-nbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:33.464 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:33.464 INFO:teuthology.orchestra.run.smithi089.stdout:Installed: 2024-04-27T03:29:33.464 INFO:teuthology.orchestra.run.smithi089.stdout: rbd-nbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:29:33.464 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:33.464 INFO:teuthology.orchestra.run.smithi089.stdout:Complete! 2024-04-27T03:29:33.551 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-mirror-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:33.551 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:33.551 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:29:33.551 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-mirror-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:29:33.551 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:33.551 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:33.702 DEBUG:teuthology.parallel:result is None 2024-04-27T03:29:33.793 DEBUG:teuthology.orchestra.run.smithi165:> sudo yum -y install rbd-nbd 2024-04-27T03:29:34.304 INFO:teuthology.orchestra.run.smithi165.stdout:Last metadata expiration check: 0:01:46 ago on Sat 27 Apr 2024 03:27:48 AM UTC. 2024-04-27T03:29:34.428 INFO:teuthology.orchestra.run.smithi165.stdout:Dependencies resolved. 2024-04-27T03:29:34.429 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: Package Architecture Version Repository Size 2024-04-27T03:29:34.429 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:34.429 INFO:teuthology.orchestra.run.smithi165.stdout:Installing: 2024-04-27T03:29:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd x86_64 2:19.0.0-2447.gb22e2ebd.el9 ceph 170 k 2024-04-27T03:29:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:34.429 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction Summary 2024-04-27T03:29:34.429 INFO:teuthology.orchestra.run.smithi165.stdout:================================================================================ 2024-04-27T03:29:34.429 INFO:teuthology.orchestra.run.smithi165.stdout:Install 1 Package 2024-04-27T03:29:34.429 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:34.429 INFO:teuthology.orchestra.run.smithi165.stdout:Total download size: 170 k 2024-04-27T03:29:34.429 INFO:teuthology.orchestra.run.smithi165.stdout:Installed size: 494 k 2024-04-27T03:29:34.430 INFO:teuthology.orchestra.run.smithi165.stdout:Downloading Packages: 2024-04-27T03:29:34.652 INFO:teuthology.orchestra.run.smithi165.stdout:rbd-nbd-19.0.0-2447.gb22e2ebd.el9.x86_64.rpm 766 kB/s | 170 kB 00:00 2024-04-27T03:29:34.653 INFO:teuthology.orchestra.run.smithi165.stdout:-------------------------------------------------------------------------------- 2024-04-27T03:29:34.653 INFO:teuthology.orchestra.run.smithi165.stdout:Total 761 kB/s | 170 kB 00:00 2024-04-27T03:29:34.653 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction check 2024-04-27T03:29:34.661 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction check succeeded. 2024-04-27T03:29:34.661 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction test 2024-04-27T03:29:34.725 INFO:teuthology.orchestra.run.smithi165.stdout:Transaction test succeeded. 2024-04-27T03:29:34.725 INFO:teuthology.orchestra.run.smithi165.stdout:Running transaction 2024-04-27T03:29:34.879 INFO:teuthology.orchestra.run.smithi165.stdout: Preparing : 1/1 2024-04-27T03:29:34.982 INFO:teuthology.orchestra.run.smithi165.stdout: Installing : rbd-nbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:35.393 INFO:teuthology.orchestra.run.smithi165.stdout: Running scriptlet: rbd-nbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:35.619 INFO:teuthology.orchestra.run.smithi165.stdout: Verifying : rbd-nbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 1/1 2024-04-27T03:29:35.619 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:35.619 INFO:teuthology.orchestra.run.smithi165.stdout:Installed: 2024-04-27T03:29:35.619 INFO:teuthology.orchestra.run.smithi165.stdout: rbd-nbd-2:19.0.0-2447.gb22e2ebd.el9.x86_64 2024-04-27T03:29:35.619 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:35.619 INFO:teuthology.orchestra.run.smithi165.stdout:Complete! 2024-04-27T03:29:35.906 DEBUG:teuthology.parallel:result is None 2024-04-27T03:29:35.907 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-27T03:29:36.123 DEBUG:teuthology.orchestra.run.smithi089:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-27T03:29:36.147 INFO:teuthology.orchestra.run.smithi089.stdout:19.0.0-2447.gb22e2ebd.el9 2024-04-27T03:29:36.148 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2447.gb22e2ebd.el9 2024-04-27T03:29:36.148 INFO:teuthology.task.install:The correct ceph version 19.0.0-2447.gb22e2ebd is installed. 2024-04-27T03:29:36.150 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=b22e2ebdeb24376882b7bda2a7329c8cccc2276a 2024-04-27T03:29:36.366 DEBUG:teuthology.orchestra.run.smithi165:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-27T03:29:36.387 INFO:teuthology.orchestra.run.smithi165.stdout:19.0.0-2447.gb22e2ebd.el9 2024-04-27T03:29:36.387 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-2447.gb22e2ebd.el9 2024-04-27T03:29:36.387 INFO:teuthology.task.install:The correct ceph version 19.0.0-2447.gb22e2ebd is installed. 2024-04-27T03:29:36.389 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-27T03:29:36.390 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:29:36.390 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-27T03:29:36.419 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:36.419 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-27T03:29:36.452 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-27T03:29:36.452 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:29:36.452 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/daemon-helper 2024-04-27T03:29:36.485 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-27T03:29:36.551 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:36.551 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/daemon-helper 2024-04-27T03:29:36.575 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-27T03:29:36.639 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-27T03:29:36.639 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:29:36.639 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-27T03:29:36.665 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-27T03:29:36.731 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:36.731 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-27T03:29:36.754 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-27T03:29:36.817 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-27T03:29:36.818 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:29:36.818 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/usr/bin/stdin-killer 2024-04-27T03:29:36.843 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-27T03:29:36.908 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:36.908 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/usr/bin/stdin-killer 2024-04-27T03:29:36.931 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-27T03:29:36.995 INFO:teuthology.run_tasks:Running task ceph... 2024-04-27T03:29:37.080 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2024-04-27T03:29:37.080 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 777 /var/log/ceph 2024-04-27T03:29:37.082 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 777 /var/log/ceph 2024-04-27T03:29:37.106 INFO:tasks.ceph:Disabling ceph logrotate... 2024-04-27T03:29:37.106 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-27T03:29:37.148 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/logrotate.d/ceph 2024-04-27T03:29:37.171 INFO:tasks.ceph:Creating extra log directories... 2024-04-27T03:29:37.171 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-27T03:29:37.213 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2024-04-27T03:29:37.246 INFO:tasks.ceph:Creating ceph cluster ceph... 2024-04-27T03:29:37.247 INFO:tasks.ceph:config {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2024-04-27T03:29:37.247 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672', 'branch': 'squid', 'description': 'rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_latest}}', 'email': 'ceph-qa@ceph.com', 'first_in_suite': False, 'job_id': '7675672', 'kernel': {'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'smithi', 'name': 'teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 3, 'size': 10}}], 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'squid'}, 'ceph': {'conf': {'client': {'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'}}, 'rgw': {'ec-data-pool': False, 'frontend': 'beast'}, 'selinux': {'allowlist': ['scontext=system_u:system_r:getty_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'squid', 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a'}}, 'owner': 'scheduled_teuthology@teuthology', 'priority': 100, 'repo': 'https://git.ceph.com/ceph.git', 'roles': [['mon.a', 'osd.0'], ['mgr.x', 'client.0', 'osd.1', 'osd.2', 'osd.3']], 'seed': 7340, 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a', 'sleep_before_teardown': 0, 'subset': '0/1', 'suite': 'rgw', 'suite_branch': 'squid', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph_b22e2ebdeb24376882b7bda2a7329c8cccc2276a/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph.git', 'suite_sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a', 'targets': {'smithi089.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDzYiMSBF4aqIe5Qm+cDeIt3txT+9fS5+hAXpK/EXm3W+qL7WXrCZfJJPTqR2VaxGfXa4aPjPJzZFh2uJ6OY6qU=', 'smithi165.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBC7Z6J+tAFj4YXchxvYjTYnocjQ/CfPgtt0ScdIakZyVglcgpY1HYvs2afnrXKPnPc9TImTaFRFe8RvhQVpI0OA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': None}, {'ceph': {'conf': {'client': {'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'osd': {'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(PG_AVAILABILITY\\)', '\\(PG_DEGRADED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'not have an application enabled'], 'sha1': 'b22e2ebdeb24376882b7bda2a7329c8cccc2276a', 'cluster': 'ceph'}}, {'rgw': {'client.0': None}}, {'radosgw-admin': None}, {'radosgw-admin-rest': None}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17', 'timestamp': '2024-04-26_21:40:02', 'tube': 'smithi', 'user': 'teuthology', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885'} 2024-04-27T03:29:37.248 DEBUG:teuthology.orchestra.run.smithi089:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-27T03:29:37.278 DEBUG:teuthology.orchestra.run.smithi165:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2024-04-27T03:29:37.292 DEBUG:teuthology.orchestra.run.smithi089:> sudo install -d -m0777 -- /var/run/ceph 2024-04-27T03:29:37.334 DEBUG:teuthology.orchestra.run.smithi165:> sudo install -d -m0777 -- /var/run/ceph 2024-04-27T03:29:37.357 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:29:37.358 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/scratch_devs of=/dev/stdout 2024-04-27T03:29:37.412 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-27T03:29:37.413 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_1 2024-04-27T03:29:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-27T03:29:37.467 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T03:29:37.467 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-27T03:29:37.467 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T03:29:37.468 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T03:29:37.468 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-04-27 03:29:30.646093240 +0000 2024-04-27T03:29:37.468 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-04-27 03:26:57.718677685 +0000 2024-04-27T03:29:37.468 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-04-27 03:26:57.718677685 +0000 2024-04-27T03:29:37.468 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-04-27 03:26:57.718677685 +0000 2024-04-27T03:29:37.468 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-27T03:29:37.530 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-04-27T03:29:37.530 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-04-27T03:29:37.530 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000204719 s, 2.5 MB/s 2024-04-27T03:29:37.531 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-27T03:29:37.587 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_2 2024-04-27T03:29:37.642 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-27T03:29:37.642 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T03:29:37.642 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-27T03:29:37.642 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T03:29:37.642 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T03:29:37.642 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-04-27 03:29:30.647093218 +0000 2024-04-27T03:29:37.643 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-04-27 03:26:58.043670000 +0000 2024-04-27T03:29:37.643 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-04-27 03:26:58.043670000 +0000 2024-04-27T03:29:37.643 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-04-27 03:26:58.043670000 +0000 2024-04-27T03:29:37.643 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-27T03:29:37.704 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-04-27T03:29:37.704 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-04-27T03:29:37.704 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.00020677 s, 2.5 MB/s 2024-04-27T03:29:37.705 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-27T03:29:37.760 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_3 2024-04-27T03:29:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-27T03:29:37.816 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T03:29:37.816 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-27T03:29:37.816 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T03:29:37.816 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T03:29:37.816 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-04-27 03:29:30.647093218 +0000 2024-04-27T03:29:37.816 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-04-27 03:26:58.373662196 +0000 2024-04-27T03:29:37.816 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-04-27 03:26:58.373662196 +0000 2024-04-27T03:29:37.817 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-04-27 03:26:58.373662196 +0000 2024-04-27T03:29:37.817 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-27T03:29:37.878 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-04-27T03:29:37.878 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-04-27T03:29:37.878 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000200389 s, 2.6 MB/s 2024-04-27T03:29:37.879 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-27T03:29:37.934 DEBUG:teuthology.orchestra.run.smithi089:> stat /dev/vg_nvme/lv_4 2024-04-27T03:29:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-27T03:29:37.989 INFO:teuthology.orchestra.run.smithi089.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T03:29:37.989 INFO:teuthology.orchestra.run.smithi089.stdout:Device: 5h/5d Inode: 720 Links: 1 2024-04-27T03:29:37.989 INFO:teuthology.orchestra.run.smithi089.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T03:29:37.989 INFO:teuthology.orchestra.run.smithi089.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T03:29:37.989 INFO:teuthology.orchestra.run.smithi089.stdout:Access: 2024-04-27 03:29:30.647093218 +0000 2024-04-27T03:29:37.989 INFO:teuthology.orchestra.run.smithi089.stdout:Modify: 2024-04-27 03:26:58.712654180 +0000 2024-04-27T03:29:37.990 INFO:teuthology.orchestra.run.smithi089.stdout:Change: 2024-04-27 03:26:58.712654180 +0000 2024-04-27T03:29:37.990 INFO:teuthology.orchestra.run.smithi089.stdout: Birth: 2024-04-27 03:26:58.712654180 +0000 2024-04-27T03:29:37.990 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-27T03:29:38.051 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records in 2024-04-27T03:29:38.051 INFO:teuthology.orchestra.run.smithi089.stderr:1+0 records out 2024-04-27T03:29:38.051 INFO:teuthology.orchestra.run.smithi089.stderr:512 bytes copied, 0.000159719 s, 3.2 MB/s 2024-04-27T03:29:38.052 DEBUG:teuthology.orchestra.run.smithi089:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-27T03:29:38.107 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-04-27T03:29:38.107 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:38.108 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/scratch_devs of=/dev/stdout 2024-04-27T03:29:38.125 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-27T03:29:38.125 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_1 2024-04-27T03:29:38.182 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-27T03:29:38.182 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T03:29:38.182 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 665 Links: 1 2024-04-27T03:29:38.182 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T03:29:38.182 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T03:29:38.182 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-27 03:29:32.752847441 +0000 2024-04-27T03:29:38.183 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-27 03:26:57.717073028 +0000 2024-04-27T03:29:38.183 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-27 03:26:57.717073028 +0000 2024-04-27T03:29:38.183 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-27 03:26:57.717073028 +0000 2024-04-27T03:29:38.183 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-27T03:29:38.246 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-27T03:29:38.247 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-27T03:29:38.247 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000217719 s, 2.4 MB/s 2024-04-27T03:29:38.248 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-27T03:29:38.305 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_2 2024-04-27T03:29:38.361 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-27T03:29:38.361 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T03:29:38.361 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-27T03:29:38.361 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T03:29:38.361 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T03:29:38.361 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-27 03:29:32.753847425 +0000 2024-04-27T03:29:38.361 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-27 03:26:58.058067248 +0000 2024-04-27T03:29:38.361 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-27 03:26:58.058067248 +0000 2024-04-27T03:29:38.362 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-27 03:26:58.058067248 +0000 2024-04-27T03:29:38.362 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-27T03:29:38.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-27T03:29:38.426 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-27T03:29:38.426 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000212662 s, 2.4 MB/s 2024-04-27T03:29:38.426 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-27T03:29:38.482 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_3 2024-04-27T03:29:38.537 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-27T03:29:38.537 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T03:29:38.537 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 705 Links: 1 2024-04-27T03:29:38.537 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T03:29:38.538 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T03:29:38.538 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-27 03:29:32.753847425 +0000 2024-04-27T03:29:38.538 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-27 03:26:58.384061723 +0000 2024-04-27T03:29:38.538 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-27 03:26:58.384061723 +0000 2024-04-27T03:29:38.538 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-27 03:26:58.384061723 +0000 2024-04-27T03:29:38.538 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-27T03:29:38.600 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-27T03:29:38.600 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-27T03:29:38.600 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.000184725 s, 2.8 MB/s 2024-04-27T03:29:38.601 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-27T03:29:38.657 DEBUG:teuthology.orchestra.run.smithi165:> stat /dev/vg_nvme/lv_4 2024-04-27T03:29:38.712 INFO:teuthology.orchestra.run.smithi165.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-27T03:29:38.712 INFO:teuthology.orchestra.run.smithi165.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-27T03:29:38.712 INFO:teuthology.orchestra.run.smithi165.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-27T03:29:38.712 INFO:teuthology.orchestra.run.smithi165.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-27T03:29:38.712 INFO:teuthology.orchestra.run.smithi165.stdout:Context: system_u:object_r:device_t:s0 2024-04-27T03:29:38.713 INFO:teuthology.orchestra.run.smithi165.stdout:Access: 2024-04-27 03:29:32.753847425 +0000 2024-04-27T03:29:38.713 INFO:teuthology.orchestra.run.smithi165.stdout:Modify: 2024-04-27 03:26:58.731055840 +0000 2024-04-27T03:29:38.713 INFO:teuthology.orchestra.run.smithi165.stdout:Change: 2024-04-27 03:26:58.731055840 +0000 2024-04-27T03:29:38.713 INFO:teuthology.orchestra.run.smithi165.stdout: Birth: 2024-04-27 03:26:58.731055840 +0000 2024-04-27T03:29:38.713 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-27T03:29:38.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records in 2024-04-27T03:29:38.774 INFO:teuthology.orchestra.run.smithi165.stderr:1+0 records out 2024-04-27T03:29:38.774 INFO:teuthology.orchestra.run.smithi165.stderr:512 bytes copied, 0.0001731 s, 3.0 MB/s 2024-04-27T03:29:38.775 DEBUG:teuthology.orchestra.run.smithi165:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-27T03:29:38.831 INFO:tasks.ceph:osd dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-27T03:29:38.831 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi165.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-04-27T03:29:38.831 INFO:tasks.ceph:Generating config... 2024-04-27T03:29:38.832 INFO:tasks.ceph:[client] debug ms = 1 2024-04-27T03:29:38.832 INFO:tasks.ceph:[client] rgw gc obj min wait = 15 2024-04-27T03:29:38.832 INFO:tasks.ceph:[client] debug rgw = 20 2024-04-27T03:29:38.832 INFO:tasks.ceph:[client] setgroup = ceph 2024-04-27T03:29:38.832 INFO:tasks.ceph:[client] setuser = ceph 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] debug ms = 1 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] debug objclass = 20 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] bdev async discard = True 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] bdev enable discard = True 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] debug osd = 20 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2024-04-27T03:29:38.833 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2024-04-27T03:29:38.834 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2024-04-27T03:29:38.834 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2024-04-27T03:29:38.834 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2024-04-27T03:29:38.834 INFO:tasks.ceph:[mgr] debug mgr = 20 2024-04-27T03:29:38.834 INFO:tasks.ceph:[mgr] debug ms = 1 2024-04-27T03:29:38.834 INFO:tasks.ceph:[mon] debug mon = 20 2024-04-27T03:29:38.834 INFO:tasks.ceph:[mon] debug ms = 1 2024-04-27T03:29:38.834 INFO:tasks.ceph:[mon] debug paxos = 20 2024-04-27T03:29:38.834 INFO:tasks.ceph:Setting up mon.a... 2024-04-27T03:29:38.835 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2024-04-27T03:29:38.883 INFO:teuthology.orchestra.run.smithi089.stdout:creating /etc/ceph/ceph.keyring 2024-04-27T03:29:38.886 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2024-04-27T03:29:38.926 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T03:29:38.989 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.89')] 2024-04-27T03:29:38.990 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.89'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug ms': 1, 'debug objclass': 20, 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false'}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'debug ms': 1, 'rgw gc obj min wait': 15, 'debug rgw': 20, 'setgroup': 'ceph', 'setuser': 'ceph'}, 'mon.a': {}} 2024-04-27T03:29:38.991 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:29:38.991 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2024-04-27T03:29:39.044 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.89 --print /home/ubuntu/cephtest/ceph.monmap 2024-04-27T03:29:39.123 INFO:teuthology.orchestra.run.smithi089.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:29:39.123 INFO:teuthology.orchestra.run.smithi089.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:29:39.123 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2024-04-27T03:29:39.124 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: generated fsid 7a1794da-6463-461c-a75b-269448ecfc86 2024-04-27T03:29:39.124 INFO:teuthology.orchestra.run.smithi089.stdout:setting min_mon_release = quincy 2024-04-27T03:29:39.124 INFO:teuthology.orchestra.run.smithi089.stdout:epoch 0 2024-04-27T03:29:39.124 INFO:teuthology.orchestra.run.smithi089.stdout:fsid 7a1794da-6463-461c-a75b-269448ecfc86 2024-04-27T03:29:39.124 INFO:teuthology.orchestra.run.smithi089.stdout:last_changed 2024-04-27T03:29:39.118959+0000 2024-04-27T03:29:39.124 INFO:teuthology.orchestra.run.smithi089.stdout:created 2024-04-27T03:29:39.118959+0000 2024-04-27T03:29:39.124 INFO:teuthology.orchestra.run.smithi089.stdout:min_mon_release 17 (quincy) 2024-04-27T03:29:39.124 INFO:teuthology.orchestra.run.smithi089.stdout:election_strategy: 1 2024-04-27T03:29:39.124 INFO:teuthology.orchestra.run.smithi089.stdout:0: [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] mon.a 2024-04-27T03:29:39.124 INFO:teuthology.orchestra.run.smithi089.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2024-04-27T03:29:39.125 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2024-04-27T03:29:39.180 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7a1794da-6463-461c-a75b-269448ecfc86... 2024-04-27T03:29:39.181 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-27T03:29:39.223 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2024-04-27T03:29:39.260 INFO:teuthology.orchestra.run.smithi089.stdout:[global] 2024-04-27T03:29:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: chdir = "" 2024-04-27T03:29:39.260 INFO:teuthology.orchestra.run.smithi089.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: auth supported = cephx 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: filestore xattr use omap = true 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: mon clock drift allowed = 1.000 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: osd crush chooseleaf type = 0 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: auth debug = true 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on old message = true 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: ms die on bug = true 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-27T03:29:39.261 INFO:teuthology.orchestra.run.smithi089.stdout: mon pg warn max object skew = 0 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: # disable pg_autoscaler by default for new pools 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default size = 2 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow primary affinity = true 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd allow pg remap = true 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on legacy crush tunables = false 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on crush straw calc version zero = false 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on no sortbitwise = false 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on osd down out interval zero = false 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: mon warn on too few osds = false 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-27T03:29:39.262 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: mon_allow_pool_size_one = true 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: osd default data pool replay window = 5 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: mon allow pool delete = true 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: mon cluster log file level = debug 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: debug asserts on shutdown = true 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: mon health detail to clog = false 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: mon host = 172.21.15.89 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: fsid = 7a1794da-6463-461c-a75b-269448ecfc86 2024-04-27T03:29:39.263 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout:[osd] 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: osd journal size = 100 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub load threshold = 5.0 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: osd scrub max interval = 600 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: osd mclock profile = high_recovery_ops 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: osd recover clone overlap = true 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: osd recovery max chunk = 1048576 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug shutdown = true 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug op order = true 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug verify stray on activate = true 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug trim objects = true 2024-04-27T03:29:39.264 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: osd open classes on start = true 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug pg log writeout = true 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: osd deep scrub update digest min age = 30 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: osd map max advance = 10 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: journal zero on create = true 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: filestore ondisk finisher threads = 3 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: filestore apply finisher threads = 3 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: bdev debug aio = true 2024-04-27T03:29:39.265 INFO:teuthology.orchestra.run.smithi089.stdout: osd debug misdirected ops = true 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: debug objclass = 20 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: bdev async discard = True 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: bdev enable discard = True 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore allocator = bitmap 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore block size = 96636764160 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: bluestore fsck on mount = True 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluefs = 1/20 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: debug bluestore = 1/20 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: debug osd = 20 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: debug rocksdb = 4/10 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd backfillfull_ratio = 0.85 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd full ratio = 0.9 2024-04-27T03:29:39.266 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd nearfull ratio = 0.8 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: osd failsafe full ratio = 0.95 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: osd objectstore = bluestore 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr] 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: debug mgr = 20 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: mgr/telemetry/nag = false 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout:[mon] 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2024-04-27T03:29:39.267 INFO:teuthology.orchestra.run.smithi089.stdout: debug mon = 20 2024-04-27T03:29:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: debug paxos = 20 2024-04-27T03:29:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: debug auth = 20 2024-04-27T03:29:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: mon data avail warn = 5 2024-04-27T03:29:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: mon mgr mkfs grace = 240 2024-04-27T03:29:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min pgs per osd = 4 2024-04-27T03:29:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd reporter subtree level = osd 2024-04-27T03:29:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: mon osd prime pg temp = true 2024-04-27T03:29:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: mon reweight min bytes per osd = 10 2024-04-27T03:29:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-27T03:29:39.268 INFO:teuthology.orchestra.run.smithi089.stdout: auth mon ticket ttl = 660 # 11m 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: auth service ticket ttl = 240 # 4m 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: # don't complain about insecure global_id in the test suite 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: # 1m isn't quite enough 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: mon_down_mkfs_grace = 2m 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: mon_warn_on_filestore_osds = false 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout:[client] 2024-04-27T03:29:39.269 INFO:teuthology.orchestra.run.smithi089.stdout: rgw cache enabled = true 2024-04-27T03:29:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable ops log = true 2024-04-27T03:29:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: rgw enable usage log = true 2024-04-27T03:29:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-27T03:29:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-27T03:29:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: debug ms = 1 2024-04-27T03:29:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: rgw gc obj min wait = 15 2024-04-27T03:29:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: debug rgw = 20 2024-04-27T03:29:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: setgroup = ceph 2024-04-27T03:29:39.270 INFO:teuthology.orchestra.run.smithi089.stdout: setuser = ceph 2024-04-27T03:29:39.270 INFO:teuthology.orchestra.run.smithi089.stdout:[mon.a] 2024-04-27T03:29:39.271 INFO:teuthology.orchestra.run.smithi165.stdout:[global] 2024-04-27T03:29:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: chdir = "" 2024-04-27T03:29:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2024-04-27T03:29:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: auth supported = cephx 2024-04-27T03:29:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: filestore xattr use omap = true 2024-04-27T03:29:39.271 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: mon clock drift allowed = 1.000 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: osd crush chooseleaf type = 0 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: auth debug = true 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on old message = true 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: ms die on bug = true 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: mon max pg per osd = 10000 # >= luminous 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: mon pg warn max object skew = 0 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: # disable pg_autoscaler by default for new pools 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: osd_pool_default_pg_autoscale_mode = off 2024-04-27T03:29:39.272 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default size = 2 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow primary affinity = true 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd allow pg remap = true 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on legacy crush tunables = false 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on crush straw calc version zero = false 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on no sortbitwise = false 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on osd down out interval zero = false 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: mon warn on too few osds = false 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_pool_no_redundancy = false 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: mon_allow_pool_size_one = true 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2024-04-27T03:29:39.273 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: osd default data pool replay window = 5 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: mon allow pool delete = true 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: mon cluster log file level = debug 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: debug asserts on shutdown = true 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: mon health detail to clog = false 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: mon host = 172.21.15.89 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: fsid = 7a1794da-6463-461c-a75b-269448ecfc86 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout:[osd] 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: osd journal size = 100 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.274 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub load threshold = 5.0 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd scrub max interval = 600 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd mclock profile = high_recovery_ops 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd recover clone overlap = true 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd recovery max chunk = 1048576 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug shutdown = true 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug op order = true 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug verify stray on activate = true 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug trim objects = true 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd open classes on start = true 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug pg log writeout = true 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.275 INFO:teuthology.orchestra.run.smithi165.stdout: osd deep scrub update digest min age = 30 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: osd map max advance = 10 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: journal zero on create = true 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: filestore ondisk finisher threads = 3 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: filestore apply finisher threads = 3 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: bdev debug aio = true 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: osd debug misdirected ops = true 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: debug objclass = 20 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: bdev async discard = True 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: bdev enable discard = True 2024-04-27T03:29:39.276 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore allocator = bitmap 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore block size = 96636764160 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: bluestore fsck on mount = True 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluefs = 1/20 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: debug bluestore = 1/20 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: debug osd = 20 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: debug rocksdb = 4/10 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd backfillfull_ratio = 0.85 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd full ratio = 0.9 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd nearfull ratio = 0.8 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd failsafe full ratio = 0.95 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: osd objectstore = bluestore 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout:[mgr] 2024-04-27T03:29:39.277 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: debug mgr = 20 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: mgr/telemetry/nag = false 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout:[mon] 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: debug mon = 20 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: debug paxos = 20 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: debug auth = 20 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: mon data avail warn = 5 2024-04-27T03:29:39.278 INFO:teuthology.orchestra.run.smithi165.stdout: mon mgr mkfs grace = 240 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min pgs per osd = 4 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd reporter subtree level = osd 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: mon osd prime pg temp = true 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: mon reweight min bytes per osd = 10 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: # rotate auth tickets quickly to exercise renewal paths 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: auth mon ticket ttl = 660 # 11m 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: auth service ticket ttl = 240 # 4m 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: # don't complain about insecure global_id in the test suite 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim = false 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.279 INFO:teuthology.orchestra.run.smithi165.stdout: # 1m isn't quite enough 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: mon_down_mkfs_grace = 2m 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: mon_warn_on_filestore_osds = false 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout:[client] 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: rgw cache enabled = true 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable ops log = true 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: rgw enable usage log = true 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: debug ms = 1 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: rgw gc obj min wait = 15 2024-04-27T03:29:39.280 INFO:teuthology.orchestra.run.smithi165.stdout: debug rgw = 20 2024-04-27T03:29:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: setgroup = ceph 2024-04-27T03:29:39.281 INFO:teuthology.orchestra.run.smithi165.stdout: setuser = ceph 2024-04-27T03:29:39.281 INFO:teuthology.orchestra.run.smithi165.stdout:[mon.a] 2024-04-27T03:29:39.282 INFO:tasks.ceph:Creating admin key on mon.a... 2024-04-27T03:29:39.282 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2024-04-27T03:29:39.374 INFO:tasks.ceph:Copying monmap to all nodes... 2024-04-27T03:29:39.374 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:29:39.374 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2024-04-27T03:29:39.389 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:29:39.389 DEBUG:teuthology.orchestra.run.smithi089:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2024-04-27T03:29:39.444 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi089.front.sepia.ceph.com 2024-04-27T03:29:39.444 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:29:39.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-27T03:29:39.444 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T03:29:39.516 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:29:39.516 DEBUG:teuthology.orchestra.run.smithi089:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-27T03:29:39.571 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi165.front.sepia.ceph.com 2024-04-27T03:29:39.571 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:39.571 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd of=/etc/ceph/ceph.keyring 2024-04-27T03:29:39.572 DEBUG:teuthology.orchestra.run.smithi165:> sudo chmod 0644 /etc/ceph/ceph.keyring 2024-04-27T03:29:39.605 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:39.605 DEBUG:teuthology.orchestra.run.smithi165:> dd of=/home/ubuntu/cephtest/ceph.monmap 2024-04-27T03:29:39.660 INFO:tasks.ceph:Setting up mon nodes... 2024-04-27T03:29:39.661 INFO:tasks.ceph:Setting up mgr nodes... 2024-04-27T03:29:39.661 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2024-04-27T03:29:39.764 INFO:teuthology.orchestra.run.smithi165.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2024-04-27T03:29:39.766 INFO:tasks.ceph:Setting up mds nodes... 2024-04-27T03:29:39.766 INFO:tasks.ceph_client:Setting up client nodes... 2024-04-27T03:29:39.767 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-27T03:29:39.805 INFO:teuthology.orchestra.run.smithi165.stdout:creating /etc/ceph/ceph.client.0.keyring 2024-04-27T03:29:39.817 INFO:tasks.ceph:Running mkfs on osd nodes... 2024-04-27T03:29:39.817 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi089.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1'}, Remote(name='ubuntu@smithi165.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'}} 2024-04-27T03:29:39.817 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2024-04-27T03:29:39.841 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1'} 2024-04-27T03:29:39.841 INFO:tasks.ceph:role: osd.0 2024-04-27T03:29:39.841 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com 2024-04-27T03:29:39.841 DEBUG:teuthology.orchestra.run.smithi089:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-27T03:29:39.912 INFO:teuthology.orchestra.run.smithi089.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T03:29:39.913 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T03:29:39.913 INFO:teuthology.orchestra.run.smithi089.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T03:29:39.913 INFO:teuthology.orchestra.run.smithi089.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-27T03:29:39.913 INFO:teuthology.orchestra.run.smithi089.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T03:29:39.913 INFO:teuthology.orchestra.run.smithi089.stdout: = sunit=0 swidth=0 blks 2024-04-27T03:29:39.913 INFO:teuthology.orchestra.run.smithi089.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T03:29:39.913 INFO:teuthology.orchestra.run.smithi089.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-27T03:29:39.913 INFO:teuthology.orchestra.run.smithi089.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T03:29:39.913 INFO:teuthology.orchestra.run.smithi089.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T03:29:40.426 INFO:teuthology.orchestra.run.smithi089.stdout:Discarding blocks...Done. 2024-04-27T03:29:40.428 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi089.front.sepia.ceph.com -o noatime 2024-04-27T03:29:40.428 DEBUG:teuthology.orchestra.run.smithi089:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2024-04-27T03:29:40.562 DEBUG:teuthology.orchestra.run.smithi089:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2024-04-27T03:29:40.595 DEBUG:teuthology.orchestra.run.smithi089:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T03:29:40.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:40.683+0000 7ff94afb2640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2024-04-27T03:29:40.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:40.686+0000 7ff94afb2640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2024-04-27T03:29:40.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:40.686+0000 7ff94afb2640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T03:29:40.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:40.686+0000 7ff94afb2640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T03:29:40.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:40.686+0000 7ff94afb2640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2024-04-27T03:29:40.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:40.689+0000 7ff94afb2640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2024-04-27T03:29:43.551 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2024-04-27T03:29:43.577 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2024-04-27T03:29:43.606 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-27T03:29:43.606 INFO:tasks.ceph:role: osd.1 2024-04-27T03:29:43.606 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-27T03:29:43.606 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2024-04-27T03:29:43.680 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T03:29:43.680 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T03:29:43.681 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T03:29:43.681 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-27T03:29:43.681 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T03:29:43.681 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-04-27T03:29:43.681 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T03:29:43.681 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-27T03:29:43.681 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T03:29:43.681 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T03:29:44.197 INFO:teuthology.orchestra.run.smithi165.stdout:Discarding blocks...Done. 2024-04-27T03:29:44.199 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-04-27T03:29:44.200 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2024-04-27T03:29:44.358 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2024-04-27T03:29:44.393 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2024-04-27T03:29:44.460 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-27T03:29:44.460 INFO:tasks.ceph:role: osd.2 2024-04-27T03:29:44.460 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-27T03:29:44.460 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2024-04-27T03:29:44.533 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T03:29:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T03:29:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T03:29:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-27T03:29:44.533 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T03:29:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-04-27T03:29:44.533 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T03:29:44.533 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-27T03:29:44.533 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T03:29:44.533 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T03:29:44.980 INFO:teuthology.orchestra.run.smithi165.stdout:Discarding blocks...Done. 2024-04-27T03:29:44.982 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-04-27T03:29:44.983 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2024-04-27T03:29:45.075 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2024-04-27T03:29:45.104 DEBUG:teuthology.orchestra.run.smithi165:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2024-04-27T03:29:45.168 INFO:tasks.ceph:roles_to_devs: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.2': '/dev/vg_nvme/lv_2', 'osd.3': '/dev/vg_nvme/lv_3'} 2024-04-27T03:29:45.168 INFO:tasks.ceph:role: osd.3 2024-04-27T03:29:45.168 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-27T03:29:45.168 DEBUG:teuthology.orchestra.run.smithi165:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2024-04-27T03:29:45.238 INFO:teuthology.orchestra.run.smithi165.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2024-04-27T03:29:45.238 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 attr=2, projid32bit=1 2024-04-27T03:29:45.239 INFO:teuthology.orchestra.run.smithi165.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2024-04-27T03:29:45.239 INFO:teuthology.orchestra.run.smithi165.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2024-04-27T03:29:45.239 INFO:teuthology.orchestra.run.smithi165.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2024-04-27T03:29:45.239 INFO:teuthology.orchestra.run.smithi165.stdout: = sunit=0 swidth=0 blks 2024-04-27T03:29:45.239 INFO:teuthology.orchestra.run.smithi165.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2024-04-27T03:29:45.239 INFO:teuthology.orchestra.run.smithi165.stdout:log =internal log bsize=4096 blocks=16384, version=2 2024-04-27T03:29:45.239 INFO:teuthology.orchestra.run.smithi165.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2024-04-27T03:29:45.239 INFO:teuthology.orchestra.run.smithi165.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2024-04-27T03:29:45.732 INFO:teuthology.orchestra.run.smithi165.stdout:Discarding blocks...Done. 2024-04-27T03:29:45.734 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi165.front.sepia.ceph.com -o noatime 2024-04-27T03:29:45.734 DEBUG:teuthology.orchestra.run.smithi165:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2024-04-27T03:29:45.883 DEBUG:teuthology.orchestra.run.smithi165:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2024-04-27T03:29:45.912 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T03:29:46.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:46.002+0000 7f123e823640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2024-04-27T03:29:46.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:46.005+0000 7f123e823640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2024-04-27T03:29:46.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:46.005+0000 7f123e823640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T03:29:46.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:46.005+0000 7f123e823640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T03:29:46.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:46.005+0000 7f123e823640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2024-04-27T03:29:46.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:46.015+0000 7f123e823640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2024-04-27T03:29:48.863 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2024-04-27T03:29:48.889 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T03:29:48.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:48.967+0000 7f3fdf030640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2024-04-27T03:29:48.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:48.968+0000 7f3fdf030640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2024-04-27T03:29:48.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:48.968+0000 7f3fdf030640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T03:29:48.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:48.968+0000 7f3fdf030640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T03:29:48.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:48.968+0000 7f3fdf030640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2024-04-27T03:29:48.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:48.968+0000 7f3fdf030640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2024-04-27T03:29:51.823 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2024-04-27T03:29:51.888 DEBUG:teuthology.orchestra.run.smithi165:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2024-04-27T03:29:51.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:51.966+0000 7f0abb8ae640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2024-04-27T03:29:51.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:51.966+0000 7f0abb8ae640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2024-04-27T03:29:51.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:51.966+0000 7f0abb8ae640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T03:29:51.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:51.966+0000 7f0abb8ae640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T03:29:51.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:51.966+0000 7f0abb8ae640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2024-04-27T03:29:51.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:51.967+0000 7f0abb8ae640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2024-04-27T03:29:54.814 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2024-04-27T03:29:54.880 INFO:tasks.ceph:Reading keys from all nodes... 2024-04-27T03:29:54.880 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:29:54.880 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2024-04-27T03:29:54.908 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:54.908 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2024-04-27T03:29:54.944 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:54.944 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2024-04-27T03:29:55.008 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:55.008 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2024-04-27T03:29:55.072 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:55.072 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2024-04-27T03:29:55.136 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:55.136 DEBUG:teuthology.orchestra.run.smithi165:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2024-04-27T03:29:55.191 INFO:tasks.ceph:Adding keys to all mons... 2024-04-27T03:29:55.191 DEBUG:teuthology.orchestra.run.smithi089:> sudo tee -a /etc/ceph/ceph.keyring 2024-04-27T03:29:55.218 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.0] 2024-04-27T03:29:55.218 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAkcSxmmnbRKBAAAKU4735r9WjOVsssFh7Yow== 2024-04-27T03:29:55.218 INFO:teuthology.orchestra.run.smithi089.stdout:[mgr.x] 2024-04-27T03:29:55.218 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAjcSxm6jxJLRAArYffc6Db/PpLDhP+oHKFIg== 2024-04-27T03:29:55.218 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.1] 2024-04-27T03:29:55.219 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAqcSxm1F01ABAAZn47af2wi/5PiD0LMFQ+mQ== 2024-04-27T03:29:55.219 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.2] 2024-04-27T03:29:55.219 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAscSxmMPy9ORAAlLpt5NqeQ42H3rG/03a7kg== 2024-04-27T03:29:55.219 INFO:teuthology.orchestra.run.smithi089.stdout:[osd.3] 2024-04-27T03:29:55.219 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAvcSxmkvmkORAAoa0EwdrwozE9XHwFnJT5Uw== 2024-04-27T03:29:55.219 INFO:teuthology.orchestra.run.smithi089.stdout:[client.0] 2024-04-27T03:29:55.219 INFO:teuthology.orchestra.run.smithi089.stdout: key = AQAjcSxmJna+LxAA8hrMJXG1JxgBKgi7nw9EtQ== 2024-04-27T03:29:55.220 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T03:29:55.315 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2024-04-27T03:29:55.366 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T03:29:55.416 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T03:29:55.464 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2024-04-27T03:29:55.513 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2024-04-27T03:29:55.586 INFO:tasks.ceph:Running mkfs on mon nodes... 2024-04-27T03:29:55.586 DEBUG:teuthology.orchestra.run.smithi089:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2024-04-27T03:29:55.609 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2024-04-27T03:29:55.702 DEBUG:teuthology.orchestra.run.smithi089:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2024-04-27T03:29:55.725 DEBUG:teuthology.orchestra.run.smithi089:> rm -- /home/ubuntu/cephtest/ceph.monmap 2024-04-27T03:29:55.780 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2024-04-27T03:29:55.780 INFO:tasks.ceph.mon.a:Restarting daemon 2024-04-27T03:29:55.780 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2024-04-27T03:29:55.822 INFO:tasks.ceph.mon.a:Started 2024-04-27T03:29:55.823 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2024-04-27T03:29:55.823 INFO:tasks.ceph.mgr.x:Restarting daemon 2024-04-27T03:29:55.823 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2024-04-27T03:29:55.825 INFO:tasks.ceph.mgr.x:Started 2024-04-27T03:29:55.825 INFO:tasks.ceph:Setting crush tunables to default 2024-04-27T03:29:55.826 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd crush tunables default 2024-04-27T03:29:55.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.915+0000 7ff7a9515640 1 Processor -- start 2024-04-27T03:29:55.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.915+0000 7ff7a9515640 1 -- start start 2024-04-27T03:29:55.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.916+0000 7ff7a9515640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7a4070ae0 0x7ff7a4131790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:55.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.916+0000 7ff7a9515640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff7a409bfe0 con 0x7ff7a406fff0 2024-04-27T03:29:55.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.916+0000 7ff7a9515640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff7a40720a0 con 0x7ff7a4070ae0 2024-04-27T03:29:55.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.916+0000 7ff7a27fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7a4070ae0 0x7ff7a4131790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:55.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.916+0000 7ff7a2ffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff7a406fff0 0x7ff7a40703d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:43996/0 (socket says 172.21.15.89:43996) 2024-04-27T03:29:55.922 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.916+0000 7ff7a27fc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff7a4070ae0 0x7ff7a4131790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39946/0 (socket says 172.21.15.89:39946) 2024-04-27T03:29:55.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.916+0000 7ff7a2ffd640 1 -- 172.21.15.89:0/564298480 learned_addr learned my addr 172.21.15.89:0/564298480 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:29:55.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.916+0000 7ff7a27fc640 1 -- 172.21.15.89:0/564298480 >> v1:172.21.15.89:6789/0 conn(0x7ff7a406fff0 legacy=0x7ff7a40703d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:55.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.916+0000 7ff7a27fc640 1 -- 172.21.15.89:0/564298480 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a40718e0 con 0x7ff7a4070ae0 2024-04-27T03:29:55.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.918+0000 7ff7a27fc640 1 --2- 172.21.15.89:0/564298480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a4070ae0 0x7ff7a4131790 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff798009720 tx=0x7ff79802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9037d5a2427847f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:55.923 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.918+0000 7ff7a1ffb640 1 -- 172.21.15.89:0/564298480 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7980316d0 con 0x7ff7a4070ae0 2024-04-27T03:29:55.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.918+0000 7ff7a1ffb640 1 -- 172.21.15.89:0/564298480 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff798037070 con 0x7ff7a4070ae0 2024-04-27T03:29:55.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.918+0000 7ff7a1ffb640 1 -- 172.21.15.89:0/564298480 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff7980319c0 con 0x7ff7a4070ae0 2024-04-27T03:29:55.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.919+0000 7ff7a9515640 1 -- 172.21.15.89:0/564298480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a4070ae0 msgr2=0x7ff7a4131790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:55.924 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.919+0000 7ff7a9515640 1 --2- 172.21.15.89:0/564298480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a4070ae0 0x7ff7a4131790 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7ff798009720 tx=0x7ff79802b840 comp rx=0 tx=0).stop 2024-04-27T03:29:55.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.919+0000 7ff7a9515640 1 -- 172.21.15.89:0/564298480 shutdown_connections 2024-04-27T03:29:55.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.919+0000 7ff7a9515640 1 --2- 172.21.15.89:0/564298480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a4070ae0 0x7ff7a4131790 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:55.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.919+0000 7ff7a9515640 1 -- 172.21.15.89:0/564298480 >> 172.21.15.89:0/564298480 conn(0x7ff7a409cb40 msgr2=0x7ff7a409cf50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:55.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.919+0000 7ff7a9515640 1 -- 172.21.15.89:0/564298480 shutdown_connections 2024-04-27T03:29:55.925 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.919+0000 7ff7a9515640 1 -- 172.21.15.89:0/564298480 wait complete. 2024-04-27T03:29:55.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.920+0000 7ff7a9515640 1 Processor -- start 2024-04-27T03:29:55.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.920+0000 7ff7a9515640 1 -- start start 2024-04-27T03:29:55.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.920+0000 7ff7a9515640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a406fff0 0x7ff7a4118e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:55.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.920+0000 7ff7a9515640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4131cd0 con 0x7ff7a406fff0 2024-04-27T03:29:55.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.920+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a406fff0 0x7ff7a4118e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:55.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.920+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a406fff0 0x7ff7a4118e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39952/0 (socket says 172.21.15.89:39952) 2024-04-27T03:29:55.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.920+0000 7ff7a2ffd640 1 -- 172.21.15.89:0/35521320 learned_addr learned my addr 172.21.15.89:0/35521320 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:29:55.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.921+0000 7ff7a2ffd640 1 -- 172.21.15.89:0/35521320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a411beb0 con 0x7ff7a406fff0 2024-04-27T03:29:55.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.921+0000 7ff7a2ffd640 1 --2- 172.21.15.89:0/35521320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a406fff0 0x7ff7a4118e00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff79400c4a0 tx=0x7ff79400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:55.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.921+0000 7ff78b7fe640 1 -- 172.21.15.89:0/35521320 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff794017070 con 0x7ff7a406fff0 2024-04-27T03:29:55.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.921+0000 7ff78b7fe640 1 -- 172.21.15.89:0/35521320 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff794005c10 con 0x7ff7a406fff0 2024-04-27T03:29:55.927 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.921+0000 7ff7a9515640 1 -- 172.21.15.89:0/35521320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a411b820 con 0x7ff7a406fff0 2024-04-27T03:29:55.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.921+0000 7ff78b7fe640 1 -- 172.21.15.89:0/35521320 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff794007070 con 0x7ff7a406fff0 2024-04-27T03:29:55.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.921+0000 7ff7a9515640 1 -- 172.21.15.89:0/35521320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a4119520 con 0x7ff7a406fff0 2024-04-27T03:29:55.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.921+0000 7ff78b7fe640 1 -- 172.21.15.89:0/35521320 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7ff79401a070 con 0x7ff7a406fff0 2024-04-27T03:29:55.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.921+0000 7ff78b7fe640 1 -- 172.21.15.89:0/35521320 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7ff794012400 con 0x7ff7a406fff0 2024-04-27T03:29:55.928 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.922+0000 7ff7a9515640 1 -- 172.21.15.89:0/35521320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff7a4062da0 con 0x7ff7a406fff0 2024-04-27T03:29:55.929 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.924+0000 7ff78b7fe640 1 -- 172.21.15.89:0/35521320 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7ff79401c440 con 0x7ff7a406fff0 2024-04-27T03:29:55.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.987+0000 7ff7a9515640 1 -- 172.21.15.89:0/35521320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7ff7a4119ff0 con 0x7ff7a406fff0 2024-04-27T03:29:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.988+0000 7ff78b7fe640 1 -- 172.21.15.89:0/35521320 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) ==== 124+0+0 (secure 0 0 0) 0x7ff794007210 con 0x7ff7a406fff0 2024-04-27T03:29:55.994 INFO:teuthology.orchestra.run.smithi089.stderr:adjusted tunables profile to default 2024-04-27T03:29:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.989+0000 7ff7a9515640 1 -- 172.21.15.89:0/35521320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a406fff0 msgr2=0x7ff7a4118e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.989+0000 7ff7a9515640 1 --2- 172.21.15.89:0/35521320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a406fff0 0x7ff7a4118e00 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7ff79400c4a0 tx=0x7ff79400c960 comp rx=0 tx=0).stop 2024-04-27T03:29:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.989+0000 7ff7a9515640 1 -- 172.21.15.89:0/35521320 shutdown_connections 2024-04-27T03:29:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.989+0000 7ff7a9515640 1 --2- 172.21.15.89:0/35521320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff7a406fff0 0x7ff7a4118e00 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:55.995 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.989+0000 7ff7a9515640 1 -- 172.21.15.89:0/35521320 >> 172.21.15.89:0/35521320 conn(0x7ff7a409cb40 msgr2=0x7ff7a412dc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.989+0000 7ff7a9515640 1 -- 172.21.15.89:0/35521320 shutdown_connections 2024-04-27T03:29:55.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:55.989+0000 7ff7a9515640 1 -- 172.21.15.89:0/35521320 wait complete. 2024-04-27T03:29:56.005 INFO:tasks.ceph:check_enable_crimson: False 2024-04-27T03:29:56.005 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2024-04-27T03:29:56.005 DEBUG:teuthology.orchestra.run.smithi089:> set -ex 2024-04-27T03:29:56.005 DEBUG:teuthology.orchestra.run.smithi089:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2024-04-27T03:29:56.028 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:56.029 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2024-04-27T03:29:56.052 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:56.052 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2024-04-27T03:29:56.115 DEBUG:teuthology.orchestra.run.smithi165:> set -ex 2024-04-27T03:29:56.115 DEBUG:teuthology.orchestra.run.smithi165:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2024-04-27T03:29:56.180 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 2643cf41-510f-4fc8-9611-7260d319e5f8 0 2024-04-27T03:29:56.210 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:56.205+0000 7f9c36419040 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-27T03:29:56.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.302+0000 7efe38752640 1 Processor -- start 2024-04-27T03:29:56.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.302+0000 7efe38752640 1 -- start start 2024-04-27T03:29:56.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.303+0000 7efe38752640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe301188e0 0x7efe3011acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:56.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.303+0000 7efe38752640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efe3011b210 con 0x7efe301149a0 2024-04-27T03:29:56.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.303+0000 7efe38752640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7efe3011c3f0 con 0x7efe301188e0 2024-04-27T03:29:56.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.303+0000 7efe35ced640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe301188e0 0x7efe3011acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:56.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.303+0000 7efe35ced640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efe301188e0 0x7efe3011acd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58678/0 (socket says 172.21.15.165:58678) 2024-04-27T03:29:56.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.303+0000 7efe35ced640 1 -- 172.21.15.165:0/501574409 learned_addr learned my addr 172.21.15.165:0/501574409 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:29:56.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.303+0000 7efe35ced640 1 -- 172.21.15.165:0/501574409 >> v1:172.21.15.89:6789/0 conn(0x7efe301149a0 legacy=0x7efe30114d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:56.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.303+0000 7efe35ced640 1 -- 172.21.15.165:0/501574409 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe3011c5a0 con 0x7efe301188e0 2024-04-27T03:29:56.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.304+0000 7efe35ced640 1 --2- 172.21.15.165:0/501574409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe301188e0 0x7efe3011acd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe200097f0 tx=0x7efe2002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e10c4f365a23b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:56.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.304+0000 7efe354ec640 1 -- 172.21.15.165:0/501574409 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe20035020 con 0x7efe301188e0 2024-04-27T03:29:56.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.304+0000 7efe354ec640 1 -- 172.21.15.165:0/501574409 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe200317a0 con 0x7efe301188e0 2024-04-27T03:29:56.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.304+0000 7efe354ec640 1 -- 172.21.15.165:0/501574409 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe20031aa0 con 0x7efe301188e0 2024-04-27T03:29:56.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.305+0000 7efe38752640 1 -- 172.21.15.165:0/501574409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe301188e0 msgr2=0x7efe3011acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:56.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.305+0000 7efe38752640 1 --2- 172.21.15.165:0/501574409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe301188e0 0x7efe3011acd0 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7efe200097f0 tx=0x7efe2002b910 comp rx=0 tx=0).stop 2024-04-27T03:29:56.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.305+0000 7efe38752640 1 -- 172.21.15.165:0/501574409 shutdown_connections 2024-04-27T03:29:56.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.305+0000 7efe38752640 1 --2- 172.21.15.165:0/501574409 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe301188e0 0x7efe3011acd0 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:56.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.305+0000 7efe38752640 1 -- 172.21.15.165:0/501574409 >> 172.21.15.165:0/501574409 conn(0x7efe30070e80 msgr2=0x7efe300732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:56.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.305+0000 7efe38752640 1 -- 172.21.15.165:0/501574409 shutdown_connections 2024-04-27T03:29:56.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.305+0000 7efe38752640 1 -- 172.21.15.165:0/501574409 wait complete. 2024-04-27T03:29:56.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.306+0000 7efe38752640 1 Processor -- start 2024-04-27T03:29:56.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.306+0000 7efe38752640 1 -- start start 2024-04-27T03:29:56.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.306+0000 7efe38752640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe301149a0 0x7efe301aa300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:56.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.306+0000 7efe38752640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efe3011b8c0 con 0x7efe301149a0 2024-04-27T03:29:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.306+0000 7efe364ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe301149a0 0x7efe301aa300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.306+0000 7efe364ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe301149a0 0x7efe301aa300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58694/0 (socket says 172.21.15.165:58694) 2024-04-27T03:29:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.306+0000 7efe364ee640 1 -- 172.21.15.165:0/775617738 learned_addr learned my addr 172.21.15.165:0/775617738 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:29:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.306+0000 7efe364ee640 1 -- 172.21.15.165:0/775617738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe301aa8d0 con 0x7efe301149a0 2024-04-27T03:29:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.306+0000 7efe364ee640 1 --2- 172.21.15.165:0/775617738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe301149a0 0x7efe301aa300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c450 tx=0x7efe2c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.307+0000 7efe26ffd640 1 -- 172.21.15.165:0/775617738 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe2c017070 con 0x7efe301149a0 2024-04-27T03:29:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.307+0000 7efe26ffd640 1 -- 172.21.15.165:0/775617738 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efe2c005220 con 0x7efe301149a0 2024-04-27T03:29:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.307+0000 7efe38752640 1 -- 172.21.15.165:0/775617738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe301aab60 con 0x7efe301149a0 2024-04-27T03:29:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.307+0000 7efe26ffd640 1 -- 172.21.15.165:0/775617738 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efe2c0035f0 con 0x7efe301149a0 2024-04-27T03:29:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.307+0000 7efe38752640 1 -- 172.21.15.165:0/775617738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe301a45f0 con 0x7efe301149a0 2024-04-27T03:29:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.307+0000 7efe26ffd640 1 -- 172.21.15.165:0/775617738 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7efe2c01a070 con 0x7efe301149a0 2024-04-27T03:29:56.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.307+0000 7efe26ffd640 1 -- 172.21.15.165:0/775617738 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 725+0+0 (secure 0 0 0) 0x7efe2c003ad0 con 0x7efe301149a0 2024-04-27T03:29:56.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.308+0000 7efe38752640 1 -- 172.21.15.165:0/775617738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe30062da0 con 0x7efe301149a0 2024-04-27T03:29:56.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.310+0000 7efe26ffd640 1 -- 172.21.15.165:0/775617738 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7efe2c006df0 con 0x7efe301149a0 2024-04-27T03:29:56.363 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:56.358+0000 7f9c36419040 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-27T03:29:56.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.382+0000 7efe38752640 1 -- 172.21.15.165:0/775617738 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd new", "uuid": "2643cf41-510f-4fc8-9611-7260d319e5f8", "id": 0} v 0) -- 0x7efe301aad90 con 0x7efe301149a0 2024-04-27T03:29:56.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.384+0000 7efe26ffd640 1 -- 172.21.15.165:0/775617738 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2643cf41-510f-4fc8-9611-7260d319e5f8", "id": 0}]=0 v3) ==== 112+0+2 (secure 0 0 0) 0x7efe2c003dd0 con 0x7efe301149a0 2024-04-27T03:29:56.389 INFO:teuthology.orchestra.run.smithi165.stdout:0 2024-04-27T03:29:56.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.384+0000 7efe38752640 1 -- 172.21.15.165:0/775617738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe301149a0 msgr2=0x7efe301aa300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:56.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.384+0000 7efe38752640 1 --2- 172.21.15.165:0/775617738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe301149a0 0x7efe301aa300 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7efe2c00c450 tx=0x7efe2c00c910 comp rx=0 tx=0).stop 2024-04-27T03:29:56.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.385+0000 7efe38752640 1 -- 172.21.15.165:0/775617738 shutdown_connections 2024-04-27T03:29:56.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.385+0000 7efe38752640 1 --2- 172.21.15.165:0/775617738 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efe301149a0 0x7efe301aa300 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:56.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.385+0000 7efe38752640 1 -- 172.21.15.165:0/775617738 >> 172.21.15.165:0/775617738 conn(0x7efe30070e80 msgr2=0x7efe30116cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:56.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.385+0000 7efe38752640 1 -- 172.21.15.165:0/775617738 shutdown_connections 2024-04-27T03:29:56.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.385+0000 7efe38752640 1 -- 172.21.15.165:0/775617738 wait complete. 2024-04-27T03:29:56.402 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 6d14d21a-3c2d-47bb-a0d9-96f58d57d612 1 2024-04-27T03:29:56.420 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:56.415+0000 7f9c36419040 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-27T03:29:56.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.494+0000 7f768d79d640 1 Processor -- start 2024-04-27T03:29:56.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.494+0000 7f768d79d640 1 -- start start 2024-04-27T03:29:56.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.494+0000 7f768d79d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f768811a8a0 0x7f768811cc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:56.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.494+0000 7f768d79d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f768811d1d0 con 0x7f768806cea0 2024-04-27T03:29:56.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.494+0000 7f768d79d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f768811e3b0 con 0x7f768811a8a0 2024-04-27T03:29:56.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.494+0000 7f76877fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f768811a8a0 0x7f768811cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:56.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.494+0000 7f76877fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f768811a8a0 0x7f768811cc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58708/0 (socket says 172.21.15.165:58708) 2024-04-27T03:29:56.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.494+0000 7f76877fe640 1 -- 172.21.15.165:0/1016480971 learned_addr learned my addr 172.21.15.165:0/1016480971 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:29:56.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.494+0000 7f7687fff640 1 --1- 172.21.15.165:0/1016480971 >> v1:172.21.15.89:6789/0 conn(0x7f768806cea0 0x7f768806d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:40384/0 (socket says 172.21.15.165:40384) 2024-04-27T03:29:56.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.494+0000 7f76877fe640 1 -- 172.21.15.165:0/1016480971 >> v1:172.21.15.89:6789/0 conn(0x7f768806cea0 legacy=0x7f768806d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:56.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.494+0000 7f76877fe640 1 -- 172.21.15.165:0/1016480971 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f768811e530 con 0x7f768811a8a0 2024-04-27T03:29:56.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.495+0000 7f76877fe640 1 --2- 172.21.15.165:0/1016480971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768811a8a0 0x7f768811cc90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f76780097f0 tx=0x7f767802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=171ef1d684b304df server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:56.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.495+0000 7f7686ffd640 1 -- 172.21.15.165:0/1016480971 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7678035020 con 0x7f768811a8a0 2024-04-27T03:29:56.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.495+0000 7f7686ffd640 1 -- 172.21.15.165:0/1016480971 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f76780317a0 con 0x7f768811a8a0 2024-04-27T03:29:56.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.495+0000 7f7686ffd640 1 -- 172.21.15.165:0/1016480971 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7678031aa0 con 0x7f768811a8a0 2024-04-27T03:29:56.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.496+0000 7f768d79d640 1 -- 172.21.15.165:0/1016480971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768811a8a0 msgr2=0x7f768811cc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:56.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.496+0000 7f768d79d640 1 --2- 172.21.15.165:0/1016480971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768811a8a0 0x7f768811cc90 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f76780097f0 tx=0x7f767802b910 comp rx=0 tx=0).stop 2024-04-27T03:29:56.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.496+0000 7f768d79d640 1 -- 172.21.15.165:0/1016480971 shutdown_connections 2024-04-27T03:29:56.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.496+0000 7f768d79d640 1 --2- 172.21.15.165:0/1016480971 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768811a8a0 0x7f768811cc90 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:56.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.496+0000 7f768d79d640 1 -- 172.21.15.165:0/1016480971 >> 172.21.15.165:0/1016480971 conn(0x7f7688070c60 msgr2=0x7f7688073080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:56.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.496+0000 7f768d79d640 1 -- 172.21.15.165:0/1016480971 shutdown_connections 2024-04-27T03:29:56.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.496+0000 7f768d79d640 1 -- 172.21.15.165:0/1016480971 wait complete. 2024-04-27T03:29:56.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.497+0000 7f768d79d640 1 Processor -- start 2024-04-27T03:29:56.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.497+0000 7f768d79d640 1 -- start start 2024-04-27T03:29:56.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.497+0000 7f768d79d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768806cea0 0x7f76881ac250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:56.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.497+0000 7f768d79d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f768811d8c0 con 0x7f768806cea0 2024-04-27T03:29:56.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.497+0000 7f7687fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768806cea0 0x7f76881ac250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:56.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.497+0000 7f7687fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768806cea0 0x7f76881ac250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58724/0 (socket says 172.21.15.165:58724) 2024-04-27T03:29:56.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.497+0000 7f7687fff640 1 -- 172.21.15.165:0/41018487 learned_addr learned my addr 172.21.15.165:0/41018487 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:29:56.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.497+0000 7f7687fff640 1 -- 172.21.15.165:0/41018487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76881ac820 con 0x7f768806cea0 2024-04-27T03:29:56.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.498+0000 7f7687fff640 1 --2- 172.21.15.165:0/41018487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768806cea0 0x7f76881ac250 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f767c00c3d0 tx=0x7f767c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:56.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.498+0000 7f7684ff9640 1 -- 172.21.15.165:0/41018487 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f767c007410 con 0x7f768806cea0 2024-04-27T03:29:56.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.498+0000 7f7684ff9640 1 -- 172.21.15.165:0/41018487 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f767c0075b0 con 0x7f768806cea0 2024-04-27T03:29:56.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.498+0000 7f768d79d640 1 -- 172.21.15.165:0/41018487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76881aca50 con 0x7f768806cea0 2024-04-27T03:29:56.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.498+0000 7f7684ff9640 1 -- 172.21.15.165:0/41018487 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f767c004910 con 0x7f768806cea0 2024-04-27T03:29:56.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.498+0000 7f768d79d640 1 -- 172.21.15.165:0/41018487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76881a6360 con 0x7f768806cea0 2024-04-27T03:29:56.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.498+0000 7f7684ff9640 1 -- 172.21.15.165:0/41018487 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f767c01a020 con 0x7f768806cea0 2024-04-27T03:29:56.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.498+0000 7f7684ff9640 1 -- 172.21.15.165:0/41018487 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 840+0+0 (secure 0 0 0) 0x7f767c01c900 con 0x7f768806cea0 2024-04-27T03:29:56.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.499+0000 7f768d79d640 1 -- 172.21.15.165:0/41018487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7688062da0 con 0x7f768806cea0 2024-04-27T03:29:56.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.501+0000 7f7684ff9640 1 -- 172.21.15.165:0/41018487 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f767c021070 con 0x7f768806cea0 2024-04-27T03:29:56.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.573+0000 7f768d79d640 1 -- 172.21.15.165:0/41018487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd new", "uuid": "6d14d21a-3c2d-47bb-a0d9-96f58d57d612", "id": 1} v 0) -- 0x7f768811a8a0 con 0x7f768806cea0 2024-04-27T03:29:56.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.574+0000 7f7684ff9640 1 -- 172.21.15.165:0/41018487 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6d14d21a-3c2d-47bb-a0d9-96f58d57d612", "id": 1}]=0 v4) ==== 112+0+2 (secure 0 0 0) 0x7f767c013400 con 0x7f768806cea0 2024-04-27T03:29:56.579 INFO:teuthology.orchestra.run.smithi165.stdout:1 2024-04-27T03:29:56.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.575+0000 7f768d79d640 1 -- 172.21.15.165:0/41018487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768806cea0 msgr2=0x7f76881ac250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:56.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.575+0000 7f768d79d640 1 --2- 172.21.15.165:0/41018487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768806cea0 0x7f76881ac250 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f767c00c3d0 tx=0x7f767c00c890 comp rx=0 tx=0).stop 2024-04-27T03:29:56.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.575+0000 7f768d79d640 1 -- 172.21.15.165:0/41018487 shutdown_connections 2024-04-27T03:29:56.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.575+0000 7f768d79d640 1 --2- 172.21.15.165:0/41018487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f768806cea0 0x7f76881ac250 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:56.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.575+0000 7f768d79d640 1 -- 172.21.15.165:0/41018487 >> 172.21.15.165:0/41018487 conn(0x7f7688070c60 msgr2=0x7f7688071040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:56.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.575+0000 7f768d79d640 1 -- 172.21.15.165:0/41018487 shutdown_connections 2024-04-27T03:29:56.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.575+0000 7f768d79d640 1 -- 172.21.15.165:0/41018487 wait complete. 2024-04-27T03:29:56.591 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 12198ef3-e304-4085-992a-2f3563d60ebc 2 2024-04-27T03:29:56.688 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:56.683+0000 7f9c36419040 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-27T03:29:56.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.723+0000 7f4a4af38640 1 Processor -- start 2024-04-27T03:29:56.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.723+0000 7f4a4af38640 1 -- start start 2024-04-27T03:29:56.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.723+0000 7f4a4af38640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4a44116c40 0x7f4a44117020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:56.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.723+0000 7f4a4af38640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4a44117680 con 0x7f4a44111b00 2024-04-27T03:29:56.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.723+0000 7f4a4af38640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f4a441126a0 con 0x7f4a44116c40 2024-04-27T03:29:56.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.723+0000 7f4a48cd4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4a44116c40 0x7f4a44117020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:56.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.724+0000 7f4a43fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4a44111b00 0x7f4a44111f90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:40388/0 (socket says 172.21.15.165:40388) 2024-04-27T03:29:56.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.724+0000 7f4a43fff640 1 -- 172.21.15.165:0/925807201 learned_addr learned my addr 172.21.15.165:0/925807201 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:29:56.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.724+0000 7f4a48cd4640 1 -- 172.21.15.165:0/925807201 >> v1:172.21.15.89:6789/0 conn(0x7f4a44111b00 legacy=0x7f4a44111f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:56.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.724+0000 7f4a48cd4640 1 -- 172.21.15.165:0/925807201 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a44112850 con 0x7f4a44116c40 2024-04-27T03:29:56.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.725+0000 7f4a48cd4640 1 --2- 172.21.15.165:0/925807201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a44116c40 0x7f4a44117020 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009720 tx=0x7f4a3802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6957b04a529087b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:56.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.725+0000 7f4a437fe640 1 -- 172.21.15.165:0/925807201 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a380316d0 con 0x7f4a44116c40 2024-04-27T03:29:56.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.725+0000 7f4a437fe640 1 -- 172.21.15.165:0/925807201 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a38037070 con 0x7f4a44116c40 2024-04-27T03:29:56.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.725+0000 7f4a437fe640 1 -- 172.21.15.165:0/925807201 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a380319c0 con 0x7f4a44116c40 2024-04-27T03:29:56.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.725+0000 7f4a4af38640 1 -- 172.21.15.165:0/925807201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a44116c40 msgr2=0x7f4a44117020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:56.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.725+0000 7f4a4af38640 1 --2- 172.21.15.165:0/925807201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a44116c40 0x7f4a44117020 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f4a38009720 tx=0x7f4a3802b840 comp rx=0 tx=0).stop 2024-04-27T03:29:56.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.725+0000 7f4a4af38640 1 -- 172.21.15.165:0/925807201 shutdown_connections 2024-04-27T03:29:56.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.725+0000 7f4a4af38640 1 --2- 172.21.15.165:0/925807201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a44116c40 0x7f4a44117020 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:56.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.725+0000 7f4a4af38640 1 -- 172.21.15.165:0/925807201 >> 172.21.15.165:0/925807201 conn(0x7f4a4406e700 msgr2=0x7f4a4410fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:56.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.726+0000 7f4a4af38640 1 -- 172.21.15.165:0/925807201 shutdown_connections 2024-04-27T03:29:56.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.726+0000 7f4a4af38640 1 -- 172.21.15.165:0/925807201 wait complete. 2024-04-27T03:29:56.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.726+0000 7f4a4af38640 1 Processor -- start 2024-04-27T03:29:56.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.726+0000 7f4a4af38640 1 -- start start 2024-04-27T03:29:56.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.727+0000 7f4a4af38640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a44111b00 0x7f4a441a69b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:56.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.727+0000 7f4a4af38640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4a4411df30 con 0x7f4a44111b00 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.727+0000 7f4a48cd4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a44111b00 0x7f4a441a69b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.727+0000 7f4a48cd4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a44111b00 0x7f4a441a69b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58754/0 (socket says 172.21.15.165:58754) 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.727+0000 7f4a48cd4640 1 -- 172.21.15.165:0/2437975746 learned_addr learned my addr 172.21.15.165:0/2437975746 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.727+0000 7f4a48cd4640 1 -- 172.21.15.165:0/2437975746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a441a6ef0 con 0x7f4a44111b00 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.727+0000 7f4a48cd4640 1 --2- 172.21.15.165:0/2437975746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a44111b00 0x7f4a441a69b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4a38034040 tx=0x7f4a380046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.727+0000 7f4a417fa640 1 -- 172.21.15.165:0/2437975746 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a38032770 con 0x7f4a44111b00 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.727+0000 7f4a417fa640 1 -- 172.21.15.165:0/2437975746 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4a38032910 con 0x7f4a44111b00 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.727+0000 7f4a417fa640 1 -- 172.21.15.165:0/2437975746 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4a38032c30 con 0x7f4a44111b00 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.727+0000 7f4a4af38640 1 -- 172.21.15.165:0/2437975746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a441a7180 con 0x7f4a44111b00 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.727+0000 7f4a4af38640 1 -- 172.21.15.165:0/2437975746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a441ab130 con 0x7f4a44111b00 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.728+0000 7f4a417fa640 1 -- 172.21.15.165:0/2437975746 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f4a3804c060 con 0x7f4a44111b00 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.728+0000 7f4a417fa640 1 -- 172.21.15.165:0/2437975746 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 955+0+0 (secure 0 0 0) 0x7f4a3802f050 con 0x7f4a44111b00 2024-04-27T03:29:56.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.728+0000 7f4a4af38640 1 -- 172.21.15.165:0/2437975746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4a44062da0 con 0x7f4a44111b00 2024-04-27T03:29:56.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.730+0000 7f4a417fa640 1 -- 172.21.15.165:0/2437975746 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f4a38040070 con 0x7f4a44111b00 2024-04-27T03:29:56.737 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:56.731+0000 7f9c36419040 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-27T03:29:56.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.803+0000 7f4a4af38640 1 -- 172.21.15.165:0/2437975746 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd new", "uuid": "12198ef3-e304-4085-992a-2f3563d60ebc", "id": 2} v 0) -- 0x7f4a44116c40 con 0x7f4a44111b00 2024-04-27T03:29:56.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.804+0000 7f4a417fa640 1 -- 172.21.15.165:0/2437975746 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "12198ef3-e304-4085-992a-2f3563d60ebc", "id": 2}]=0 v5) ==== 112+0+2 (secure 0 0 0) 0x7f4a3804e7e0 con 0x7f4a44111b00 2024-04-27T03:29:56.810 INFO:teuthology.orchestra.run.smithi165.stdout:2 2024-04-27T03:29:56.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.805+0000 7f4a4af38640 1 -- 172.21.15.165:0/2437975746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a44111b00 msgr2=0x7f4a441a69b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:56.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.805+0000 7f4a4af38640 1 --2- 172.21.15.165:0/2437975746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a44111b00 0x7f4a441a69b0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f4a38034040 tx=0x7f4a380046c0 comp rx=0 tx=0).stop 2024-04-27T03:29:56.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.805+0000 7f4a4af38640 1 -- 172.21.15.165:0/2437975746 shutdown_connections 2024-04-27T03:29:56.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.805+0000 7f4a4af38640 1 --2- 172.21.15.165:0/2437975746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4a44111b00 0x7f4a441a69b0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:56.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.805+0000 7f4a4af38640 1 -- 172.21.15.165:0/2437975746 >> 172.21.15.165:0/2437975746 conn(0x7f4a4406e700 msgr2=0x7f4a4411cb80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:56.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.805+0000 7f4a4af38640 1 -- 172.21.15.165:0/2437975746 shutdown_connections 2024-04-27T03:29:56.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.805+0000 7f4a4af38640 1 -- 172.21.15.165:0/2437975746 wait complete. 2024-04-27T03:29:56.822 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph --cluster ceph osd new 5ecd16bd-fe0b-4b0d-8028-00b613ff7ec7 3 2024-04-27T03:29:56.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.914+0000 7fcb4841a640 1 Processor -- start 2024-04-27T03:29:56.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.914+0000 7fcb4841a640 1 -- start start 2024-04-27T03:29:56.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.914+0000 7fcb4841a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb4011ab90 0x7fcb4011cf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:56.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.914+0000 7fcb4841a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fcb4011d4c0 con 0x7fcb4006d060 2024-04-27T03:29:56.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.914+0000 7fcb4841a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fcb4011e6a0 con 0x7fcb4011ab90 2024-04-27T03:29:56.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.914+0000 7fcb459b5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb4011ab90 0x7fcb4011cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:56.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.914+0000 7fcb459b5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fcb4011ab90 0x7fcb4011cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58770/0 (socket says 172.21.15.165:58770) 2024-04-27T03:29:56.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.914+0000 7fcb461b6640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fcb4006d060 0x7fcb4006d440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:40404/0 (socket says 172.21.15.165:40404) 2024-04-27T03:29:56.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.914+0000 7fcb459b5640 1 -- 172.21.15.165:0/3343692268 learned_addr learned my addr 172.21.15.165:0/3343692268 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:29:56.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.915+0000 7fcb459b5640 1 -- 172.21.15.165:0/3343692268 >> v1:172.21.15.89:6789/0 conn(0x7fcb4006d060 legacy=0x7fcb4006d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:56.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.915+0000 7fcb459b5640 1 -- 172.21.15.165:0/3343692268 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb4011e820 con 0x7fcb4011ab90 2024-04-27T03:29:56.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.915+0000 7fcb459b5640 1 --2- 172.21.15.165:0/3343692268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb4011ab90 0x7fcb4011cf80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcb300097f0 tx=0x7fcb3002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c83fdb7a901eeb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:56.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.918+0000 7fcb451b4640 1 -- 172.21.15.165:0/3343692268 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb30035020 con 0x7fcb4011ab90 2024-04-27T03:29:56.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.918+0000 7fcb451b4640 1 -- 172.21.15.165:0/3343692268 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb300317a0 con 0x7fcb4011ab90 2024-04-27T03:29:56.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.918+0000 7fcb451b4640 1 -- 172.21.15.165:0/3343692268 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb30031aa0 con 0x7fcb4011ab90 2024-04-27T03:29:56.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.918+0000 7fcb4841a640 1 -- 172.21.15.165:0/3343692268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb4011ab90 msgr2=0x7fcb4011cf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:56.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.918+0000 7fcb4841a640 1 --2- 172.21.15.165:0/3343692268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb4011ab90 0x7fcb4011cf80 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcb300097f0 tx=0x7fcb3002b910 comp rx=0 tx=0).stop 2024-04-27T03:29:56.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.919+0000 7fcb4841a640 1 -- 172.21.15.165:0/3343692268 shutdown_connections 2024-04-27T03:29:56.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.919+0000 7fcb4841a640 1 --2- 172.21.15.165:0/3343692268 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb4011ab90 0x7fcb4011cf80 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:56.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.919+0000 7fcb4841a640 1 -- 172.21.15.165:0/3343692268 >> 172.21.15.165:0/3343692268 conn(0x7fcb40070e80 msgr2=0x7fcb400732a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:56.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.919+0000 7fcb4841a640 1 -- 172.21.15.165:0/3343692268 shutdown_connections 2024-04-27T03:29:56.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.919+0000 7fcb4841a640 1 -- 172.21.15.165:0/3343692268 wait complete. 2024-04-27T03:29:56.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.919+0000 7fcb4841a640 1 Processor -- start 2024-04-27T03:29:56.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.919+0000 7fcb4841a640 1 -- start start 2024-04-27T03:29:56.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.919+0000 7fcb4841a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb4006d060 0x7fcb401a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:56.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.919+0000 7fcb4841a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fcb4011db80 con 0x7fcb4006d060 2024-04-27T03:29:56.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.920+0000 7fcb461b6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb4006d060 0x7fcb401a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:56.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.920+0000 7fcb461b6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb4006d060 0x7fcb401a69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58782/0 (socket says 172.21.15.165:58782) 2024-04-27T03:29:56.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.920+0000 7fcb461b6640 1 -- 172.21.15.165:0/4186625588 learned_addr learned my addr 172.21.15.165:0/4186625588 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:29:56.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.920+0000 7fcb461b6640 1 -- 172.21.15.165:0/4186625588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb401a6ee0 con 0x7fcb4006d060 2024-04-27T03:29:56.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.920+0000 7fcb461b6640 1 --2- 172.21.15.165:0/4186625588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb4006d060 0x7fcb401a69a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00c3d0 tx=0x7fcb3c00c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:56.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.920+0000 7fcb36ffd640 1 -- 172.21.15.165:0/4186625588 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c007410 con 0x7fcb4006d060 2024-04-27T03:29:56.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.920+0000 7fcb36ffd640 1 -- 172.21.15.165:0/4186625588 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcb3c0075b0 con 0x7fcb4006d060 2024-04-27T03:29:56.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.920+0000 7fcb36ffd640 1 -- 172.21.15.165:0/4186625588 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcb3c004910 con 0x7fcb4006d060 2024-04-27T03:29:56.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.920+0000 7fcb4841a640 1 -- 172.21.15.165:0/4186625588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb401a7170 con 0x7fcb4006d060 2024-04-27T03:29:56.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.920+0000 7fcb4841a640 1 -- 172.21.15.165:0/4186625588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb401ab0a0 con 0x7fcb4006d060 2024-04-27T03:29:56.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.921+0000 7fcb36ffd640 1 -- 172.21.15.165:0/4186625588 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fcb3c01a020 con 0x7fcb4006d060 2024-04-27T03:29:56.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.921+0000 7fcb36ffd640 1 -- 172.21.15.165:0/4186625588 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(5..5 src has 1..5) ==== 1070+0+0 (secure 0 0 0) 0x7fcb3c012880 con 0x7fcb4006d060 2024-04-27T03:29:56.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.921+0000 7fcb4841a640 1 -- 172.21.15.165:0/4186625588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb40062da0 con 0x7fcb4006d060 2024-04-27T03:29:56.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.923+0000 7fcb36ffd640 1 -- 172.21.15.165:0/4186625588 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fcb3c021070 con 0x7fcb4006d060 2024-04-27T03:29:57.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.995+0000 7fcb4841a640 1 -- 172.21.15.165:0/4186625588 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd new", "uuid": "5ecd16bd-fe0b-4b0d-8028-00b613ff7ec7", "id": 3} v 0) -- 0x7fcb4011ab90 con 0x7fcb4006d060 2024-04-27T03:29:57.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:56.999+0000 7fcb36ffd640 1 -- 172.21.15.165:0/4186625588 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "5ecd16bd-fe0b-4b0d-8028-00b613ff7ec7", "id": 3}]=0 v6) ==== 112+0+2 (secure 0 0 0) 0x7fcb3c012cd0 con 0x7fcb4006d060 2024-04-27T03:29:57.005 INFO:teuthology.orchestra.run.smithi165.stdout:3 2024-04-27T03:29:57.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:57.000+0000 7fcb4841a640 1 -- 172.21.15.165:0/4186625588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb4006d060 msgr2=0x7fcb401a69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:57.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:57.000+0000 7fcb4841a640 1 --2- 172.21.15.165:0/4186625588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb4006d060 0x7fcb401a69a0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c00c3d0 tx=0x7fcb3c00c890 comp rx=0 tx=0).stop 2024-04-27T03:29:57.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:57.000+0000 7fcb4841a640 1 -- 172.21.15.165:0/4186625588 shutdown_connections 2024-04-27T03:29:57.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:57.000+0000 7fcb4841a640 1 --2- 172.21.15.165:0/4186625588 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fcb4006d060 0x7fcb401a69a0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:57.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:57.000+0000 7fcb4841a640 1 -- 172.21.15.165:0/4186625588 >> 172.21.15.165:0/4186625588 conn(0x7fcb40070e80 msgr2=0x7fcb4011af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:57.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:57.000+0000 7fcb4841a640 1 -- 172.21.15.165:0/4186625588 shutdown_connections 2024-04-27T03:29:57.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:29:57.000+0000 7fcb4841a640 1 -- 172.21.15.165:0/4186625588 wait complete. 2024-04-27T03:29:57.016 INFO:tasks.ceph.osd.0:Restarting daemon 2024-04-27T03:29:57.016 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2024-04-27T03:29:57.018 INFO:tasks.ceph.osd.0:Started 2024-04-27T03:29:57.019 INFO:tasks.ceph.osd.1:Restarting daemon 2024-04-27T03:29:57.019 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2024-04-27T03:29:57.022 INFO:tasks.ceph.osd.1:Started 2024-04-27T03:29:57.022 INFO:tasks.ceph.osd.2:Restarting daemon 2024-04-27T03:29:57.022 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2024-04-27T03:29:57.024 INFO:tasks.ceph.osd.2:Started 2024-04-27T03:29:57.024 INFO:tasks.ceph.osd.3:Restarting daemon 2024-04-27T03:29:57.024 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2024-04-27T03:29:57.026 INFO:tasks.ceph.osd.3:Started 2024-04-27T03:29:57.027 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T03:29:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.123+0000 7f6fd6f84640 1 Processor -- start 2024-04-27T03:29:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.123+0000 7f6fd6f84640 1 -- start start 2024-04-27T03:29:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.123+0000 7f6fd6f84640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6fd006f260 0x7f6fd006f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.123+0000 7f6fd6f84640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6fd006fff0 con 0x7f6fd006fb80 2024-04-27T03:29:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.123+0000 7f6fd6f84640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f6fd0070bc0 con 0x7f6fd006f260 2024-04-27T03:29:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.124+0000 7f6fd4d20640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6fd006f260 0x7f6fd006f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.124+0000 7f6fcffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6fd006fb80 0x7f6fd012e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44010/0 (socket says 172.21.15.89:44010) 2024-04-27T03:29:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.124+0000 7f6fcffff640 1 -- 172.21.15.89:0/588885378 learned_addr learned my addr 172.21.15.89:0/588885378 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:29:57.130 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.124+0000 7f6fd4d20640 1 -- 172.21.15.89:0/588885378 >> v1:172.21.15.89:6789/0 conn(0x7f6fd006fb80 legacy=0x7f6fd012e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.124+0000 7f6fd4d20640 1 -- 172.21.15.89:0/588885378 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd012ec50 con 0x7f6fd006f260 2024-04-27T03:29:57.131 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.125+0000 7f6fd4d20640 1 --2- 172.21.15.89:0/588885378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fd006f260 0x7f6fd006f640 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009720 tx=0x7f6fc402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed33e01f9cb2ad0d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.125+0000 7f6fcf7fe640 1 -- 172.21.15.89:0/588885378 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fc40316d0 con 0x7f6fd006f260 2024-04-27T03:29:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.125+0000 7f6fcf7fe640 1 -- 172.21.15.89:0/588885378 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc4037070 con 0x7f6fd006f260 2024-04-27T03:29:57.133 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.125+0000 7f6fcf7fe640 1 -- 172.21.15.89:0/588885378 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fc40319c0 con 0x7f6fd006f260 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.125+0000 7f6fd6f84640 1 -- 172.21.15.89:0/588885378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fd006f260 msgr2=0x7f6fd006f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.125+0000 7f6fd6f84640 1 --2- 172.21.15.89:0/588885378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fd006f260 0x7f6fd006f640 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009720 tx=0x7f6fc402b840 comp rx=0 tx=0).stop 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.125+0000 7f6fd6f84640 1 -- 172.21.15.89:0/588885378 shutdown_connections 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.125+0000 7f6fd6f84640 1 --2- 172.21.15.89:0/588885378 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fd006f260 0x7f6fd006f640 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.125+0000 7f6fd6f84640 1 -- 172.21.15.89:0/588885378 >> 172.21.15.89:0/588885378 conn(0x7f6fd009e170 msgr2=0x7f6fd006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.125+0000 7f6fd6f84640 1 -- 172.21.15.89:0/588885378 shutdown_connections 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.125+0000 7f6fd6f84640 1 -- 172.21.15.89:0/588885378 wait complete. 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.126+0000 7f6fd6f84640 1 Processor -- start 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.126+0000 7f6fd6f84640 1 -- start start 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fd6f84640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fd006f260 0x7f6fd0117480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fd6f84640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6fd012f400 con 0x7f6fd006f260 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fd4d20640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fd006f260 0x7f6fd0117480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:57.134 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fd4d20640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fd006f260 0x7f6fd0117480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:39980/0 (socket says 172.21.15.89:39980) 2024-04-27T03:29:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fd4d20640 1 -- 172.21.15.89:0/2054636643 learned_addr learned my addr 172.21.15.89:0/2054636643 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:29:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fd4d20640 1 -- 172.21.15.89:0/2054636643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6fd011a8a0 con 0x7f6fd006f260 2024-04-27T03:29:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fd4d20640 1 --2- 172.21.15.89:0/2054636643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fd006f260 0x7f6fd0117480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009840 tx=0x7f6fc40046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fcd7fa640 1 -- 172.21.15.89:0/2054636643 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fc4032640 con 0x7f6fd006f260 2024-04-27T03:29:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fcd7fa640 1 -- 172.21.15.89:0/2054636643 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6fc40327e0 con 0x7f6fd006f260 2024-04-27T03:29:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fd6f84640 1 -- 172.21.15.89:0/2054636643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6fd0119c70 con 0x7f6fd006f260 2024-04-27T03:29:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fcd7fa640 1 -- 172.21.15.89:0/2054636643 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6fc4032b00 con 0x7f6fd006f260 2024-04-27T03:29:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fd6f84640 1 -- 172.21.15.89:0/2054636643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6fd011a3f0 con 0x7f6fd006f260 2024-04-27T03:29:57.135 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fcd7fa640 1 -- 172.21.15.89:0/2054636643 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6fc404c060 con 0x7f6fd006f260 2024-04-27T03:29:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.127+0000 7f6fcd7fa640 1 -- 172.21.15.89:0/2054636643 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7f6fc402f050 con 0x7f6fd006f260 2024-04-27T03:29:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.128+0000 7f6fd6f84640 1 -- 172.21.15.89:0/2054636643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6fd00a18a0 con 0x7f6fd006f260 2024-04-27T03:29:57.136 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.130+0000 7f6fcd7fa640 1 -- 172.21.15.89:0/2054636643 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7f6fd00a18a0 con 0x7f6fd006f260 2024-04-27T03:29:57.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.188+0000 7f6fd6f84640 1 -- 172.21.15.89:0/2054636643 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6fd01347f0 con 0x7f6fd006f260 2024-04-27T03:29:57.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.189+0000 7f6fcd7fa640 1 -- 172.21.15.89:0/2054636643 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) ==== 74+0+4153 (secure 0 0 0) 0x7f6fc4040070 con 0x7f6fd006f260 2024-04-27T03:29:57.195 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:29:57.195 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":6,"fsid":"7a1794da-6463-461c-a75b-269448ecfc86","created":"2024-04-27T03:29:55.910567+0000","modified":"2024-04-27T03:29:56.997311+0000","last_up_change":"0.000000","last_in_change":"2024-04-27T03:29:56.997311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2643cf41-510f-4fc8-9611-7260d319e5f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6d14d21a-3c2d-47bb-a0d9-96f58d57d612","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"12198ef3-e304-4085-992a-2f3563d60ebc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5ecd16bd-fe0b-4b0d-8028-00b613ff7ec7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T03:29:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.189+0000 7f6fd6f84640 1 -- 172.21.15.89:0/2054636643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fd006f260 msgr2=0x7f6fd0117480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.189+0000 7f6fd6f84640 1 --2- 172.21.15.89:0/2054636643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fd006f260 0x7f6fd0117480 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f6fc4009840 tx=0x7f6fc40046c0 comp rx=0 tx=0).stop 2024-04-27T03:29:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.190+0000 7f6fd6f84640 1 -- 172.21.15.89:0/2054636643 shutdown_connections 2024-04-27T03:29:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.190+0000 7f6fd6f84640 1 --2- 172.21.15.89:0/2054636643 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6fd006f260 0x7f6fd0117480 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.190+0000 7f6fd6f84640 1 -- 172.21.15.89:0/2054636643 >> 172.21.15.89:0/2054636643 conn(0x7f6fd009e170 msgr2=0x7f6fd012b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.190+0000 7f6fd6f84640 1 -- 172.21.15.89:0/2054636643 shutdown_connections 2024-04-27T03:29:57.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:29:57.190+0000 7f6fd6f84640 1 -- 172.21.15.89:0/2054636643 wait complete. 2024-04-27T03:29:57.206 INFO:tasks.ceph.ceph_manager.ceph:[] 2024-04-27T03:29:57.206 INFO:tasks.ceph:Waiting for OSDs to come up 2024-04-27T03:29:57.498 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:57.493+0000 7f9c36419040 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-27T03:29:57.585 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:57.579+0000 7f9c36419040 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-27T03:29:57.609 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-27T03:29:57.610 INFO:tasks.ceph.osd.0.smithi089.stderr:2024-04-27T03:29:57.604+0000 7f1881161640 -1 Falling back to public interface 2024-04-27T03:29:57.624 INFO:tasks.ceph.osd.1.smithi165.stderr:2024-04-27T03:29:57.618+0000 7fa0d4387640 -1 Falling back to public interface 2024-04-27T03:29:57.628 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-27T03:29:57.622+0000 7f6927094640 -1 Falling back to public interface 2024-04-27T03:29:57.629 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:57.624+0000 7f9c36419040 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-27T03:29:57.633 INFO:tasks.ceph.osd.2.smithi165.stderr:2024-04-27T03:29:57.627+0000 7f887762e640 -1 Falling back to public interface 2024-04-27T03:29:57.700 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.694+0000 7fd231385640 1 Processor -- start 2024-04-27T03:29:57.700 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.694+0000 7fd231385640 1 -- start start 2024-04-27T03:29:57.701 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.694+0000 7fd231385640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd22c096350 0x7fd22c096730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:57.701 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.694+0000 7fd231385640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd22c071e10 con 0x7fd22c071570 2024-04-27T03:29:57.701 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.694+0000 7fd231385640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fd22c06ecf0 con 0x7fd22c096350 2024-04-27T03:29:57.701 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.694+0000 7fd22affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd22c096350 0x7fd22c096730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:57.701 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.695+0000 7fd223fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd22c071570 0x7fd22c12c050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44014/0 (socket says 172.21.15.89:44014) 2024-04-27T03:29:57.701 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.695+0000 7fd223fff640 1 -- 172.21.15.89:0/4112222746 learned_addr learned my addr 172.21.15.89:0/4112222746 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:29:57.701 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.695+0000 7fd22affd640 1 -- 172.21.15.89:0/4112222746 >> v1:172.21.15.89:6789/0 conn(0x7fd22c071570 legacy=0x7fd22c12c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:57.701 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.695+0000 7fd22affd640 1 -- 172.21.15.89:0/4112222746 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c09c5e0 con 0x7fd22c096350 2024-04-27T03:29:57.701 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.695+0000 7fd22a7fc640 1 -- 172.21.15.89:0/4112222746 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 215721703 0 0) 0x7fd22c071e10 con 0x7fd22c071570 2024-04-27T03:29:57.704 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.695+0000 7fd22affd640 1 --2- 172.21.15.89:0/4112222746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c096350 0x7fd22c096730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009720 tx=0x7fd21c02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ff278decff37d12 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:57.704 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.696+0000 7fd22a7fc640 1 -- 172.21.15.89:0/4112222746 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd21c0316d0 con 0x7fd22c096350 2024-04-27T03:29:57.704 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.696+0000 7fd22a7fc640 1 -- 172.21.15.89:0/4112222746 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd21c037070 con 0x7fd22c096350 2024-04-27T03:29:57.704 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.696+0000 7fd22a7fc640 1 -- 172.21.15.89:0/4112222746 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd21c0319c0 con 0x7fd22c096350 2024-04-27T03:29:57.704 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.696+0000 7fd231385640 1 -- 172.21.15.89:0/4112222746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c096350 msgr2=0x7fd22c096730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.696+0000 7fd231385640 1 --2- 172.21.15.89:0/4112222746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c096350 0x7fd22c096730 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009720 tx=0x7fd21c02b840 comp rx=0 tx=0).stop 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.696+0000 7fd231385640 1 -- 172.21.15.89:0/4112222746 shutdown_connections 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.696+0000 7fd231385640 1 --2- 172.21.15.89:0/4112222746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c096350 0x7fd22c096730 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.696+0000 7fd231385640 1 -- 172.21.15.89:0/4112222746 >> 172.21.15.89:0/4112222746 conn(0x7fd22c09a840 msgr2=0x7fd22c09ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.696+0000 7fd231385640 1 -- 172.21.15.89:0/4112222746 shutdown_connections 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.696+0000 7fd231385640 1 -- 172.21.15.89:0/4112222746 wait complete. 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.697+0000 7fd231385640 1 Processor -- start 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.697+0000 7fd231385640 1 -- start start 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.697+0000 7fd231385640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c071570 0x7fd22c126030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.697+0000 7fd231385640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd22c12c760 con 0x7fd22c071570 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.697+0000 7fd22affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c071570 0x7fd22c126030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.697+0000 7fd22affd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c071570 0x7fd22c126030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:40012/0 (socket says 172.21.15.89:40012) 2024-04-27T03:29:57.705 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.697+0000 7fd22affd640 1 -- 172.21.15.89:0/3998207432 learned_addr learned my addr 172.21.15.89:0/3998207432 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:29:57.706 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.698+0000 7fd22affd640 1 -- 172.21.15.89:0/3998207432 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd22c12b8d0 con 0x7fd22c071570 2024-04-27T03:29:57.706 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.698+0000 7fd22affd640 1 --2- 172.21.15.89:0/3998207432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c071570 0x7fd22c126030 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009720 tx=0x7fd21c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:29:57.706 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.698+0000 7fd2237fe640 1 -- 172.21.15.89:0/3998207432 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd21c032770 con 0x7fd22c071570 2024-04-27T03:29:57.706 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.698+0000 7fd2237fe640 1 -- 172.21.15.89:0/3998207432 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd21c032910 con 0x7fd22c071570 2024-04-27T03:29:57.706 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.698+0000 7fd231385640 1 -- 172.21.15.89:0/3998207432 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd22c128050 con 0x7fd22c071570 2024-04-27T03:29:57.706 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.698+0000 7fd2237fe640 1 -- 172.21.15.89:0/3998207432 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd21c032bf0 con 0x7fd22c071570 2024-04-27T03:29:57.706 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.698+0000 7fd231385640 1 -- 172.21.15.89:0/3998207432 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd22c128ce0 con 0x7fd22c071570 2024-04-27T03:29:57.706 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.698+0000 7fd2237fe640 1 -- 172.21.15.89:0/3998207432 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fd21c04c060 con 0x7fd22c071570 2024-04-27T03:29:57.706 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.698+0000 7fd2237fe640 1 -- 172.21.15.89:0/3998207432 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(6..6 src has 1..6) ==== 1185+0+0 (secure 0 0 0) 0x7fd21c02f050 con 0x7fd22c071570 2024-04-27T03:29:57.707 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.699+0000 7fd231385640 1 -- 172.21.15.89:0/3998207432 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd22c0737f0 con 0x7fd22c071570 2024-04-27T03:29:57.707 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.701+0000 7fd2237fe640 1 -- 172.21.15.89:0/3998207432 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73141 (secure 0 0 0) 0x7fd22c12b8d0 con 0x7fd22c071570 2024-04-27T03:29:57.765 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.759+0000 7fd231385640 1 -- 172.21.15.89:0/3998207432 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd22c0990c0 con 0x7fd22c071570 2024-04-27T03:29:57.765 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.759+0000 7fd2237fe640 1 -- 172.21.15.89:0/3998207432 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v6) ==== 74+0+4153 (secure 0 0 0) 0x7fd22c0990c0 con 0x7fd22c071570 2024-04-27T03:29:57.766 INFO:teuthology.misc.health.smithi089.stdout: 2024-04-27T03:29:57.766 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":6,"fsid":"7a1794da-6463-461c-a75b-269448ecfc86","created":"2024-04-27T03:29:55.910567+0000","modified":"2024-04-27T03:29:56.997311+0000","last_up_change":"0.000000","last_in_change":"2024-04-27T03:29:56.997311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2643cf41-510f-4fc8-9611-7260d319e5f8","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6d14d21a-3c2d-47bb-a0d9-96f58d57d612","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"12198ef3-e304-4085-992a-2f3563d60ebc","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"5ecd16bd-fe0b-4b0d-8028-00b613ff7ec7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T03:29:57.767 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.760+0000 7fd231385640 1 -- 172.21.15.89:0/3998207432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c071570 msgr2=0x7fd22c126030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:29:57.767 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.760+0000 7fd231385640 1 --2- 172.21.15.89:0/3998207432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c071570 0x7fd22c126030 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fd21c009720 tx=0x7fd21c0046c0 comp rx=0 tx=0).stop 2024-04-27T03:29:57.767 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.760+0000 7fd231385640 1 -- 172.21.15.89:0/3998207432 shutdown_connections 2024-04-27T03:29:57.767 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.760+0000 7fd231385640 1 --2- 172.21.15.89:0/3998207432 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd22c071570 0x7fd22c126030 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:29:57.767 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.760+0000 7fd231385640 1 -- 172.21.15.89:0/3998207432 >> 172.21.15.89:0/3998207432 conn(0x7fd22c09a840 msgr2=0x7fd22c1b41f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:29:57.767 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.760+0000 7fd231385640 1 -- 172.21.15.89:0/3998207432 shutdown_connections 2024-04-27T03:29:57.767 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:29:57.760+0000 7fd231385640 1 -- 172.21.15.89:0/3998207432 wait complete. 2024-04-27T03:29:57.775 DEBUG:teuthology.misc:0 of 4 OSDs are up 2024-04-27T03:29:57.797 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:57.792+0000 7f9c36419040 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-27T03:29:57.849 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:57.843+0000 7f9c36419040 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-27T03:29:57.902 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:57.896+0000 7f9c36419040 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-27T03:29:58.015 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:58.010+0000 7f9c36419040 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-27T03:29:58.395 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:58.390+0000 7f9c36419040 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-27T03:29:58.655 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:58.649+0000 7f9c36419040 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-27T03:29:58.708 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:58.702+0000 7f9c36419040 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-27T03:29:58.827 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:58.821+0000 7f9c36419040 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-27T03:29:59.105 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:59.100+0000 7f9c36419040 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-27T03:29:59.153 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:59.147+0000 7f9c36419040 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-27T03:29:59.310 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:59.303+0000 7f9c36419040 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-27T03:29:59.444 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:59.439+0000 7f9c36419040 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-27T03:29:59.495 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:59.489+0000 7f9c36419040 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-27T03:29:59.565 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:59.559+0000 7f9c36419040 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-27T03:29:59.612 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:29:59.606+0000 7f9c36419040 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-27T03:30:00.088 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:30:00.082+0000 7f9c36419040 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-27T03:30:00.210 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:30:00.204+0000 7f9c36419040 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-27T03:30:01.041 INFO:tasks.ceph.osd.0.smithi089.stderr:2024-04-27T03:30:01.035+0000 7f1881161640 -1 osd.0 0 log_to_monitors true 2024-04-27T03:30:01.159 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-27T03:30:01.154+0000 7f6927094640 -1 osd.3 0 log_to_monitors true 2024-04-27T03:30:01.208 INFO:tasks.ceph.osd.1.smithi165.stderr:2024-04-27T03:30:01.203+0000 7fa0d4387640 -1 osd.1 0 log_to_monitors true 2024-04-27T03:30:01.212 INFO:tasks.ceph.osd.2.smithi165.stderr:2024-04-27T03:30:01.206+0000 7f887762e640 -1 osd.2 0 log_to_monitors true 2024-04-27T03:30:01.217 INFO:tasks.ceph.mgr.x.smithi165.stderr:2024-04-27T03:30:01.212+0000 7f9bd6dd9640 -1 mgr.server handle_report got status from non-daemon mon.a 2024-04-27T03:30:03.094 INFO:tasks.ceph.osd.0.smithi089.stderr:2024-04-27T03:30:03.089+0000 7f187e10c640 -1 osd.0 0 waiting for initial osdmap 2024-04-27T03:30:03.104 INFO:tasks.ceph.osd.0.smithi089.stderr:2024-04-27T03:30:03.099+0000 7f1874f06640 -1 osd.0 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T03:30:03.980 INFO:tasks.ceph.osd.1.smithi165.stderr:2024-04-27T03:30:03.974+0000 7fa0cfafa640 -1 osd.1 0 waiting for initial osdmap 2024-04-27T03:30:03.989 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-27T03:30:03.984+0000 7f6922807640 -1 osd.3 0 waiting for initial osdmap 2024-04-27T03:30:03.994 INFO:tasks.ceph.osd.1.smithi165.stderr:2024-04-27T03:30:03.989+0000 7fa0c812c640 -1 osd.1 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T03:30:03.998 INFO:tasks.ceph.osd.2.smithi165.stderr:2024-04-27T03:30:03.992+0000 7f8873bc7640 -1 osd.2 0 waiting for initial osdmap 2024-04-27T03:30:04.005 INFO:tasks.ceph.osd.3.smithi165.stderr:2024-04-27T03:30:04.000+0000 7f691ae39640 -1 osd.3 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T03:30:04.011 INFO:tasks.ceph.osd.2.smithi165.stderr:2024-04-27T03:30:04.006+0000 7f886a9c1640 -1 osd.2 8 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-27T03:30:04.178 DEBUG:teuthology.orchestra.run.smithi089:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2024-04-27T03:30:04.279 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.273+0000 7fe5144c8640 1 Processor -- start 2024-04-27T03:30:04.279 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.273+0000 7fe5144c8640 1 -- start start 2024-04-27T03:30:04.280 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.274+0000 7fe5144c8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe50c12d9e0 0x7fe50c12fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:04.280 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.274+0000 7fe5144c8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe50c127110 con 0x7fe50c1275d0 2024-04-27T03:30:04.280 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.274+0000 7fe5144c8640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fe50c1280c0 con 0x7fe50c12d9e0 2024-04-27T03:30:04.280 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.274+0000 7fe511a63640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe50c12d9e0 0x7fe50c12fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:04.280 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.274+0000 7fe512264640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe50c1275d0 0x7fe50c1279b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41642/0 (socket says 172.21.15.89:41642) 2024-04-27T03:30:04.280 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.274+0000 7fe512264640 1 -- 172.21.15.89:0/2912855287 learned_addr learned my addr 172.21.15.89:0/2912855287 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:04.281 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.274+0000 7fe511a63640 1 -- 172.21.15.89:0/2912855287 >> v1:172.21.15.89:6789/0 conn(0x7fe50c1275d0 legacy=0x7fe50c1279b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:04.281 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.274+0000 7fe511a63640 1 -- 172.21.15.89:0/2912855287 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c09c5e0 con 0x7fe50c12d9e0 2024-04-27T03:30:04.282 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.275+0000 7fe511a63640 1 --2- 172.21.15.89:0/2912855287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c12d9e0 0x7fe50c12fdd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009720 tx=0x7fe4fc02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85e2bc923a0b8d14 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:04.282 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.275+0000 7fe511262640 1 -- 172.21.15.89:0/2912855287 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4fc035020 con 0x7fe50c12d9e0 2024-04-27T03:30:04.282 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.275+0000 7fe511262640 1 -- 172.21.15.89:0/2912855287 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4fc031760 con 0x7fe50c12d9e0 2024-04-27T03:30:04.282 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.275+0000 7fe511262640 1 -- 172.21.15.89:0/2912855287 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe4fc031a60 con 0x7fe50c12d9e0 2024-04-27T03:30:04.282 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.276+0000 7fe5144c8640 1 -- 172.21.15.89:0/2912855287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c12d9e0 msgr2=0x7fe50c12fdd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:04.282 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.276+0000 7fe5144c8640 1 --2- 172.21.15.89:0/2912855287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c12d9e0 0x7fe50c12fdd0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc009720 tx=0x7fe4fc02b8d0 comp rx=0 tx=0).stop 2024-04-27T03:30:04.282 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.276+0000 7fe5144c8640 1 -- 172.21.15.89:0/2912855287 shutdown_connections 2024-04-27T03:30:04.282 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.276+0000 7fe5144c8640 1 --2- 172.21.15.89:0/2912855287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c12d9e0 0x7fe50c12fdd0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:04.282 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.276+0000 7fe5144c8640 1 -- 172.21.15.89:0/2912855287 >> 172.21.15.89:0/2912855287 conn(0x7fe50c09a840 msgr2=0x7fe50c09ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:04.282 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.276+0000 7fe5144c8640 1 -- 172.21.15.89:0/2912855287 shutdown_connections 2024-04-27T03:30:04.282 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.276+0000 7fe5144c8640 1 -- 172.21.15.89:0/2912855287 wait complete. 2024-04-27T03:30:04.284 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.277+0000 7fe5144c8640 1 Processor -- start 2024-04-27T03:30:04.284 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.277+0000 7fe5144c8640 1 -- start start 2024-04-27T03:30:04.284 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.277+0000 7fe5144c8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c1275d0 0x7fe50c122bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.277+0000 7fe5144c8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe50c070710 con 0x7fe50c1275d0 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.277+0000 7fe512264640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c1275d0 0x7fe50c122bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.277+0000 7fe512264640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c1275d0 0x7fe50c122bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55204/0 (socket says 172.21.15.89:55204) 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.277+0000 7fe512264640 1 -- 172.21.15.89:0/707558147 learned_addr learned my addr 172.21.15.89:0/707558147 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.278+0000 7fe512264640 1 -- 172.21.15.89:0/707558147 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe50c096b10 con 0x7fe50c1275d0 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.278+0000 7fe512264640 1 --2- 172.21.15.89:0/707558147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c1275d0 0x7fe50c122bb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c480 tx=0x7fe50800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.278+0000 7fe502ffd640 1 -- 172.21.15.89:0/707558147 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe508017070 con 0x7fe50c1275d0 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.278+0000 7fe502ffd640 1 -- 172.21.15.89:0/707558147 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe508005c10 con 0x7fe50c1275d0 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.278+0000 7fe5144c8640 1 -- 172.21.15.89:0/707558147 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe50c096d40 con 0x7fe50c1275d0 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.278+0000 7fe502ffd640 1 -- 172.21.15.89:0/707558147 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe5080070d0 con 0x7fe50c1275d0 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.278+0000 7fe5144c8640 1 -- 172.21.15.89:0/707558147 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe50c08ac40 con 0x7fe50c1275d0 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.279+0000 7fe502ffd640 1 -- 172.21.15.89:0/707558147 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7fe50801a070 con 0x7fe50c1275d0 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.279+0000 7fe502ffd640 1 --2- 172.21.15.89:0/707558147 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe4f0037fd0 0x7fe4f003a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:04.285 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.279+0000 7fe502ffd640 1 -- 172.21.15.89:0/707558147 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 2743+0+0 (secure 0 0 0) 0x7fe508049da0 con 0x7fe50c1275d0 2024-04-27T03:30:04.286 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.279+0000 7fe5144c8640 1 -- 172.21.15.89:0/707558147 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe50c0737f0 con 0x7fe50c1275d0 2024-04-27T03:30:04.286 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.279+0000 7fe511a63640 1 --2- 172.21.15.89:0/707558147 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe4f0037fd0 0x7fe4f003a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:04.286 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.279+0000 7fe511a63640 1 --2- 172.21.15.89:0/707558147 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe4f0037fd0 0x7fe4f003a490 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc036000 tx=0x7fe4fc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:04.289 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.284+0000 7fe502ffd640 1 -- 172.21.15.89:0/707558147 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe50801a520 con 0x7fe50c1275d0 2024-04-27T03:30:04.462 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.456+0000 7fe5144c8640 1 -- 172.21.15.89:0/707558147 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe50c1246c0 con 0x7fe50c1275d0 2024-04-27T03:30:04.462 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.457+0000 7fe502ffd640 1 -- 172.21.15.89:0/707558147 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v9) ==== 74+0+6068 (secure 0 0 0) 0x7fe50801a780 con 0x7fe50c1275d0 2024-04-27T03:30:04.462 INFO:teuthology.misc.health.smithi089.stdout: 2024-04-27T03:30:04.463 INFO:teuthology.misc.health.smithi089.stdout:{"epoch":9,"fsid":"7a1794da-6463-461c-a75b-269448ecfc86","created":"2024-04-27T03:29:55.910567+0000","modified":"2024-04-27T03:30:04.092854+0000","last_up_change":"2024-04-27T03:30:04.092854+0000","last_in_change":"2024-04-27T03:29:56.997311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"2643cf41-510f-4fc8-9611-7260d319e5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6801","nonce":2353668432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6803","nonce":2353668432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6807","nonce":2353668432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6805","nonce":2353668432}]},"public_addr":"172.21.15.89:6801/2353668432","cluster_addr":"172.21.15.89:6803/2353668432","heartbeat_back_addr":"172.21.15.89:6807/2353668432","heartbeat_front_addr":"172.21.15.89:6805/2353668432","state":["exists","up"]},{"osd":1,"uuid":"6d14d21a-3c2d-47bb-a0d9-96f58d57d612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6801","nonce":79879041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6803","nonce":79879041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6807","nonce":79879041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6805","nonce":79879041}]},"public_addr":"172.21.15.165:6801/79879041","cluster_addr":"172.21.15.165:6803/79879041","heartbeat_back_addr":"172.21.15.165:6807/79879041","heartbeat_front_addr":"172.21.15.165:6805/79879041","state":["exists","up"]},{"osd":2,"uuid":"12198ef3-e304-4085-992a-2f3563d60ebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6817","nonce":975457786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6819","nonce":975457786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6823","nonce":975457786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6821","nonce":975457786}]},"public_addr":"172.21.15.165:6817/975457786","cluster_addr":"172.21.15.165:6819/975457786","heartbeat_back_addr":"172.21.15.165:6823/975457786","heartbeat_front_addr":"172.21.15.165:6821/975457786","state":["exists","up"]},{"osd":3,"uuid":"5ecd16bd-fe0b-4b0d-8028-00b613ff7ec7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6809","nonce":2380918637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6811","nonce":2380918637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6815","nonce":2380918637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6813","nonce":2380918637}]},"public_addr":"172.21.15.165:6809/2380918637","cluster_addr":"172.21.15.165:6811/2380918637","heartbeat_back_addr":"172.21.15.165:6815/2380918637","heartbeat_front_addr":"172.21.15.165:6813/2380918637","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T03:30:04.464 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.459+0000 7fe5144c8640 1 -- 172.21.15.89:0/707558147 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe4f0037fd0 msgr2=0x7fe4f003a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:04.465 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.459+0000 7fe5144c8640 1 --2- 172.21.15.89:0/707558147 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe4f0037fd0 0x7fe4f003a490 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fe4fc036000 tx=0x7fe4fc008030 comp rx=0 tx=0).stop 2024-04-27T03:30:04.465 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.459+0000 7fe5144c8640 1 -- 172.21.15.89:0/707558147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c1275d0 msgr2=0x7fe50c122bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:04.465 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.459+0000 7fe5144c8640 1 --2- 172.21.15.89:0/707558147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c1275d0 0x7fe50c122bb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe50800c480 tx=0x7fe50800c940 comp rx=0 tx=0).stop 2024-04-27T03:30:04.465 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.459+0000 7fe5144c8640 1 -- 172.21.15.89:0/707558147 shutdown_connections 2024-04-27T03:30:04.465 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.459+0000 7fe5144c8640 1 --2- 172.21.15.89:0/707558147 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe4f0037fd0 0x7fe4f003a490 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:04.465 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.459+0000 7fe5144c8640 1 --2- 172.21.15.89:0/707558147 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe50c1275d0 0x7fe50c122bb0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:04.465 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.459+0000 7fe5144c8640 1 -- 172.21.15.89:0/707558147 >> 172.21.15.89:0/707558147 conn(0x7fe50c09a840 msgr2=0x7fe50c1256d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:04.465 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.459+0000 7fe5144c8640 1 -- 172.21.15.89:0/707558147 shutdown_connections 2024-04-27T03:30:04.465 INFO:teuthology.misc.health.smithi089.stderr:2024-04-27T03:30:04.459+0000 7fe5144c8640 1 -- 172.21.15.89:0/707558147 wait complete. 2024-04-27T03:30:04.474 DEBUG:teuthology.misc:4 of 4 OSDs are up 2024-04-27T03:30:04.474 INFO:tasks.ceph:Creating RBD pool 2024-04-27T03:30:04.474 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd pool create rbd 8 2024-04-27T03:30:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.566+0000 7f37ec152640 1 Processor -- start 2024-04-27T03:30:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.566+0000 7f37ec152640 1 -- start start 2024-04-27T03:30:04.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.566+0000 7f37ec152640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37e41287a0 0x7f37e41303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.566+0000 7f37ec152640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f37e4098800 con 0x7f37e4127cb0 2024-04-27T03:30:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.566+0000 7f37ec152640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f37e4127b00 con 0x7f37e41287a0 2024-04-27T03:30:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.567+0000 7f37e96ed640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37e41287a0 0x7f37e41303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.567+0000 7f37e96ed640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f37e41287a0 0x7f37e41303c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55214/0 (socket says 172.21.15.89:55214) 2024-04-27T03:30:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.567+0000 7f37e96ed640 1 -- 172.21.15.89:0/2672083022 learned_addr learned my addr 172.21.15.89:0/2672083022 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.567+0000 7f37e9eee640 1 --1- 172.21.15.89:0/2672083022 >> v1:172.21.15.89:6789/0 conn(0x7f37e4127cb0 0x7f37e4128090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41646/0 (socket says 172.21.15.89:41646) 2024-04-27T03:30:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.567+0000 7f37e96ed640 1 -- 172.21.15.89:0/2672083022 >> v1:172.21.15.89:6789/0 conn(0x7f37e4127cb0 legacy=0x7f37e4128090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.567+0000 7f37e96ed640 1 -- 172.21.15.89:0/2672083022 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e406f940 con 0x7f37e41287a0 2024-04-27T03:30:04.573 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.567+0000 7f37e8eec640 1 -- 172.21.15.89:0/2672083022 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3951392504 0 0) 0x7f37e4098800 con 0x7f37e4127cb0 2024-04-27T03:30:04.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.568+0000 7f37e96ed640 1 --2- 172.21.15.89:0/2672083022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e41287a0 0x7f37e41303c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f37d40097f0 tx=0x7f37d402b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3b29a19b67e4cf9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:04.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.568+0000 7f37e8eec640 1 -- 172.21.15.89:0/2672083022 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37d4035020 con 0x7f37e41287a0 2024-04-27T03:30:04.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.568+0000 7f37e8eec640 1 -- 172.21.15.89:0/2672083022 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37d40317a0 con 0x7f37e41287a0 2024-04-27T03:30:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.568+0000 7f37e8eec640 1 -- 172.21.15.89:0/2672083022 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37d4031aa0 con 0x7f37e41287a0 2024-04-27T03:30:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.568+0000 7f37ec152640 1 -- 172.21.15.89:0/2672083022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e41287a0 msgr2=0x7f37e41303c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.568+0000 7f37ec152640 1 --2- 172.21.15.89:0/2672083022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e41287a0 0x7f37e41303c0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f37d40097f0 tx=0x7f37d402b910 comp rx=0 tx=0).stop 2024-04-27T03:30:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.568+0000 7f37ec152640 1 -- 172.21.15.89:0/2672083022 shutdown_connections 2024-04-27T03:30:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.568+0000 7f37ec152640 1 --2- 172.21.15.89:0/2672083022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e41287a0 0x7f37e41303c0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.568+0000 7f37ec152640 1 -- 172.21.15.89:0/2672083022 >> 172.21.15.89:0/2672083022 conn(0x7f37e4099680 msgr2=0x7f37e4099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.568+0000 7f37ec152640 1 -- 172.21.15.89:0/2672083022 shutdown_connections 2024-04-27T03:30:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.568+0000 7f37ec152640 1 -- 172.21.15.89:0/2672083022 wait complete. 2024-04-27T03:30:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.569+0000 7f37ec152640 1 Processor -- start 2024-04-27T03:30:04.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.569+0000 7f37ec152640 1 -- start start 2024-04-27T03:30:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.569+0000 7f37ec152640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e4127cb0 0x7f37e4095890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.569+0000 7f37ec152640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f37e4130900 con 0x7f37e4127cb0 2024-04-27T03:30:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.570+0000 7f37e9eee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e4127cb0 0x7f37e4095890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.570+0000 7f37e9eee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e4127cb0 0x7f37e4095890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55216/0 (socket says 172.21.15.89:55216) 2024-04-27T03:30:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.570+0000 7f37e9eee640 1 -- 172.21.15.89:0/2703633422 learned_addr learned my addr 172.21.15.89:0/2703633422 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.570+0000 7f37e9eee640 1 -- 172.21.15.89:0/2703633422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37e406e0f0 con 0x7f37e4127cb0 2024-04-27T03:30:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.570+0000 7f37e9eee640 1 --2- 172.21.15.89:0/2703633422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e4127cb0 0x7f37e4095890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f37e000c840 tx=0x7f37e000cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:04.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.570+0000 7f37da7fc640 1 -- 172.21.15.89:0/2703633422 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37e0018070 con 0x7f37e4127cb0 2024-04-27T03:30:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.570+0000 7f37da7fc640 1 -- 172.21.15.89:0/2703633422 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f37e000a660 con 0x7f37e4127cb0 2024-04-27T03:30:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.570+0000 7f37ec152640 1 -- 172.21.15.89:0/2703633422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37e406d690 con 0x7f37e4127cb0 2024-04-27T03:30:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.570+0000 7f37da7fc640 1 -- 172.21.15.89:0/2703633422 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f37e000a980 con 0x7f37e4127cb0 2024-04-27T03:30:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.570+0000 7f37ec152640 1 -- 172.21.15.89:0/2703633422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37e4125880 con 0x7f37e4127cb0 2024-04-27T03:30:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.571+0000 7f37da7fc640 1 -- 172.21.15.89:0/2703633422 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7f37e001b070 con 0x7f37e4127cb0 2024-04-27T03:30:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.571+0000 7f37da7fc640 1 --2- 172.21.15.89:0/2703633422 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37c8037fd0 0x7f37c803a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.571+0000 7f37ec152640 1 -- 172.21.15.89:0/2703633422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f37e4062da0 con 0x7f37e4127cb0 2024-04-27T03:30:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.571+0000 7f37da7fc640 1 -- 172.21.15.89:0/2703633422 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 2743+0+0 (secure 0 0 0) 0x7f37e004a240 con 0x7f37e4127cb0 2024-04-27T03:30:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.571+0000 7f37e96ed640 1 --2- 172.21.15.89:0/2703633422 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37c8037fd0 0x7f37c803a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:04.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.572+0000 7f37e96ed640 1 --2- 172.21.15.89:0/2703633422 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37c8037fd0 0x7f37c803a490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f37d40097f0 tx=0x7f37d4008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:04.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.576+0000 7f37da7fc640 1 -- 172.21.15.89:0/2703633422 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f37e001c3b0 con 0x7f37e4127cb0 2024-04-27T03:30:04.753 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:04.748+0000 7f37ec152640 1 -- 172.21.15.89:0/2703633422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) -- 0x7f37e4071f80 con 0x7f37e4127cb0 2024-04-27T03:30:05.101 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.096+0000 7f37da7fc640 1 -- 172.21.15.89:0/2703633422 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v10) ==== 109+0+0 (secure 0 0 0) 0x7f37e000ad00 con 0x7f37e4127cb0 2024-04-27T03:30:05.102 INFO:teuthology.orchestra.run.smithi089.stderr:pool 'rbd' created 2024-04-27T03:30:05.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.098+0000 7f37ec152640 1 -- 172.21.15.89:0/2703633422 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37c8037fd0 msgr2=0x7f37c803a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:05.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.098+0000 7f37ec152640 1 --2- 172.21.15.89:0/2703633422 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37c8037fd0 0x7f37c803a490 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f37d40097f0 tx=0x7f37d4008030 comp rx=0 tx=0).stop 2024-04-27T03:30:05.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.098+0000 7f37ec152640 1 -- 172.21.15.89:0/2703633422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e4127cb0 msgr2=0x7f37e4095890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:05.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.098+0000 7f37ec152640 1 --2- 172.21.15.89:0/2703633422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e4127cb0 0x7f37e4095890 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f37e000c840 tx=0x7f37e000cd00 comp rx=0 tx=0).stop 2024-04-27T03:30:05.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.098+0000 7f37ec152640 1 -- 172.21.15.89:0/2703633422 shutdown_connections 2024-04-27T03:30:05.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.098+0000 7f37ec152640 1 --2- 172.21.15.89:0/2703633422 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37c8037fd0 0x7f37c803a490 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:05.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.098+0000 7f37ec152640 1 --2- 172.21.15.89:0/2703633422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f37e4127cb0 0x7f37e4095890 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:05.104 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.098+0000 7f37ec152640 1 -- 172.21.15.89:0/2703633422 >> 172.21.15.89:0/2703633422 conn(0x7f37e4099680 msgr2=0x7f37e4128e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:05.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.098+0000 7f37ec152640 1 -- 172.21.15.89:0/2703633422 shutdown_connections 2024-04-27T03:30:05.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.098+0000 7f37ec152640 1 -- 172.21.15.89:0/2703633422 wait complete. 2024-04-27T03:30:05.115 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2024-04-27T03:30:05.213 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.207+0000 7f2b6bd59640 1 Processor -- start 2024-04-27T03:30:05.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.207+0000 7f2b6bd59640 1 -- start start 2024-04-27T03:30:05.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.208+0000 7f2b6bd59640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2b64099d70 0x7f2b64099100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:05.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.208+0000 7f2b6bd59640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2b64099640 con 0x7f2b64098610 2024-04-27T03:30:05.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.208+0000 7f2b6bd59640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f2b640734b0 con 0x7f2b64099d70 2024-04-27T03:30:05.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.208+0000 7f2b692f4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2b64099d70 0x7f2b64099100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:05.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.208+0000 7f2b69af5640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2b64098610 0x7f2b640989f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41650/0 (socket says 172.21.15.89:41650) 2024-04-27T03:30:05.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.208+0000 7f2b69af5640 1 -- 172.21.15.89:0/3389871692 learned_addr learned my addr 172.21.15.89:0/3389871692 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:05.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.208+0000 7f2b692f4640 1 -- 172.21.15.89:0/3389871692 >> v1:172.21.15.89:6789/0 conn(0x7f2b64098610 legacy=0x7f2b640989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:05.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.208+0000 7f2b692f4640 1 -- 172.21.15.89:0/3389871692 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b64134ab0 con 0x7f2b64099d70 2024-04-27T03:30:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.209+0000 7f2b692f4640 1 --2- 172.21.15.89:0/3389871692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b64099d70 0x7f2b64099100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009720 tx=0x7f2b5402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34effd3a1c2e8ef server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.209+0000 7f2b68af3640 1 -- 172.21.15.89:0/3389871692 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b540316d0 con 0x7f2b64099d70 2024-04-27T03:30:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.209+0000 7f2b68af3640 1 -- 172.21.15.89:0/3389871692 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b54037070 con 0x7f2b64099d70 2024-04-27T03:30:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.209+0000 7f2b68af3640 1 -- 172.21.15.89:0/3389871692 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b540319c0 con 0x7f2b64099d70 2024-04-27T03:30:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.209+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3389871692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b64099d70 msgr2=0x7f2b64099100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.209+0000 7f2b6bd59640 1 --2- 172.21.15.89:0/3389871692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b64099d70 0x7f2b64099100 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f2b54009720 tx=0x7f2b5402b840 comp rx=0 tx=0).stop 2024-04-27T03:30:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.209+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3389871692 shutdown_connections 2024-04-27T03:30:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.209+0000 7f2b6bd59640 1 --2- 172.21.15.89:0/3389871692 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b64099d70 0x7f2b64099100 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.209+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3389871692 >> 172.21.15.89:0/3389871692 conn(0x7f2b6409ec10 msgr2=0x7f2b6409f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.210+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3389871692 shutdown_connections 2024-04-27T03:30:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.210+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3389871692 wait complete. 2024-04-27T03:30:05.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.210+0000 7f2b6bd59640 1 Processor -- start 2024-04-27T03:30:05.217 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.210+0000 7f2b6bd59640 1 -- start start 2024-04-27T03:30:05.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.211+0000 7f2b6bd59640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b64098610 0x7f2b640904e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:05.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.211+0000 7f2b6bd59640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2b64133e30 con 0x7f2b64098610 2024-04-27T03:30:05.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.211+0000 7f2b69af5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b64098610 0x7f2b640904e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:05.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.211+0000 7f2b69af5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b64098610 0x7f2b640904e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55222/0 (socket says 172.21.15.89:55222) 2024-04-27T03:30:05.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.211+0000 7f2b69af5640 1 -- 172.21.15.89:0/3172755092 learned_addr learned my addr 172.21.15.89:0/3172755092 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:05.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.211+0000 7f2b69af5640 1 -- 172.21.15.89:0/3172755092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2b64093fb0 con 0x7f2b64098610 2024-04-27T03:30:05.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.211+0000 7f2b69af5640 1 --2- 172.21.15.89:0/3172755092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b64098610 0x7f2b640904e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000c4a0 tx=0x7f2b6000c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:05.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.211+0000 7f2b5a7fc640 1 -- 172.21.15.89:0/3172755092 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b60017070 con 0x7f2b64098610 2024-04-27T03:30:05.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.211+0000 7f2b5a7fc640 1 -- 172.21.15.89:0/3172755092 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b60005c10 con 0x7f2b64098610 2024-04-27T03:30:05.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.211+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3172755092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2b640930f0 con 0x7f2b64098610 2024-04-27T03:30:05.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.211+0000 7f2b5a7fc640 1 -- 172.21.15.89:0/3172755092 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b60007070 con 0x7f2b64098610 2024-04-27T03:30:05.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.211+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3172755092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2b64133360 con 0x7f2b64098610 2024-04-27T03:30:05.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.212+0000 7f2b5a7fc640 1 -- 172.21.15.89:0/3172755092 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7f2b6001a070 con 0x7f2b64098610 2024-04-27T03:30:05.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.212+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3172755092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2b64093b10 con 0x7f2b64098610 2024-04-27T03:30:05.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.212+0000 7f2b5a7fc640 1 --2- 172.21.15.89:0/3172755092 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2b48037fd0 0x7f2b4803a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:05.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.212+0000 7f2b5a7fc640 1 -- 172.21.15.89:0/3172755092 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(10..10 src has 1..10) ==== 3460+0+0 (secure 0 0 0) 0x7f2b6004a1a0 con 0x7f2b64098610 2024-04-27T03:30:05.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.213+0000 7f2b692f4640 1 --2- 172.21.15.89:0/3172755092 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2b48037fd0 0x7f2b4803a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:05.220 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.213+0000 7f2b692f4640 1 --2- 172.21.15.89:0/3172755092 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2b48037fd0 0x7f2b4803a490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2b54036000 tx=0x7f2b54008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:05.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.217+0000 7f2b5a7fc640 1 -- 172.21.15.89:0/3172755092 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f2b6001b3b0 con 0x7f2b64098610 2024-04-27T03:30:05.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:05.393+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3172755092 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) -- 0x7f2b64062da0 con 0x7f2b64098610 2024-04-27T03:30:06.102 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.096+0000 7f2b5a7fc640 1 -- 172.21.15.89:0/3172755092 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v11) ==== 173+0+0 (secure 0 0 0) 0x7f2b6004c0d0 con 0x7f2b64098610 2024-04-27T03:30:06.102 INFO:teuthology.orchestra.run.smithi089.stderr:enabled application 'rbd' on pool 'rbd' 2024-04-27T03:30:06.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.099+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3172755092 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2b48037fd0 msgr2=0x7f2b4803a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:06.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.099+0000 7f2b6bd59640 1 --2- 172.21.15.89:0/3172755092 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2b48037fd0 0x7f2b4803a490 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f2b54036000 tx=0x7f2b54008030 comp rx=0 tx=0).stop 2024-04-27T03:30:06.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.099+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3172755092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b64098610 msgr2=0x7f2b640904e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:06.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.099+0000 7f2b6bd59640 1 --2- 172.21.15.89:0/3172755092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b64098610 0x7f2b640904e0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f2b6000c4a0 tx=0x7f2b6000c960 comp rx=0 tx=0).stop 2024-04-27T03:30:06.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.099+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3172755092 shutdown_connections 2024-04-27T03:30:06.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.099+0000 7f2b6bd59640 1 --2- 172.21.15.89:0/3172755092 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2b48037fd0 0x7f2b4803a490 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:06.105 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.099+0000 7f2b6bd59640 1 --2- 172.21.15.89:0/3172755092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2b64098610 0x7f2b640904e0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:06.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.099+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3172755092 >> 172.21.15.89:0/3172755092 conn(0x7f2b6409ec10 msgr2=0x7f2b6406d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:06.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.099+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3172755092 shutdown_connections 2024-04-27T03:30:06.106 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.099+0000 7f2b6bd59640 1 -- 172.21.15.89:0/3172755092 wait complete. 2024-04-27T03:30:06.117 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2024-04-27T03:30:06.118 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-27T03:30:06.118 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2024-04-27T03:30:06.118 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2024-04-27T03:30:06.118 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2024-04-27T03:30:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.209+0000 7f35a9a06640 1 Processor -- start 2024-04-27T03:30:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.209+0000 7f35a9a06640 1 -- start start 2024-04-27T03:30:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.210+0000 7f35a9a06640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f35a406f260 0x7f35a406f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.210+0000 7f35a9a06640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f35a406fff0 con 0x7f35a406fb80 2024-04-27T03:30:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.210+0000 7f35a9a06640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f35a4070bc0 con 0x7f35a406f260 2024-04-27T03:30:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.210+0000 7f35a2ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f35a406f260 0x7f35a406f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.210+0000 7f35a27fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f35a406fb80 0x7f35a412e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41666/0 (socket says 172.21.15.89:41666) 2024-04-27T03:30:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.210+0000 7f35a27fc640 1 -- 172.21.15.89:0/4030299225 learned_addr learned my addr 172.21.15.89:0/4030299225 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.210+0000 7f35a2ffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f35a406f260 0x7f35a406f640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55234/0 (socket says 172.21.15.89:55234) 2024-04-27T03:30:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.210+0000 7f35a2ffd640 1 -- 172.21.15.89:0/4030299225 >> v1:172.21.15.89:6789/0 conn(0x7f35a406fb80 legacy=0x7f35a412e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:06.216 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.210+0000 7f35a2ffd640 1 -- 172.21.15.89:0/4030299225 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a412ec50 con 0x7f35a406f260 2024-04-27T03:30:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.211+0000 7f35a2ffd640 1 --2- 172.21.15.89:0/4030299225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a406f260 0x7f35a406f640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3598009720 tx=0x7f359802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca69ef0a37f49701 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.211+0000 7f35a1ffb640 1 -- 172.21.15.89:0/4030299225 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35980316d0 con 0x7f35a406f260 2024-04-27T03:30:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.211+0000 7f35a1ffb640 1 -- 172.21.15.89:0/4030299225 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3598037070 con 0x7f35a406f260 2024-04-27T03:30:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.211+0000 7f35a1ffb640 1 -- 172.21.15.89:0/4030299225 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f35980319c0 con 0x7f35a406f260 2024-04-27T03:30:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.211+0000 7f35a9a06640 1 -- 172.21.15.89:0/4030299225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a406f260 msgr2=0x7f35a406f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.211+0000 7f35a9a06640 1 --2- 172.21.15.89:0/4030299225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a406f260 0x7f35a406f640 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f3598009720 tx=0x7f359802b840 comp rx=0 tx=0).stop 2024-04-27T03:30:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.212+0000 7f35a9a06640 1 -- 172.21.15.89:0/4030299225 shutdown_connections 2024-04-27T03:30:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.212+0000 7f35a9a06640 1 --2- 172.21.15.89:0/4030299225 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a406f260 0x7f35a406f640 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:06.218 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.212+0000 7f35a9a06640 1 -- 172.21.15.89:0/4030299225 >> 172.21.15.89:0/4030299225 conn(0x7f35a409e170 msgr2=0x7f35a406d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:06.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.212+0000 7f35a9a06640 1 -- 172.21.15.89:0/4030299225 shutdown_connections 2024-04-27T03:30:06.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.212+0000 7f35a9a06640 1 -- 172.21.15.89:0/4030299225 wait complete. 2024-04-27T03:30:06.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.212+0000 7f35a9a06640 1 Processor -- start 2024-04-27T03:30:06.219 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.213+0000 7f35a9a06640 1 -- start start 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.213+0000 7f35a9a06640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a406f260 0x7f35a4117480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.213+0000 7f35a9a06640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f35a412f400 con 0x7f35a406f260 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.213+0000 7f35a2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a406f260 0x7f35a4117480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.213+0000 7f35a2ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a406f260 0x7f35a4117480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55236/0 (socket says 172.21.15.89:55236) 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.213+0000 7f35a2ffd640 1 -- 172.21.15.89:0/1916552913 learned_addr learned my addr 172.21.15.89:0/1916552913 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.213+0000 7f35a2ffd640 1 -- 172.21.15.89:0/1916552913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f35a411a8a0 con 0x7f35a406f260 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.213+0000 7f35a2ffd640 1 --2- 172.21.15.89:0/1916552913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a406f260 0x7f35a4117480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3598009840 tx=0x7f35980046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.213+0000 7f358bfff640 1 -- 172.21.15.89:0/1916552913 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3598032640 con 0x7f35a406f260 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.213+0000 7f358bfff640 1 -- 172.21.15.89:0/1916552913 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35980327e0 con 0x7f35a406f260 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.213+0000 7f35a9a06640 1 -- 172.21.15.89:0/1916552913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f35a4119c70 con 0x7f35a406f260 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.214+0000 7f358bfff640 1 -- 172.21.15.89:0/1916552913 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3598032b00 con 0x7f35a406f260 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.214+0000 7f35a9a06640 1 -- 172.21.15.89:0/1916552913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f35a411a3f0 con 0x7f35a406f260 2024-04-27T03:30:06.221 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.214+0000 7f358bfff640 1 -- 172.21.15.89:0/1916552913 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7f359804c060 con 0x7f35a406f260 2024-04-27T03:30:06.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.214+0000 7f35a9a06640 1 -- 172.21.15.89:0/1916552913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f35a40a18a0 con 0x7f35a406f260 2024-04-27T03:30:06.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.215+0000 7f358bfff640 1 --2- 172.21.15.89:0/1916552913 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3590037fd0 0x7f359003a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:06.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.215+0000 7f358bfff640 1 -- 172.21.15.89:0/1916552913 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 3482+0+0 (secure 0 0 0) 0x7f359802f050 con 0x7f35a406f260 2024-04-27T03:30:06.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.215+0000 7f35a27fc640 1 --2- 172.21.15.89:0/1916552913 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3590037fd0 0x7f359003a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:06.222 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.215+0000 7f35a27fc640 1 --2- 172.21.15.89:0/1916552913 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3590037fd0 0x7f359003a490 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3594007fd0 tx=0x7f3594007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:06.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.219+0000 7f358bfff640 1 -- 172.21.15.89:0/1916552913 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f35a40a18a0 con 0x7f35a406f260 2024-04-27T03:30:06.450 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.444+0000 7f35a9a06640 1 -- 172.21.15.89:0/1916552913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7f35a41347f0 con 0x7f35a406f260 2024-04-27T03:30:06.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.446+0000 7f358bfff640 1 -- 172.21.15.89:0/1916552913 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v4) ==== 74+0+87112 (secure 0 0 0) 0x7f3598040070 con 0x7f35a406f260 2024-04-27T03:30:06.453 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:30:06.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.448+0000 7f35a9a06640 1 -- 172.21.15.89:0/1916552913 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3590037fd0 msgr2=0x7f359003a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:06.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.448+0000 7f35a9a06640 1 --2- 172.21.15.89:0/1916552913 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3590037fd0 0x7f359003a490 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f3594007fd0 tx=0x7f3594007ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:06.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.448+0000 7f35a9a06640 1 -- 172.21.15.89:0/1916552913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a406f260 msgr2=0x7f35a4117480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:06.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.448+0000 7f35a9a06640 1 --2- 172.21.15.89:0/1916552913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a406f260 0x7f35a4117480 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f3598009840 tx=0x7f35980046c0 comp rx=0 tx=0).stop 2024-04-27T03:30:06.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.448+0000 7f35a9a06640 1 -- 172.21.15.89:0/1916552913 shutdown_connections 2024-04-27T03:30:06.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.448+0000 7f35a9a06640 1 --2- 172.21.15.89:0/1916552913 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3590037fd0 0x7f359003a490 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:06.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.448+0000 7f35a9a06640 1 --2- 172.21.15.89:0/1916552913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f35a406f260 0x7f35a4117480 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:06.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.448+0000 7f35a9a06640 1 -- 172.21.15.89:0/1916552913 >> 172.21.15.89:0/1916552913 conn(0x7f35a409e170 msgr2=0x7f35a412b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:06.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.448+0000 7f35a9a06640 1 -- 172.21.15.89:0/1916552913 shutdown_connections 2024-04-27T03:30:06.455 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.448+0000 7f35a9a06640 1 -- 172.21.15.89:0/1916552913 wait complete. 2024-04-27T03:30:06.464 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":4,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6824","nonce":3121580967},{"type":"v1","addr":"172.21.15.165:6825","nonce":3121580967}]},"active_addr":"172.21.15.165:6825/3121580967","active_change":"2024-04-27T03:30:00.206854+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.165:0","nonce":2336948550}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.165:0","nonce":3618199520}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.165:0","nonce":931940668}]}]} 2024-04-27T03:30:06.466 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2024-04-27T03:30:06.466 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2024-04-27T03:30:06.466 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T03:30:06.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.556+0000 7ff378406640 1 Processor -- start 2024-04-27T03:30:06.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.556+0000 7ff378406640 1 -- start start 2024-04-27T03:30:06.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.557+0000 7ff378406640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff37006fd50 0x7ff370133860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:06.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.557+0000 7ff378406640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7ff370133da0 con 0x7ff37006f260 2024-04-27T03:30:06.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.557+0000 7ff378406640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7ff370070bc0 con 0x7ff37006fd50 2024-04-27T03:30:06.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.557+0000 7ff3759a1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7ff37006fd50 0x7ff370133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:06.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.557+0000 7ff3761a2640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7ff37006f260 0x7ff37006f640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41682/0 (socket says 172.21.15.89:41682) 2024-04-27T03:30:06.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.557+0000 7ff3761a2640 1 -- 172.21.15.89:0/1493148775 learned_addr learned my addr 172.21.15.89:0/1493148775 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:06.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.557+0000 7ff3759a1640 1 -- 172.21.15.89:0/1493148775 >> v1:172.21.15.89:6789/0 conn(0x7ff37006f260 legacy=0x7ff37006f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:06.563 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.557+0000 7ff3759a1640 1 -- 172.21.15.89:0/1493148775 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff370134f80 con 0x7ff37006fd50 2024-04-27T03:30:06.564 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.558+0000 7ff3759a1640 1 --2- 172.21.15.89:0/1493148775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff37006fd50 0x7ff370133860 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff360009720 tx=0x7ff36002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3343a88170c9b1d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:06.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.558+0000 7ff3751a0640 1 -- 172.21.15.89:0/1493148775 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3600316d0 con 0x7ff37006fd50 2024-04-27T03:30:06.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.558+0000 7ff3751a0640 1 -- 172.21.15.89:0/1493148775 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff360037070 con 0x7ff37006fd50 2024-04-27T03:30:06.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.558+0000 7ff3751a0640 1 -- 172.21.15.89:0/1493148775 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff3600319c0 con 0x7ff37006fd50 2024-04-27T03:30:06.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.558+0000 7ff378406640 1 -- 172.21.15.89:0/1493148775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff37006fd50 msgr2=0x7ff370133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:06.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.558+0000 7ff378406640 1 --2- 172.21.15.89:0/1493148775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff37006fd50 0x7ff370133860 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff360009720 tx=0x7ff36002b840 comp rx=0 tx=0).stop 2024-04-27T03:30:06.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.558+0000 7ff378406640 1 -- 172.21.15.89:0/1493148775 shutdown_connections 2024-04-27T03:30:06.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.558+0000 7ff378406640 1 --2- 172.21.15.89:0/1493148775 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff37006fd50 0x7ff370133860 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:06.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.558+0000 7ff378406640 1 -- 172.21.15.89:0/1493148775 >> 172.21.15.89:0/1493148775 conn(0x7ff37009e170 msgr2=0x7ff37006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:06.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.558+0000 7ff378406640 1 -- 172.21.15.89:0/1493148775 shutdown_connections 2024-04-27T03:30:06.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.559+0000 7ff378406640 1 -- 172.21.15.89:0/1493148775 wait complete. 2024-04-27T03:30:06.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.559+0000 7ff378406640 1 Processor -- start 2024-04-27T03:30:06.565 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.559+0000 7ff378406640 1 -- start start 2024-04-27T03:30:06.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.560+0000 7ff378406640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff37006f260 0x7ff37012aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:06.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.560+0000 7ff378406640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7ff370134300 con 0x7ff37006f260 2024-04-27T03:30:06.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.560+0000 7ff3761a2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff37006f260 0x7ff37012aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:06.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.560+0000 7ff3761a2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff37006f260 0x7ff37012aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55250/0 (socket says 172.21.15.89:55250) 2024-04-27T03:30:06.567 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.560+0000 7ff3761a2640 1 -- 172.21.15.89:0/995119996 learned_addr learned my addr 172.21.15.89:0/995119996 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.560+0000 7ff3761a2640 1 -- 172.21.15.89:0/995119996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff37011b5e0 con 0x7ff37006f260 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.560+0000 7ff3761a2640 1 --2- 172.21.15.89:0/995119996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff37006f260 0x7ff37012aa30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c4a0 tx=0x7ff36c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.560+0000 7ff366ffd640 1 -- 172.21.15.89:0/995119996 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff36c017070 con 0x7ff37006f260 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.560+0000 7ff366ffd640 1 -- 172.21.15.89:0/995119996 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff36c005c10 con 0x7ff37006f260 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.560+0000 7ff378406640 1 -- 172.21.15.89:0/995119996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff37011c250 con 0x7ff37006f260 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.560+0000 7ff366ffd640 1 -- 172.21.15.89:0/995119996 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff36c007070 con 0x7ff37006f260 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.560+0000 7ff378406640 1 -- 172.21.15.89:0/995119996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff37011c910 con 0x7ff37006f260 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.561+0000 7ff366ffd640 1 -- 172.21.15.89:0/995119996 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7ff36c01a070 con 0x7ff37006f260 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.561+0000 7ff366ffd640 1 --2- 172.21.15.89:0/995119996 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff354037fd0 0x7ff35403a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.561+0000 7ff378406640 1 -- 172.21.15.89:0/995119996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff3700a18a0 con 0x7ff37006f260 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.561+0000 7ff366ffd640 1 -- 172.21.15.89:0/995119996 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 3482+0+0 (secure 0 0 0) 0x7ff36c04a0f0 con 0x7ff37006f260 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.561+0000 7ff3759a1640 1 --2- 172.21.15.89:0/995119996 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff354037fd0 0x7ff35403a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:06.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.562+0000 7ff3759a1640 1 --2- 172.21.15.89:0/995119996 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff354037fd0 0x7ff35403a490 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff360009720 tx=0x7ff360008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:06.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.566+0000 7ff366ffd640 1 -- 172.21.15.89:0/995119996 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7ff3700a18a0 con 0x7ff37006f260 2024-04-27T03:30:06.748 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.743+0000 7ff378406640 1 -- 172.21.15.89:0/995119996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff370062da0 con 0x7ff37006f260 2024-04-27T03:30:06.749 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.743+0000 7ff366ffd640 1 -- 172.21.15.89:0/995119996 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+9745 (secure 0 0 0) 0x7ff36c01b3b0 con 0x7ff37006f260 2024-04-27T03:30:06.749 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:30:06.749 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":11,"fsid":"7a1794da-6463-461c-a75b-269448ecfc86","created":"2024-04-27T03:29:55.910567+0000","modified":"2024-04-27T03:30:06.097036+0000","last_up_change":"2024-04-27T03:30:04.092854+0000","last_in_change":"2024-04-27T03:29:56.997311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T03:30:04.239623+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-27T03:30:04.766593+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2643cf41-510f-4fc8-9611-7260d319e5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6801","nonce":2353668432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6803","nonce":2353668432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6807","nonce":2353668432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6805","nonce":2353668432}]},"public_addr":"172.21.15.89:6801/2353668432","cluster_addr":"172.21.15.89:6803/2353668432","heartbeat_back_addr":"172.21.15.89:6807/2353668432","heartbeat_front_addr":"172.21.15.89:6805/2353668432","state":["exists","up"]},{"osd":1,"uuid":"6d14d21a-3c2d-47bb-a0d9-96f58d57d612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6801","nonce":79879041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6803","nonce":79879041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6807","nonce":79879041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6805","nonce":79879041}]},"public_addr":"172.21.15.165:6801/79879041","cluster_addr":"172.21.15.165:6803/79879041","heartbeat_back_addr":"172.21.15.165:6807/79879041","heartbeat_front_addr":"172.21.15.165:6805/79879041","state":["exists","up"]},{"osd":2,"uuid":"12198ef3-e304-4085-992a-2f3563d60ebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6817","nonce":975457786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6819","nonce":975457786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6823","nonce":975457786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6821","nonce":975457786}]},"public_addr":"172.21.15.165:6817/975457786","cluster_addr":"172.21.15.165:6819/975457786","heartbeat_back_addr":"172.21.15.165:6823/975457786","heartbeat_front_addr":"172.21.15.165:6821/975457786","state":["exists","up"]},{"osd":3,"uuid":"5ecd16bd-fe0b-4b0d-8028-00b613ff7ec7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6809","nonce":2380918637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6811","nonce":2380918637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6815","nonce":2380918637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6813","nonce":2380918637}]},"public_addr":"172.21.15.165:6809/2380918637","cluster_addr":"172.21.15.165:6811/2380918637","heartbeat_back_addr":"172.21.15.165:6815/2380918637","heartbeat_front_addr":"172.21.15.165:6813/2380918637","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T03:30:02.036887+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T03:30:02.229251+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T03:30:02.245310+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T03:30:02.141804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T03:30:06.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.745+0000 7ff378406640 1 -- 172.21.15.89:0/995119996 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff354037fd0 msgr2=0x7ff35403a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:06.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.745+0000 7ff378406640 1 --2- 172.21.15.89:0/995119996 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff354037fd0 0x7ff35403a490 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7ff360009720 tx=0x7ff360008030 comp rx=0 tx=0).stop 2024-04-27T03:30:06.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.745+0000 7ff378406640 1 -- 172.21.15.89:0/995119996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff37006f260 msgr2=0x7ff37012aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:06.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.745+0000 7ff378406640 1 --2- 172.21.15.89:0/995119996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff37006f260 0x7ff37012aa30 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00c4a0 tx=0x7ff36c00c960 comp rx=0 tx=0).stop 2024-04-27T03:30:06.751 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.745+0000 7ff378406640 1 -- 172.21.15.89:0/995119996 shutdown_connections 2024-04-27T03:30:06.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.745+0000 7ff378406640 1 --2- 172.21.15.89:0/995119996 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff354037fd0 0x7ff35403a490 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:06.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.745+0000 7ff378406640 1 --2- 172.21.15.89:0/995119996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7ff37006f260 0x7ff37012aa30 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:06.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.745+0000 7ff378406640 1 -- 172.21.15.89:0/995119996 >> 172.21.15.89:0/995119996 conn(0x7ff37009e170 msgr2=0x7ff37012edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:06.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.746+0000 7ff378406640 1 -- 172.21.15.89:0/995119996 shutdown_connections 2024-04-27T03:30:06.752 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.746+0000 7ff378406640 1 -- 172.21.15.89:0/995119996 wait complete. 2024-04-27T03:30:06.762 INFO:tasks.ceph.ceph_manager.ceph:all up! 2024-04-27T03:30:06.762 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T03:30:06.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.852+0000 7f3d9c0d2640 1 Processor -- start 2024-04-27T03:30:06.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.852+0000 7f3d9c0d2640 1 -- start start 2024-04-27T03:30:06.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.852+0000 7f3d9c0d2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3d9406d690 0x7f3d9406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:06.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.852+0000 7f3d9c0d2640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f3d9406f0f0 con 0x7f3d9406dfb0 2024-04-27T03:30:06.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.852+0000 7f3d9c0d2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f3d94071a60 con 0x7f3d9406d690 2024-04-27T03:30:06.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.852+0000 7f3d99e6e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f3d9406d690 0x7f3d9406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:06.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.852+0000 7f3d9966d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f3d9406dfb0 0x7f3d94070620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41692/0 (socket says 172.21.15.89:41692) 2024-04-27T03:30:06.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.852+0000 7f3d9966d640 1 -- 172.21.15.89:0/2691741315 learned_addr learned my addr 172.21.15.89:0/2691741315 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:06.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.853+0000 7f3d99e6e640 1 -- 172.21.15.89:0/2691741315 >> v1:172.21.15.89:6789/0 conn(0x7f3d9406dfb0 legacy=0x7f3d94070620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:06.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.853+0000 7f3d99e6e640 1 -- 172.21.15.89:0/2691741315 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d9406f8b0 con 0x7f3d9406d690 2024-04-27T03:30:06.859 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.853+0000 7f3d98e6c640 1 -- 172.21.15.89:0/2691741315 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3454551332 0 0) 0x7f3d9406f0f0 con 0x7f3d9406dfb0 2024-04-27T03:30:06.862 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.853+0000 7f3d99e6e640 1 --2- 172.21.15.89:0/2691741315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9406d690 0x7f3d9406da70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00aea0 tx=0x7f3d8c02cfc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=766af319f552553a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.853+0000 7f3d98e6c640 1 -- 172.21.15.89:0/2691741315 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d8c02d8e0 con 0x7f3d9406d690 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.853+0000 7f3d98e6c640 1 -- 172.21.15.89:0/2691741315 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8c02da80 con 0x7f3d9406d690 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.853+0000 7f3d98e6c640 1 -- 172.21.15.89:0/2691741315 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d8c02dd80 con 0x7f3d9406d690 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.854+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/2691741315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9406d690 msgr2=0x7f3d9406da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.854+0000 7f3d9c0d2640 1 --2- 172.21.15.89:0/2691741315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9406d690 0x7f3d9406da70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00aea0 tx=0x7f3d8c02cfc0 comp rx=0 tx=0).stop 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.854+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/2691741315 shutdown_connections 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.854+0000 7f3d9c0d2640 1 --2- 172.21.15.89:0/2691741315 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9406d690 0x7f3d9406da70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.854+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/2691741315 >> 172.21.15.89:0/2691741315 conn(0x7f3d94099680 msgr2=0x7f3d94099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.854+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/2691741315 shutdown_connections 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.854+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/2691741315 wait complete. 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.855+0000 7f3d9c0d2640 1 Processor -- start 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.855+0000 7f3d9c0d2640 1 -- start start 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.855+0000 7f3d9c0d2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9406d690 0x7f3d941bb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:06.863 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.855+0000 7f3d9c0d2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f3d94070050 con 0x7f3d9406d690 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.855+0000 7f3d99e6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9406d690 0x7f3d941bb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.855+0000 7f3d99e6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9406d690 0x7f3d941bb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55270/0 (socket says 172.21.15.89:55270) 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.855+0000 7f3d99e6e640 1 -- 172.21.15.89:0/589696621 learned_addr learned my addr 172.21.15.89:0/589696621 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.856+0000 7f3d99e6e640 1 -- 172.21.15.89:0/589696621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d941a6900 con 0x7f3d9406d690 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.856+0000 7f3d99e6e640 1 --2- 172.21.15.89:0/589696621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9406d690 0x7f3d941bb6e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00aea0 tx=0x7f3d8c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.856+0000 7f3d927fc640 1 -- 172.21.15.89:0/589696621 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d8c044030 con 0x7f3d9406d690 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.856+0000 7f3d927fc640 1 -- 172.21.15.89:0/589696621 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3d8c03f7b0 con 0x7f3d9406d690 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.856+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/589696621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d94096980 con 0x7f3d9406d690 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.856+0000 7f3d927fc640 1 -- 172.21.15.89:0/589696621 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3d8c03fad0 con 0x7f3d9406d690 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.856+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/589696621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d94095f30 con 0x7f3d9406d690 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.857+0000 7f3d927fc640 1 -- 172.21.15.89:0/589696621 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7f3d8c005d10 con 0x7f3d9406d690 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.857+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/589696621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d94062da0 con 0x7f3d9406d690 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.857+0000 7f3d927fc640 1 --2- 172.21.15.89:0/589696621 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d80037fd0 0x7f3d8003a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.857+0000 7f3d927fc640 1 -- 172.21.15.89:0/589696621 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(11..11 src has 1..11) ==== 3482+0+0 (secure 0 0 0) 0x7f3d8c045070 con 0x7f3d9406d690 2024-04-27T03:30:06.864 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.857+0000 7f3d9966d640 1 --2- 172.21.15.89:0/589696621 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d80037fd0 0x7f3d8003a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:06.865 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.858+0000 7f3d9966d640 1 --2- 172.21.15.89:0/589696621 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d80037fd0 0x7f3d8003a490 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3d88000c00 tx=0x7f3d88007970 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:06.867 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:06.862+0000 7f3d927fc640 1 -- 172.21.15.89:0/589696621 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f3d8c04e020 con 0x7f3d9406d690 2024-04-27T03:30:07.041 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.036+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/589696621 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3d940963e0 con 0x7f3d9406d690 2024-04-27T03:30:07.042 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.036+0000 7f3d927fc640 1 -- 172.21.15.89:0/589696621 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v11) ==== 74+0+9745 (secure 0 0 0) 0x7f3d941a6900 con 0x7f3d9406d690 2024-04-27T03:30:07.042 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:30:07.042 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":11,"fsid":"7a1794da-6463-461c-a75b-269448ecfc86","created":"2024-04-27T03:29:55.910567+0000","modified":"2024-04-27T03:30:06.097036+0000","last_up_change":"2024-04-27T03:30:04.092854+0000","last_in_change":"2024-04-27T03:29:56.997311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T03:30:04.239623+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-27T03:30:04.766593+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"11","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2643cf41-510f-4fc8-9611-7260d319e5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6801","nonce":2353668432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6803","nonce":2353668432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6807","nonce":2353668432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6805","nonce":2353668432}]},"public_addr":"172.21.15.89:6801/2353668432","cluster_addr":"172.21.15.89:6803/2353668432","heartbeat_back_addr":"172.21.15.89:6807/2353668432","heartbeat_front_addr":"172.21.15.89:6805/2353668432","state":["exists","up"]},{"osd":1,"uuid":"6d14d21a-3c2d-47bb-a0d9-96f58d57d612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6801","nonce":79879041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6803","nonce":79879041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6807","nonce":79879041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6805","nonce":79879041}]},"public_addr":"172.21.15.165:6801/79879041","cluster_addr":"172.21.15.165:6803/79879041","heartbeat_back_addr":"172.21.15.165:6807/79879041","heartbeat_front_addr":"172.21.15.165:6805/79879041","state":["exists","up"]},{"osd":2,"uuid":"12198ef3-e304-4085-992a-2f3563d60ebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6817","nonce":975457786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6819","nonce":975457786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6823","nonce":975457786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6821","nonce":975457786}]},"public_addr":"172.21.15.165:6817/975457786","cluster_addr":"172.21.15.165:6819/975457786","heartbeat_back_addr":"172.21.15.165:6823/975457786","heartbeat_front_addr":"172.21.15.165:6821/975457786","state":["exists","up"]},{"osd":3,"uuid":"5ecd16bd-fe0b-4b0d-8028-00b613ff7ec7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":10,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6809","nonce":2380918637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6811","nonce":2380918637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6815","nonce":2380918637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6813","nonce":2380918637}]},"public_addr":"172.21.15.165:6809/2380918637","cluster_addr":"172.21.15.165:6811/2380918637","heartbeat_back_addr":"172.21.15.165:6815/2380918637","heartbeat_front_addr":"172.21.15.165:6813/2380918637","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T03:30:02.036887+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T03:30:02.229251+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T03:30:02.245310+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T03:30:02.141804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T03:30:07.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.038+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/589696621 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d80037fd0 msgr2=0x7f3d8003a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.038+0000 7f3d9c0d2640 1 --2- 172.21.15.89:0/589696621 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d80037fd0 0x7f3d8003a490 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f3d88000c00 tx=0x7f3d88007970 comp rx=0 tx=0).stop 2024-04-27T03:30:07.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.038+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/589696621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9406d690 msgr2=0x7f3d941bb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.038+0000 7f3d9c0d2640 1 --2- 172.21.15.89:0/589696621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9406d690 0x7f3d941bb6e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f3d8c00aea0 tx=0x7f3d8c004a40 comp rx=0 tx=0).stop 2024-04-27T03:30:07.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.038+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/589696621 shutdown_connections 2024-04-27T03:30:07.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.038+0000 7f3d9c0d2640 1 --2- 172.21.15.89:0/589696621 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d80037fd0 0x7f3d8003a490 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.038+0000 7f3d9c0d2640 1 --2- 172.21.15.89:0/589696621 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f3d9406d690 0x7f3d941bb6e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.038+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/589696621 >> 172.21.15.89:0/589696621 conn(0x7f3d94099680 msgr2=0x7f3d9408c350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.038+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/589696621 shutdown_connections 2024-04-27T03:30:07.045 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.038+0000 7f3d9c0d2640 1 -- 172.21.15.89:0/589696621 wait complete. 2024-04-27T03:30:07.055 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2024-04-27T03:30:07.055 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2024-04-27T03:30:07.056 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2024-04-27T03:30:07.056 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2024-04-27T03:30:07.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.152+0000 7fa454bed640 1 Processor -- start 2024-04-27T03:30:07.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.152+0000 7fadcba9a640 1 Processor -- start 2024-04-27T03:30:07.158 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.152+0000 7fa454bed640 1 -- start start 2024-04-27T03:30:07.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.152+0000 7f6d3e704640 1 Processor -- start 2024-04-27T03:30:07.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.152+0000 7f4216c25640 1 Processor -- start 2024-04-27T03:30:07.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.152+0000 7f6d3e704640 1 -- start start 2024-04-27T03:30:07.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.152+0000 7f4216c25640 1 -- start start 2024-04-27T03:30:07.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.152+0000 7fadcba9a640 1 -- start start 2024-04-27T03:30:07.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fadcba9a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fadc406d690 0x7fadc406da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f4216c25640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f42101281e0 0x7f42101285c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f6d3e704640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6d3812bae0 0x7f6d38130580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fa454bed640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa450127ce0 0x7fa4501280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fadcba9a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fadc406f1d0 con 0x7fadc406dfb0 2024-04-27T03:30:07.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fadcba9a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fadc40703c0 con 0x7fadc406d690 2024-04-27T03:30:07.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f6d3e704640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6d3806fac0 con 0x7f6d3812aff0 2024-04-27T03:30:07.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f6d3e704640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f6d38098800 con 0x7f6d3812bae0 2024-04-27T03:30:07.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f4216c25640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4210072960 con 0x7f421012c410 2024-04-27T03:30:07.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f4216c25640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f4210072f00 con 0x7f42101281e0 2024-04-27T03:30:07.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fa454bed640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa450127b00 con 0x7fa450070100 2024-04-27T03:30:07.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fa454bed640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fa4500708e0 con 0x7fa450127ce0 2024-04-27T03:30:07.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f42149c1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f42101281e0 0x7f42101285c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fa44e59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa450127ce0 0x7fa4501280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f42149c1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f42101281e0 0x7f42101285c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55276/0 (socket says 172.21.15.89:55276) 2024-04-27T03:30:07.163 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f420ffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f421012c410 0x7f421012c870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41702/0 (socket says 172.21.15.89:41702) 2024-04-27T03:30:07.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f420ffff640 1 -- 172.21.15.89:0/3349141245 learned_addr learned my addr 172.21.15.89:0/3349141245 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:07.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f6d377fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6d3812bae0 0x7f6d38130580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f6d377fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6d3812bae0 0x7f6d38130580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55296/0 (socket says 172.21.15.89:55296) 2024-04-27T03:30:07.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f6d377fe640 1 -- 172.21.15.89:0/963577151 learned_addr learned my addr 172.21.15.89:0/963577151 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:07.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fadc9836640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fadc406d690 0x7fadc406da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fa44e59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa450127ce0 0x7fa4501280c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55312/0 (socket says 172.21.15.89:55312) 2024-04-27T03:30:07.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fa44e59c640 1 -- 172.21.15.89:0/4294430489 learned_addr learned my addr 172.21.15.89:0/4294430489 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:07.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fadc9836640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fadc406d690 0x7fadc406da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55274/0 (socket says 172.21.15.89:55274) 2024-04-27T03:30:07.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fadc9836640 1 -- 172.21.15.89:0/2725816511 learned_addr learned my addr 172.21.15.89:0/2725816511 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:07.165 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7fa44dd9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa450070100 0x7fa45012b000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41720/0 (socket says 172.21.15.89:41720) 2024-04-27T03:30:07.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f6d377fe640 1 -- 172.21.15.89:0/963577151 >> v1:172.21.15.89:6789/0 conn(0x7f6d3812aff0 legacy=0x7f6d3812b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f6d377fe640 1 -- 172.21.15.89:0/963577151 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d38070b60 con 0x7f6d3812bae0 2024-04-27T03:30:07.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.153+0000 7f42149c1640 1 -- 172.21.15.89:0/3349141245 >> v1:172.21.15.89:6789/0 conn(0x7f421012c410 legacy=0x7f421012c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.166 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.154+0000 7f42149c1640 1 -- 172.21.15.89:0/3349141245 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4210071990 con 0x7f42101281e0 2024-04-27T03:30:07.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.154+0000 7fa44d59a640 1 -- 172.21.15.89:0/4294430489 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2571329327 0 0) 0x7fa444001110 con 0x7fa450070100 2024-04-27T03:30:07.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.154+0000 7fa44d59a640 1 -- 172.21.15.89:0/4294430489 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa438003520 con 0x7fa450070100 2024-04-27T03:30:07.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.154+0000 7fadc9836640 1 -- 172.21.15.89:0/2725816511 >> v1:172.21.15.89:6789/0 conn(0x7fadc406dfb0 legacy=0x7fadc40705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.167 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.154+0000 7fadc9836640 1 -- 172.21.15.89:0/2725816511 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc406f730 con 0x7fadc406d690 2024-04-27T03:30:07.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.154+0000 7fa44e59c640 1 -- 172.21.15.89:0/4294430489 >> v1:172.21.15.89:6789/0 conn(0x7fa450070100 legacy=0x7fa45012b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.154+0000 7fa44e59c640 1 -- 172.21.15.89:0/4294430489 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa450098800 con 0x7fa450127ce0 2024-04-27T03:30:07.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7f6d377fe640 1 --2- 172.21.15.89:0/963577151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3812bae0 0x7f6d38130580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00ba90 tx=0x7f6d2c02e0e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc5d83d24d8065c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.168 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7f42149c1640 1 --2- 172.21.15.89:0/3349141245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42101281e0 0x7f42101285c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4204009720 tx=0x7f420402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a091c523021ca4d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7fa44e59c640 1 --2- 172.21.15.89:0/4294430489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa450127ce0 0x7fa4501280c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4400097f0 tx=0x7fa44002b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5629dee0552b7f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7fadc9836640 1 --2- 172.21.15.89:0/2725816511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc406d690 0x7fadc406da70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fadb400d150 tx=0x7fadb402e180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736aaef0f45da34c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7f6d36ffd640 1 -- 172.21.15.89:0/963577151 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d2c03b070 con 0x7f6d3812bae0 2024-04-27T03:30:07.169 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7f6d36ffd640 1 -- 172.21.15.89:0/963577151 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d2c02edc0 con 0x7f6d3812bae0 2024-04-27T03:30:07.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7fa44d59a640 1 -- 172.21.15.89:0/4294430489 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa440035020 con 0x7fa450127ce0 2024-04-27T03:30:07.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7f6d36ffd640 1 -- 172.21.15.89:0/963577151 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d2c036590 con 0x7f6d3812bae0 2024-04-27T03:30:07.170 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7fadc8834640 1 -- 172.21.15.89:0/2725816511 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadb4034070 con 0x7fadc406d690 2024-04-27T03:30:07.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7f420f7fe640 1 -- 172.21.15.89:0/3349141245 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42040316d0 con 0x7f42101281e0 2024-04-27T03:30:07.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7fa44d59a640 1 -- 172.21.15.89:0/4294430489 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa4400317a0 con 0x7fa450127ce0 2024-04-27T03:30:07.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7fa44d59a640 1 -- 172.21.15.89:0/4294430489 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa440031aa0 con 0x7fa450127ce0 2024-04-27T03:30:07.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7fadc8834640 1 -- 172.21.15.89:0/2725816511 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb402ede0 con 0x7fadc406d690 2024-04-27T03:30:07.171 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7fadc8834640 1 -- 172.21.15.89:0/2725816511 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadb4038550 con 0x7fadc406d690 2024-04-27T03:30:07.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7f420f7fe640 1 -- 172.21.15.89:0/3349141245 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4204037070 con 0x7f42101281e0 2024-04-27T03:30:07.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.155+0000 7f420f7fe640 1 -- 172.21.15.89:0/3349141245 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f42040319c0 con 0x7f42101281e0 2024-04-27T03:30:07.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f6d3e704640 1 -- 172.21.15.89:0/963577151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3812bae0 msgr2=0x7f6d38130580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fa454bed640 1 -- 172.21.15.89:0/4294430489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa450127ce0 msgr2=0x7fa4501280c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fa454bed640 1 --2- 172.21.15.89:0/4294430489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa450127ce0 0x7fa4501280c0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa4400097f0 tx=0x7fa44002b910 comp rx=0 tx=0).stop 2024-04-27T03:30:07.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f6d3e704640 1 --2- 172.21.15.89:0/963577151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3812bae0 0x7f6d38130580 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c00ba90 tx=0x7f6d2c02e0e0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f4216c25640 1 -- 172.21.15.89:0/3349141245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42101281e0 msgr2=0x7f42101285c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f6d3e704640 1 -- 172.21.15.89:0/963577151 shutdown_connections 2024-04-27T03:30:07.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f6d3e704640 1 --2- 172.21.15.89:0/963577151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3812bae0 0x7f6d38130580 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.173 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f6d3e704640 1 -- 172.21.15.89:0/963577151 >> 172.21.15.89:0/963577151 conn(0x7f6d38099680 msgr2=0x7f6d38099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fa454bed640 1 -- 172.21.15.89:0/4294430489 shutdown_connections 2024-04-27T03:30:07.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f4216c25640 1 --2- 172.21.15.89:0/3349141245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42101281e0 0x7f42101285c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f4204009720 tx=0x7f420402b840 comp rx=0 tx=0).stop 2024-04-27T03:30:07.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f6d3e704640 1 -- 172.21.15.89:0/963577151 shutdown_connections 2024-04-27T03:30:07.174 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f6d3e704640 1 -- 172.21.15.89:0/963577151 wait complete. 2024-04-27T03:30:07.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fa454bed640 1 --2- 172.21.15.89:0/4294430489 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa450127ce0 0x7fa4501280c0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fa454bed640 1 -- 172.21.15.89:0/4294430489 >> 172.21.15.89:0/4294430489 conn(0x7fa450099680 msgr2=0x7fa450099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fadcba9a640 1 -- 172.21.15.89:0/2725816511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc406d690 msgr2=0x7fadc406da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f4216c25640 1 -- 172.21.15.89:0/3349141245 shutdown_connections 2024-04-27T03:30:07.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fadcba9a640 1 --2- 172.21.15.89:0/2725816511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc406d690 0x7fadc406da70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fadb400d150 tx=0x7fadb402e180 comp rx=0 tx=0).stop 2024-04-27T03:30:07.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f4216c25640 1 --2- 172.21.15.89:0/3349141245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42101281e0 0x7f42101285c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f4216c25640 1 -- 172.21.15.89:0/3349141245 >> 172.21.15.89:0/3349141245 conn(0x7f421009c7f0 msgr2=0x7f421009cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fa454bed640 1 -- 172.21.15.89:0/4294430489 shutdown_connections 2024-04-27T03:30:07.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fa454bed640 1 -- 172.21.15.89:0/4294430489 wait complete. 2024-04-27T03:30:07.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fadcba9a640 1 -- 172.21.15.89:0/2725816511 shutdown_connections 2024-04-27T03:30:07.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fadcba9a640 1 --2- 172.21.15.89:0/2725816511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc406d690 0x7fadc406da70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fadcba9a640 1 -- 172.21.15.89:0/2725816511 >> 172.21.15.89:0/2725816511 conn(0x7fadc4099680 msgr2=0x7fadc4099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f4216c25640 1 -- 172.21.15.89:0/3349141245 shutdown_connections 2024-04-27T03:30:07.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7f4216c25640 1 -- 172.21.15.89:0/3349141245 wait complete. 2024-04-27T03:30:07.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fadcba9a640 1 -- 172.21.15.89:0/2725816511 shutdown_connections 2024-04-27T03:30:07.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.156+0000 7fadcba9a640 1 -- 172.21.15.89:0/2725816511 wait complete. 2024-04-27T03:30:07.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.157+0000 7f6d3e704640 1 Processor -- start 2024-04-27T03:30:07.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.157+0000 7f4216c25640 1 Processor -- start 2024-04-27T03:30:07.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.157+0000 7f6d3e704640 1 -- start start 2024-04-27T03:30:07.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.157+0000 7f4216c25640 1 -- start start 2024-04-27T03:30:07.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.157+0000 7fa454bed640 1 Processor -- start 2024-04-27T03:30:07.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.157+0000 7fadcba9a640 1 Processor -- start 2024-04-27T03:30:07.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.157+0000 7fa454bed640 1 -- start start 2024-04-27T03:30:07.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.157+0000 7fadcba9a640 1 -- start start 2024-04-27T03:30:07.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.157+0000 7f6d3e704640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3812aff0 0x7f6d381be9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.180 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.157+0000 7f6d3e704640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6d380708a0 con 0x7f6d3812aff0 2024-04-27T03:30:07.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fa454bed640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa450070100 0x7fa45006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fa454bed640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa450128780 con 0x7fa450070100 2024-04-27T03:30:07.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fadcba9a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc406d690 0x7fadc41bea60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fadcba9a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fadc4098750 con 0x7fadc406d690 2024-04-27T03:30:07.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7f6d37fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3812aff0 0x7f6d381be9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7f6d37fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3812aff0 0x7f6d381be9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55322/0 (socket says 172.21.15.89:55322) 2024-04-27T03:30:07.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7f6d37fff640 1 -- 172.21.15.89:0/3382620053 learned_addr learned my addr 172.21.15.89:0/3382620053 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:07.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fa44e59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa450070100 0x7fa45006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fadc9836640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc406d690 0x7fadc41bea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fadc9836640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc406d690 0x7fadc41bea60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55354/0 (socket says 172.21.15.89:55354) 2024-04-27T03:30:07.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fadc9836640 1 -- 172.21.15.89:0/3660636065 learned_addr learned my addr 172.21.15.89:0/3660636065 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:07.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7f4216c25640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42101281e0 0x7f4210115e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fa44e59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa450070100 0x7fa45006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55338/0 (socket says 172.21.15.89:55338) 2024-04-27T03:30:07.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fa44e59c640 1 -- 172.21.15.89:0/2566960103 learned_addr learned my addr 172.21.15.89:0/2566960103 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:07.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7f4216c25640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4210072280 con 0x7f42101281e0 2024-04-27T03:30:07.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7f42149c1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42101281e0 0x7f4210115e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7f6d37fff640 1 -- 172.21.15.89:0/3382620053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d38127f30 con 0x7f6d3812aff0 2024-04-27T03:30:07.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fa44e59c640 1 -- 172.21.15.89:0/2566960103 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa4501255b0 con 0x7fa450070100 2024-04-27T03:30:07.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7f42149c1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42101281e0 0x7f4210115e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55360/0 (socket says 172.21.15.89:55360) 2024-04-27T03:30:07.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7f42149c1640 1 -- 172.21.15.89:0/3135448833 learned_addr learned my addr 172.21.15.89:0/3135448833 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:07.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fadc9836640 1 -- 172.21.15.89:0/3660636065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fadc4094af0 con 0x7fadc406d690 2024-04-27T03:30:07.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fa44e59c640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa450070100 0x7fa45006d420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa4400097f0 tx=0x7fa4400046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7f42149c1640 1 -- 172.21.15.89:0/3135448833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4210119040 con 0x7f42101281e0 2024-04-27T03:30:07.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7f6d37fff640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3812aff0 0x7f6d381be9b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800c450 tx=0x7f6d2800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fadc9836640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc406d690 0x7fadc41bea60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fadb400d270 tx=0x7fadb4004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.158+0000 7fa43effd640 1 -- 172.21.15.89:0/2566960103 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa440031e10 con 0x7fa450070100 2024-04-27T03:30:07.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7fa43effd640 1 -- 172.21.15.89:0/2566960103 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa44003ad00 con 0x7fa450070100 2024-04-27T03:30:07.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7fa43effd640 1 -- 172.21.15.89:0/2566960103 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4400443f0 con 0x7fa450070100 2024-04-27T03:30:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7f42149c1640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42101281e0 0x7f4210115e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4204034040 tx=0x7f42040046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7fa454bed640 1 -- 172.21.15.89:0/2566960103 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45012b140 con 0x7fa450070100 2024-04-27T03:30:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7fa454bed640 1 -- 172.21.15.89:0/2566960103 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45006d960 con 0x7fa450070100 2024-04-27T03:30:07.188 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7f6d34ff9640 1 -- 172.21.15.89:0/3382620053 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d28017070 con 0x7f6d3812aff0 2024-04-27T03:30:07.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7f6d34ff9640 1 -- 172.21.15.89:0/3382620053 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6d28005220 con 0x7f6d3812aff0 2024-04-27T03:30:07.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7f6d34ff9640 1 -- 172.21.15.89:0/3382620053 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6d280035f0 con 0x7f6d3812aff0 2024-04-27T03:30:07.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7f6d3e704640 1 -- 172.21.15.89:0/3382620053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d38127bd0 con 0x7f6d3812aff0 2024-04-27T03:30:07.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7f6d3e704640 1 -- 172.21.15.89:0/3382620053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d3808a580 con 0x7f6d3812aff0 2024-04-27T03:30:07.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7f420d7fa640 1 -- 172.21.15.89:0/3135448833 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4204032640 con 0x7f42101281e0 2024-04-27T03:30:07.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7fadc27fc640 1 -- 172.21.15.89:0/3660636065 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadb4047070 con 0x7fadc406d690 2024-04-27T03:30:07.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7f420d7fa640 1 -- 172.21.15.89:0/3135448833 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f42040327e0 con 0x7f42101281e0 2024-04-27T03:30:07.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7fadc27fc640 1 -- 172.21.15.89:0/3660636065 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fadb40378c0 con 0x7fadc406d690 2024-04-27T03:30:07.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7f420d7fa640 1 -- 172.21.15.89:0/3135448833 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4204032640 con 0x7f42101281e0 2024-04-27T03:30:07.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7fadc27fc640 1 -- 172.21.15.89:0/3660636065 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fadb4037c00 con 0x7fadc406d690 2024-04-27T03:30:07.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7fadcba9a640 1 -- 172.21.15.89:0/3660636065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fadc4095d00 con 0x7fadc406d690 2024-04-27T03:30:07.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7fadcba9a640 1 -- 172.21.15.89:0/3660636065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fadc41a6a90 con 0x7fadc406d690 2024-04-27T03:30:07.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7f4216c25640 1 -- 172.21.15.89:0/3135448833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42101b4690 con 0x7f42101281e0 2024-04-27T03:30:07.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.159+0000 7f4216c25640 1 -- 172.21.15.89:0/3135448833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4210118b90 con 0x7f42101281e0 2024-04-27T03:30:07.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.160+0000 7fa43effd640 1 -- 172.21.15.89:0/2566960103 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7fa44004b060 con 0x7fa450070100 2024-04-27T03:30:07.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.160+0000 7fa454bed640 1 -- 172.21.15.89:0/2566960103 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa450062da0 con 0x7fa450070100 2024-04-27T03:30:07.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.160+0000 7fa43effd640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa430038000 0x7fa43003a4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.160+0000 7fa43effd640 1 -- 172.21.15.89:0/2566960103 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7fa44006cf30 con 0x7fa450070100 2024-04-27T03:30:07.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.160+0000 7fadcba9a640 1 -- 172.21.15.89:0/3660636065 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fadc4094380 con 0x7fadc406d690 2024-04-27T03:30:07.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.160+0000 7fa43effd640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa43003d780 0x7fa43003fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.160+0000 7fa43effd640 1 -- 172.21.15.89:0/2566960103 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa44004ee50 con 0x7fa43003d780 2024-04-27T03:30:07.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.160+0000 7fa44dd9b640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa430038000 0x7fa43003a4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7fa43effd640 1 -- 172.21.15.89:0/2566960103 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7fa44004b760 con 0x7fa450070100 2024-04-27T03:30:07.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7f6d3e704640 1 -- 172.21.15.89:0/3382620053 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f6d3808ab10 con 0x7f6d3812aff0 2024-04-27T03:30:07.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7f6d34ff9640 1 -- 172.21.15.89:0/3382620053 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7f6d2801a070 con 0x7f6d3812aff0 2024-04-27T03:30:07.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7fa44ed9d640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa43003d780 0x7fa43003fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7fa44dd9b640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa430038000 0x7fa43003a4c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa444007fd0 tx=0x7fa444001210 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.194 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7fa44ed9d640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa43003d780 0x7fa43003fbc0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.160+0000 7f4216c25640 1 -- 172.21.15.89:0/3135448833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f4210118410 con 0x7f42101281e0 2024-04-27T03:30:07.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7f6d34ff9640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6d24038070 0x7f6d2403a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7f6d34ff9640 1 -- 172.21.15.89:0/3382620053 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f6d2804a260 con 0x7f6d3812aff0 2024-04-27T03:30:07.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7f6d377fe640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6d24038070 0x7f6d2403a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7f6d34ff9640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6d2403d7f0 0x7f6d2403fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7f6d34ff9640 1 -- 172.21.15.89:0/3382620053 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6d28015e70 con 0x7f6d2403d7f0 2024-04-27T03:30:07.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7fadc27fc640 1 -- 172.21.15.89:0/3660636065 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7fadb404a070 con 0x7fadc406d690 2024-04-27T03:30:07.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7f420d7fa640 1 -- 172.21.15.89:0/3135448833 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7f420404c060 con 0x7f42101281e0 2024-04-27T03:30:07.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7f6d3cca1640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6d2403d7f0 0x7f6d2403fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.196 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7f420d7fa640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f41f4038360 0x7f41f403a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7f6d377fe640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6d24038070 0x7f6d2403a530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c038000 tx=0x7f6d2c032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.161+0000 7fadc27fc640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fadb0038070 0x7fadb003a530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7f420d7fa640 1 -- 172.21.15.89:0/3135448833 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f420402f050 con 0x7f42101281e0 2024-04-27T03:30:07.197 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7fadc27fc640 1 -- 172.21.15.89:0/3660636065 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7fadb406fb30 con 0x7fadc406d690 2024-04-27T03:30:07.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7f420ffff640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f41f4038360 0x7f41f403a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7f420d7fa640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x7f41f403e300 0x7f41f40407a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7f420d7fa640 1 -- 172.21.15.89:0/3135448833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f41f4040e70 con 0x7f41f403e300 2024-04-27T03:30:07.198 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7fadc9035640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fadb0038070 0x7fadb003a530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7f6d34ff9640 1 -- 172.21.15.89:0/3382620053 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7f6d280199b0 con 0x7f6d3812aff0 2024-04-27T03:30:07.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7f420d7fa640 1 -- 172.21.15.89:0/3135448833 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7f420404dab0 con 0x7f42101281e0 2024-04-27T03:30:07.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7fadc27fc640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x7fadb003d7f0 0x7fadb003fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7fadc27fc640 1 -- 172.21.15.89:0/3660636065 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fadb403fe50 con 0x7fadb003d7f0 2024-04-27T03:30:07.199 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7fadc27fc640 1 -- 172.21.15.89:0/3660636065 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=12) ==== 24+0+0 (secure 0 0 0) 0x7fadb404a360 con 0x7fadc406d690 2024-04-27T03:30:07.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7fa43effd640 1 -- 172.21.15.89:0/2566960103 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa44004ee50 con 0x7fa43003d780 2024-04-27T03:30:07.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7f42151c2640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x7f41f403e300 0x7f41f40407a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7f6d3cca1640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6d2403d7f0 0x7f6d2403fc30 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.200 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7fadca037640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x7fadb003d7f0 0x7fadb003fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7f420ffff640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f41f4038360 0x7f41f403a820 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f41fc007fd0 tx=0x7f41fc007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7fadc9035640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fadb0038070 0x7fadb003a530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fadb8000f60 tx=0x7fadb8007a30 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7f42151c2640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x7f41f403e300 0x7f41f40407a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.162+0000 7fadca037640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x7fadb003d7f0 0x7fadb003fc30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.163+0000 7f6d34ff9640 1 -- 172.21.15.89:0/3382620053 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6d28015e70 con 0x7f6d2403d7f0 2024-04-27T03:30:07.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.163+0000 7f420d7fa640 1 -- 172.21.15.89:0/3135448833 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f41f4040e70 con 0x7f41f403e300 2024-04-27T03:30:07.202 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.164+0000 7fadc27fc640 1 -- 172.21.15.89:0/3660636065 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fadb403fe50 con 0x7fadb003d7f0 2024-04-27T03:30:07.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.176+0000 7fa454bed640 1 -- 172.21.15.89:0/2566960103 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa450062da0 con 0x7fa43003d780 2024-04-27T03:30:07.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa43effd640 1 -- 172.21.15.89:0/2566960103 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa450062da0 con 0x7fa43003d780 2024-04-27T03:30:07.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 -- 172.21.15.89:0/2566960103 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa43003d780 msgr2=0x7fa43003fbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa43003d780 0x7fa43003fbc0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 -- 172.21.15.89:0/2566960103 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa430038000 msgr2=0x7fa43003a4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa430038000 0x7fa43003a4c0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa444007fd0 tx=0x7fa444001210 comp rx=0 tx=0).stop 2024-04-27T03:30:07.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 -- 172.21.15.89:0/2566960103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa450070100 msgr2=0x7fa45006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa450070100 0x7fa45006d420 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fa4400097f0 tx=0x7fa4400046c0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 -- 172.21.15.89:0/2566960103 shutdown_connections 2024-04-27T03:30:07.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa43003d780 0x7fa43003fbc0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa430038000 0x7fa43003a4c0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 --2- 172.21.15.89:0/2566960103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa450070100 0x7fa45006d420 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 -- 172.21.15.89:0/2566960103 >> 172.21.15.89:0/2566960103 conn(0x7fa450099680 msgr2=0x7fa45006e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 -- 172.21.15.89:0/2566960103 shutdown_connections 2024-04-27T03:30:07.205 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.177+0000 7fa454bed640 1 -- 172.21.15.89:0/2566960103 wait complete. 2024-04-27T03:30:07.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.178+0000 7f4216c25640 1 -- 172.21.15.89:0/3135448833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f42101177a0 con 0x7f41f403e300 2024-04-27T03:30:07.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.178+0000 7f6d3e704640 1 -- 172.21.15.89:0/3382620053 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6d3809adc0 con 0x7f6d2403d7f0 2024-04-27T03:30:07.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.178+0000 7f420d7fa640 1 -- 172.21.15.89:0/3135448833 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f420800ad10 con 0x7f41f403e300 2024-04-27T03:30:07.206 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.178+0000 7f6d34ff9640 1 -- 172.21.15.89:0/3382620053 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f6d3809adc0 con 0x7f6d2403d7f0 2024-04-27T03:30:07.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.178+0000 7f6d3e704640 1 -- 172.21.15.89:0/3382620053 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6d2403d7f0 msgr2=0x7f6d2403fc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f6d3e704640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6d2403d7f0 0x7f6d2403fc30 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.207 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f6d3e704640 1 -- 172.21.15.89:0/3382620053 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6d24038070 msgr2=0x7f6d2403a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f6d3e704640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6d24038070 0x7f6d2403a530 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f6d2c038000 tx=0x7f6d2c032000 comp rx=0 tx=0).stop 2024-04-27T03:30:07.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f6d3e704640 1 -- 172.21.15.89:0/3382620053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3812aff0 msgr2=0x7f6d381be9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f6d3e704640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3812aff0 0x7f6d381be9b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800c450 tx=0x7f6d2800c910 comp rx=0 tx=0).stop 2024-04-27T03:30:07.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 -- 172.21.15.89:0/3135448833 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x7f41f403e300 msgr2=0x7f41f40407a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.208 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x7f41f403e300 0x7f41f40407a0 crc :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f6d3e704640 1 -- 172.21.15.89:0/3382620053 shutdown_connections 2024-04-27T03:30:07.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f6d3e704640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6d2403d7f0 0x7f6d2403fc30 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f6d3e704640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6d24038070 0x7f6d2403a530 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f6d3e704640 1 --2- 172.21.15.89:0/3382620053 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6d3812aff0 0x7f6d381be9b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f6d3e704640 1 -- 172.21.15.89:0/3382620053 >> 172.21.15.89:0/3382620053 conn(0x7f6d38099680 msgr2=0x7f6d3806d7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.209 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 -- 172.21.15.89:0/3135448833 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f41f4038360 msgr2=0x7f41f403a820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f41f4038360 0x7f41f403a820 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f41fc007fd0 tx=0x7f41fc007ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f6d3e704640 1 -- 172.21.15.89:0/3382620053 shutdown_connections 2024-04-27T03:30:07.210 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 -- 172.21.15.89:0/3135448833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42101281e0 msgr2=0x7f4210115e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42101281e0 0x7f4210115e40 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f4204034040 tx=0x7f42040046c0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 -- 172.21.15.89:0/3135448833 shutdown_connections 2024-04-27T03:30:07.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x7f41f403e300 0x7f41f40407a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.211 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f6d3e704640 1 -- 172.21.15.89:0/3382620053 wait complete. 2024-04-27T03:30:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f41f4038360 0x7f41f403a820 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 --2- 172.21.15.89:0/3135448833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f42101281e0 0x7f4210115e40 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 -- 172.21.15.89:0/3135448833 >> 172.21.15.89:0/3135448833 conn(0x7f421009c7f0 msgr2=0x7f42101b48a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 -- 172.21.15.89:0/3135448833 shutdown_connections 2024-04-27T03:30:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.179+0000 7f4216c25640 1 -- 172.21.15.89:0/3135448833 wait complete. 2024-04-27T03:30:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.183+0000 7fadcba9a640 1 -- 172.21.15.89:0/3660636065 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fadc4062da0 con 0x7fadb003d7f0 2024-04-27T03:30:07.212 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.183+0000 7fadc27fc640 1 -- 172.21.15.89:0/3660636065 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fadc4062da0 con 0x7fadb003d7f0 2024-04-27T03:30:07.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 -- 172.21.15.89:0/3660636065 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x7fadb003d7f0 msgr2=0x7fadb003fc30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x7fadb003d7f0 0x7fadb003fc30 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 -- 172.21.15.89:0/3660636065 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fadb0038070 msgr2=0x7fadb003a530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fadb0038070 0x7fadb003a530 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fadb8000f60 tx=0x7fadb8007a30 comp rx=0 tx=0).stop 2024-04-27T03:30:07.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 -- 172.21.15.89:0/3660636065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc406d690 msgr2=0x7fadc41bea60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc406d690 0x7fadc41bea60 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fadb400d270 tx=0x7fadb4004ab0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 -- 172.21.15.89:0/3660636065 shutdown_connections 2024-04-27T03:30:07.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x7fadb003d7f0 0x7fadb003fc30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fadb0038070 0x7fadb003a530 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 --2- 172.21.15.89:0/3660636065 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fadc406d690 0x7fadc41bea60 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 -- 172.21.15.89:0/3660636065 >> 172.21.15.89:0/3660636065 conn(0x7fadc4099680 msgr2=0x7fadc408c1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.214 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 -- 172.21.15.89:0/3660636065 shutdown_connections 2024-04-27T03:30:07.215 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.184+0000 7fadcba9a640 1 -- 172.21.15.89:0/3660636065 wait complete. 2024-04-27T03:30:07.215 INFO:teuthology.orchestra.run.smithi089.stdout:38654705667 2024-04-27T03:30:07.216 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-27T03:30:07.216 INFO:teuthology.orchestra.run.smithi089.stdout:38654705666 2024-04-27T03:30:07.217 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-27T03:30:07.218 INFO:teuthology.orchestra.run.smithi089.stdout:38654705668 2024-04-27T03:30:07.218 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-27T03:30:07.219 INFO:teuthology.orchestra.run.smithi089.stdout:38654705666 2024-04-27T03:30:07.219 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-27T03:30:07.357 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.352+0000 7efef729a640 1 Processor -- start 2024-04-27T03:30:07.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.352+0000 7f6e49f32640 1 Processor -- start 2024-04-27T03:30:07.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.352+0000 7efef729a640 1 -- start start 2024-04-27T03:30:07.358 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.352+0000 7f40cf14d640 1 Processor -- start 2024-04-27T03:30:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.352+0000 7f6e49f32640 1 -- start start 2024-04-27T03:30:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.352+0000 7f40cf14d640 1 -- start start 2024-04-27T03:30:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.352+0000 7efef729a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efef006d7d0 0x7efef006dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.352+0000 7efef729a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efef00a08a0 con 0x7efef006e0f0 2024-04-27T03:30:07.359 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.352+0000 7efef729a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7efef006e560 con 0x7efef006d7d0 2024-04-27T03:30:07.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.352+0000 7f6e49f32640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6e440989f0 0x7f6e4406dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.352+0000 7f6e49f32640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6e4406f0f0 con 0x7f6e44098610 2024-04-27T03:30:07.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.352+0000 7f6e49f32640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f6e44071a60 con 0x7f6e440989f0 2024-04-27T03:30:07.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7efef4835640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efef006e0f0 0x7efef012e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41736/0 (socket says 172.21.15.89:41736) 2024-04-27T03:30:07.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7efef4835640 1 -- 172.21.15.89:0/282856755 learned_addr learned my addr 172.21.15.89:0/282856755 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:07.360 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7efef5036640 1 --2- 172.21.15.89:0/282856755 >> v2:172.21.15.89:3300/0 conn(0x7efef006d7d0 0x7efef006dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f40cf14d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f40c806fff0 0x7f40c80703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f6e437fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6e44098610 0x7f6e4406d690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41742/0 (socket says 172.21.15.89:41742) 2024-04-27T03:30:07.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f6e437fe640 1 -- 172.21.15.89:0/1164694568 learned_addr learned my addr 172.21.15.89:0/1164694568 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:07.361 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f6e42ffd640 1 --2- 172.21.15.89:0/1164694568 >> v2:172.21.15.89:3300/0 conn(0x7f6e440989f0 0x7f6e4406dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f40cf14d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f40c8071f20 con 0x7f40c8070910 2024-04-27T03:30:07.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f40cf14d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f40c8072780 con 0x7f40c806fff0 2024-04-27T03:30:07.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f23f0e96640 1 Processor -- start 2024-04-27T03:30:07.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f40c7fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f40c8070910 0x7f40c81290b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41758/0 (socket says 172.21.15.89:41758) 2024-04-27T03:30:07.362 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f40c7fff640 1 -- 172.21.15.89:0/245023589 learned_addr learned my addr 172.21.15.89:0/245023589 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:07.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f23f0e96640 1 -- start start 2024-04-27T03:30:07.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f40ccee9640 1 --2- 172.21.15.89:0/245023589 >> v2:172.21.15.89:3300/0 conn(0x7f40c806fff0 0x7f40c80703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7efef5036640 1 -- 172.21.15.89:0/282856755 >> v1:172.21.15.89:6789/0 conn(0x7efef006e0f0 legacy=0x7efef012e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.363 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7efef5036640 1 -- 172.21.15.89:0/282856755 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef006f6d0 con 0x7efef006d7d0 2024-04-27T03:30:07.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f6e427fc640 1 -- 172.21.15.89:0/1164694568 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1921288077 0 0) 0x7f6e4406f0f0 con 0x7f6e44098610 2024-04-27T03:30:07.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f6e427fc640 1 -- 172.21.15.89:0/1164694568 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f6e2c003520 con 0x7f6e44098610 2024-04-27T03:30:07.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f6e42ffd640 1 -- 172.21.15.89:0/1164694568 >> v1:172.21.15.89:6789/0 conn(0x7f6e44098610 legacy=0x7f6e4406d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f6e42ffd640 1 -- 172.21.15.89:0/1164694568 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e4406f8b0 con 0x7f6e440989f0 2024-04-27T03:30:07.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f40ccee9640 1 -- 172.21.15.89:0/245023589 >> v1:172.21.15.89:6789/0 conn(0x7f40c8070910 legacy=0x7f40c81290b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f40c77fe640 1 -- 172.21.15.89:0/245023589 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2375944181 0 0) 0x7f40c8071f20 con 0x7f40c8070910 2024-04-27T03:30:07.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.353+0000 7f40ccee9640 1 -- 172.21.15.89:0/245023589 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c80714c0 con 0x7f40c806fff0 2024-04-27T03:30:07.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7f23f0e96640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f23ec06e2c0 0x7f23ec133640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7f23f0e96640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f23ec0a08a0 con 0x7f23ec06d7d0 2024-04-27T03:30:07.365 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7f23f0e96640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f23ec06f6d0 con 0x7f23ec06e2c0 2024-04-27T03:30:07.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7f23ea59c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f23ec06d7d0 0x7f23ec06dbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41774/0 (socket says 172.21.15.89:41774) 2024-04-27T03:30:07.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7f23ea59c640 1 -- 172.21.15.89:0/1932173305 learned_addr learned my addr 172.21.15.89:0/1932173305 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:07.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7f23e9d9b640 1 --2- 172.21.15.89:0/1932173305 >> v2:172.21.15.89:3300/0 conn(0x7f23ec06e2c0 0x7f23ec133640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7f23e9d9b640 1 -- 172.21.15.89:0/1932173305 >> v1:172.21.15.89:6789/0 conn(0x7f23ec06d7d0 legacy=0x7f23ec06dbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7f23e9d9b640 1 -- 172.21.15.89:0/1932173305 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec06faa0 con 0x7f23ec06e2c0 2024-04-27T03:30:07.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7efef5036640 1 --2- 172.21.15.89:0/282856755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef006d7d0 0x7efef006dbb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efeec009750 tx=0x7efeec02b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc5bc1b166c8bb8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.366 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7f6e42ffd640 1 --2- 172.21.15.89:0/1164694568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e440989f0 0x7f6e4406dda0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009720 tx=0x7f6e3802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed894a4f6ab9ef45 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7efee7fff640 1 -- 172.21.15.89:0/282856755 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeec035020 con 0x7efef006d7d0 2024-04-27T03:30:07.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7f40ccee9640 1 --2- 172.21.15.89:0/245023589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c806fff0 0x7f40c80703d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009750 tx=0x7f40bc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39ba22386f518c58 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7efee7fff640 1 -- 172.21.15.89:0/282856755 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeec031760 con 0x7efef006d7d0 2024-04-27T03:30:07.367 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7efee7fff640 1 -- 172.21.15.89:0/282856755 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeec035020 con 0x7efef006d7d0 2024-04-27T03:30:07.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.354+0000 7f6e427fc640 1 -- 172.21.15.89:0/1164694568 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e380316d0 con 0x7f6e440989f0 2024-04-27T03:30:07.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f40c77fe640 1 -- 172.21.15.89:0/245023589 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40bc0316d0 con 0x7f40c806fff0 2024-04-27T03:30:07.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f6e427fc640 1 -- 172.21.15.89:0/1164694568 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e38037070 con 0x7f6e440989f0 2024-04-27T03:30:07.368 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f40c77fe640 1 -- 172.21.15.89:0/245023589 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40bc037070 con 0x7f40c806fff0 2024-04-27T03:30:07.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f6e427fc640 1 -- 172.21.15.89:0/1164694568 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e380319c0 con 0x7f6e440989f0 2024-04-27T03:30:07.369 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f40c77fe640 1 -- 172.21.15.89:0/245023589 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40bc0316d0 con 0x7f40c806fff0 2024-04-27T03:30:07.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f23e9d9b640 1 --2- 172.21.15.89:0/1932173305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ec06e2c0 0x7f23ec133640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009750 tx=0x7f23e002b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6c622e21923bbe server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f23e959a640 1 -- 172.21.15.89:0/1932173305 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23e0035020 con 0x7f23ec06e2c0 2024-04-27T03:30:07.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f23e959a640 1 -- 172.21.15.89:0/1932173305 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23e0031760 con 0x7f23ec06e2c0 2024-04-27T03:30:07.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f23e959a640 1 -- 172.21.15.89:0/1932173305 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23e0031a60 con 0x7f23ec06e2c0 2024-04-27T03:30:07.370 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7efef729a640 1 -- 172.21.15.89:0/282856755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef006d7d0 msgr2=0x7efef006dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7efef729a640 1 --2- 172.21.15.89:0/282856755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef006d7d0 0x7efef006dbb0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7efeec009750 tx=0x7efeec02b8d0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7efef729a640 1 -- 172.21.15.89:0/282856755 shutdown_connections 2024-04-27T03:30:07.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7efef729a640 1 --2- 172.21.15.89:0/282856755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef006d7d0 0x7efef006dbb0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7efef729a640 1 -- 172.21.15.89:0/282856755 >> 172.21.15.89:0/282856755 conn(0x7efef009ea70 msgr2=0x7efef009ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f40cf14d640 1 -- 172.21.15.89:0/245023589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c806fff0 msgr2=0x7f40c80703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.371 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f40cf14d640 1 --2- 172.21.15.89:0/245023589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c806fff0 0x7f40c80703d0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f40bc009750 tx=0x7f40bc02b840 comp rx=0 tx=0).stop 2024-04-27T03:30:07.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f6e49f32640 1 -- 172.21.15.89:0/1164694568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e440989f0 msgr2=0x7f6e4406dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f6e49f32640 1 --2- 172.21.15.89:0/1164694568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e440989f0 0x7f6e4406dda0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f6e38009720 tx=0x7f6e3802b840 comp rx=0 tx=0).stop 2024-04-27T03:30:07.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7efef729a640 1 -- 172.21.15.89:0/282856755 shutdown_connections 2024-04-27T03:30:07.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f40cf14d640 1 -- 172.21.15.89:0/245023589 shutdown_connections 2024-04-27T03:30:07.372 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f40cf14d640 1 --2- 172.21.15.89:0/245023589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c806fff0 0x7f40c80703d0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.355+0000 7f40cf14d640 1 -- 172.21.15.89:0/245023589 >> 172.21.15.89:0/245023589 conn(0x7f40c809b840 msgr2=0x7f40c809bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7efef729a640 1 -- 172.21.15.89:0/282856755 wait complete. 2024-04-27T03:30:07.373 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f40cf14d640 1 -- 172.21.15.89:0/245023589 shutdown_connections 2024-04-27T03:30:07.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f6e49f32640 1 -- 172.21.15.89:0/1164694568 shutdown_connections 2024-04-27T03:30:07.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f6e49f32640 1 --2- 172.21.15.89:0/1164694568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e440989f0 0x7f6e4406dda0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f6e49f32640 1 -- 172.21.15.89:0/1164694568 >> 172.21.15.89:0/1164694568 conn(0x7f6e44099680 msgr2=0x7f6e44099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f6e49f32640 1 -- 172.21.15.89:0/1164694568 shutdown_connections 2024-04-27T03:30:07.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f40cf14d640 1 -- 172.21.15.89:0/245023589 wait complete. 2024-04-27T03:30:07.374 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f23f0e96640 1 -- 172.21.15.89:0/1932173305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ec06e2c0 msgr2=0x7f23ec133640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f23f0e96640 1 --2- 172.21.15.89:0/1932173305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ec06e2c0 0x7f23ec133640 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f23e0009750 tx=0x7f23e002b8d0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.375 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f6e49f32640 1 -- 172.21.15.89:0/1164694568 wait complete. 2024-04-27T03:30:07.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f23f0e96640 1 -- 172.21.15.89:0/1932173305 shutdown_connections 2024-04-27T03:30:07.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f23f0e96640 1 --2- 172.21.15.89:0/1932173305 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ec06e2c0 0x7f23ec133640 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f23f0e96640 1 -- 172.21.15.89:0/1932173305 >> 172.21.15.89:0/1932173305 conn(0x7f23ec09ea70 msgr2=0x7f23ec09ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.376 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f23f0e96640 1 -- 172.21.15.89:0/1932173305 shutdown_connections 2024-04-27T03:30:07.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.356+0000 7f23f0e96640 1 -- 172.21.15.89:0/1932173305 wait complete. 2024-04-27T03:30:07.377 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.357+0000 7f6e49f32640 1 Processor -- start 2024-04-27T03:30:07.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.357+0000 7f40cf14d640 1 Processor -- start 2024-04-27T03:30:07.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.357+0000 7f40cf14d640 1 -- start start 2024-04-27T03:30:07.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.357+0000 7efef729a640 1 Processor -- start 2024-04-27T03:30:07.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.357+0000 7f6e49f32640 1 -- start start 2024-04-27T03:30:07.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.357+0000 7efef729a640 1 -- start start 2024-04-27T03:30:07.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.357+0000 7f23f0e96640 1 Processor -- start 2024-04-27T03:30:07.379 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.357+0000 7f23f0e96640 1 -- start start 2024-04-27T03:30:07.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.357+0000 7f6e49f32640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e44098610 0x7f6e441313f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.357+0000 7f6e49f32640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6e44070050 con 0x7f6e44098610 2024-04-27T03:30:07.380 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f23f0e96640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ec06d7d0 0x7f23ec09a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f6e437fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e44098610 0x7f6e441313f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f6e437fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e44098610 0x7f6e441313f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55392/0 (socket says 172.21.15.89:55392) 2024-04-27T03:30:07.381 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f6e437fe640 1 -- 172.21.15.89:0/3723673351 learned_addr learned my addr 172.21.15.89:0/3723673351 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:07.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f23f0e96640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f23ec133f10 con 0x7f23ec06d7d0 2024-04-27T03:30:07.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f23ea59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ec06d7d0 0x7f23ec09a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f23ea59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ec06d7d0 0x7f23ec09a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55400/0 (socket says 172.21.15.89:55400) 2024-04-27T03:30:07.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f23ea59c640 1 -- 172.21.15.89:0/2421363568 learned_addr learned my addr 172.21.15.89:0/2421363568 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:07.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7efef729a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef006d7d0 0x7efef01c3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.382 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7efef729a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efef012ebb0 con 0x7efef006d7d0 2024-04-27T03:30:07.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f40cf14d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c806fff0 0x7f40c819ecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f40cf14d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f40c8072260 con 0x7f40c806fff0 2024-04-27T03:30:07.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f6e437fe640 1 -- 172.21.15.89:0/3723673351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e44094640 con 0x7f6e44098610 2024-04-27T03:30:07.383 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f23ea59c640 1 -- 172.21.15.89:0/2421363568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f23ec129840 con 0x7f23ec06d7d0 2024-04-27T03:30:07.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7efef5036640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef006d7d0 0x7efef01c3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7efef5036640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef006d7d0 0x7efef01c3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55408/0 (socket says 172.21.15.89:55408) 2024-04-27T03:30:07.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7efef5036640 1 -- 172.21.15.89:0/2047007670 learned_addr learned my addr 172.21.15.89:0/2047007670 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:07.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f40ccee9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c806fff0 0x7f40c819ecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.384 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f40ccee9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c806fff0 0x7f40c819ecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55410/0 (socket says 172.21.15.89:55410) 2024-04-27T03:30:07.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f40ccee9640 1 -- 172.21.15.89:0/2959439635 learned_addr learned my addr 172.21.15.89:0/2959439635 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:07.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f23ea59c640 1 --2- 172.21.15.89:0/2421363568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ec06d7d0 0x7f23ec09a6d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00c4a0 tx=0x7f23dc00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.385 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f6e437fe640 1 --2- 172.21.15.89:0/3723673351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e44098610 0x7f6e441313f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400c3b0 tx=0x7f6e3400c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7efef5036640 1 -- 172.21.15.89:0/2047007670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efef0090c20 con 0x7efef006d7d0 2024-04-27T03:30:07.386 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.358+0000 7f6e2bfff640 1 -- 172.21.15.89:0/3723673351 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e34017070 con 0x7f6e44098610 2024-04-27T03:30:07.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7f23daffd640 1 -- 172.21.15.89:0/2421363568 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23dc017070 con 0x7f23ec06d7d0 2024-04-27T03:30:07.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7f6e49f32640 1 -- 172.21.15.89:0/3723673351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e44121d80 con 0x7f6e44098610 2024-04-27T03:30:07.387 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7f6e2bfff640 1 -- 172.21.15.89:0/3723673351 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6e34005c10 con 0x7f6e44098610 2024-04-27T03:30:07.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7f6e2bfff640 1 -- 172.21.15.89:0/3723673351 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6e340070d0 con 0x7f6e44098610 2024-04-27T03:30:07.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7f6e49f32640 1 -- 172.21.15.89:0/3723673351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e44131930 con 0x7f6e44098610 2024-04-27T03:30:07.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7f23f0e96640 1 -- 172.21.15.89:0/2421363568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f23ec070890 con 0x7f23ec06d7d0 2024-04-27T03:30:07.388 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7efef5036640 1 --2- 172.21.15.89:0/2047007670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef006d7d0 0x7efef01c3130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efeec009870 tx=0x7efeec0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7f40ccee9640 1 -- 172.21.15.89:0/2959439635 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40c81a1170 con 0x7f40c806fff0 2024-04-27T03:30:07.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7f23f0e96640 1 -- 172.21.15.89:0/2421363568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23ec12a4d0 con 0x7f23ec06d7d0 2024-04-27T03:30:07.389 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7f23daffd640 1 -- 172.21.15.89:0/2421363568 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f23dc005c10 con 0x7f23ec06d7d0 2024-04-27T03:30:07.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7f23daffd640 1 -- 172.21.15.89:0/2421363568 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f23dc007070 con 0x7f23ec06d7d0 2024-04-27T03:30:07.390 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7f40ccee9640 1 --2- 172.21.15.89:0/2959439635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c806fff0 0x7f40c819ecb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f40bc034040 tx=0x7f40bc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.359+0000 7efee5ffb640 1 -- 172.21.15.89:0/2047007670 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeec035020 con 0x7efef006d7d0 2024-04-27T03:30:07.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.360+0000 7efef729a640 1 -- 172.21.15.89:0/2047007670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efef0091b90 con 0x7efef006d7d0 2024-04-27T03:30:07.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.360+0000 7efef729a640 1 -- 172.21.15.89:0/2047007670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efef01c3670 con 0x7efef006d7d0 2024-04-27T03:30:07.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.360+0000 7efee5ffb640 1 -- 172.21.15.89:0/2047007670 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efeec03ba10 con 0x7efef006d7d0 2024-04-27T03:30:07.391 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.360+0000 7efee5ffb640 1 -- 172.21.15.89:0/2047007670 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efeec03bcf0 con 0x7efef006d7d0 2024-04-27T03:30:07.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.360+0000 7f6e49f32640 1 -- 172.21.15.89:0/3723673351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e44062da0 con 0x7f6e44098610 2024-04-27T03:30:07.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.360+0000 7f40c57fa640 1 -- 172.21.15.89:0/2959439635 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40bc0316d0 con 0x7f40c806fff0 2024-04-27T03:30:07.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.360+0000 7f40cf14d640 1 -- 172.21.15.89:0/2959439635 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40c81a16a0 con 0x7f40c806fff0 2024-04-27T03:30:07.392 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.360+0000 7f40cf14d640 1 -- 172.21.15.89:0/2959439635 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40c819f3d0 con 0x7f40c806fff0 2024-04-27T03:30:07.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.360+0000 7f6e2bfff640 1 -- 172.21.15.89:0/3723673351 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7f6e3401a070 con 0x7f6e44098610 2024-04-27T03:30:07.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.361+0000 7f6e2bfff640 1 --2- 172.21.15.89:0/3723673351 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6e30038020 0x7f6e3003a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.361+0000 7f6e2bfff640 1 -- 172.21.15.89:0/3723673351 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f6e3404a390 con 0x7f6e44098610 2024-04-27T03:30:07.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.361+0000 7efee5ffb640 1 -- 172.21.15.89:0/2047007670 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7efeec038020 con 0x7efef006d7d0 2024-04-27T03:30:07.393 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.360+0000 7f40c57fa640 1 -- 172.21.15.89:0/2959439635 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f40bc0093c0 con 0x7f40c806fff0 2024-04-27T03:30:07.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.361+0000 7f23f0e96640 1 -- 172.21.15.89:0/2421363568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f23d0005080 con 0x7f23ec06d7d0 2024-04-27T03:30:07.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.361+0000 7efef729a640 1 -- 172.21.15.89:0/2047007670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efef0062da0 con 0x7efef006d7d0 2024-04-27T03:30:07.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.361+0000 7efee5ffb640 1 --2- 172.21.15.89:0/2047007670 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efed4038020 0x7efed403a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.361+0000 7efee5ffb640 1 -- 172.21.15.89:0/2047007670 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7efeec06ccd0 con 0x7efef006d7d0 2024-04-27T03:30:07.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.361+0000 7f6e42ffd640 1 --2- 172.21.15.89:0/3723673351 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6e30038020 0x7f6e3003a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.361+0000 7f40c57fa640 1 -- 172.21.15.89:0/2959439635 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f40bc0316d0 con 0x7f40c806fff0 2024-04-27T03:30:07.396 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.361+0000 7f23daffd640 1 -- 172.21.15.89:0/2421363568 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7f23dc01a070 con 0x7f23ec06d7d0 2024-04-27T03:30:07.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.361+0000 7efef4835640 1 --2- 172.21.15.89:0/2047007670 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efed4038020 0x7efed403a4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.362+0000 7f23daffd640 1 --2- 172.21.15.89:0/2421363568 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f23cc037fd0 0x7f23cc03a490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.397 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.362+0000 7f40cf14d640 1 -- 172.21.15.89:0/2959439635 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40a8005080 con 0x7f40c806fff0 2024-04-27T03:30:07.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.362+0000 7f23daffd640 1 -- 172.21.15.89:0/2421363568 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f23dc04a180 con 0x7f23ec06d7d0 2024-04-27T03:30:07.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.362+0000 7f23e9d9b640 1 --2- 172.21.15.89:0/2421363568 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f23cc037fd0 0x7f23cc03a490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.398 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.362+0000 7efef4835640 1 --2- 172.21.15.89:0/2047007670 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efed4038020 0x7efed403a4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efee00015c0 tx=0x7efee0007a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.362+0000 7f6e42ffd640 1 --2- 172.21.15.89:0/3723673351 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6e30038020 0x7f6e3003a4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6e38036000 tx=0x7f6e38008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.362+0000 7f23e9d9b640 1 --2- 172.21.15.89:0/2421363568 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f23cc037fd0 0x7f23cc03a490 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f23e0036000 tx=0x7f23e0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.399 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.366+0000 7f6e2bfff640 1 -- 172.21.15.89:0/3723673351 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f6e34019e10 con 0x7f6e44098610 2024-04-27T03:30:07.400 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.370+0000 7efee5ffb640 1 -- 172.21.15.89:0/2047007670 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efeec040070 con 0x7efef006d7d0 2024-04-27T03:30:07.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.371+0000 7f40c57fa640 1 -- 172.21.15.89:0/2959439635 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 4) ==== 45507+0+0 (secure 0 0 0) 0x7f40bc004030 con 0x7f40c806fff0 2024-04-27T03:30:07.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.371+0000 7f40c57fa640 1 --2- 172.21.15.89:0/2959439635 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f40ac038160 0x7f40ac03a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:07.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.371+0000 7f40c7fff640 1 --2- 172.21.15.89:0/2959439635 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f40ac038160 0x7f40ac03a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:07.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.372+0000 7f40c7fff640 1 --2- 172.21.15.89:0/2959439635 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f40ac038160 0x7f40ac03a620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f40b4000c00 tx=0x7f40b4007910 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:07.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.375+0000 7f40c57fa640 1 -- 172.21.15.89:0/2959439635 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f40bc02f050 con 0x7f40c806fff0 2024-04-27T03:30:07.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.375+0000 7f23daffd640 1 -- 172.21.15.89:0/2421363568 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f23dc012400 con 0x7f23ec06d7d0 2024-04-27T03:30:07.402 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.379+0000 7f40c57fa640 1 -- 172.21.15.89:0/2959439635 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f40bc041070 con 0x7f40c806fff0 2024-04-27T03:30:07.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.549+0000 7f6e49f32640 1 -- 172.21.15.89:0/3723673351 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6e44070580 con 0x7f6e44098610 2024-04-27T03:30:07.555 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.549+0000 7f6e2bfff640 1 -- 172.21.15.89:0/3723673351 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f6e34010b90 con 0x7f6e44098610 2024-04-27T03:30:07.555 INFO:teuthology.orchestra.run.smithi089.stdout:38654705665 2024-04-27T03:30:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.552+0000 7f6e49f32640 1 -- 172.21.15.89:0/3723673351 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6e30038020 msgr2=0x7f6e3003a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.552+0000 7f6e49f32640 1 --2- 172.21.15.89:0/3723673351 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6e30038020 0x7f6e3003a4e0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6e38036000 tx=0x7f6e38008030 comp rx=0 tx=0).stop 2024-04-27T03:30:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.552+0000 7f6e49f32640 1 -- 172.21.15.89:0/3723673351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e44098610 msgr2=0x7f6e441313f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.552+0000 7f6e49f32640 1 --2- 172.21.15.89:0/3723673351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e44098610 0x7f6e441313f0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6e3400c3b0 tx=0x7f6e3400c870 comp rx=0 tx=0).stop 2024-04-27T03:30:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.552+0000 7f6e49f32640 1 -- 172.21.15.89:0/3723673351 shutdown_connections 2024-04-27T03:30:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.552+0000 7f6e49f32640 1 --2- 172.21.15.89:0/3723673351 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6e30038020 0x7f6e3003a4e0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.552+0000 7f6e49f32640 1 --2- 172.21.15.89:0/3723673351 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6e44098610 0x7f6e441313f0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.558 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.552+0000 7f6e49f32640 1 -- 172.21.15.89:0/3723673351 >> 172.21.15.89:0/3723673351 conn(0x7f6e44099680 msgr2=0x7f6e4409a560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.552+0000 7f6e49f32640 1 -- 172.21.15.89:0/3723673351 shutdown_connections 2024-04-27T03:30:07.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.552+0000 7f6e49f32640 1 -- 172.21.15.89:0/3723673351 wait complete. 2024-04-27T03:30:07.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.553+0000 7efef729a640 1 -- 172.21.15.89:0/2047007670 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7efef0091380 con 0x7efef006d7d0 2024-04-27T03:30:07.559 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.553+0000 7efee5ffb640 1 -- 172.21.15.89:0/2047007670 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7efeec0309a0 con 0x7efef006d7d0 2024-04-27T03:30:07.559 INFO:teuthology.orchestra.run.smithi089.stdout:38654705665 2024-04-27T03:30:07.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.555+0000 7efef729a640 1 -- 172.21.15.89:0/2047007670 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efed4038020 msgr2=0x7efed403a4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.555+0000 7efef729a640 1 --2- 172.21.15.89:0/2047007670 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efed4038020 0x7efed403a4e0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7efee00015c0 tx=0x7efee0007a80 comp rx=0 tx=0).stop 2024-04-27T03:30:07.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.555+0000 7efef729a640 1 -- 172.21.15.89:0/2047007670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef006d7d0 msgr2=0x7efef01c3130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.561 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.555+0000 7efef729a640 1 --2- 172.21.15.89:0/2047007670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef006d7d0 0x7efef01c3130 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7efeec009870 tx=0x7efeec0046c0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.555+0000 7efef729a640 1 -- 172.21.15.89:0/2047007670 shutdown_connections 2024-04-27T03:30:07.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.555+0000 7efef729a640 1 --2- 172.21.15.89:0/2047007670 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efed4038020 0x7efed403a4e0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.555+0000 7efef729a640 1 --2- 172.21.15.89:0/2047007670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efef006d7d0 0x7efef01c3130 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.555+0000 7efef729a640 1 -- 172.21.15.89:0/2047007670 >> 172.21.15.89:0/2047007670 conn(0x7efef009ea70 msgr2=0x7efef009c540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.556+0000 7efef729a640 1 -- 172.21.15.89:0/2047007670 shutdown_connections 2024-04-27T03:30:07.562 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.556+0000 7efef729a640 1 -- 172.21.15.89:0/2047007670 wait complete. 2024-04-27T03:30:07.568 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.563+0000 7f40cf14d640 1 -- 172.21.15.89:0/2959439635 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f40a8005640 con 0x7f40c806fff0 2024-04-27T03:30:07.569 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.563+0000 7f40c57fa640 1 -- 172.21.15.89:0/2959439635 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f40bc032d50 con 0x7f40c806fff0 2024-04-27T03:30:07.569 INFO:teuthology.orchestra.run.smithi089.stdout:38654705666 2024-04-27T03:30:07.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.565+0000 7f40cf14d640 1 -- 172.21.15.89:0/2959439635 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f40ac038160 msgr2=0x7f40ac03a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.565+0000 7f40cf14d640 1 --2- 172.21.15.89:0/2959439635 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f40ac038160 0x7f40ac03a620 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f40b4000c00 tx=0x7f40b4007910 comp rx=0 tx=0).stop 2024-04-27T03:30:07.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.565+0000 7f40cf14d640 1 -- 172.21.15.89:0/2959439635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c806fff0 msgr2=0x7f40c819ecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.565+0000 7f40cf14d640 1 --2- 172.21.15.89:0/2959439635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c806fff0 0x7f40c819ecb0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f40bc034040 tx=0x7f40bc0046c0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.565+0000 7f40cf14d640 1 -- 172.21.15.89:0/2959439635 shutdown_connections 2024-04-27T03:30:07.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.565+0000 7f40cf14d640 1 --2- 172.21.15.89:0/2959439635 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f40ac038160 0x7f40ac03a620 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.565+0000 7f40cf14d640 1 --2- 172.21.15.89:0/2959439635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f40c806fff0 0x7f40c819ecb0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.571 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.565+0000 7f40cf14d640 1 -- 172.21.15.89:0/2959439635 >> 172.21.15.89:0/2959439635 conn(0x7f40c809b840 msgr2=0x7f40c80998c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.565+0000 7f40cf14d640 1 -- 172.21.15.89:0/2959439635 shutdown_connections 2024-04-27T03:30:07.572 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.565+0000 7f40cf14d640 1 -- 172.21.15.89:0/2959439635 wait complete. 2024-04-27T03:30:07.572 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705666 got 38654705665 for osd.1 2024-04-27T03:30:07.574 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705666 got 38654705665 for osd.3 2024-04-27T03:30:07.574 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.568+0000 7f23f0e96640 1 -- 172.21.15.89:0/2421363568 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f23d0005370 con 0x7f23ec06d7d0 2024-04-27T03:30:07.575 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.568+0000 7f23daffd640 1 -- 172.21.15.89:0/2421363568 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f23dc01a360 con 0x7f23ec06d7d0 2024-04-27T03:30:07.575 INFO:teuthology.orchestra.run.smithi089.stdout:38654705667 2024-04-27T03:30:07.576 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.571+0000 7f23f0e96640 1 -- 172.21.15.89:0/2421363568 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f23cc037fd0 msgr2=0x7f23cc03a490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.571+0000 7f23f0e96640 1 --2- 172.21.15.89:0/2421363568 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f23cc037fd0 0x7f23cc03a490 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f23e0036000 tx=0x7f23e0008030 comp rx=0 tx=0).stop 2024-04-27T03:30:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.571+0000 7f23f0e96640 1 -- 172.21.15.89:0/2421363568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ec06d7d0 msgr2=0x7f23ec09a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.571+0000 7f23f0e96640 1 --2- 172.21.15.89:0/2421363568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ec06d7d0 0x7f23ec09a6d0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f23dc00c4a0 tx=0x7f23dc00c960 comp rx=0 tx=0).stop 2024-04-27T03:30:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.571+0000 7f23f0e96640 1 -- 172.21.15.89:0/2421363568 shutdown_connections 2024-04-27T03:30:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.571+0000 7f23f0e96640 1 --2- 172.21.15.89:0/2421363568 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f23cc037fd0 0x7f23cc03a490 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.571+0000 7f23f0e96640 1 --2- 172.21.15.89:0/2421363568 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f23ec06d7d0 0x7f23ec09a6d0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.571+0000 7f23f0e96640 1 -- 172.21.15.89:0/2421363568 >> 172.21.15.89:0/2421363568 conn(0x7f23ec09ea70 msgr2=0x7f23ec12e130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.571+0000 7f23f0e96640 1 -- 172.21.15.89:0/2421363568 shutdown_connections 2024-04-27T03:30:07.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:07.571+0000 7f23f0e96640 1 -- 172.21.15.89:0/2421363568 wait complete. 2024-04-27T03:30:07.582 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705666 for osd.2 2024-04-27T03:30:07.587 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705667 for osd.0 2024-04-27T03:30:08.573 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2024-04-27T03:30:08.574 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2024-04-27T03:30:08.583 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2024-04-27T03:30:08.588 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2024-04-27T03:30:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.681+0000 7f033d20f640 1 Processor -- start 2024-04-27T03:30:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.681+0000 7f033d20f640 1 -- start start 2024-04-27T03:30:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.681+0000 7f033d20f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f033806d7d0 0x7f033806dbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:08.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.681+0000 7f033d20f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f03380a08a0 con 0x7f033806e0f0 2024-04-27T03:30:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.681+0000 7f033d20f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f033806e560 con 0x7f033806d7d0 2024-04-27T03:30:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.681+0000 7f0336d9d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f033806d7d0 0x7f033806dbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.681+0000 7f033659c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f033806e0f0 0x7f033812e320 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41786/0 (socket says 172.21.15.89:41786) 2024-04-27T03:30:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.681+0000 7f033659c640 1 -- 172.21.15.89:0/2934167627 learned_addr learned my addr 172.21.15.89:0/2934167627 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.682+0000 7f0336d9d640 1 -- 172.21.15.89:0/2934167627 >> v1:172.21.15.89:6789/0 conn(0x7f033806e0f0 legacy=0x7f033812e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.682+0000 7f0336d9d640 1 -- 172.21.15.89:0/2934167627 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f033806f6d0 con 0x7f033806d7d0 2024-04-27T03:30:08.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.683+0000 7fc9f364e640 1 Processor -- start 2024-04-27T03:30:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.683+0000 7f0336d9d640 1 --2- 172.21.15.89:0/2934167627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033806d7d0 0x7f033806dbb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0328009750 tx=0x7f032802b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c519f0433f751708 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.683+0000 7fc9f364e640 1 -- start start 2024-04-27T03:30:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.683+0000 7f0335d9b640 1 -- 172.21.15.89:0/2934167627 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0328035020 con 0x7f033806d7d0 2024-04-27T03:30:08.689 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.683+0000 7f0335d9b640 1 -- 172.21.15.89:0/2934167627 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0328031760 con 0x7f033806d7d0 2024-04-27T03:30:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.683+0000 7f0335d9b640 1 -- 172.21.15.89:0/2934167627 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0328031a60 con 0x7f033806d7d0 2024-04-27T03:30:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7fc9f364e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc9ec06fd50 0x7fc9ec133860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7fc9f13ea640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc9ec06f260 0x7fc9ec06f640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41790/0 (socket says 172.21.15.89:41790) 2024-04-27T03:30:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7fc9f364e640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9ec133da0 con 0x7fc9ec06f260 2024-04-27T03:30:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7fc9f13ea640 1 -- 172.21.15.89:0/2379701098 learned_addr learned my addr 172.21.15.89:0/2379701098 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:08.690 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7fc9f364e640 1 -- 172.21.15.89:0/2379701098 --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fc9ec070bc0 con 0x7fc9ec06fd50 2024-04-27T03:30:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7f033d20f640 1 -- 172.21.15.89:0/2934167627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033806d7d0 msgr2=0x7f033806dbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7fc9f0be9640 1 --2- 172.21.15.89:0/2379701098 >> v2:172.21.15.89:3300/0 conn(0x7fc9ec06fd50 0x7fc9ec133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7f033d20f640 1 --2- 172.21.15.89:0/2934167627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033806d7d0 0x7f033806dbb0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f0328009750 tx=0x7f032802b8d0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7f033d20f640 1 -- 172.21.15.89:0/2934167627 shutdown_connections 2024-04-27T03:30:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7f033d20f640 1 --2- 172.21.15.89:0/2934167627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033806d7d0 0x7f033806dbb0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7f033d20f640 1 -- 172.21.15.89:0/2934167627 >> 172.21.15.89:0/2934167627 conn(0x7f033809ea70 msgr2=0x7f033809ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7f033d20f640 1 -- 172.21.15.89:0/2934167627 shutdown_connections 2024-04-27T03:30:08.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7f033d20f640 1 -- 172.21.15.89:0/2934167627 wait complete. 2024-04-27T03:30:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7fc9e3fff640 1 -- 172.21.15.89:0/2379701098 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1573472410 0 0) 0x7fc9ec133da0 con 0x7fc9ec06f260 2024-04-27T03:30:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7fc9e3fff640 1 -- 172.21.15.89:0/2379701098 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc9d8003520 con 0x7fc9ec06f260 2024-04-27T03:30:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7fc9f0be9640 1 -- 172.21.15.89:0/2379701098 >> v1:172.21.15.89:6789/0 conn(0x7fc9ec06f260 legacy=0x7fc9ec06f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.684+0000 7fc9f0be9640 1 -- 172.21.15.89:0/2379701098 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec134f80 con 0x7fc9ec06fd50 2024-04-27T03:30:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.685+0000 7f033d20f640 1 Processor -- start 2024-04-27T03:30:08.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.685+0000 7f033d20f640 1 -- start start 2024-04-27T03:30:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.685+0000 7fc9f0be9640 1 --2- 172.21.15.89:0/2379701098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9ec06fd50 0x7fc9ec133860 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009720 tx=0x7fc9dc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db3d87dd5c530fe7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.686+0000 7fc9e3fff640 1 -- 172.21.15.89:0/2379701098 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9dc0316d0 con 0x7fc9ec06fd50 2024-04-27T03:30:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.686+0000 7fc9e3fff640 1 -- 172.21.15.89:0/2379701098 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9dc037070 con 0x7fc9ec06fd50 2024-04-27T03:30:08.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.686+0000 7fc9e3fff640 1 -- 172.21.15.89:0/2379701098 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9dc0319c0 con 0x7fc9ec06fd50 2024-04-27T03:30:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.686+0000 7f033d20f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033806d7d0 0x7f03381c3310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.686+0000 7f033d20f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f033812ebb0 con 0x7f033806d7d0 2024-04-27T03:30:08.695 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.686+0000 7f0336d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033806d7d0 0x7f03381c3310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.686+0000 7f0336d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033806d7d0 0x7f03381c3310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55424/0 (socket says 172.21.15.89:55424) 2024-04-27T03:30:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.686+0000 7f0336d9d640 1 -- 172.21.15.89:0/4094725431 learned_addr learned my addr 172.21.15.89:0/4094725431 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.686+0000 7f0336d9d640 1 -- 172.21.15.89:0/4094725431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0338091640 con 0x7f033806d7d0 2024-04-27T03:30:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.686+0000 7f0336d9d640 1 --2- 172.21.15.89:0/4094725431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033806d7d0 0x7f03381c3310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0328009750 tx=0x7f03280046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.686+0000 7f031f7fe640 1 -- 172.21.15.89:0/4094725431 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f032803b980 con 0x7f033806d7d0 2024-04-27T03:30:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.687+0000 7f031f7fe640 1 -- 172.21.15.89:0/4094725431 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f032803bb40 con 0x7f033806d7d0 2024-04-27T03:30:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.687+0000 7fc9f364e640 1 -- 172.21.15.89:0/2379701098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9ec06fd50 msgr2=0x7fc9ec133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.687+0000 7fc9f364e640 1 --2- 172.21.15.89:0/2379701098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9ec06fd50 0x7fc9ec133860 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009720 tx=0x7fc9dc02b840 comp rx=0 tx=0).stop 2024-04-27T03:30:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.687+0000 7f031f7fe640 1 -- 172.21.15.89:0/4094725431 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f032803be60 con 0x7f033806d7d0 2024-04-27T03:30:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.687+0000 7f033d20f640 1 -- 172.21.15.89:0/4094725431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0338091b00 con 0x7f033806d7d0 2024-04-27T03:30:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.687+0000 7f033d20f640 1 -- 172.21.15.89:0/4094725431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03381c3850 con 0x7f033806d7d0 2024-04-27T03:30:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.687+0000 7fc9f364e640 1 -- 172.21.15.89:0/2379701098 shutdown_connections 2024-04-27T03:30:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.687+0000 7fc9f364e640 1 --2- 172.21.15.89:0/2379701098 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9ec06fd50 0x7fc9ec133860 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.687+0000 7fc9f364e640 1 -- 172.21.15.89:0/2379701098 >> 172.21.15.89:0/2379701098 conn(0x7fc9ec09e170 msgr2=0x7fc9ec06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:08.697 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.687+0000 7fc9f364e640 1 -- 172.21.15.89:0/2379701098 shutdown_connections 2024-04-27T03:30:08.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.687+0000 7fc9f364e640 1 -- 172.21.15.89:0/2379701098 wait complete. 2024-04-27T03:30:08.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.688+0000 7f031f7fe640 1 -- 172.21.15.89:0/4094725431 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f032804c060 con 0x7f033806d7d0 2024-04-27T03:30:08.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.688+0000 7f033d20f640 1 -- 172.21.15.89:0/4094725431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f033809d2f0 con 0x7f033806d7d0 2024-04-27T03:30:08.698 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.688+0000 7f031f7fe640 1 --2- 172.21.15.89:0/4094725431 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0324038060 0x7f032403a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:08.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.688+0000 7fc9f364e640 1 Processor -- start 2024-04-27T03:30:08.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.688+0000 7f031f7fe640 1 -- 172.21.15.89:0/4094725431 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f032806cc70 con 0x7f033806d7d0 2024-04-27T03:30:08.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.688+0000 7fc9f364e640 1 -- start start 2024-04-27T03:30:08.699 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.688+0000 7f033659c640 1 --2- 172.21.15.89:0/4094725431 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0324038060 0x7f032403a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:08.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.689+0000 7fc9f364e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9ec06f260 0x7fc9ec131840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:08.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.689+0000 7fc9f364e640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc9ec134300 con 0x7fc9ec06f260 2024-04-27T03:30:08.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.689+0000 7fc9f13ea640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9ec06f260 0x7fc9ec131840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:08.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.689+0000 7fc9f13ea640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9ec06f260 0x7fc9ec131840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55436/0 (socket says 172.21.15.89:55436) 2024-04-27T03:30:08.700 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.689+0000 7fc9f13ea640 1 -- 172.21.15.89:0/3372635527 learned_addr learned my addr 172.21.15.89:0/3372635527 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:08.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.689+0000 7f033659c640 1 --2- 172.21.15.89:0/4094725431 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0324038060 0x7f032403a520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f032c005880 tx=0x7f032c007a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:08.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.689+0000 7fc9f13ea640 1 -- 172.21.15.89:0/3372635527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9ec120f50 con 0x7fc9ec06f260 2024-04-27T03:30:08.701 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.689+0000 7fc9f13ea640 1 --2- 172.21.15.89:0/3372635527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9ec06f260 0x7fc9ec131840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800c480 tx=0x7fc9e800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:08.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.692+0000 7fc9e1ffb640 1 -- 172.21.15.89:0/3372635527 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9e8017070 con 0x7fc9ec06f260 2024-04-27T03:30:08.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.693+0000 7fc9e1ffb640 1 -- 172.21.15.89:0/3372635527 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9e8005c10 con 0x7fc9ec06f260 2024-04-27T03:30:08.702 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.693+0000 7f031f7fe640 1 -- 172.21.15.89:0/4094725431 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f033809d2f0 con 0x7f033806d7d0 2024-04-27T03:30:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.693+0000 7fc9e1ffb640 1 -- 172.21.15.89:0/3372635527 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9e8007070 con 0x7fc9ec06f260 2024-04-27T03:30:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.693+0000 7fc9f364e640 1 -- 172.21.15.89:0/3372635527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9ec11ff20 con 0x7fc9ec06f260 2024-04-27T03:30:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.694+0000 7fc9f364e640 1 -- 172.21.15.89:0/3372635527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9ec121390 con 0x7fc9ec06f260 2024-04-27T03:30:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.694+0000 7fc9e1ffb640 1 -- 172.21.15.89:0/3372635527 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc9e801a070 con 0x7fc9ec06f260 2024-04-27T03:30:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.694+0000 7fc9e1ffb640 1 --2- 172.21.15.89:0/3372635527 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc9d0037ff0 0x7fc9d003a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.694+0000 7fc9e1ffb640 1 -- 172.21.15.89:0/3372635527 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7fc9e804c560 con 0x7fc9ec06f260 2024-04-27T03:30:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.695+0000 7fc9f0be9640 1 --2- 172.21.15.89:0/3372635527 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc9d0037ff0 0x7fc9d003a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:08.703 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.695+0000 7fc9f364e640 1 -- 172.21.15.89:0/3372635527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc9ec0a18a0 con 0x7fc9ec06f260 2024-04-27T03:30:08.704 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.696+0000 7fc9f0be9640 1 --2- 172.21.15.89:0/3372635527 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc9d0037ff0 0x7fc9d003a4b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009720 tx=0x7fc9dc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:08.709 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.704+0000 7fc9e1ffb640 1 -- 172.21.15.89:0/3372635527 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc9ec0a18a0 con 0x7fc9ec06f260 2024-04-27T03:30:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.722+0000 7fc86b577640 1 Processor -- start 2024-04-27T03:30:08.728 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.723+0000 7fc86b577640 1 -- start start 2024-04-27T03:30:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.723+0000 7fc86b577640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc86412d690 0x7fc86412fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.723+0000 7fc86b577640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc8640724c0 con 0x7fc8641272a0 2024-04-27T03:30:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.723+0000 7fc86b577640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fc864126de0 con 0x7fc86412d690 2024-04-27T03:30:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.723+0000 7fc868b12640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc86412d690 0x7fc86412fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.723+0000 7fc868b12640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc86412d690 0x7fc86412fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55446/0 (socket says 172.21.15.89:55446) 2024-04-27T03:30:08.729 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.723+0000 7fc869313640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc8641272a0 0x7fc864127680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41796/0 (socket says 172.21.15.89:41796) 2024-04-27T03:30:08.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.723+0000 7fc868b12640 1 -- 172.21.15.89:0/1609468443 learned_addr learned my addr 172.21.15.89:0/1609468443 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:08.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.723+0000 7fc868b12640 1 -- 172.21.15.89:0/1609468443 >> v1:172.21.15.89:6789/0 conn(0x7fc8641272a0 legacy=0x7fc864127680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.730 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.724+0000 7fc868b12640 1 -- 172.21.15.89:0/1609468443 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86406ecb0 con 0x7fc86412d690 2024-04-27T03:30:08.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.724+0000 7fc868b12640 1 --2- 172.21.15.89:0/1609468443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc86412d690 0x7fc86412fa80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc85400b1e0 tx=0x7fc85402c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb5ee2da378736a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:08.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.724+0000 7fc85bfff640 1 -- 172.21.15.89:0/1609468443 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc854039070 con 0x7fc86412d690 2024-04-27T03:30:08.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.724+0000 7fc85bfff640 1 -- 172.21.15.89:0/1609468443 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc854034440 con 0x7fc86412d690 2024-04-27T03:30:08.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.725+0000 7fc85bfff640 1 -- 172.21.15.89:0/1609468443 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc854034740 con 0x7fc86412d690 2024-04-27T03:30:08.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.725+0000 7fc86b577640 1 -- 172.21.15.89:0/1609468443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc86412d690 msgr2=0x7fc86412fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.725+0000 7fc86b577640 1 --2- 172.21.15.89:0/1609468443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc86412d690 0x7fc86412fa80 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7fc85400b1e0 tx=0x7fc85402c230 comp rx=0 tx=0).stop 2024-04-27T03:30:08.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.725+0000 7fc86b577640 1 -- 172.21.15.89:0/1609468443 shutdown_connections 2024-04-27T03:30:08.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.725+0000 7fc86b577640 1 --2- 172.21.15.89:0/1609468443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc86412d690 0x7fc86412fa80 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.725+0000 7fc86b577640 1 -- 172.21.15.89:0/1609468443 >> 172.21.15.89:0/1609468443 conn(0x7fc864099680 msgr2=0x7fc864099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:08.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.725+0000 7fc86b577640 1 -- 172.21.15.89:0/1609468443 shutdown_connections 2024-04-27T03:30:08.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.726+0000 7fc86b577640 1 -- 172.21.15.89:0/1609468443 wait complete. 2024-04-27T03:30:08.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.726+0000 7fe15d288640 1 Processor -- start 2024-04-27T03:30:08.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.726+0000 7fe15d288640 1 -- start start 2024-04-27T03:30:08.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.726+0000 7fe15d288640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fe15806fff0 0x7fe1580703d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:08.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fc86b577640 1 Processor -- start 2024-04-27T03:30:08.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fe15d288640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fe158071f20 con 0x7fe158070910 2024-04-27T03:30:08.733 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fe15d288640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fe158072780 con 0x7fe15806fff0 2024-04-27T03:30:08.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fc86b577640 1 -- start start 2024-04-27T03:30:08.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fe1567fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fe158070910 0x7fe1581290b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41812/0 (socket says 172.21.15.89:41812) 2024-04-27T03:30:08.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fe1567fc640 1 -- 172.21.15.89:0/1352497261 learned_addr learned my addr 172.21.15.89:0/1352497261 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:08.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fe156ffd640 1 --2- 172.21.15.89:0/1352497261 >> v2:172.21.15.89:3300/0 conn(0x7fe15806fff0 0x7fe1580703d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:08.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fe156ffd640 1 -- 172.21.15.89:0/1352497261 >> v1:172.21.15.89:6789/0 conn(0x7fe158070910 legacy=0x7fe1581290b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.734 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fe156ffd640 1 -- 172.21.15.89:0/1352497261 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1580714c0 con 0x7fe15806fff0 2024-04-27T03:30:08.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fc86b577640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8641272a0 0x7fc86412b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:08.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fc86b577640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc86406fc80 con 0x7fc8641272a0 2024-04-27T03:30:08.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fc869313640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8641272a0 0x7fc86412b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:08.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fc869313640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8641272a0 0x7fc86412b0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55472/0 (socket says 172.21.15.89:55472) 2024-04-27T03:30:08.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.727+0000 7fc869313640 1 -- 172.21.15.89:0/4036625552 learned_addr learned my addr 172.21.15.89:0/4036625552 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:08.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.728+0000 7fc869313640 1 -- 172.21.15.89:0/4036625552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc86408a390 con 0x7fc8641272a0 2024-04-27T03:30:08.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.728+0000 7fc869313640 1 --2- 172.21.15.89:0/4036625552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8641272a0 0x7fc86412b0f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc86000c380 tx=0x7fc86000c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:08.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.728+0000 7fe156ffd640 1 --2- 172.21.15.89:0/1352497261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe15806fff0 0x7fe1580703d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe148009750 tx=0x7fe14802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6c50f73cbe1ee43 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:08.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.728+0000 7fc859ffb640 1 -- 172.21.15.89:0/4036625552 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc860007410 con 0x7fc8641272a0 2024-04-27T03:30:08.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.728+0000 7fc859ffb640 1 -- 172.21.15.89:0/4036625552 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc8600075b0 con 0x7fc8641272a0 2024-04-27T03:30:08.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.728+0000 7fc86b577640 1 -- 172.21.15.89:0/4036625552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc86408a8e0 con 0x7fc8641272a0 2024-04-27T03:30:08.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.728+0000 7fe155ffb640 1 -- 172.21.15.89:0/1352497261 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1480316d0 con 0x7fe15806fff0 2024-04-27T03:30:08.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.728+0000 7fc859ffb640 1 -- 172.21.15.89:0/4036625552 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc8600047e0 con 0x7fc8641272a0 2024-04-27T03:30:08.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.728+0000 7fe155ffb640 1 -- 172.21.15.89:0/1352497261 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe148037070 con 0x7fe15806fff0 2024-04-27T03:30:08.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.728+0000 7fe155ffb640 1 -- 172.21.15.89:0/1352497261 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe1480319c0 con 0x7fe15806fff0 2024-04-27T03:30:08.738 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.728+0000 7fc86b577640 1 -- 172.21.15.89:0/4036625552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc86412b810 con 0x7fc8641272a0 2024-04-27T03:30:08.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.729+0000 7fe15d288640 1 -- 172.21.15.89:0/1352497261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe15806fff0 msgr2=0x7fe1580703d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.729+0000 7fe15d288640 1 --2- 172.21.15.89:0/1352497261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe15806fff0 0x7fe1580703d0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fe148009750 tx=0x7fe14802b840 comp rx=0 tx=0).stop 2024-04-27T03:30:08.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.729+0000 7fe15d288640 1 -- 172.21.15.89:0/1352497261 shutdown_connections 2024-04-27T03:30:08.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.729+0000 7fe15d288640 1 --2- 172.21.15.89:0/1352497261 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe15806fff0 0x7fe1580703d0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.729+0000 7fe15d288640 1 -- 172.21.15.89:0/1352497261 >> 172.21.15.89:0/1352497261 conn(0x7fe15809b840 msgr2=0x7fe15809bc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:08.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.729+0000 7fc86b577640 1 -- 172.21.15.89:0/4036625552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc8641277f0 con 0x7fc8641272a0 2024-04-27T03:30:08.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.729+0000 7fc859ffb640 1 -- 172.21.15.89:0/4036625552 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc86001a020 con 0x7fc8641272a0 2024-04-27T03:30:08.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.729+0000 7fe15d288640 1 -- 172.21.15.89:0/1352497261 shutdown_connections 2024-04-27T03:30:08.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.729+0000 7fe15d288640 1 -- 172.21.15.89:0/1352497261 wait complete. 2024-04-27T03:30:08.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.729+0000 7fc859ffb640 1 --2- 172.21.15.89:0/4036625552 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc848038010 0x7fc84803a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:08.740 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.730+0000 7fc859ffb640 1 -- 172.21.15.89:0/4036625552 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7fc86004a7e0 con 0x7fc8641272a0 2024-04-27T03:30:08.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.730+0000 7fc868b12640 1 --2- 172.21.15.89:0/4036625552 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc848038010 0x7fc84803a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:08.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.730+0000 7fe15d288640 1 Processor -- start 2024-04-27T03:30:08.741 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.730+0000 7fc868b12640 1 --2- 172.21.15.89:0/4036625552 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc848038010 0x7fc84803a4d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc854036000 tx=0x7fc854030000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:08.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.730+0000 7fe15d288640 1 -- start start 2024-04-27T03:30:08.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.731+0000 7fe15d288640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe15806fff0 0x7fe15819ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:08.742 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.731+0000 7fe15d288640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fe158072260 con 0x7fe15806fff0 2024-04-27T03:30:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.731+0000 7fe156ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe15806fff0 0x7fe15819ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.731+0000 7fe156ffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe15806fff0 0x7fe15819ecc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55476/0 (socket says 172.21.15.89:55476) 2024-04-27T03:30:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.731+0000 7fe156ffd640 1 -- 172.21.15.89:0/235690049 learned_addr learned my addr 172.21.15.89:0/235690049 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.731+0000 7fe156ffd640 1 -- 172.21.15.89:0/235690049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe1581a0530 con 0x7fe15806fff0 2024-04-27T03:30:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.731+0000 7fe156ffd640 1 --2- 172.21.15.89:0/235690049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe15806fff0 0x7fe15819ecc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe148034040 tx=0x7fe1480046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.734+0000 7fe13f7fe640 1 -- 172.21.15.89:0/235690049 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe148032770 con 0x7fe15806fff0 2024-04-27T03:30:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.734+0000 7fe13f7fe640 1 -- 172.21.15.89:0/235690049 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe148032910 con 0x7fe15806fff0 2024-04-27T03:30:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.734+0000 7fe13f7fe640 1 -- 172.21.15.89:0/235690049 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe148032bf0 con 0x7fe15806fff0 2024-04-27T03:30:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.734+0000 7fe15d288640 1 -- 172.21.15.89:0/235690049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1581b4220 con 0x7fe15806fff0 2024-04-27T03:30:08.743 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.734+0000 7fc859ffb640 1 -- 172.21.15.89:0/4036625552 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fc8641277f0 con 0x7fc8641272a0 2024-04-27T03:30:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.734+0000 7fe15d288640 1 -- 172.21.15.89:0/235690049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1581a1840 con 0x7fe15806fff0 2024-04-27T03:30:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.735+0000 7fe13f7fe640 1 -- 172.21.15.89:0/235690049 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fe14804c060 con 0x7fe15806fff0 2024-04-27T03:30:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.735+0000 7fe13f7fe640 1 --2- 172.21.15.89:0/235690049 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe14403c510 0x7fe14403e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:08.744 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.735+0000 7fe13f7fe640 1 -- 172.21.15.89:0/235690049 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7fe14802f050 con 0x7fe15806fff0 2024-04-27T03:30:08.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.736+0000 7fe1567fc640 1 --2- 172.21.15.89:0/235690049 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe14403c510 0x7fe14403e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:08.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.736+0000 7fe15d288640 1 -- 172.21.15.89:0/235690049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe1581a2310 con 0x7fe15806fff0 2024-04-27T03:30:08.745 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.736+0000 7fe1567fc640 1 --2- 172.21.15.89:0/235690049 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe14403c510 0x7fe14403e9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe14c000f60 tx=0x7fe14c007a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:08.750 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.745+0000 7fe13f7fe640 1 -- 172.21.15.89:0/235690049 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fe148040070 con 0x7fe15806fff0 2024-04-27T03:30:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.942+0000 7fc86b577640 1 -- 172.21.15.89:0/4036625552 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7fc864062da0 con 0x7fc8641272a0 2024-04-27T03:30:08.948 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.943+0000 7fc859ffb640 1 -- 172.21.15.89:0/4036625552 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc86001bc30 con 0x7fc8641272a0 2024-04-27T03:30:08.948 INFO:teuthology.orchestra.run.smithi089.stdout:38654705668 2024-04-27T03:30:08.950 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.945+0000 7fc86b577640 1 -- 172.21.15.89:0/4036625552 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc848038010 msgr2=0x7fc84803a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.945+0000 7fc86b577640 1 --2- 172.21.15.89:0/4036625552 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc848038010 0x7fc84803a4d0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc854036000 tx=0x7fc854030000 comp rx=0 tx=0).stop 2024-04-27T03:30:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.945+0000 7fc86b577640 1 -- 172.21.15.89:0/4036625552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8641272a0 msgr2=0x7fc86412b0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.945+0000 7fc86b577640 1 --2- 172.21.15.89:0/4036625552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8641272a0 0x7fc86412b0f0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fc86000c380 tx=0x7fc86000c840 comp rx=0 tx=0).stop 2024-04-27T03:30:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.945+0000 7fc86b577640 1 -- 172.21.15.89:0/4036625552 shutdown_connections 2024-04-27T03:30:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.945+0000 7fc86b577640 1 --2- 172.21.15.89:0/4036625552 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc848038010 0x7fc84803a4d0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.945+0000 7fc86b577640 1 --2- 172.21.15.89:0/4036625552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc8641272a0 0x7fc86412b0f0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.945+0000 7fc86b577640 1 -- 172.21.15.89:0/4036625552 >> 172.21.15.89:0/4036625552 conn(0x7fc864099680 msgr2=0x7fc8640701e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.945+0000 7fc86b577640 1 -- 172.21.15.89:0/4036625552 shutdown_connections 2024-04-27T03:30:08.951 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.945+0000 7fc86b577640 1 -- 172.21.15.89:0/4036625552 wait complete. 2024-04-27T03:30:08.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.947+0000 7f033d20f640 1 -- 172.21.15.89:0/4094725431 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f0338092270 con 0x7f033806d7d0 2024-04-27T03:30:08.953 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.948+0000 7f031f7fe640 1 -- 172.21.15.89:0/4094725431 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f0338092270 con 0x7f033806d7d0 2024-04-27T03:30:08.953 INFO:teuthology.orchestra.run.smithi089.stdout:38654705666 2024-04-27T03:30:08.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.948+0000 7fe15d288640 1 -- 172.21.15.89:0/235690049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7fe158062da0 con 0x7fe15806fff0 2024-04-27T03:30:08.954 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.948+0000 7fe13f7fe640 1 -- 172.21.15.89:0/235690049 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fe148052020 con 0x7fe15806fff0 2024-04-27T03:30:08.954 INFO:teuthology.orchestra.run.smithi089.stdout:38654705667 2024-04-27T03:30:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7f033d20f640 1 -- 172.21.15.89:0/4094725431 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0324038060 msgr2=0x7f032403a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7f033d20f640 1 --2- 172.21.15.89:0/4094725431 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0324038060 0x7f032403a520 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f032c005880 tx=0x7f032c007a80 comp rx=0 tx=0).stop 2024-04-27T03:30:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7f033d20f640 1 -- 172.21.15.89:0/4094725431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033806d7d0 msgr2=0x7f03381c3310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7f033d20f640 1 --2- 172.21.15.89:0/4094725431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033806d7d0 0x7f03381c3310 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0328009750 tx=0x7f03280046c0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7f033d20f640 1 -- 172.21.15.89:0/4094725431 shutdown_connections 2024-04-27T03:30:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7f033d20f640 1 --2- 172.21.15.89:0/4094725431 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0324038060 0x7f032403a520 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7f033d20f640 1 --2- 172.21.15.89:0/4094725431 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f033806d7d0 0x7f03381c3310 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7f033d20f640 1 -- 172.21.15.89:0/4094725431 >> 172.21.15.89:0/4094725431 conn(0x7f033809ea70 msgr2=0x7f033809c590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7f033d20f640 1 -- 172.21.15.89:0/4094725431 shutdown_connections 2024-04-27T03:30:08.956 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7f033d20f640 1 -- 172.21.15.89:0/4094725431 wait complete. 2024-04-27T03:30:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7fe15d288640 1 -- 172.21.15.89:0/235690049 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe14403c510 msgr2=0x7fe14403e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7fe15d288640 1 --2- 172.21.15.89:0/235690049 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe14403c510 0x7fe14403e9d0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe14c000f60 tx=0x7fe14c007a80 comp rx=0 tx=0).stop 2024-04-27T03:30:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7fe15d288640 1 -- 172.21.15.89:0/235690049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe15806fff0 msgr2=0x7fe15819ecc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7fe15d288640 1 --2- 172.21.15.89:0/235690049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe15806fff0 0x7fe15819ecc0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fe148034040 tx=0x7fe1480046c0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7fe15d288640 1 -- 172.21.15.89:0/235690049 shutdown_connections 2024-04-27T03:30:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7fe15d288640 1 --2- 172.21.15.89:0/235690049 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe14403c510 0x7fe14403e9d0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.957 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7fe15d288640 1 --2- 172.21.15.89:0/235690049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fe15806fff0 0x7fe15819ecc0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.950+0000 7fe15d288640 1 -- 172.21.15.89:0/235690049 >> 172.21.15.89:0/235690049 conn(0x7fe15809b840 msgr2=0x7fe1581b44a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.951+0000 7fe15d288640 1 -- 172.21.15.89:0/235690049 shutdown_connections 2024-04-27T03:30:08.958 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.951+0000 7fe15d288640 1 -- 172.21.15.89:0/235690049 wait complete. 2024-04-27T03:30:08.963 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705668 got 38654705668 for osd.0 2024-04-27T03:30:08.963 DEBUG:teuthology.parallel:result is None 2024-04-27T03:30:08.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.959+0000 7fc9f364e640 1 -- 172.21.15.89:0/3372635527 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7fc9ec062da0 con 0x7fc9ec06f260 2024-04-27T03:30:08.965 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.959+0000 7fc9e1ffb640 1 -- 172.21.15.89:0/3372635527 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7fc9e8049030 con 0x7fc9ec06f260 2024-04-27T03:30:08.965 INFO:teuthology.orchestra.run.smithi089.stdout:38654705666 2024-04-27T03:30:08.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.961+0000 7fc9f364e640 1 -- 172.21.15.89:0/3372635527 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc9d0037ff0 msgr2=0x7fc9d003a4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.961+0000 7fc9f364e640 1 --2- 172.21.15.89:0/3372635527 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc9d0037ff0 0x7fc9d003a4b0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc9dc009720 tx=0x7fc9dc008030 comp rx=0 tx=0).stop 2024-04-27T03:30:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.961+0000 7fc9f364e640 1 -- 172.21.15.89:0/3372635527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9ec06f260 msgr2=0x7fc9ec131840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.961+0000 7fc9f364e640 1 --2- 172.21.15.89:0/3372635527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9ec06f260 0x7fc9ec131840 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fc9e800c480 tx=0x7fc9e800c940 comp rx=0 tx=0).stop 2024-04-27T03:30:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.962+0000 7fc9f364e640 1 -- 172.21.15.89:0/3372635527 shutdown_connections 2024-04-27T03:30:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.962+0000 7fc9f364e640 1 --2- 172.21.15.89:0/3372635527 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc9d0037ff0 0x7fc9d003a4b0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.962+0000 7fc9f364e640 1 --2- 172.21.15.89:0/3372635527 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9ec06f260 0x7fc9ec131840 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.962+0000 7fc9f364e640 1 -- 172.21.15.89:0/3372635527 >> 172.21.15.89:0/3372635527 conn(0x7fc9ec09e170 msgr2=0x7fc9ec08cd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.962+0000 7fc9f364e640 1 -- 172.21.15.89:0/3372635527 shutdown_connections 2024-04-27T03:30:08.968 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:08.962+0000 7fc9f364e640 1 -- 172.21.15.89:0/3372635527 wait complete. 2024-04-27T03:30:08.969 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705666 got 38654705666 for osd.3 2024-04-27T03:30:08.969 DEBUG:teuthology.parallel:result is None 2024-04-27T03:30:08.970 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705667 got 38654705667 for osd.2 2024-04-27T03:30:08.970 DEBUG:teuthology.parallel:result is None 2024-04-27T03:30:08.978 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705666 got 38654705666 for osd.1 2024-04-27T03:30:08.978 DEBUG:teuthology.parallel:result is None 2024-04-27T03:30:08.978 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-27T03:30:08.979 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T03:30:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.107+0000 7f1b2894b640 1 Processor -- start 2024-04-27T03:30:09.113 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.107+0000 7f1b2894b640 1 -- start start 2024-04-27T03:30:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.107+0000 7f1b2894b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1b24096430 0x7f1b24096810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.107+0000 7f1b2894b640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f1b24098800 con 0x7f1b24071090 2024-04-27T03:30:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.107+0000 7f1b2894b640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f1b24071930 con 0x7f1b24096430 2024-04-27T03:30:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.107+0000 7f1b2259c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f1b24096430 0x7f1b24096810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.107+0000 7f1b1bfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f1b24071090 0x7f1b2406d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41814/0 (socket says 172.21.15.89:41814) 2024-04-27T03:30:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.107+0000 7f1b1bfff640 1 -- 172.21.15.89:0/654734780 learned_addr learned my addr 172.21.15.89:0/654734780 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.108+0000 7f1b2259c640 1 -- 172.21.15.89:0/654734780 >> v1:172.21.15.89:6789/0 conn(0x7f1b24071090 legacy=0x7f1b2406d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.108+0000 7f1b2259c640 1 -- 172.21.15.89:0/654734780 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b24071f50 con 0x7f1b24096430 2024-04-27T03:30:09.114 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.108+0000 7f1b21d9b640 1 -- 172.21.15.89:0/654734780 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2991176144 0 0) 0x7f1b24098800 con 0x7f1b24071090 2024-04-27T03:30:09.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.108+0000 7f1b2259c640 1 --2- 172.21.15.89:0/654734780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b24096430 0x7f1b24096810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009720 tx=0x7f1b1402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b3b1d567f31201 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:09.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.108+0000 7f1b21d9b640 1 -- 172.21.15.89:0/654734780 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b140316d0 con 0x7f1b24096430 2024-04-27T03:30:09.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.108+0000 7f1b21d9b640 1 -- 172.21.15.89:0/654734780 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b14037070 con 0x7f1b24096430 2024-04-27T03:30:09.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.109+0000 7f1b21d9b640 1 -- 172.21.15.89:0/654734780 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b140319c0 con 0x7f1b24096430 2024-04-27T03:30:09.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.109+0000 7f1b2894b640 1 -- 172.21.15.89:0/654734780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b24096430 msgr2=0x7f1b24096810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:09.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.109+0000 7f1b2894b640 1 --2- 172.21.15.89:0/654734780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b24096430 0x7f1b24096810 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f1b14009720 tx=0x7f1b1402b840 comp rx=0 tx=0).stop 2024-04-27T03:30:09.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.109+0000 7f1b2894b640 1 -- 172.21.15.89:0/654734780 shutdown_connections 2024-04-27T03:30:09.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.109+0000 7f1b2894b640 1 --2- 172.21.15.89:0/654734780 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b24096430 0x7f1b24096810 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:09.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.109+0000 7f1b2894b640 1 -- 172.21.15.89:0/654734780 >> 172.21.15.89:0/654734780 conn(0x7f1b24099680 msgr2=0x7f1b24099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:09.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.109+0000 7f1b2894b640 1 -- 172.21.15.89:0/654734780 shutdown_connections 2024-04-27T03:30:09.117 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.109+0000 7f1b2894b640 1 -- 172.21.15.89:0/654734780 wait complete. 2024-04-27T03:30:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.110+0000 7f1b2894b640 1 Processor -- start 2024-04-27T03:30:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.110+0000 7f1b2894b640 1 -- start start 2024-04-27T03:30:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.110+0000 7f1b2894b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b24071090 0x7f1b24089f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.110+0000 7f1b2894b640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f1b24096d50 con 0x7f1b24071090 2024-04-27T03:30:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.110+0000 7f1b2259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b24071090 0x7f1b24089f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.110+0000 7f1b2259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b24071090 0x7f1b24089f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55486/0 (socket says 172.21.15.89:55486) 2024-04-27T03:30:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.110+0000 7f1b2259c640 1 -- 172.21.15.89:0/3947377701 learned_addr learned my addr 172.21.15.89:0/3947377701 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.111+0000 7f1b2259c640 1 -- 172.21.15.89:0/3947377701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b241261d0 con 0x7f1b24071090 2024-04-27T03:30:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.111+0000 7f1b2259c640 1 --2- 172.21.15.89:0/3947377701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b24071090 0x7f1b24089f90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1b14034040 tx=0x7f1b140046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.111+0000 7f1b1affd640 1 -- 172.21.15.89:0/3947377701 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b14032770 con 0x7f1b24071090 2024-04-27T03:30:09.118 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.111+0000 7f1b1affd640 1 -- 172.21.15.89:0/3947377701 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b14032910 con 0x7f1b24071090 2024-04-27T03:30:09.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.111+0000 7f1b2894b640 1 -- 172.21.15.89:0/3947377701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b24095eb0 con 0x7f1b24071090 2024-04-27T03:30:09.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.111+0000 7f1b1affd640 1 -- 172.21.15.89:0/3947377701 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b14032c30 con 0x7f1b24071090 2024-04-27T03:30:09.119 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.111+0000 7f1b2894b640 1 -- 172.21.15.89:0/3947377701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b2408c230 con 0x7f1b24071090 2024-04-27T03:30:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.112+0000 7f1b1affd640 1 -- 172.21.15.89:0/3947377701 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f1b1404c060 con 0x7f1b24071090 2024-04-27T03:30:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.112+0000 7f1b1affd640 1 --2- 172.21.15.89:0/3947377701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1b04038010 0x7f1b0403a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.112+0000 7f1b1affd640 1 -- 172.21.15.89:0/3947377701 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f1b1402f050 con 0x7f1b24071090 2024-04-27T03:30:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.112+0000 7f1b2894b640 1 -- 172.21.15.89:0/3947377701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b24062da0 con 0x7f1b24071090 2024-04-27T03:30:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.112+0000 7f1b1bfff640 1 --2- 172.21.15.89:0/3947377701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1b04038010 0x7f1b0403a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:09.120 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.113+0000 7f1b1bfff640 1 --2- 172.21.15.89:0/3947377701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1b04038010 0x7f1b0403a4d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c000c00 tx=0x7f1b0c007910 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:09.122 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.117+0000 7f1b1affd640 1 -- 172.21.15.89:0/3947377701 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f1b14040070 con 0x7f1b24071090 2024-04-27T03:30:09.290 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.284+0000 7f1b2894b640 1 -- 172.21.15.89:0/3947377701 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1b24129010 con 0x7f1b04038010 2024-04-27T03:30:09.291 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.286+0000 7f1b1affd640 1 -- 172.21.15.89:0/3947377701 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+31861 (secure 0 0 0) 0x7f1b0c0070d0 con 0x7f1b04038010 2024-04-27T03:30:09.292 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:30:09.292 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-04-27T03:30:09.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.288+0000 7f1b2894b640 1 -- 172.21.15.89:0/3947377701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1b04038010 msgr2=0x7f1b0403a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:09.293 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.288+0000 7f1b2894b640 1 --2- 172.21.15.89:0/3947377701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1b04038010 0x7f1b0403a4d0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f1b0c000c00 tx=0x7f1b0c007910 comp rx=0 tx=0).stop 2024-04-27T03:30:09.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.288+0000 7f1b2894b640 1 -- 172.21.15.89:0/3947377701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b24071090 msgr2=0x7f1b24089f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:09.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.288+0000 7f1b2894b640 1 --2- 172.21.15.89:0/3947377701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b24071090 0x7f1b24089f90 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1b14034040 tx=0x7f1b140046c0 comp rx=0 tx=0).stop 2024-04-27T03:30:09.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.288+0000 7f1b2894b640 1 -- 172.21.15.89:0/3947377701 shutdown_connections 2024-04-27T03:30:09.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.288+0000 7f1b2894b640 1 --2- 172.21.15.89:0/3947377701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1b04038010 0x7f1b0403a4d0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:09.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.288+0000 7f1b2894b640 1 --2- 172.21.15.89:0/3947377701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1b24071090 0x7f1b24089f90 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:09.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.288+0000 7f1b2894b640 1 -- 172.21.15.89:0/3947377701 >> 172.21.15.89:0/3947377701 conn(0x7f1b24099680 msgr2=0x7f1b24128590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:09.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.288+0000 7f1b2894b640 1 -- 172.21.15.89:0/3947377701 shutdown_connections 2024-04-27T03:30:09.294 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.288+0000 7f1b2894b640 1 -- 172.21.15.89:0/3947377701 wait complete. 2024-04-27T03:30:09.304 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-04-27T03:30:08.211800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":110240,"kb_used_data":1288,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377377120,"statfs":{"total":386547056640,"available":386434170880,"internally_reserved":0,"allocated":1318912,"data_stored":1025204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":110618390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-27T03:30:07.104109+0000","last_change":"2024-04-27T03:30:06.107954+0000","last_active":"2024-04-27T03:30:07.104109+0000","last_peered":"2024-04-27T03:30:07.104109+0000","last_clean":"2024-04-27T03:30:07.104109+0000","last_became_active":"2024-04-27T03:30:06.107600+0000","last_became_peered":"2024-04-27T03:30:06.107600+0000","last_unstale":"2024-04-27T03:30:07.104109+0000","last_undegraded":"2024-04-27T03:30:07.104109+0000","last_fullsized":"2024-04-27T03:30:07.104109+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:44.876194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-04-27T03:30:06.104702+0000","last_change":"2024-04-27T03:30:06.104067+0000","last_active":"2024-04-27T03:30:06.104702+0000","last_peered":"2024-04-27T03:30:06.104702+0000","last_clean":"2024-04-27T03:30:06.104702+0000","last_became_active":"2024-04-27T03:30:06.103759+0000","last_became_peered":"2024-04-27T03:30:06.103759+0000","last_unstale":"2024-04-27T03:30:06.104702+0000","last_undegraded":"2024-04-27T03:30:06.104702+0000","last_fullsized":"2024-04-27T03:30:06.104702+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:34:06.835461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-27T03:30:07.104105+0000","last_change":"2024-04-27T03:30:06.106816+0000","last_active":"2024-04-27T03:30:07.104105+0000","last_peered":"2024-04-27T03:30:07.104105+0000","last_clean":"2024-04-27T03:30:07.104105+0000","last_became_active":"2024-04-27T03:30:06.106475+0000","last_became_peered":"2024-04-27T03:30:06.106475+0000","last_unstale":"2024-04-27T03:30:07.104105+0000","last_undegraded":"2024-04-27T03:30:07.104105+0000","last_fullsized":"2024-04-27T03:30:07.104105+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:36.362210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-04-27T03:30:06.108522+0000","last_change":"2024-04-27T03:30:06.107861+0000","last_active":"2024-04-27T03:30:06.108522+0000","last_peered":"2024-04-27T03:30:06.108522+0000","last_clean":"2024-04-27T03:30:06.108522+0000","last_became_active":"2024-04-27T03:30:06.107594+0000","last_became_peered":"2024-04-27T03:30:06.107594+0000","last_unstale":"2024-04-27T03:30:06.108522+0000","last_undegraded":"2024-04-27T03:30:06.108522+0000","last_fullsized":"2024-04-27T03:30:06.108522+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:59.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-27T03:30:07.104075+0000","last_change":"2024-04-27T03:30:06.107772+0000","last_active":"2024-04-27T03:30:07.104075+0000","last_peered":"2024-04-27T03:30:07.104075+0000","last_clean":"2024-04-27T03:30:07.104075+0000","last_became_active":"2024-04-27T03:30:06.107356+0000","last_became_peered":"2024-04-27T03:30:06.107356+0000","last_unstale":"2024-04-27T03:30:07.104075+0000","last_undegraded":"2024-04-27T03:30:07.104075+0000","last_fullsized":"2024-04-27T03:30:07.104075+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:05:28.414626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-27T03:30:07.110169+0000","last_change":"2024-04-27T03:30:06.103722+0000","last_active":"2024-04-27T03:30:07.110169+0000","last_peered":"2024-04-27T03:30:07.110169+0000","last_clean":"2024-04-27T03:30:07.110169+0000","last_became_active":"2024-04-27T03:30:06.103441+0000","last_became_peered":"2024-04-27T03:30:06.103441+0000","last_unstale":"2024-04-27T03:30:07.110169+0000","last_undegraded":"2024-04-27T03:30:07.110169+0000","last_fullsized":"2024-04-27T03:30:07.110169+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:59.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-27T03:30:07.104066+0000","last_change":"2024-04-27T03:30:06.106890+0000","last_active":"2024-04-27T03:30:07.104066+0000","last_peered":"2024-04-27T03:30:07.104066+0000","last_clean":"2024-04-27T03:30:07.104066+0000","last_became_active":"2024-04-27T03:30:06.106474+0000","last_became_peered":"2024-04-27T03:30:06.106474+0000","last_unstale":"2024-04-27T03:30:07.104066+0000","last_undegraded":"2024-04-27T03:30:07.104066+0000","last_fullsized":"2024-04-27T03:30:07.104066+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:12.652758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-27T03:30:07.104181+0000","last_change":"2024-04-27T03:30:06.106875+0000","last_active":"2024-04-27T03:30:07.104181+0000","last_peered":"2024-04-27T03:30:07.104181+0000","last_clean":"2024-04-27T03:30:07.104181+0000","last_became_active":"2024-04-27T03:30:06.106511+0000","last_became_peered":"2024-04-27T03:30:06.106511+0000","last_unstale":"2024-04-27T03:30:07.104181+0000","last_undegraded":"2024-04-27T03:30:07.104181+0000","last_fullsized":"2024-04-27T03:30:07.104181+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:56:59.055441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":159,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-04-27T03:30:06.676693+0000","last_change":"2024-04-27T03:30:06.104151+0000","last_active":"2024-04-27T03:30:06.676693+0000","last_peered":"2024-04-27T03:30:06.676693+0000","last_clean":"2024-04-27T03:30:06.676693+0000","last_became_active":"2024-04-27T03:30:06.102893+0000","last_became_peered":"2024-04-27T03:30:06.102893+0000","last_unstale":"2024-04-27T03:30:06.676693+0000","last_undegraded":"2024-04-27T03:30:06.676693+0000","last_fullsized":"2024-04-27T03:30:06.676693+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:59.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705666,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":98304,"data_stored":26661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":561152,"data_stored":485941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705666,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":561152,"data_stored":485941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":98304,"data_stored":26661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T03:30:09.305 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T03:30:09.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.395+0000 7f13c44e6640 1 Processor -- start 2024-04-27T03:30:09.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.395+0000 7f13c44e6640 1 -- start start 2024-04-27T03:30:09.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.395+0000 7f13c44e6640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f13bc1287d0 0x7f13bc130360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:09.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.395+0000 7f13c44e6640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13bc127b00 con 0x7f13bc127ce0 2024-04-27T03:30:09.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.395+0000 7f13c44e6640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f13bc0708e0 con 0x7f13bc1287d0 2024-04-27T03:30:09.403 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.395+0000 7f13c2282640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f13bc127ce0 0x7f13bc1280c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:41830/0 (socket says 172.21.15.89:41830) 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.395+0000 7f13c2282640 1 -- 172.21.15.89:0/2972652792 learned_addr learned my addr 172.21.15.89:0/2972652792 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.395+0000 7f13c1a81640 1 --2- 172.21.15.89:0/2972652792 >> v2:172.21.15.89:3300/0 conn(0x7f13bc1287d0 0x7f13bc130360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.395+0000 7f13c1280640 1 -- 172.21.15.89:0/2972652792 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3471014004 0 0) 0x7f13b4001490 con 0x7f13bc127ce0 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.396+0000 7f13c1280640 1 -- 172.21.15.89:0/2972652792 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f13a4003520 con 0x7f13bc127ce0 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.396+0000 7f13c1a81640 1 -- 172.21.15.89:0/2972652792 >> v1:172.21.15.89:6789/0 conn(0x7f13bc127ce0 legacy=0x7f13bc1280c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.396+0000 7f13c1a81640 1 -- 172.21.15.89:0/2972652792 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc098800 con 0x7f13bc1287d0 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.396+0000 7f13c1a81640 1 --2- 172.21.15.89:0/2972652792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13bc1287d0 0x7f13bc130360 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009750 tx=0x7f13b002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4be5057f2204e828 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.396+0000 7f13c1280640 1 -- 172.21.15.89:0/2972652792 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13b00316d0 con 0x7f13bc1287d0 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.396+0000 7f13c1280640 1 -- 172.21.15.89:0/2972652792 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b0037070 con 0x7f13bc1287d0 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.396+0000 7f13c1280640 1 -- 172.21.15.89:0/2972652792 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13b00319a0 con 0x7f13bc1287d0 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.397+0000 7f13c44e6640 1 -- 172.21.15.89:0/2972652792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13bc1287d0 msgr2=0x7f13bc130360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.397+0000 7f13c44e6640 1 --2- 172.21.15.89:0/2972652792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13bc1287d0 0x7f13bc130360 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f13b0009750 tx=0x7f13b002b840 comp rx=0 tx=0).stop 2024-04-27T03:30:09.404 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.397+0000 7f13c44e6640 1 -- 172.21.15.89:0/2972652792 shutdown_connections 2024-04-27T03:30:09.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.397+0000 7f13c44e6640 1 --2- 172.21.15.89:0/2972652792 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13bc1287d0 0x7f13bc130360 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:09.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.397+0000 7f13c44e6640 1 -- 172.21.15.89:0/2972652792 >> 172.21.15.89:0/2972652792 conn(0x7f13bc099680 msgr2=0x7f13bc099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:09.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.397+0000 7f13c44e6640 1 -- 172.21.15.89:0/2972652792 shutdown_connections 2024-04-27T03:30:09.405 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.397+0000 7f13c44e6640 1 -- 172.21.15.89:0/2972652792 wait complete. 2024-04-27T03:30:09.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.398+0000 7f13c44e6640 1 Processor -- start 2024-04-27T03:30:09.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.398+0000 7f13c44e6640 1 -- start start 2024-04-27T03:30:09.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.398+0000 7f13c44e6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13bc127ce0 0x7f13bc123c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:09.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.398+0000 7f13c44e6640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f13bc06faa0 con 0x7f13bc127ce0 2024-04-27T03:30:09.407 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.398+0000 7f13c2282640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13bc127ce0 0x7f13bc123c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.398+0000 7f13c2282640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13bc127ce0 0x7f13bc123c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:55506/0 (socket says 172.21.15.89:55506) 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.398+0000 7f13c2282640 1 -- 172.21.15.89:0/2495192285 learned_addr learned my addr 172.21.15.89:0/2495192285 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.398+0000 7f13c2282640 1 -- 172.21.15.89:0/2495192285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13bc125970 con 0x7f13bc127ce0 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.399+0000 7f13c2282640 1 --2- 172.21.15.89:0/2495192285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13bc127ce0 0x7f13bc123c30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f13b400cad0 tx=0x7f13b400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.399+0000 7f13baffd640 1 -- 172.21.15.89:0/2495192285 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13b4017070 con 0x7f13bc127ce0 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.399+0000 7f13baffd640 1 -- 172.21.15.89:0/2495192285 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13b400ab80 con 0x7f13bc127ce0 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.399+0000 7f13c44e6640 1 -- 172.21.15.89:0/2495192285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13bc1254d0 con 0x7f13bc127ce0 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.399+0000 7f13baffd640 1 -- 172.21.15.89:0/2495192285 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13b4012400 con 0x7f13bc127ce0 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.399+0000 7f13c44e6640 1 -- 172.21.15.89:0/2495192285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13bc124170 con 0x7f13bc127ce0 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.400+0000 7f13baffd640 1 -- 172.21.15.89:0/2495192285 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f13b401a070 con 0x7f13bc127ce0 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.400+0000 7f13c44e6640 1 -- 172.21.15.89:0/2495192285 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13bc062da0 con 0x7f13bc127ce0 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.400+0000 7f13baffd640 1 --2- 172.21.15.89:0/2495192285 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f13a8038060 0x7f13a803a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:09.408 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.400+0000 7f13baffd640 1 -- 172.21.15.89:0/2495192285 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f13b404acd0 con 0x7f13bc127ce0 2024-04-27T03:30:09.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.400+0000 7f13c1a81640 1 --2- 172.21.15.89:0/2495192285 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f13a8038060 0x7f13a803a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:09.409 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.401+0000 7f13c1a81640 1 --2- 172.21.15.89:0/2495192285 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f13a8038060 0x7f13a803a520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f13b0036000 tx=0x7f13b0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:09.410 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.404+0000 7f13baffd640 1 -- 172.21.15.89:0/2495192285 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f13bc125970 con 0x7f13bc127ce0 2024-04-27T03:30:09.577 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.571+0000 7f13c44e6640 1 -- 172.21.15.89:0/2495192285 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f13bc124d90 con 0x7f13a8038060 2024-04-27T03:30:09.578 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.573+0000 7f13baffd640 1 -- 172.21.15.89:0/2495192285 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+31861 (secure 0 0 0) 0x7f13bc124d90 con 0x7f13a8038060 2024-04-27T03:30:09.579 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:30:09.579 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-04-27T03:30:09.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.574+0000 7f13c44e6640 1 -- 172.21.15.89:0/2495192285 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f13a8038060 msgr2=0x7f13a803a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.574+0000 7f13c44e6640 1 --2- 172.21.15.89:0/2495192285 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f13a8038060 0x7f13a803a520 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f13b0036000 tx=0x7f13b0008030 comp rx=0 tx=0).stop 2024-04-27T03:30:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.574+0000 7f13c44e6640 1 -- 172.21.15.89:0/2495192285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13bc127ce0 msgr2=0x7f13bc123c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.574+0000 7f13c44e6640 1 --2- 172.21.15.89:0/2495192285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13bc127ce0 0x7f13bc123c30 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f13b400cad0 tx=0x7f13b400cf90 comp rx=0 tx=0).stop 2024-04-27T03:30:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.574+0000 7f13c44e6640 1 -- 172.21.15.89:0/2495192285 shutdown_connections 2024-04-27T03:30:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.574+0000 7f13c44e6640 1 --2- 172.21.15.89:0/2495192285 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f13a8038060 0x7f13a803a520 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.574+0000 7f13c44e6640 1 --2- 172.21.15.89:0/2495192285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13bc127ce0 0x7f13bc123c30 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.574+0000 7f13c44e6640 1 -- 172.21.15.89:0/2495192285 >> 172.21.15.89:0/2495192285 conn(0x7f13bc099680 msgr2=0x7f13bc06d420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.575+0000 7f13c44e6640 1 -- 172.21.15.89:0/2495192285 shutdown_connections 2024-04-27T03:30:09.581 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.575+0000 7f13c44e6640 1 -- 172.21.15.89:0/2495192285 wait complete. 2024-04-27T03:30:09.591 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2024-04-27T03:30:08.211800+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":6,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":110240,"kb_used_data":1288,"kb_used_omap":6,"kb_used_meta":108025,"kb_avail":377377120,"statfs":{"total":386547056640,"available":386434170880,"internally_reserved":0,"allocated":1318912,"data_stored":1025204,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":110618390},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-27T03:30:07.104109+0000","last_change":"2024-04-27T03:30:06.107954+0000","last_active":"2024-04-27T03:30:07.104109+0000","last_peered":"2024-04-27T03:30:07.104109+0000","last_clean":"2024-04-27T03:30:07.104109+0000","last_became_active":"2024-04-27T03:30:06.107600+0000","last_became_peered":"2024-04-27T03:30:06.107600+0000","last_unstale":"2024-04-27T03:30:07.104109+0000","last_undegraded":"2024-04-27T03:30:07.104109+0000","last_fullsized":"2024-04-27T03:30:07.104109+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:44.876194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-04-27T03:30:06.104702+0000","last_change":"2024-04-27T03:30:06.104067+0000","last_active":"2024-04-27T03:30:06.104702+0000","last_peered":"2024-04-27T03:30:06.104702+0000","last_clean":"2024-04-27T03:30:06.104702+0000","last_became_active":"2024-04-27T03:30:06.103759+0000","last_became_peered":"2024-04-27T03:30:06.103759+0000","last_unstale":"2024-04-27T03:30:06.104702+0000","last_undegraded":"2024-04-27T03:30:06.104702+0000","last_fullsized":"2024-04-27T03:30:06.104702+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:34:06.835461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-27T03:30:07.104105+0000","last_change":"2024-04-27T03:30:06.106816+0000","last_active":"2024-04-27T03:30:07.104105+0000","last_peered":"2024-04-27T03:30:07.104105+0000","last_clean":"2024-04-27T03:30:07.104105+0000","last_became_active":"2024-04-27T03:30:06.106475+0000","last_became_peered":"2024-04-27T03:30:06.106475+0000","last_unstale":"2024-04-27T03:30:07.104105+0000","last_undegraded":"2024-04-27T03:30:07.104105+0000","last_fullsized":"2024-04-27T03:30:07.104105+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:36.362210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":9,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-04-27T03:30:06.108522+0000","last_change":"2024-04-27T03:30:06.107861+0000","last_active":"2024-04-27T03:30:06.108522+0000","last_peered":"2024-04-27T03:30:06.108522+0000","last_clean":"2024-04-27T03:30:06.108522+0000","last_became_active":"2024-04-27T03:30:06.107594+0000","last_became_peered":"2024-04-27T03:30:06.107594+0000","last_unstale":"2024-04-27T03:30:06.108522+0000","last_undegraded":"2024-04-27T03:30:06.108522+0000","last_fullsized":"2024-04-27T03:30:06.108522+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:59.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-27T03:30:07.104075+0000","last_change":"2024-04-27T03:30:06.107772+0000","last_active":"2024-04-27T03:30:07.104075+0000","last_peered":"2024-04-27T03:30:07.104075+0000","last_clean":"2024-04-27T03:30:07.104075+0000","last_became_active":"2024-04-27T03:30:06.107356+0000","last_became_peered":"2024-04-27T03:30:06.107356+0000","last_unstale":"2024-04-27T03:30:07.104075+0000","last_undegraded":"2024-04-27T03:30:07.104075+0000","last_fullsized":"2024-04-27T03:30:07.104075+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:05:28.414626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-27T03:30:07.110169+0000","last_change":"2024-04-27T03:30:06.103722+0000","last_active":"2024-04-27T03:30:07.110169+0000","last_peered":"2024-04-27T03:30:07.110169+0000","last_clean":"2024-04-27T03:30:07.110169+0000","last_became_active":"2024-04-27T03:30:06.103441+0000","last_became_peered":"2024-04-27T03:30:06.103441+0000","last_unstale":"2024-04-27T03:30:07.110169+0000","last_undegraded":"2024-04-27T03:30:07.110169+0000","last_fullsized":"2024-04-27T03:30:07.110169+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:59.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-27T03:30:07.104066+0000","last_change":"2024-04-27T03:30:06.106890+0000","last_active":"2024-04-27T03:30:07.104066+0000","last_peered":"2024-04-27T03:30:07.104066+0000","last_clean":"2024-04-27T03:30:07.104066+0000","last_became_active":"2024-04-27T03:30:06.106474+0000","last_became_peered":"2024-04-27T03:30:06.106474+0000","last_unstale":"2024-04-27T03:30:07.104066+0000","last_undegraded":"2024-04-27T03:30:07.104066+0000","last_fullsized":"2024-04-27T03:30:07.104066+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:12.652758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":12,"state":"active+clean","last_fresh":"2024-04-27T03:30:07.104181+0000","last_change":"2024-04-27T03:30:06.106875+0000","last_active":"2024-04-27T03:30:07.104181+0000","last_peered":"2024-04-27T03:30:07.104181+0000","last_clean":"2024-04-27T03:30:07.104181+0000","last_became_active":"2024-04-27T03:30:06.106511+0000","last_became_peered":"2024-04-27T03:30:06.106511+0000","last_unstale":"2024-04-27T03:30:07.104181+0000","last_undegraded":"2024-04-27T03:30:07.104181+0000","last_fullsized":"2024-04-27T03:30:07.104181+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:56:59.055441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":159,"reported_epoch":11,"state":"active+clean","last_fresh":"2024-04-27T03:30:06.676693+0000","last_change":"2024-04-27T03:30:06.104151+0000","last_active":"2024-04-27T03:30:06.676693+0000","last_peered":"2024-04-27T03:30:06.676693+0000","last_clean":"2024-04-27T03:30:06.676693+0000","last_became_active":"2024-04-27T03:30:06.102893+0000","last_became_peered":"2024-04-27T03:30:06.102893+0000","last_unstale":"2024-04-27T03:30:06.676693+0000","last_undegraded":"2024-04-27T03:30:06.676693+0000","last_fullsized":"2024-04-27T03:30:06.676693+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:59.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705666,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27124,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344716,"statfs":{"total":96636764160,"available":96608989184,"internally_reserved":0,"allocated":98304,"data_stored":26661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705667,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":561152,"data_stored":485941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705666,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28000,"kb_used_data":548,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94343840,"statfs":{"total":96636764160,"available":96608092160,"internally_reserved":0,"allocated":561152,"data_stored":485941,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705668,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27116,"kb_used_data":96,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":94344724,"statfs":{"total":96636764160,"available":96608997376,"internally_reserved":0,"allocated":98304,"data_stored":26661,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T03:30:09.591 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-27T03:30:09.592 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-27T03:30:09.592 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2024-04-27T03:30:09.592 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2024-04-27T03:30:09.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.681+0000 7f9008406640 1 Processor -- start 2024-04-27T03:30:09.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.681+0000 7f9008406640 1 -- start start 2024-04-27T03:30:09.687 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.681+0000 7f9008406640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90001287d0 0x7f9000130360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:09.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.681+0000 7f9008406640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9000127b00 con 0x7f9000127ce0 2024-04-27T03:30:09.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.681+0000 7f9008406640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f90000708e0 con 0x7f90001287d0 2024-04-27T03:30:09.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.681+0000 7f90059a1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f90001287d0 0x7f9000130360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:09.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.681+0000 7f90061a2640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9000127ce0 0x7f90001280c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:60390/0 (socket says 172.21.15.89:60390) 2024-04-27T03:30:09.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.681+0000 7f90061a2640 1 -- 172.21.15.89:0/753965824 learned_addr learned my addr 172.21.15.89:0/753965824 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:30:09.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.682+0000 7f90059a1640 1 -- 172.21.15.89:0/753965824 >> v1:172.21.15.89:6789/0 conn(0x7f9000127ce0 legacy=0x7f90001280c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:09.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.682+0000 7f90059a1640 1 -- 172.21.15.89:0/753965824 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9000098800 con 0x7f90001287d0 2024-04-27T03:30:09.688 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.682+0000 7f90051a0640 1 -- 172.21.15.89:0/753965824 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1559710948 0 0) 0x7f8ffc001110 con 0x7f9000127ce0 2024-04-27T03:30:09.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.682+0000 7f90059a1640 1 --2- 172.21.15.89:0/753965824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90001287d0 0x7f9000130360 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009750 tx=0x7f8ff002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48e26bcd6e9ea343 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:09.691 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.683+0000 7f90051a0640 1 -- 172.21.15.89:0/753965824 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ff00316d0 con 0x7f90001287d0 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.683+0000 7f90051a0640 1 -- 172.21.15.89:0/753965824 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ff0037070 con 0x7f90001287d0 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.683+0000 7f90051a0640 1 -- 172.21.15.89:0/753965824 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ff00319c0 con 0x7f90001287d0 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.683+0000 7f9008406640 1 -- 172.21.15.89:0/753965824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90001287d0 msgr2=0x7f9000130360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.683+0000 7f9008406640 1 --2- 172.21.15.89:0/753965824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90001287d0 0x7f9000130360 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0009750 tx=0x7f8ff002b840 comp rx=0 tx=0).stop 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.683+0000 7f9008406640 1 -- 172.21.15.89:0/753965824 shutdown_connections 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.683+0000 7f9008406640 1 --2- 172.21.15.89:0/753965824 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f90001287d0 0x7f9000130360 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.683+0000 7f9008406640 1 -- 172.21.15.89:0/753965824 >> 172.21.15.89:0/753965824 conn(0x7f9000099680 msgr2=0x7f9000099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.683+0000 7f9008406640 1 -- 172.21.15.89:0/753965824 shutdown_connections 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.683+0000 7f9008406640 1 -- 172.21.15.89:0/753965824 wait complete. 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.684+0000 7f9008406640 1 Processor -- start 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.684+0000 7f9008406640 1 -- start start 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.684+0000 7f9008406640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9000127ce0 0x7f90001229b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:09.692 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.684+0000 7f9008406640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f900006faa0 con 0x7f9000127ce0 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.684+0000 7f90061a2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9000127ce0 0x7f90001229b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.684+0000 7f90061a2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9000127ce0 0x7f90001229b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:38320/0 (socket says 172.21.15.89:38320) 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.684+0000 7f90061a2640 1 -- 172.21.15.89:0/1314170656 learned_addr learned my addr 172.21.15.89:0/1314170656 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.685+0000 7f90061a2640 1 -- 172.21.15.89:0/1314170656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9000124900 con 0x7f9000127ce0 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.685+0000 7f90061a2640 1 --2- 172.21.15.89:0/1314170656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9000127ce0 0x7f90001229b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c700 tx=0x7f8ffc00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.685+0000 7f8ff6ffd640 1 -- 172.21.15.89:0/1314170656 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ffc018070 con 0x7f9000127ce0 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.685+0000 7f8ff6ffd640 1 -- 172.21.15.89:0/1314170656 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ffc0070d0 con 0x7f9000127ce0 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.685+0000 7f8ff6ffd640 1 -- 172.21.15.89:0/1314170656 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ffc0073f0 con 0x7f9000127ce0 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.685+0000 7f9008406640 1 -- 172.21.15.89:0/1314170656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900008a270 con 0x7f9000127ce0 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.685+0000 7f9008406640 1 -- 172.21.15.89:0/1314170656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f90001230d0 con 0x7f9000127ce0 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.686+0000 7f8ff6ffd640 1 -- 172.21.15.89:0/1314170656 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f8ffc01b070 con 0x7f9000127ce0 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.686+0000 7f8ff6ffd640 1 --2- 172.21.15.89:0/1314170656 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fe4038060 0x7f8fe403a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.686+0000 7f9008406640 1 -- 172.21.15.89:0/1314170656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9000062da0 con 0x7f9000127ce0 2024-04-27T03:30:09.693 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.686+0000 7f8ff6ffd640 1 -- 172.21.15.89:0/1314170656 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f8ffc04a2c0 con 0x7f9000127ce0 2024-04-27T03:30:09.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.686+0000 7f90059a1640 1 --2- 172.21.15.89:0/1314170656 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fe4038060 0x7f8fe403a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:09.694 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.687+0000 7f90059a1640 1 --2- 172.21.15.89:0/1314170656 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fe4038060 0x7f8fe403a520 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0036000 tx=0x7f8ff0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:09.696 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.691+0000 7f8ff6ffd640 1 -- 172.21.15.89:0/1314170656 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9000127b00 con 0x7f9000127ce0 2024-04-27T03:30:09.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.899+0000 7f9008406640 1 -- 172.21.15.89:0/1314170656 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7f900009c340 con 0x7f9000127ce0 2024-04-27T03:30:09.905 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.899+0000 7f8ff6ffd640 1 -- 172.21.15.89:0/1314170656 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7f8ffc0047e0 con 0x7f9000127ce0 2024-04-27T03:30:09.905 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:30:09.905 INFO:teuthology.orchestra.run.smithi089.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-27T03:30:09.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.901+0000 7f9008406640 1 -- 172.21.15.89:0/1314170656 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fe4038060 msgr2=0x7f8fe403a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:09.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.901+0000 7f9008406640 1 --2- 172.21.15.89:0/1314170656 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fe4038060 0x7f8fe403a520 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f8ff0036000 tx=0x7f8ff0008030 comp rx=0 tx=0).stop 2024-04-27T03:30:09.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.901+0000 7f9008406640 1 -- 172.21.15.89:0/1314170656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9000127ce0 msgr2=0x7f90001229b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:09.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.901+0000 7f9008406640 1 --2- 172.21.15.89:0/1314170656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9000127ce0 0x7f90001229b0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00c700 tx=0x7f8ffc00cbc0 comp rx=0 tx=0).stop 2024-04-27T03:30:09.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.901+0000 7f9008406640 1 -- 172.21.15.89:0/1314170656 shutdown_connections 2024-04-27T03:30:09.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.901+0000 7f9008406640 1 --2- 172.21.15.89:0/1314170656 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fe4038060 0x7f8fe403a520 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:09.907 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.901+0000 7f9008406640 1 --2- 172.21.15.89:0/1314170656 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9000127ce0 0x7f90001229b0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:09.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.901+0000 7f9008406640 1 -- 172.21.15.89:0/1314170656 >> 172.21.15.89:0/1314170656 conn(0x7f9000099680 msgr2=0x7f900009b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:09.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.901+0000 7f9008406640 1 -- 172.21.15.89:0/1314170656 shutdown_connections 2024-04-27T03:30:09.908 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:30:09.901+0000 7f9008406640 1 -- 172.21.15.89:0/1314170656 wait complete. 2024-04-27T03:30:09.918 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2024-04-27T03:30:09.918 INFO:teuthology.run_tasks:Running task rgw... 2024-04-27T03:30:09.933 DEBUG:tasks.rgw:config is {'client.0': None} 2024-04-27T03:30:09.933 DEBUG:tasks.rgw:client list is dict_keys(['client.0']) 2024-04-27T03:30:09.933 INFO:tasks.rgw:Creating data pools 2024-04-27T03:30:09.933 DEBUG:tasks.rgw:Obtaining remote for client client.0 2024-04-27T03:30:09.933 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph osd pool create default.rgw.buckets.data 64 64 --cluster ceph 2024-04-27T03:30:10.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.037+0000 7f9b5a68c640 1 Processor -- start 2024-04-27T03:30:10.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.037+0000 7f9b5a68c640 1 -- start start 2024-04-27T03:30:10.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.037+0000 7f9b5a68c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9b54118960 0x7f9b5411ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:10.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.037+0000 7f9b5a68c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f9b5410eb40 con 0x7f9b5410dfc0 2024-04-27T03:30:10.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.037+0000 7f9b5a68c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f9b5411c2a0 con 0x7f9b54118960 2024-04-27T03:30:10.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.038+0000 7f9b537fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9b54118960 0x7f9b5411ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:10.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.038+0000 7f9b53fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f9b5410dfc0 0x7f9b5410e3a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:47480/0 (socket says 172.21.15.165:47480) 2024-04-27T03:30:10.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.038+0000 7f9b537fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f9b54118960 0x7f9b5411ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:35848/0 (socket says 172.21.15.165:35848) 2024-04-27T03:30:10.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.038+0000 7f9b53fff640 1 -- 172.21.15.165:0/1881309496 learned_addr learned my addr 172.21.15.165:0/1881309496 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:10.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.038+0000 7f9b537fe640 1 -- 172.21.15.165:0/1881309496 >> v1:172.21.15.89:6789/0 conn(0x7f9b5410dfc0 legacy=0x7f9b5410e3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:10.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.038+0000 7f9b537fe640 1 -- 172.21.15.165:0/1881309496 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b5411c420 con 0x7f9b54118960 2024-04-27T03:30:10.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.038+0000 7f9b52ffd640 1 -- 172.21.15.165:0/1881309496 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1895567721 0 0) 0x7f9b5410eb40 con 0x7f9b5410dfc0 2024-04-27T03:30:10.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.039+0000 7f9b537fe640 1 --2- 172.21.15.165:0/1881309496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b54118960 0x7f9b5411ad50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009720 tx=0x7f9b4802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5de8baf93cf51f8c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:10.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.039+0000 7f9b52ffd640 1 -- 172.21.15.165:0/1881309496 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b480316d0 con 0x7f9b54118960 2024-04-27T03:30:10.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.039+0000 7f9b52ffd640 1 -- 172.21.15.165:0/1881309496 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b48037070 con 0x7f9b54118960 2024-04-27T03:30:10.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.039+0000 7f9b52ffd640 1 -- 172.21.15.165:0/1881309496 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b480319c0 con 0x7f9b54118960 2024-04-27T03:30:10.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.039+0000 7f9b5a68c640 1 -- 172.21.15.165:0/1881309496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b54118960 msgr2=0x7f9b5411ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.039+0000 7f9b5a68c640 1 --2- 172.21.15.165:0/1881309496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b54118960 0x7f9b5411ad50 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f9b48009720 tx=0x7f9b4802b840 comp rx=0 tx=0).stop 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.040+0000 7f9b5a68c640 1 -- 172.21.15.165:0/1881309496 shutdown_connections 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.040+0000 7f9b5a68c640 1 --2- 172.21.15.165:0/1881309496 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b54118960 0x7f9b5411ad50 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.040+0000 7f9b5a68c640 1 -- 172.21.15.165:0/1881309496 >> 172.21.15.165:0/1881309496 conn(0x7f9b5406f2b0 msgr2=0x7f9b540716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.040+0000 7f9b5a68c640 1 -- 172.21.15.165:0/1881309496 shutdown_connections 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.040+0000 7f9b5a68c640 1 -- 172.21.15.165:0/1881309496 wait complete. 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.040+0000 7f9b5a68c640 1 Processor -- start 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.041+0000 7f9b5a68c640 1 -- start start 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.041+0000 7f9b5a68c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5410dfc0 0x7f9b541aa0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.041+0000 7f9b5a68c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f9b5411b7b0 con 0x7f9b5410dfc0 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.041+0000 7f9b53fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5410dfc0 0x7f9b541aa0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.041+0000 7f9b53fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5410dfc0 0x7f9b541aa0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:35854/0 (socket says 172.21.15.165:35854) 2024-04-27T03:30:10.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.041+0000 7f9b53fff640 1 -- 172.21.15.165:0/730195223 learned_addr learned my addr 172.21.15.165:0/730195223 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:10.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.041+0000 7f9b53fff640 1 -- 172.21.15.165:0/730195223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b541aa670 con 0x7f9b5410dfc0 2024-04-27T03:30:10.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.041+0000 7f9b53fff640 1 --2- 172.21.15.165:0/730195223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5410dfc0 0x7f9b541aa0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400c3e0 tx=0x7f9b4400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:10.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.041+0000 7f9b50ff9640 1 -- 172.21.15.165:0/730195223 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b44007410 con 0x7f9b5410dfc0 2024-04-27T03:30:10.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.042+0000 7f9b50ff9640 1 -- 172.21.15.165:0/730195223 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9b440075d0 con 0x7f9b5410dfc0 2024-04-27T03:30:10.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.042+0000 7f9b5a68c640 1 -- 172.21.15.165:0/730195223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b541aa900 con 0x7f9b5410dfc0 2024-04-27T03:30:10.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.042+0000 7f9b50ff9640 1 -- 172.21.15.165:0/730195223 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9b44004950 con 0x7f9b5410dfc0 2024-04-27T03:30:10.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.042+0000 7f9b5a68c640 1 -- 172.21.15.165:0/730195223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b541a4310 con 0x7f9b5410dfc0 2024-04-27T03:30:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.042+0000 7f9b50ff9640 1 -- 172.21.15.165:0/730195223 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9b44011070 con 0x7f9b5410dfc0 2024-04-27T03:30:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.042+0000 7f9b5a68c640 1 -- 172.21.15.165:0/730195223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b54062da0 con 0x7f9b5410dfc0 2024-04-27T03:30:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.043+0000 7f9b50ff9640 1 --2- 172.21.15.165:0/730195223 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9b40038060 0x7f9b4003a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.043+0000 7f9b50ff9640 1 -- 172.21.15.165:0/730195223 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(12..12 src has 1..12) ==== 3482+0+0 (secure 0 0 0) 0x7f9b4404abf0 con 0x7f9b5410dfc0 2024-04-27T03:30:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.043+0000 7f9b537fe640 1 --2- 172.21.15.165:0/730195223 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9b40038060 0x7f9b4003a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:10.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.043+0000 7f9b537fe640 1 --2- 172.21.15.165:0/730195223 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9b40038060 0x7f9b4003a520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b48036000 tx=0x7f9b48008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:10.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.047+0000 7f9b50ff9640 1 -- 172.21.15.165:0/730195223 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f9b44011c60 con 0x7f9b5410dfc0 2024-04-27T03:30:10.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.220+0000 7f9b5a68c640 1 -- 172.21.15.165:0/730195223 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64} v 0) -- 0x7f9b541aab30 con 0x7f9b5410dfc0 2024-04-27T03:30:10.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.341+0000 7f9b50ff9640 1 -- 172.21.15.165:0/730195223 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.data", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.data' created v13) ==== 167+0+0 (secure 0 0 0) 0x7f9b4401c990 con 0x7f9b5410dfc0 2024-04-27T03:30:10.347 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'default.rgw.buckets.data' created 2024-04-27T03:30:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.344+0000 7f9b5a68c640 1 -- 172.21.15.165:0/730195223 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9b40038060 msgr2=0x7f9b4003a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.344+0000 7f9b5a68c640 1 --2- 172.21.15.165:0/730195223 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9b40038060 0x7f9b4003a520 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f9b48036000 tx=0x7f9b48008030 comp rx=0 tx=0).stop 2024-04-27T03:30:10.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.344+0000 7f9b5a68c640 1 -- 172.21.15.165:0/730195223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5410dfc0 msgr2=0x7f9b541aa0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:10.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.344+0000 7f9b5a68c640 1 --2- 172.21.15.165:0/730195223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5410dfc0 0x7f9b541aa0a0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9b4400c3e0 tx=0x7f9b4400c8a0 comp rx=0 tx=0).stop 2024-04-27T03:30:10.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.344+0000 7f9b5a68c640 1 -- 172.21.15.165:0/730195223 shutdown_connections 2024-04-27T03:30:10.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.344+0000 7f9b5a68c640 1 --2- 172.21.15.165:0/730195223 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9b40038060 0x7f9b4003a520 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:10.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.344+0000 7f9b5a68c640 1 --2- 172.21.15.165:0/730195223 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f9b5410dfc0 0x7f9b541aa0a0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:10.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.344+0000 7f9b5a68c640 1 -- 172.21.15.165:0/730195223 >> 172.21.15.165:0/730195223 conn(0x7f9b5406f2b0 msgr2=0x7f9b540714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:10.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.344+0000 7f9b5a68c640 1 -- 172.21.15.165:0/730195223 shutdown_connections 2024-04-27T03:30:10.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.344+0000 7f9b5a68c640 1 -- 172.21.15.165:0/730195223 wait complete. 2024-04-27T03:30:10.367 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2024-04-27T03:30:10.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.458+0000 7f139079d640 1 Processor -- start 2024-04-27T03:30:10.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.458+0000 7f139079d640 1 -- start start 2024-04-27T03:30:10.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.458+0000 7f139079d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f138811aaf0 0x7f138811cee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:10.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.458+0000 7f139079d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f138811d420 con 0x7f1388116c40 2024-04-27T03:30:10.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.458+0000 7f139079d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f138811e600 con 0x7f138811aaf0 2024-04-27T03:30:10.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.459+0000 7f138dd38640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f138811aaf0 0x7f138811cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:10.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.459+0000 7f138dd38640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f138811aaf0 0x7f138811cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:35866/0 (socket says 172.21.15.165:35866) 2024-04-27T03:30:10.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.459+0000 7f138dd38640 1 -- 172.21.15.165:0/3242946149 learned_addr learned my addr 172.21.15.165:0/3242946149 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:10.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.459+0000 7f138dd38640 1 -- 172.21.15.165:0/3242946149 >> v1:172.21.15.89:6789/0 conn(0x7f1388116c40 legacy=0x7f1388117020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:10.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.459+0000 7f138dd38640 1 -- 172.21.15.165:0/3242946149 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f138811e7b0 con 0x7f138811aaf0 2024-04-27T03:30:10.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.459+0000 7f138d537640 1 -- 172.21.15.165:0/3242946149 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1427888231 0 0) 0x7f138811d420 con 0x7f1388116c40 2024-04-27T03:30:10.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.460+0000 7f138dd38640 1 --2- 172.21.15.165:0/3242946149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f138811aaf0 0x7f138811cee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f13780097f0 tx=0x7f137802b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7133409df2e3027 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:10.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.460+0000 7f138d537640 1 -- 172.21.15.165:0/3242946149 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1378035020 con 0x7f138811aaf0 2024-04-27T03:30:10.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.460+0000 7f138d537640 1 -- 172.21.15.165:0/3242946149 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13780317a0 con 0x7f138811aaf0 2024-04-27T03:30:10.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.460+0000 7f138d537640 1 -- 172.21.15.165:0/3242946149 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1378031aa0 con 0x7f138811aaf0 2024-04-27T03:30:10.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.461+0000 7f139079d640 1 -- 172.21.15.165:0/3242946149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f138811aaf0 msgr2=0x7f138811cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:10.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.461+0000 7f139079d640 1 --2- 172.21.15.165:0/3242946149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f138811aaf0 0x7f138811cee0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f13780097f0 tx=0x7f137802b910 comp rx=0 tx=0).stop 2024-04-27T03:30:10.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.461+0000 7f139079d640 1 -- 172.21.15.165:0/3242946149 shutdown_connections 2024-04-27T03:30:10.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.461+0000 7f139079d640 1 --2- 172.21.15.165:0/3242946149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f138811aaf0 0x7f138811cee0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:10.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.461+0000 7f139079d640 1 -- 172.21.15.165:0/3242946149 >> 172.21.15.165:0/3242946149 conn(0x7f138806e700 msgr2=0x7f138810fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:10.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.461+0000 7f139079d640 1 -- 172.21.15.165:0/3242946149 shutdown_connections 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.461+0000 7f139079d640 1 -- 172.21.15.165:0/3242946149 wait complete. 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.461+0000 7f139079d640 1 Processor -- start 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.461+0000 7f139079d640 1 -- start start 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.461+0000 7f139079d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1388116c40 0x7f13881ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.461+0000 7f139079d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f138811dad0 con 0x7f1388116c40 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.462+0000 7f138e539640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1388116c40 0x7f13881ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.462+0000 7f138e539640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1388116c40 0x7f13881ac2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:35874/0 (socket says 172.21.15.165:35874) 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.462+0000 7f138e539640 1 -- 172.21.15.165:0/472948947 learned_addr learned my addr 172.21.15.165:0/472948947 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.462+0000 7f138e539640 1 -- 172.21.15.165:0/472948947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13881ac8c0 con 0x7f1388116c40 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.462+0000 7f138e539640 1 --2- 172.21.15.165:0/472948947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1388116c40 0x7f13881ac2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f138400c3e0 tx=0x7f138400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.462+0000 7f137effd640 1 -- 172.21.15.165:0/472948947 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1384017070 con 0x7f1388116c40 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.462+0000 7f137effd640 1 -- 172.21.15.165:0/472948947 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1384005430 con 0x7f1388116c40 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.462+0000 7f139079d640 1 -- 172.21.15.165:0/472948947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13881acb50 con 0x7f1388116c40 2024-04-27T03:30:10.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.462+0000 7f137effd640 1 -- 172.21.15.165:0/472948947 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1384003760 con 0x7f1388116c40 2024-04-27T03:30:10.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.462+0000 7f139079d640 1 -- 172.21.15.165:0/472948947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13881a65e0 con 0x7f1388116c40 2024-04-27T03:30:10.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.463+0000 7f137effd640 1 -- 172.21.15.165:0/472948947 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f138401a070 con 0x7f1388116c40 2024-04-27T03:30:10.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.463+0000 7f139079d640 1 -- 172.21.15.165:0/472948947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1388062da0 con 0x7f1388116c40 2024-04-27T03:30:10.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.463+0000 7f137effd640 1 --2- 172.21.15.165:0/472948947 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f136c038010 0x7f136c03a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:10.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.463+0000 7f137effd640 1 -- 172.21.15.165:0/472948947 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(13..13 src has 1..13) ==== 3845+0+0 (secure 0 0 0) 0x7f138404a5a0 con 0x7f1388116c40 2024-04-27T03:30:10.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.463+0000 7f138dd38640 1 --2- 172.21.15.165:0/472948947 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f136c038010 0x7f136c03a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:10.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.464+0000 7f138dd38640 1 --2- 172.21.15.165:0/472948947 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f136c038010 0x7f136c03a4d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1378036000 tx=0x7f1378008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:10.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.468+0000 7f137effd640 1 -- 172.21.15.165:0/472948947 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f13840127a0 con 0x7f1388116c40 2024-04-27T03:30:10.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:10.643+0000 7f139079d640 1 -- 172.21.15.165:0/472948947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"} v 0) -- 0x7f13881acd80 con 0x7f1388116c40 2024-04-27T03:30:11.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.342+0000 7f137effd640 1 -- 172.21.15.165:0/472948947 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.data", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.data' v14) ==== 185+0+0 (secure 0 0 0) 0x7f138404a350 con 0x7f1388116c40 2024-04-27T03:30:11.348 INFO:teuthology.orchestra.run.smithi165.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2024-04-27T03:30:11.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.346+0000 7f139079d640 1 -- 172.21.15.165:0/472948947 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f136c038010 msgr2=0x7f136c03a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:11.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.346+0000 7f139079d640 1 --2- 172.21.15.165:0/472948947 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f136c038010 0x7f136c03a4d0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1378036000 tx=0x7f1378008030 comp rx=0 tx=0).stop 2024-04-27T03:30:11.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.346+0000 7f139079d640 1 -- 172.21.15.165:0/472948947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1388116c40 msgr2=0x7f13881ac2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:11.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.346+0000 7f139079d640 1 --2- 172.21.15.165:0/472948947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1388116c40 0x7f13881ac2f0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f138400c3e0 tx=0x7f138400c8a0 comp rx=0 tx=0).stop 2024-04-27T03:30:11.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.346+0000 7f139079d640 1 -- 172.21.15.165:0/472948947 shutdown_connections 2024-04-27T03:30:11.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.346+0000 7f139079d640 1 --2- 172.21.15.165:0/472948947 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f136c038010 0x7f136c03a4d0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f1378036000 tx=0x7f1378008030 comp rx=0 tx=0).stop 2024-04-27T03:30:11.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.346+0000 7f139079d640 1 --2- 172.21.15.165:0/472948947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f1388116c40 0x7f13881ac2f0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:11.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.346+0000 7f139079d640 1 -- 172.21.15.165:0/472948947 >> 172.21.15.165:0/472948947 conn(0x7f138806e700 msgr2=0x7f13881157a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:11.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.347+0000 7f139079d640 1 -- 172.21.15.165:0/472948947 shutdown_connections 2024-04-27T03:30:11.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.348+0000 7f139079d640 1 -- 172.21.15.165:0/472948947 wait complete. 2024-04-27T03:30:11.376 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2024-04-27T03:30:11.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.468+0000 7f13cbddb640 1 Processor -- start 2024-04-27T03:30:11.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.469+0000 7f13cbddb640 1 -- start start 2024-04-27T03:30:11.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.469+0000 7f13cbddb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f13c410dfc0 0x7f13c410e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:11.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.469+0000 7f13cbddb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f13c411b110 con 0x7f13c410e8e0 2024-04-27T03:30:11.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.469+0000 7f13cbddb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f13c411c2f0 con 0x7f13c410dfc0 2024-04-27T03:30:11.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.469+0000 7f13c9b77640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f13c410dfc0 0x7f13c410e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:11.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.469+0000 7f13c9376640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f13c410e8e0 0x7f13c4119960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:47506/0 (socket says 172.21.15.165:47506) 2024-04-27T03:30:11.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.469+0000 7f13c9376640 1 -- 172.21.15.165:0/1232295183 learned_addr learned my addr 172.21.15.165:0/1232295183 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:11.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.469+0000 7f13c9b77640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f13c410dfc0 0x7f13c410e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:35880/0 (socket says 172.21.15.165:35880) 2024-04-27T03:30:11.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.470+0000 7f13c9b77640 1 -- 172.21.15.165:0/1232295183 >> v1:172.21.15.89:6789/0 conn(0x7f13c410e8e0 legacy=0x7f13c4119960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:11.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.470+0000 7f13c9b77640 1 -- 172.21.15.165:0/1232295183 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c411c470 con 0x7f13c410dfc0 2024-04-27T03:30:11.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.470+0000 7f13c9b77640 1 --2- 172.21.15.165:0/1232295183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13c410dfc0 0x7f13c410e3a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009720 tx=0x7f13c002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a21a8e5272953d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:11.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.470+0000 7f13c8b75640 1 -- 172.21.15.165:0/1232295183 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13c00316d0 con 0x7f13c410dfc0 2024-04-27T03:30:11.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.471+0000 7f13c8b75640 1 -- 172.21.15.165:0/1232295183 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c0037070 con 0x7f13c410dfc0 2024-04-27T03:30:11.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.471+0000 7f13c8b75640 1 -- 172.21.15.165:0/1232295183 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13c00319c0 con 0x7f13c410dfc0 2024-04-27T03:30:11.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.471+0000 7f13cbddb640 1 -- 172.21.15.165:0/1232295183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13c410dfc0 msgr2=0x7f13c410e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:11.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.471+0000 7f13cbddb640 1 --2- 172.21.15.165:0/1232295183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13c410dfc0 0x7f13c410e3a0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f13c0009720 tx=0x7f13c002b840 comp rx=0 tx=0).stop 2024-04-27T03:30:11.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.471+0000 7f13cbddb640 1 -- 172.21.15.165:0/1232295183 shutdown_connections 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.471+0000 7f13cbddb640 1 --2- 172.21.15.165:0/1232295183 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13c410dfc0 0x7f13c410e3a0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.471+0000 7f13cbddb640 1 -- 172.21.15.165:0/1232295183 >> 172.21.15.165:0/1232295183 conn(0x7f13c406f2b0 msgr2=0x7f13c40716d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.471+0000 7f13cbddb640 1 -- 172.21.15.165:0/1232295183 shutdown_connections 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.471+0000 7f13cbddb640 1 -- 172.21.15.165:0/1232295183 wait complete. 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.472+0000 7f13cbddb640 1 Processor -- start 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.472+0000 7f13cbddb640 1 -- start start 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.472+0000 7f13cbddb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13c410dfc0 0x7f13c410f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.472+0000 7f13cbddb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f13c411b800 con 0x7f13c410dfc0 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.472+0000 7f13c9b77640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13c410dfc0 0x7f13c410f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.472+0000 7f13c9b77640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13c410dfc0 0x7f13c410f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:35888/0 (socket says 172.21.15.165:35888) 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.472+0000 7f13c9b77640 1 -- 172.21.15.165:0/2221565638 learned_addr learned my addr 172.21.15.165:0/2221565638 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.473+0000 7f13c9b77640 1 -- 172.21.15.165:0/2221565638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c410fd00 con 0x7f13c410dfc0 2024-04-27T03:30:11.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.473+0000 7f13c9b77640 1 --2- 172.21.15.165:0/2221565638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13c410dfc0 0x7f13c410f7c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f13c002b880 tx=0x7f13c00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:11.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.473+0000 7f13ba7fc640 1 -- 172.21.15.165:0/2221565638 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13c0032770 con 0x7f13c410dfc0 2024-04-27T03:30:11.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.473+0000 7f13ba7fc640 1 -- 172.21.15.165:0/2221565638 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f13c0032910 con 0x7f13c410dfc0 2024-04-27T03:30:11.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.473+0000 7f13cbddb640 1 -- 172.21.15.165:0/2221565638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c410ff30 con 0x7f13c410dfc0 2024-04-27T03:30:11.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.473+0000 7f13ba7fc640 1 -- 172.21.15.165:0/2221565638 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f13c0032bf0 con 0x7f13c410dfc0 2024-04-27T03:30:11.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.473+0000 7f13cbddb640 1 -- 172.21.15.165:0/2221565638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c4110420 con 0x7f13c410dfc0 2024-04-27T03:30:11.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.474+0000 7f13ba7fc640 1 -- 172.21.15.165:0/2221565638 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f13c004c060 con 0x7f13c410dfc0 2024-04-27T03:30:11.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.474+0000 7f13cbddb640 1 -- 172.21.15.165:0/2221565638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c4062da0 con 0x7f13c410dfc0 2024-04-27T03:30:11.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.474+0000 7f13ba7fc640 1 --2- 172.21.15.165:0/2221565638 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f13a8038010 0x7f13a803a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:11.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.474+0000 7f13ba7fc640 1 -- 172.21.15.165:0/2221565638 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 3856+0+0 (secure 0 0 0) 0x7f13c002f050 con 0x7f13c410dfc0 2024-04-27T03:30:11.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.474+0000 7f13c9376640 1 --2- 172.21.15.165:0/2221565638 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f13a8038010 0x7f13a803a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:11.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.475+0000 7f13c9376640 1 --2- 172.21.15.165:0/2221565638 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f13a8038010 0x7f13a803a4d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f13c410ed30 tx=0x7f13b4007a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:11.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.479+0000 7f13ba7fc640 1 -- 172.21.15.165:0/2221565638 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f13c0040070 con 0x7f13c410dfc0 2024-04-27T03:30:11.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:11.654+0000 7f13cbddb640 1 -- 172.21.15.165:0/2221565638 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64} v 0) -- 0x7f13c410e8e0 con 0x7f13c410dfc0 2024-04-27T03:30:12.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.349+0000 7f13ba7fc640 1 -- 172.21.15.165:0/2221565638 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "default.rgw.buckets.index", "pg_num": 64, "pgp_num": 64}]=0 pool 'default.rgw.buckets.index' created v15) ==== 169+0+0 (secure 0 0 0) 0x7f13c004de20 con 0x7f13c410dfc0 2024-04-27T03:30:12.354 INFO:teuthology.orchestra.run.smithi165.stderr:pool 'default.rgw.buckets.index' created 2024-04-27T03:30:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.351+0000 7f13cbddb640 1 -- 172.21.15.165:0/2221565638 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f13a8038010 msgr2=0x7f13a803a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.351+0000 7f13cbddb640 1 --2- 172.21.15.165:0/2221565638 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f13a8038010 0x7f13a803a4d0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f13c410ed30 tx=0x7f13b4007a80 comp rx=0 tx=0).stop 2024-04-27T03:30:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.351+0000 7f13cbddb640 1 -- 172.21.15.165:0/2221565638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13c410dfc0 msgr2=0x7f13c410f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.351+0000 7f13cbddb640 1 --2- 172.21.15.165:0/2221565638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13c410dfc0 0x7f13c410f7c0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f13c002b880 tx=0x7f13c00046c0 comp rx=0 tx=0).stop 2024-04-27T03:30:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.351+0000 7f13cbddb640 1 -- 172.21.15.165:0/2221565638 shutdown_connections 2024-04-27T03:30:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.351+0000 7f13cbddb640 1 --2- 172.21.15.165:0/2221565638 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f13a8038010 0x7f13a803a4d0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.351+0000 7f13cbddb640 1 --2- 172.21.15.165:0/2221565638 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f13c410dfc0 0x7f13c410f7c0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.351+0000 7f13cbddb640 1 -- 172.21.15.165:0/2221565638 >> 172.21.15.165:0/2221565638 conn(0x7f13c406f2b0 msgr2=0x7f13c411a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.351+0000 7f13cbddb640 1 -- 172.21.15.165:0/2221565638 shutdown_connections 2024-04-27T03:30:12.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.351+0000 7f13cbddb640 1 -- 172.21.15.165:0/2221565638 wait complete. 2024-04-27T03:30:12.374 DEBUG:teuthology.orchestra.run.smithi165:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2024-04-27T03:30:12.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.467+0000 7fbd9884d640 1 Processor -- start 2024-04-27T03:30:12.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.467+0000 7fbd9884d640 1 -- start start 2024-04-27T03:30:12.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.468+0000 7fbd9884d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbd9411aa80 0x7fbd9411ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:12.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.468+0000 7fbd9884d640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fbd9411d3b0 con 0x7fbd9406cea0 2024-04-27T03:30:12.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.468+0000 7fbd9884d640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fbd9411e590 con 0x7fbd9411aa80 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.468+0000 7fbd91d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbd9411aa80 0x7fbd9411ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.468+0000 7fbd9259c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fbd9406cea0 0x7fbd9406d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:47508/0 (socket says 172.21.15.165:47508) 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.468+0000 7fbd91d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fbd9411aa80 0x7fbd9411ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:35898/0 (socket says 172.21.15.165:35898) 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.468+0000 7fbd9259c640 1 -- 172.21.15.165:0/1212138814 learned_addr learned my addr 172.21.15.165:0/1212138814 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.468+0000 7fbd91d9b640 1 -- 172.21.15.165:0/1212138814 >> v1:172.21.15.89:6789/0 conn(0x7fbd9406cea0 legacy=0x7fbd9406d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.468+0000 7fbd91d9b640 1 -- 172.21.15.165:0/1212138814 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd9411e710 con 0x7fbd9411aa80 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.469+0000 7fbd91d9b640 1 --2- 172.21.15.165:0/1212138814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9411aa80 0x7fbd9411ce70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009720 tx=0x7fbd8802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3806736f7f1afcd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.469+0000 7fbd9159a640 1 -- 172.21.15.165:0/1212138814 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd880316d0 con 0x7fbd9411aa80 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.469+0000 7fbd9159a640 1 -- 172.21.15.165:0/1212138814 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd88037070 con 0x7fbd9411aa80 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.469+0000 7fbd9159a640 1 -- 172.21.15.165:0/1212138814 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd880319c0 con 0x7fbd9411aa80 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.470+0000 7fbd9884d640 1 -- 172.21.15.165:0/1212138814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9411aa80 msgr2=0x7fbd9411ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.470+0000 7fbd9884d640 1 --2- 172.21.15.165:0/1212138814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9411aa80 0x7fbd9411ce70 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbd88009720 tx=0x7fbd8802b840 comp rx=0 tx=0).stop 2024-04-27T03:30:12.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.470+0000 7fbd9884d640 1 -- 172.21.15.165:0/1212138814 shutdown_connections 2024-04-27T03:30:12.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.470+0000 7fbd9884d640 1 --2- 172.21.15.165:0/1212138814 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9411aa80 0x7fbd9411ce70 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:12.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.470+0000 7fbd9884d640 1 -- 172.21.15.165:0/1212138814 >> 172.21.15.165:0/1212138814 conn(0x7fbd94070e10 msgr2=0x7fbd94073230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:12.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.470+0000 7fbd9884d640 1 -- 172.21.15.165:0/1212138814 shutdown_connections 2024-04-27T03:30:12.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.470+0000 7fbd9884d640 1 -- 172.21.15.165:0/1212138814 wait complete. 2024-04-27T03:30:12.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.471+0000 7fbd9884d640 1 Processor -- start 2024-04-27T03:30:12.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.471+0000 7fbd9884d640 1 -- start start 2024-04-27T03:30:12.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.471+0000 7fbd9884d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9406cea0 0x7fbd941ac470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.471+0000 7fbd9884d640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fbd9411daa0 con 0x7fbd9406cea0 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.471+0000 7fbd9259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9406cea0 0x7fbd941ac470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.471+0000 7fbd9259c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9406cea0 0x7fbd941ac470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:35914/0 (socket says 172.21.15.165:35914) 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.471+0000 7fbd9259c640 1 -- 172.21.15.165:0/638076570 learned_addr learned my addr 172.21.15.165:0/638076570 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.471+0000 7fbd9259c640 1 -- 172.21.15.165:0/638076570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd941aca40 con 0x7fbd9406cea0 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.472+0000 7fbd9259c640 1 --2- 172.21.15.165:0/638076570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9406cea0 0x7fbd941ac470 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400c340 tx=0x7fbd84001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.472+0000 7fbd82ffd640 1 -- 172.21.15.165:0/638076570 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd84005c10 con 0x7fbd9406cea0 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.472+0000 7fbd82ffd640 1 -- 172.21.15.165:0/638076570 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbd84005db0 con 0x7fbd9406cea0 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.472+0000 7fbd82ffd640 1 -- 172.21.15.165:0/638076570 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbd84004930 con 0x7fbd9406cea0 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.472+0000 7fbd9884d640 1 -- 172.21.15.165:0/638076570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd941acc70 con 0x7fbd9406cea0 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.472+0000 7fbd9884d640 1 -- 172.21.15.165:0/638076570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd941a6500 con 0x7fbd9406cea0 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.473+0000 7fbd82ffd640 1 -- 172.21.15.165:0/638076570 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fbd8401a020 con 0x7fbd9406cea0 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.473+0000 7fbd9884d640 1 -- 172.21.15.165:0/638076570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd94062da0 con 0x7fbd9406cea0 2024-04-27T03:30:12.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.473+0000 7fbd82ffd640 1 --2- 172.21.15.165:0/638076570 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbd74038010 0x7fbd7403a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:12.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.473+0000 7fbd82ffd640 1 -- 172.21.15.165:0/638076570 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(15..15 src has 1..15) ==== 4220+0+0 (secure 0 0 0) 0x7fbd8404aae0 con 0x7fbd9406cea0 2024-04-27T03:30:12.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.473+0000 7fbd91d9b640 1 --2- 172.21.15.165:0/638076570 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbd74038010 0x7fbd7403a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:12.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.473+0000 7fbd91d9b640 1 --2- 172.21.15.165:0/638076570 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbd74038010 0x7fbd7403a4d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbd88036000 tx=0x7fbd88008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:12.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.478+0000 7fbd82ffd640 1 -- 172.21.15.165:0/638076570 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fbd840127f0 con 0x7fbd9406cea0 2024-04-27T03:30:12.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:12.653+0000 7fbd9884d640 1 -- 172.21.15.165:0/638076570 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"} v 0) -- 0x7fbd9411aa80 con 0x7fbd9406cea0 2024-04-27T03:30:13.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:13.356+0000 7fbd82ffd640 1 -- 172.21.15.165:0/638076570 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "default.rgw.buckets.index", "app": "rgw"}]=0 enabled application 'rgw' on pool 'default.rgw.buckets.index' v16) ==== 187+0+0 (secure 0 0 0) 0x7fbd84012a50 con 0x7fbd9406cea0 2024-04-27T03:30:13.362 INFO:teuthology.orchestra.run.smithi165.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2024-04-27T03:30:13.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:13.359+0000 7fbd9884d640 1 -- 172.21.15.165:0/638076570 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbd74038010 msgr2=0x7fbd7403a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:13.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:13.359+0000 7fbd9884d640 1 --2- 172.21.15.165:0/638076570 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbd74038010 0x7fbd7403a4d0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fbd88036000 tx=0x7fbd88008030 comp rx=0 tx=0).stop 2024-04-27T03:30:13.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:13.359+0000 7fbd9884d640 1 -- 172.21.15.165:0/638076570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9406cea0 msgr2=0x7fbd941ac470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:13.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:13.359+0000 7fbd9884d640 1 --2- 172.21.15.165:0/638076570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9406cea0 0x7fbd941ac470 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbd8400c340 tx=0x7fbd84001e70 comp rx=0 tx=0).stop 2024-04-27T03:30:13.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:13.359+0000 7fbd9884d640 1 -- 172.21.15.165:0/638076570 shutdown_connections 2024-04-27T03:30:13.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:13.359+0000 7fbd9884d640 1 --2- 172.21.15.165:0/638076570 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbd74038010 0x7fbd7403a4d0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:13.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:13.359+0000 7fbd9884d640 1 --2- 172.21.15.165:0/638076570 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fbd9406cea0 0x7fbd941ac470 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:13.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:13.359+0000 7fbd9884d640 1 -- 172.21.15.165:0/638076570 >> 172.21.15.165:0/638076570 conn(0x7fbd94070e10 msgr2=0x7fbd940711f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:13.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:13.359+0000 7fbd9884d640 1 -- 172.21.15.165:0/638076570 shutdown_connections 2024-04-27T03:30:13.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:13.359+0000 7fbd9884d640 1 -- 172.21.15.165:0/638076570 wait complete. 2024-04-27T03:30:13.391 DEBUG:tasks.rgw:Pools created 2024-04-27T03:30:13.391 INFO:tasks.rgw:Starting rgw... 2024-04-27T03:30:13.391 INFO:tasks.rgw:rgw client.0 config is {} 2024-04-27T03:30:13.391 INFO:tasks.rgw:Using beast as radosgw frontend 2024-04-27T03:30:13.391 DEBUG:teuthology.orchestra.run.smithi165:> sudo echo -n http://smithi165.front.sepia.ceph.com:80 | sudo tee /home/ubuntu/cephtest/url_file 2024-04-27T03:30:13.418 INFO:teuthology.orchestra.run.smithi165.stdout:http://smithi165.front.sepia.ceph.com:80 2024-04-27T03:30:13.418 DEBUG:teuthology.orchestra.run.smithi165:> sudo chown ceph /home/ubuntu/cephtest/url_file 2024-04-27T03:30:13.483 INFO:tasks.rgw.client.0:Restarting daemon 2024-04-27T03:30:13.483 DEBUG:teuthology.orchestra.run.smithi165:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term radosgw --rgw-frontends 'beast port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2024-04-27T03:30:13.526 INFO:tasks.rgw.client.0:Started 2024-04-27T03:30:13.526 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi165.front.sepia.ceph.com:80/ 2024-04-27T03:30:13.526 DEBUG:teuthology.orchestra.run.smithi165:> curl http://smithi165.front.sepia.ceph.com:80/ 2024-04-27T03:30:13.557 INFO:teuthology.orchestra.run.smithi165.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T03:30:13.557 INFO:teuthology.orchestra.run.smithi165.stderr: Dload Upload Total Spent Left Speed 2024-04-27T03:30:13.558 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-27T03:30:13.558 INFO:teuthology.orchestra.run.smithi165.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-27T03:30:13.558 INFO:teuthology.orchestra.run.smithi165.stderr:curl: (7) Failed to connect to smithi165.front.sepia.ceph.com port 80: Connection refused 2024-04-27T03:30:14.560 DEBUG:teuthology.orchestra.run.smithi165:> curl http://smithi165.front.sepia.ceph.com:80/ 2024-04-27T03:30:14.581 INFO:teuthology.orchestra.run.smithi165.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T03:30:14.581 INFO:teuthology.orchestra.run.smithi165.stderr: Dload Upload Total Spent Left Speed 2024-04-27T03:30:14.582 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-27T03:30:14.583 INFO:teuthology.orchestra.run.smithi165.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-27T03:30:14.583 INFO:teuthology.orchestra.run.smithi165.stderr:curl: (7) Failed to connect to smithi165.front.sepia.ceph.com port 80: Connection refused 2024-04-27T03:30:16.583 DEBUG:teuthology.orchestra.run.smithi165:> curl http://smithi165.front.sepia.ceph.com:80/ 2024-04-27T03:30:16.606 INFO:teuthology.orchestra.run.smithi165.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T03:30:16.606 INFO:teuthology.orchestra.run.smithi165.stderr: Dload Upload Total Spent Left Speed 2024-04-27T03:30:16.607 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-27T03:30:16.607 INFO:teuthology.orchestra.run.smithi165.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-27T03:30:16.608 INFO:teuthology.orchestra.run.smithi165.stderr:curl: (7) Failed to connect to smithi165.front.sepia.ceph.com port 80: Connection refused 2024-04-27T03:30:20.609 DEBUG:teuthology.orchestra.run.smithi165:> curl http://smithi165.front.sepia.ceph.com:80/ 2024-04-27T03:30:20.633 DEBUG:teuthology.orchestra.run:got remote process result: 7 2024-04-27T03:30:20.634 INFO:teuthology.orchestra.run.smithi165.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T03:30:20.634 INFO:teuthology.orchestra.run.smithi165.stderr: Dload Upload Total Spent Left Speed 2024-04-27T03:30:20.634 INFO:teuthology.orchestra.run.smithi165.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 2024-04-27T03:30:20.634 INFO:teuthology.orchestra.run.smithi165.stderr:curl: (7) Failed to connect to smithi165.front.sepia.ceph.com port 80: Connection refused 2024-04-27T03:30:22.478 INFO:tasks.rgw.client.0.smithi165.stdout:2024-04-27T03:30:22.473+0000 7f9204c6b880 -1 LDAP not started since no server URIs were provided in the configuration. 2024-04-27T03:30:28.635 DEBUG:teuthology.orchestra.run.smithi165:> curl http://smithi165.front.sepia.ceph.com:80/ 2024-04-27T03:30:28.658 INFO:teuthology.orchestra.run.smithi165.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2024-04-27T03:30:28.658 INFO:teuthology.orchestra.run.smithi165.stderr: Dload Upload Total Spent Left Speed 2024-04-27T03:30:28.660 INFO:teuthology.orchestra.run.smithi165.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 187 0 187 0 0 62333 0 --:--:-- --:--:-- --:--:-- 93500 2024-04-27T03:30:28.661 INFO:teuthology.orchestra.run.smithi165.stdout:anonymous 2024-04-27T03:30:28.661 INFO:teuthology.run_tasks:Running task radosgw-admin... 2024-04-27T03:30:28.751 DEBUG:tasks.radosgw_admin:config is: {'client.0': None} 2024-04-27T03:30:28.752 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-04-27T03:30:28.752 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-04-27T03:30:28.752 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-04-27T03:30:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:28.798 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:28.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.793+0000 7fdb051a2800 1 Processor -- start 2024-04-27T03:30:28.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.793+0000 7fdb051a2800 1 -- start start 2024-04-27T03:30:28.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.793+0000 7fdb051a2800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ffabcfd250 0x55ffabd05d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.793+0000 7fdb051a2800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ffabbbbd40 con 0x55ffabcfb710 2024-04-27T03:30:28.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.793+0000 7fdb051a2800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55ffabb8e4d0 con 0x55ffabcfd250 2024-04-27T03:30:28.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.794+0000 7fdafbfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ffabcfd250 0x55ffabd05d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.794+0000 7fdb02ca8640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55ffabcfb710 0x55ffabcfbb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53506/0 (socket says 172.21.15.165:53506) 2024-04-27T03:30:28.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.794+0000 7fdafbfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ffabcfd250 0x55ffabd05d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51034/0 (socket says 172.21.15.165:51034) 2024-04-27T03:30:28.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.794+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1525966301 learned_addr learned my addr 172.21.15.165:0/1525966301 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:28.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.794+0000 7fdafbfff640 1 -- 172.21.15.165:0/1525966301 >> v1:172.21.15.89:6789/0 conn(0x55ffabcfb710 legacy=0x55ffabcfbb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.794+0000 7fdafbfff640 1 -- 172.21.15.165:0/1525966301 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffabbba590 con 0x55ffabcfd250 2024-04-27T03:30:28.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.796+0000 7fdafbfff640 1 --2- 172.21.15.165:0/1525966301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfd250 0x55ffabd05d50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00a470 tx=0x7fdaec01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3cb6a01b00ec242 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.796+0000 7fdb024a7640 1 -- 172.21.15.165:0/1525966301 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaec057070 con 0x55ffabcfd250 2024-04-27T03:30:28.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.796+0000 7fdb024a7640 1 -- 172.21.15.165:0/1525966301 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaec04adc0 con 0x55ffabcfd250 2024-04-27T03:30:28.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.796+0000 7fdb024a7640 1 -- 172.21.15.165:0/1525966301 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaec052590 con 0x55ffabcfd250 2024-04-27T03:30:28.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.797+0000 7fdb051a2800 1 -- 172.21.15.165:0/1525966301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfd250 msgr2=0x55ffabd05d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.797+0000 7fdb051a2800 1 --2- 172.21.15.165:0/1525966301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfd250 0x55ffabd05d50 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fdaec00a470 tx=0x7fdaec01ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:28.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.797+0000 7fdb051a2800 1 -- 172.21.15.165:0/1525966301 shutdown_connections 2024-04-27T03:30:28.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.797+0000 7fdb051a2800 1 --2- 172.21.15.165:0/1525966301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfd250 0x55ffabd05d50 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.797+0000 7fdb051a2800 1 -- 172.21.15.165:0/1525966301 >> 172.21.15.165:0/1525966301 conn(0x55ffabcf7370 msgr2=0x55ffabcf9790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:28.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.797+0000 7fdb051a2800 1 -- 172.21.15.165:0/1525966301 shutdown_connections 2024-04-27T03:30:28.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.797+0000 7fdb051a2800 1 -- 172.21.15.165:0/1525966301 wait complete. 2024-04-27T03:30:28.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.811+0000 7fdb051a2800 1 Processor -- start 2024-04-27T03:30:28.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.811+0000 7fdb051a2800 1 -- start start 2024-04-27T03:30:28.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.811+0000 7fdb051a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 0x55ffabdf05b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.811+0000 7fdb051a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ffabc483b0 con 0x55ffabcfb710 2024-04-27T03:30:28.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.811+0000 7fdb02ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 0x55ffabdf05b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.811+0000 7fdb02ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 0x55ffabdf05b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51038/0 (socket says 172.21.15.165:51038) 2024-04-27T03:30:28.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.811+0000 7fdb02ca8640 1 -- 172.21.15.165:0/411862078 learned_addr learned my addr 172.21.15.165:0/411862078 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:28.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.811+0000 7fdb02ca8640 1 -- 172.21.15.165:0/411862078 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffabdf0b80 con 0x55ffabcfb710 2024-04-27T03:30:28.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.812+0000 7fdb02ca8640 1 --2- 172.21.15.165:0/411862078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 0x55ffabdf05b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4001de0 tx=0x7fdaf400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ecdbb82ae6da4b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.812+0000 7fd9f0e09640 1 -- 172.21.15.165:0/411862078 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf4012020 con 0x55ffabcfb710 2024-04-27T03:30:28.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.812+0000 7fd9f0e09640 1 -- 172.21.15.165:0/411862078 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf400e5a0 con 0x55ffabcfb710 2024-04-27T03:30:28.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.812+0000 7fd9f0e09640 1 -- 172.21.15.165:0/411862078 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf400e8c0 con 0x55ffabcfb710 2024-04-27T03:30:28.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.812+0000 7fdb051a2800 1 -- 172.21.15.165:0/411862078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 msgr2=0x55ffabdf05b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb051a2800 1 --2- 172.21.15.165:0/411862078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 0x55ffabdf05b0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4001de0 tx=0x7fdaf400b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb051a2800 1 -- 172.21.15.165:0/411862078 shutdown_connections 2024-04-27T03:30:28.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb051a2800 1 --2- 172.21.15.165:0/411862078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 0x55ffabdf05b0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb051a2800 1 -- 172.21.15.165:0/411862078 >> 172.21.15.165:0/411862078 conn(0x55ffabcf7370 msgr2=0x55ffabdebed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:28.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb051a2800 1 -- 172.21.15.165:0/411862078 shutdown_connections 2024-04-27T03:30:28.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb051a2800 1 -- 172.21.15.165:0/411862078 wait complete. 2024-04-27T03:30:28.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb051a2800 1 Processor -- start 2024-04-27T03:30:28.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb051a2800 1 -- start start 2024-04-27T03:30:28.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb051a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 0x55ffabdf80d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb051a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ffabd08570 con 0x55ffabcfb710 2024-04-27T03:30:28.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb02ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 0x55ffabdf80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb02ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 0x55ffabdf80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51052/0 (socket says 172.21.15.165:51052) 2024-04-27T03:30:28.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.813+0000 7fdb02ca8640 1 -- 172.21.15.165:0/2891926058 learned_addr learned my addr 172.21.15.165:0/2891926058 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:28.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.814+0000 7fdb02ca8640 1 -- 172.21.15.165:0/2891926058 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffabdf8610 con 0x55ffabcfb710 2024-04-27T03:30:28.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.814+0000 7fdb02ca8640 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 0x55ffabdf80d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400c010 tx=0x7fdaf4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.814+0000 7fd9ef606640 1 -- 172.21.15.165:0/2891926058 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf4012070 con 0x55ffabcfb710 2024-04-27T03:30:28.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.814+0000 7fd9ef606640 1 -- 172.21.15.165:0/2891926058 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf401a7a0 con 0x55ffabcfb710 2024-04-27T03:30:28.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.814+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ffabdf88a0 con 0x55ffabcfb710 2024-04-27T03:30:28.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.815+0000 7fd9ef606640 1 -- 172.21.15.165:0/2891926058 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf401aa80 con 0x55ffabcfb710 2024-04-27T03:30:28.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.815+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffabdfb550 con 0x55ffabcfb710 2024-04-27T03:30:28.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.815+0000 7fd9ef606640 1 -- 172.21.15.165:0/2891926058 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fdaf4015020 con 0x55ffabcfb710 2024-04-27T03:30:28.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.815+0000 7fd9ef606640 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9e8036890 0x7fd9e8038d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.815+0000 7fd9ef606640 1 -- 172.21.15.165:0/2891926058 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fdaf4020070 con 0x55ffabcfb710 2024-04-27T03:30:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.815+0000 7fdafbfff640 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9e8036890 0x7fd9e8038d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.815+0000 7fdb051a2800 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ffabcfd250 0x55ffabdff310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.815+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabdff8a0 con 0x55ffabcfd250 2024-04-27T03:30:28.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.816+0000 7fdb034a9640 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ffabcfd250 0x55ffabdff310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.816+0000 7fdb034a9640 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ffabcfd250 0x55ffabdff310 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.816+0000 7fdafbfff640 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9e8036890 0x7fd9e8038d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdaec054000 tx=0x7fdaec04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.816+0000 7fdb034a9640 1 -- 172.21.15.165:0/2891926058 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabcfd250 2024-04-27T03:30:28.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.816+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabdfc8e0 con 0x55ffabcfd250 2024-04-27T03:30:28.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.816+0000 7fdb034a9640 1 -- 172.21.15.165:0/2891926058 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabcfd250 2024-04-27T03:30:28.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.816+0000 7fdb051a2800 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ffabe01bf0 0x55ffabe03fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.816+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe04680 con 0x55ffabe01bf0 2024-04-27T03:30:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.817+0000 7fdb02ca8640 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ffabe01bf0 0x55ffabe03fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.817+0000 7fdb02ca8640 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ffabe01bf0 0x55ffabe03fe0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.817+0000 7fdb02ca8640 1 -- 172.21.15.165:0/2891926058 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fdaf4018430 con 0x55ffabe01bf0 2024-04-27T03:30:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.817+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe00ba0 con 0x55ffabcfd250 2024-04-27T03:30:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.817+0000 7fdb034a9640 1 -- 172.21.15.165:0/2891926058 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabcfd250 2024-04-27T03:30:28.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.818+0000 7fdb051a2800 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ffabe05b30 0x55ffabe07f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.818+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe086e0 con 0x55ffabe05b30 2024-04-27T03:30:28.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.818+0000 7fdafbfff640 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ffabe05b30 0x55ffabe07f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.818+0000 7fdafbfff640 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ffabe05b30 0x55ffabe07f80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.819+0000 7fdafbfff640 1 -- 172.21.15.165:0/2891926058 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fdaec05a040 con 0x55ffabe05b30 2024-04-27T03:30:28.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.819+0000 7fdb051a2800 1 Processor -- start 2024-04-27T03:30:28.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.819+0000 7fdb051a2800 1 -- start start 2024-04-27T03:30:28.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.819+0000 7fdb051a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabe16b20 0x55ffabe16f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.819+0000 7fdb051a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ffabbee6b0 con 0x55ffabe16b20 2024-04-27T03:30:28.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.819+0000 7fdb02ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabe16b20 0x55ffabe16f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.820+0000 7fdb02ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabe16b20 0x55ffabe16f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51068/0 (socket says 172.21.15.165:51068) 2024-04-27T03:30:28.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.820+0000 7fdb02ca8640 1 -- 172.21.15.165:0/3237640421 learned_addr learned my addr 172.21.15.165:0/3237640421 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:28.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.820+0000 7fdb02ca8640 1 -- 172.21.15.165:0/3237640421 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffabe17480 con 0x55ffabe16b20 2024-04-27T03:30:28.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.820+0000 7fdb02ca8640 1 --2- 172.21.15.165:0/3237640421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabe16b20 0x55ffabe16f40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdaf404a080 tx=0x7fdaf4022ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.820+0000 7fd9ece01640 1 -- 172.21.15.165:0/3237640421 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf4029070 con 0x55ffabe16b20 2024-04-27T03:30:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.820+0000 7fd9ece01640 1 -- 172.21.15.165:0/3237640421 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf40439f0 con 0x55ffabe16b20 2024-04-27T03:30:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.820+0000 7fd9ece01640 1 -- 172.21.15.165:0/3237640421 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf404b1d0 con 0x55ffabe16b20 2024-04-27T03:30:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.821+0000 7fdb051a2800 1 -- 172.21.15.165:0/3237640421 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ffabe17ef0 con 0x55ffabe16b20 2024-04-27T03:30:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.821+0000 7fdb051a2800 1 -- 172.21.15.165:0/3237640421 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffabe17ef0 con 0x55ffabe16b20 2024-04-27T03:30:28.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.821+0000 7fd9ece01640 1 -- 172.21.15.165:0/3237640421 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fdaf402c070 con 0x55ffabe16b20 2024-04-27T03:30:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.822+0000 7fd9ece01640 1 --2- 172.21.15.165:0/3237640421 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9dc036900 0x7fd9dc038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.822+0000 7fd9ece01640 1 -- 172.21.15.165:0/3237640421 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fdaf402fb00 con 0x55ffabe16b20 2024-04-27T03:30:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.822+0000 7fdafbfff640 1 --2- 172.21.15.165:0/3237640421 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9dc036900 0x7fd9dc038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.822+0000 7fdb051a2800 1 Processor -- start 2024-04-27T03:30:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.822+0000 7fdb051a2800 1 -- start start 2024-04-27T03:30:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.822+0000 7fdafbfff640 1 --2- 172.21.15.165:0/3237640421 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9dc036900 0x7fd9dc038dc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdaec055040 tx=0x7fdaec005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.822+0000 7fdb051a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 0x55ffabcfdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.822+0000 7fdb051a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ffabdf0f00 con 0x55ffabcfdbc0 2024-04-27T03:30:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.823+0000 7fdb02ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 0x55ffabcfdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.823+0000 7fdb02ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 0x55ffabcfdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51076/0 (socket says 172.21.15.165:51076) 2024-04-27T03:30:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.823+0000 7fdb02ca8640 1 -- 172.21.15.165:0/2517131660 learned_addr learned my addr 172.21.15.165:0/2517131660 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.823+0000 7fdb02ca8640 1 -- 172.21.15.165:0/2517131660 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffabcfe5b0 con 0x55ffabcfdbc0 2024-04-27T03:30:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.823+0000 7fdb02ca8640 1 --2- 172.21.15.165:0/2517131660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 0x55ffabcfdfe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009ed0 tx=0x7fdaf4040f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646d60b4801c00ef server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.823+0000 7fd9e67fc640 1 -- 172.21.15.165:0/2517131660 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf405c070 con 0x55ffabcfdbc0 2024-04-27T03:30:28.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.823+0000 7fd9e67fc640 1 -- 172.21.15.165:0/2517131660 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf404b370 con 0x55ffabcfdbc0 2024-04-27T03:30:28.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.823+0000 7fd9e67fc640 1 -- 172.21.15.165:0/2517131660 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf4033040 con 0x55ffabcfdbc0 2024-04-27T03:30:28.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb051a2800 1 -- 172.21.15.165:0/2517131660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 msgr2=0x55ffabcfdfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb051a2800 1 --2- 172.21.15.165:0/2517131660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 0x55ffabcfdfe0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fdaf4009ed0 tx=0x7fdaf4040f70 comp rx=0 tx=0).stop 2024-04-27T03:30:28.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb051a2800 1 -- 172.21.15.165:0/2517131660 shutdown_connections 2024-04-27T03:30:28.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb051a2800 1 --2- 172.21.15.165:0/2517131660 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 0x55ffabcfdfe0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb051a2800 1 -- 172.21.15.165:0/2517131660 >> 172.21.15.165:0/2517131660 conn(0x55ffabe1fdd0 msgr2=0x55ffabe1de30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:28.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb051a2800 1 -- 172.21.15.165:0/2517131660 shutdown_connections 2024-04-27T03:30:28.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb051a2800 1 -- 172.21.15.165:0/2517131660 wait complete. 2024-04-27T03:30:28.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb051a2800 1 Processor -- start 2024-04-27T03:30:28.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb051a2800 1 -- start start 2024-04-27T03:30:28.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb051a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 0x55ffabe2f310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb051a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ffabcfeb80 con 0x55ffabcfdbc0 2024-04-27T03:30:28.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb02ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 0x55ffabe2f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb02ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 0x55ffabe2f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51084/0 (socket says 172.21.15.165:51084) 2024-04-27T03:30:28.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.824+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 learned_addr learned my addr 172.21.15.165:0/1443928452 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:28.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.825+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ffabe2f850 con 0x55ffabcfdbc0 2024-04-27T03:30:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.825+0000 7fdb02ca8640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 0x55ffabe2f310 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdaf405a040 tx=0x7fdaf4038e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.825+0000 7fd9e4ff9640 1 -- 172.21.15.165:0/1443928452 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf4067020 con 0x55ffabcfdbc0 2024-04-27T03:30:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.825+0000 7fd9e4ff9640 1 -- 172.21.15.165:0/1443928452 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdaf4023cc0 con 0x55ffabcfdbc0 2024-04-27T03:30:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.825+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ffabe2f850 con 0x55ffabcfdbc0 2024-04-27T03:30:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.825+0000 7fd9e4ff9640 1 -- 172.21.15.165:0/1443928452 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdaf405c030 con 0x55ffabcfdbc0 2024-04-27T03:30:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.825+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ffabe2f850 con 0x55ffabcfdbc0 2024-04-27T03:30:28.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.826+0000 7fd9e4ff9640 1 -- 172.21.15.165:0/1443928452 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fdaf406a020 con 0x55ffabcfdbc0 2024-04-27T03:30:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.826+0000 7fd9e4ff9640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9cc036950 0x7fd9cc038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.826+0000 7fd9e4ff9640 1 -- 172.21.15.165:0/1443928452 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fdaf405d070 con 0x55ffabcfdbc0 2024-04-27T03:30:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.826+0000 7fdb051a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.826+0000 7fdafbfff640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9cc036950 0x7fd9cc038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.826+0000 7fdb051a2800 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ffabe3aa50 0x55ffabe3cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.826+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3d660 con 0x55ffabe3aa50 2024-04-27T03:30:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdb034a9640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ffabe3aa50 0x55ffabe3cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdafbfff640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9cc036950 0x7fd9cc038e10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55ffabe08ae0 tx=0x7fdaec04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdb034a9640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ffabe3aa50 0x55ffabe3cf10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdb051a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdb051a2800 20 realm 2024-04-27T03:30:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdb051a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3da70 con 0x55ffabe3aa50 2024-04-27T03:30:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdb051a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:28.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdb051a2800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:28.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdb051a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:28.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.827+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e310 con 0x55ffabe3aa50 2024-04-27T03:30:28.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.828+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.828+0000 7fdb051a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:28.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.828+0000 7fdb051a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.828+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.828+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.828+0000 7fdb051a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.828+0000 7fdb051a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.828+0000 7fdb051a2800 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ffabe3f410 0x55ffabe41800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.828+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe41d80 con 0x55ffabe3f410 2024-04-27T03:30:28.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.828+0000 7fdb02ca8640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ffabe3f410 0x55ffabe41800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.828+0000 7fdb02ca8640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ffabe3f410 0x55ffabe41800 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.829+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.829+0000 7fdb051a2800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.829+0000 7fdb051a2800 20 searching for the correct realm 2024-04-27T03:30:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.829+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe436e0 con 0x55ffabe3f410 2024-04-27T03:30:28.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.829+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.829+0000 7fdb02ca8640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fdaf4077040 0x7fdaf402f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.829+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe436e0 con 0x7fdaf4077040 2024-04-27T03:30:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.829+0000 7fdafbfff640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fdaf4077040 0x7fdaf402f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.830+0000 7fdafbfff640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fdaf4077040 0x7fdaf402f4d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.830+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.830+0000 7fdafbfff640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fdaec05c410 0x7fdaec05c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:28.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.830+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe436e0 con 0x7fdaec05c410 2024-04-27T03:30:28.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.830+0000 7fdb034a9640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fdaec05c410 0x7fdaec05c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:28.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.831+0000 7fdb034a9640 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fdaec05c410 0x7fdaec05c850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:28.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.831+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.831+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.831+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.831+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.832+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.832+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x7fdaec05c410 2024-04-27T03:30:28.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.832+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.832+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x7fdaec05c410 2024-04-27T03:30:28.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.832+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.832+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x7fdaf4077040 2024-04-27T03:30:28.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.832+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.832+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3f410 2024-04-27T03:30:28.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.833+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.833+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.833+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.833+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x7fdaec05c410 2024-04-27T03:30:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.833+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.833+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.833+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.833+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x7fdaf4077040 2024-04-27T03:30:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.834+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.834+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3f410 2024-04-27T03:30:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.834+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.834+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3f410 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.834+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.834+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x7fdaec05c410 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.835+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.835+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3f410 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.835+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v18'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.835+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3f410 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.835+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.835+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.835+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.835+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.836+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.836+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x7fdaec05c410 2024-04-27T03:30:28.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.836+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.836+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3f410 2024-04-27T03:30:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.836+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.836+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x7fdaf4077040 2024-04-27T03:30:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.836+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.836+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.837+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.837+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x7fdaec05c410 2024-04-27T03:30:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.837+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.837+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3f410 2024-04-27T03:30:28.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.837+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.837+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x7fdaec05c410 2024-04-27T03:30:28.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.838+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.838+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x7fdaf4077040 2024-04-27T03:30:28.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.838+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.838+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3f410 2024-04-27T03:30:28.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.838+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.838+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x7fdaec05c410 2024-04-27T03:30:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe3e6b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fdafc038040 con 0x7fdaf4077040 2024-04-27T03:30:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb051a2800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb051a2800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:28.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb051a2800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb051a2800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb051a2800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb051a2800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb051a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe426d0 con 0x55ffabe3aa50 2024-04-27T03:30:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.839+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe42ac0 con 0x55ffabe3aa50 2024-04-27T03:30:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:28.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe42ac0 con 0x7fdaec05c410 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 20 zone default found 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 4 Realm: () 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 20 zonegroup default 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.840+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe42ee0 con 0x7fdaec05c410 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb051a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb051a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe43300 con 0x55ffabe3f410 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb051a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb051a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe45590 con 0x55ffabe3f410 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb051a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb051a2800 20 started sync module instance, tier type = 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb051a2800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.841+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe44a30 con 0x7fdaf4077040 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.842+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.842+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94556588885376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe45930 con 0x7fdaf4077040 2024-04-27T03:30:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.842+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe46180 con 0x55ffabe3aa50 2024-04-27T03:30:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.844+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94556588885376] v25'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.844+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.844+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94556588891344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe465a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.844+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe46df0 con 0x55ffabe3f410 2024-04-27T03:30:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.846+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.846+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94556588894160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe472a0 con 0x55ffabe3f410 2024-04-27T03:30:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.846+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe47dc0 con 0x7fdaf4077040 2024-04-27T03:30:28.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.848+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94556588891344] v25'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.849+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.849+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94556588898448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaf4077040 2024-04-27T03:30:28.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.849+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe48c00 con 0x55ffabe3aa50 2024-04-27T03:30:28.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.850+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94556588898448] v25'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.851+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94556588894160] v25'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.852+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.852+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94556588902128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe46cf0 con 0x55ffabe3aa50 2024-04-27T03:30:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.852+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe49610 con 0x7fdaec05c410 2024-04-27T03:30:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.853+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94556588902128] v25'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.854+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.854+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94556588892400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe49610 con 0x7fdaec05c410 2024-04-27T03:30:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.854+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe49ea0 con 0x7fdaec05c410 2024-04-27T03:30:28.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.855+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94556588892400] v25'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.856+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.856+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94556588904800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe49a30 con 0x7fdaec05c410 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.856+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe4a8b0 con 0x7fdaec05c410 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.857+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94556588904800] v25'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.857+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'3 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.857+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94556588905856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe4aa90 con 0x7fdaec05c410 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.857+0000 7fdb051a2800 20 add_watcher() i=0 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.857+0000 7fdb051a2800 20 add_watcher() i=1 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.857+0000 7fdb051a2800 20 add_watcher() i=2 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.857+0000 7fdb051a2800 20 add_watcher() i=3 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.857+0000 7fdb051a2800 20 add_watcher() i=4 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.857+0000 7fdb051a2800 20 add_watcher() i=5 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.857+0000 7fdb051a2800 20 add_watcher() i=6 2024-04-27T03:30:28.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.858+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94556588905856] v25'4 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.858+0000 7fdb051a2800 20 add_watcher() i=7 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.858+0000 7fdb051a2800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.858+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe4dc30 con 0x55ffabe3aa50 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.858+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.858+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94556588922272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe4a9b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.859+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94556588922272] v25'30 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.860+0000 7fd9b97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.860+0000 7fdb051a2800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.860+0000 7fdb051a2800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.860+0000 7fdb051a2800 5 note: GC not initialized 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.860+0000 7fd933fff640 20 reqs_thread_entry: start 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.860+0000 7fdb051a2800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.860+0000 7fdb051a2800 20 Filter name: none 2024-04-27T03:30:28.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.860+0000 7fd931ffb640 20 reqs_thread_entry: start 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.860+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe807e0 con 0x55ffabe3f410 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.861+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(62 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.861+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe45590 con 0x55ffabe3f410 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.861+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(63 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.861+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe45590 con 0x55ffabe3aa50 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.861+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(64 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.861+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe426d0 con 0x55ffabe3aa50 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.862+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.862+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe426d0 con 0x7fdaf4077040 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.862+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.862+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3f410 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.862+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(67 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.862+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.862+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.862+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3f410 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.863+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(69 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.863+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaf4077040 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.863+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.863+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3f410 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.863+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(71 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.863+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.863+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(72 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.863+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaec05c410 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.864+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.864+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaec05c410 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.864+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.864+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3f410 2024-04-27T03:30:28.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.864+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(75 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.864+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaec05c410 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.865+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(76 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.865+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaec05c410 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.865+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(77 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.865+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaec05c410 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.865+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(78 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.865+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3f410 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.865+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(79 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.865+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaf4077040 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.866+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(80 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.866+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.866+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(81 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.866+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.866+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(82 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.866+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3f410 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.866+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(83 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.866+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaf4077040 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.867+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(84 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.867+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaec05c410 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.867+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(85 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.867+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.867+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(86 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.867+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaec05c410 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.868+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(87 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.868+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3f410 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.868+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(88 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.868+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaf4077040 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.868+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(89 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.868+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaec05c410 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.868+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(90 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.868+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3aa50 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.869+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(91 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.869+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x55ffabe3f410 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.869+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(92 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.869+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe481b0 con 0x7fdaf4077040 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.869+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(93 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.870+0000 7fd9b97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.870+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94556588922272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe45590 con 0x55ffabe3aa50 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.871+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94556588922272] v25'31 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.871+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94556588885376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe426d0 con 0x7fdaf4077040 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.872+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94556588885376] v25'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.872+0000 7fdb051a2800 20 remove_watcher() i=0 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.872+0000 7fdb051a2800 2 removed watcher, disabling cache 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.872+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94556588891344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe426d0 con 0x55ffabe3aa50 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.874+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94556588891344] v25'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.874+0000 7fdb051a2800 20 remove_watcher() i=1 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.874+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94556588894160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe426d0 con 0x55ffabe3f410 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.875+0000 7fdb02ca8640 1 -- 172.21.15.165:0/1443928452 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94556588894160] v25'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdaf4035070 con 0x55ffabe3f410 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.875+0000 7fdb051a2800 20 remove_watcher() i=2 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.875+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94556588898448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe45cd0 con 0x7fdaf4077040 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.876+0000 7fdafbfff640 1 -- 172.21.15.165:0/1443928452 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94556588898448] v25'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdaec05a040 con 0x7fdaf4077040 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.876+0000 7fdb051a2800 20 remove_watcher() i=3 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.876+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94556588902128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe47890 con 0x55ffabe3aa50 2024-04-27T03:30:28.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.877+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94556588902128] v25'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x55ffabe3aa50 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.877+0000 7fdb051a2800 20 remove_watcher() i=4 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.877+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94556588892400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe45cd0 con 0x7fdaec05c410 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.878+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94556588892400] v25'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.878+0000 7fdb051a2800 20 remove_watcher() i=5 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.878+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94556588904800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe486f0 con 0x7fdaec05c410 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.879+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94556588904800] v25'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.879+0000 7fdb051a2800 20 remove_watcher() i=6 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.879+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94556588905856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ffabe47890 con 0x7fdaec05c410 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94556588905856] v25'5 uv3 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdafc00a4a0 con 0x7fdaec05c410 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 20 remove_watcher() i=7 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fdaec05c410 msgr2=0x7fdaec05c850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fdaec05c410 0x7fdaec05c850 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ffabe3aa50 msgr2=0x55ffabe3cf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ffabe3aa50 0x55ffabe3cf10 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fdaf4077040 msgr2=0x7fdaf402f4d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fdaf4077040 0x7fdaf402f4d0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ffabe3f410 msgr2=0x55ffabe41800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ffabe3f410 0x55ffabe41800 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9cc036950 msgr2=0x7fd9cc038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9cc036950 0x7fd9cc038e10 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x55ffabe08ae0 tx=0x7fdaec04f000 comp rx=0 tx=0).stop 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 msgr2=0x55ffabe2f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb034a9640 1 -- 172.21.15.165:0/1443928452 reap_dead start 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 0x55ffabe2f310 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fdaf405a040 tx=0x7fdaf4038e80 comp rx=0 tx=0).stop 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 shutdown_connections 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 --2- 172.21.15.165:0/1443928452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfdbc0 0x55ffabe2f310 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 >> 172.21.15.165:0/1443928452 conn(0x55ffabe1fdd0 msgr2=0x55ffabe1d430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:28.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 shutdown_connections 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/1443928452 wait complete. 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/3237640421 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9dc036900 msgr2=0x7fd9dc038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 --2- 172.21.15.165:0/3237640421 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9dc036900 0x7fd9dc038dc0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdaec055040 tx=0x7fdaec005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/3237640421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabe16b20 msgr2=0x55ffabe16f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 --2- 172.21.15.165:0/3237640421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabe16b20 0x55ffabe16f40 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fdaf404a080 tx=0x7fdaf4022ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 -- 172.21.15.165:0/3237640421 shutdown_connections 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.881+0000 7fdb051a2800 1 --2- 172.21.15.165:0/3237640421 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9dc036900 0x7fd9dc038dc0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 --2- 172.21.15.165:0/3237640421 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabe16b20 0x55ffabe16f40 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 -- 172.21.15.165:0/3237640421 >> 172.21.15.165:0/3237640421 conn(0x55ffabe0abc0 msgr2=0x55ffabe0afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 -- 172.21.15.165:0/3237640421 shutdown_connections 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 -- 172.21.15.165:0/3237640421 wait complete. 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ffabe05b30 msgr2=0x55ffabe07f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ffabe05b30 0x55ffabe07f80 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ffabcfd250 msgr2=0x55ffabdff310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ffabcfd250 0x55ffabdff310 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ffabe01bf0 msgr2=0x55ffabe03fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ffabe01bf0 0x55ffabe03fe0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9e8036890 msgr2=0x7fd9e8038d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd9e8036890 0x7fd9e8038d50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdaec054000 tx=0x7fdaec04e000 comp rx=0 tx=0).stop 2024-04-27T03:30:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 msgr2=0x55ffabdf80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 --2- 172.21.15.165:0/2891926058 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ffabcfb710 0x55ffabdf80d0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fdaf400c010 tx=0x7fdaf4000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb034a9640 1 -- 172.21.15.165:0/2891926058 reap_dead start 2024-04-27T03:30:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 shutdown_connections 2024-04-27T03:30:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 >> 172.21.15.165:0/2891926058 conn(0x55ffabcf7370 msgr2=0x55ffabdea790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 shutdown_connections 2024-04-27T03:30:28.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:28.882+0000 7fdb051a2800 1 -- 172.21.15.165:0/2891926058 wait complete. 2024-04-27T03:30:28.998 INFO:teuthology.orchestra.run.smithi165.stdout:[] 2024-04-27T03:30:28.998 DEBUG:tasks.util.rgw: json result: [] 2024-04-27T03:30:28.998 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud'] 2024-04-27T03:30:28.998 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud'] 2024-04-27T03:30:28.999 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud 2024-04-27T03:30:29.042 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:29.042 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:29.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.037+0000 7ffb6550a800 1 Processor -- start 2024-04-27T03:30:29.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.037+0000 7ffb6550a800 1 -- start start 2024-04-27T03:30:29.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.037+0000 7ffb6550a800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560077eb6d20 0x560077eb7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.037+0000 7ffb6550a800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x560077d76d40 con 0x560077eb7f80 2024-04-27T03:30:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.037+0000 7ffb6550a800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x560077d494d0 con 0x560077eb6d20 2024-04-27T03:30:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.038+0000 7ffb63009640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560077eb6d20 0x560077eb7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.038+0000 7ffb5bfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x560077eb7f80 0x560077eb8400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53522/0 (socket says 172.21.15.165:53522) 2024-04-27T03:30:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.038+0000 7ffb5bfff640 1 -- 172.21.15.165:0/2917250766 learned_addr learned my addr 172.21.15.165:0/2917250766 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.038+0000 7ffb63009640 1 -- 172.21.15.165:0/2917250766 >> v1:172.21.15.89:6789/0 conn(0x560077eb7f80 legacy=0x560077eb8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.038+0000 7ffb63009640 1 -- 172.21.15.165:0/2917250766 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x560077d75590 con 0x560077eb6d20 2024-04-27T03:30:29.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.040+0000 7ffb63009640 1 --2- 172.21.15.165:0/2917250766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077eb7140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400a600 tx=0x7ffb5401ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3d0013ceac55844 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.040+0000 7ffb62808640 1 -- 172.21.15.165:0/2917250766 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5400a200 con 0x560077eb6d20 2024-04-27T03:30:29.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.040+0000 7ffb62808640 1 -- 172.21.15.165:0/2917250766 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5400a3a0 con 0x560077eb6d20 2024-04-27T03:30:29.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.040+0000 7ffb62808640 1 -- 172.21.15.165:0/2917250766 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb540086e0 con 0x560077eb6d20 2024-04-27T03:30:29.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.040+0000 7ffb6550a800 1 -- 172.21.15.165:0/2917250766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 msgr2=0x560077eb7140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.040+0000 7ffb6550a800 1 --2- 172.21.15.165:0/2917250766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077eb7140 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7ffb5400a600 tx=0x7ffb5401ed70 comp rx=0 tx=0).stop 2024-04-27T03:30:29.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.040+0000 7ffb6550a800 1 -- 172.21.15.165:0/2917250766 shutdown_connections 2024-04-27T03:30:29.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.040+0000 7ffb6550a800 1 --2- 172.21.15.165:0/2917250766 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077eb7140 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.040+0000 7ffb6550a800 1 -- 172.21.15.165:0/2917250766 >> 172.21.15.165:0/2917250766 conn(0x560077eb2380 msgr2=0x560077eb47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.040+0000 7ffb6550a800 1 -- 172.21.15.165:0/2917250766 shutdown_connections 2024-04-27T03:30:29.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.040+0000 7ffb6550a800 1 -- 172.21.15.165:0/2917250766 wait complete. 2024-04-27T03:30:29.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.052+0000 7ffb6550a800 1 Processor -- start 2024-04-27T03:30:29.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.052+0000 7ffb6550a800 1 -- start start 2024-04-27T03:30:29.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.052+0000 7ffb6550a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077fa7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.052+0000 7ffb6550a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560077e033c0 con 0x560077eb6d20 2024-04-27T03:30:29.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.052+0000 7ffb63009640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077fa7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.053+0000 7ffb63009640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077fa7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51096/0 (socket says 172.21.15.165:51096) 2024-04-27T03:30:29.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.053+0000 7ffb63009640 1 -- 172.21.15.165:0/4176246230 learned_addr learned my addr 172.21.15.165:0/4176246230 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.053+0000 7ffb63009640 1 -- 172.21.15.165:0/4176246230 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560077fa7930 con 0x560077eb6d20 2024-04-27T03:30:29.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.053+0000 7ffb63009640 1 --2- 172.21.15.165:0/4176246230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077fa7360 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffb54056040 tx=0x7ffb5404e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6777620bffd7ed6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.053+0000 7ffa5160a640 1 -- 172.21.15.165:0/4176246230 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54065070 con 0x560077eb6d20 2024-04-27T03:30:29.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.053+0000 7ffa5160a640 1 -- 172.21.15.165:0/4176246230 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb54008590 con 0x560077eb6d20 2024-04-27T03:30:29.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.053+0000 7ffa5160a640 1 -- 172.21.15.165:0/4176246230 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54008950 con 0x560077eb6d20 2024-04-27T03:30:29.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.054+0000 7ffb6550a800 1 -- 172.21.15.165:0/4176246230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 msgr2=0x560077fa7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.054+0000 7ffb6550a800 1 --2- 172.21.15.165:0/4176246230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077fa7360 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7ffb54056040 tx=0x7ffb5404e000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.054+0000 7ffb6550a800 1 -- 172.21.15.165:0/4176246230 shutdown_connections 2024-04-27T03:30:29.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.054+0000 7ffb6550a800 1 --2- 172.21.15.165:0/4176246230 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077fa7360 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.054+0000 7ffb6550a800 1 -- 172.21.15.165:0/4176246230 >> 172.21.15.165:0/4176246230 conn(0x560077eb2380 msgr2=0x560077fa2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.054+0000 7ffb6550a800 1 -- 172.21.15.165:0/4176246230 shutdown_connections 2024-04-27T03:30:29.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.054+0000 7ffb6550a800 1 -- 172.21.15.165:0/4176246230 wait complete. 2024-04-27T03:30:29.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.054+0000 7ffb6550a800 1 Processor -- start 2024-04-27T03:30:29.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.054+0000 7ffb6550a800 1 -- start start 2024-04-27T03:30:29.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.054+0000 7ffb6550a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077fb31d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.054+0000 7ffb6550a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560077e2a6f0 con 0x560077eb6d20 2024-04-27T03:30:29.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.055+0000 7ffb63009640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077fb31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.055+0000 7ffb63009640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077fb31d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51106/0 (socket says 172.21.15.165:51106) 2024-04-27T03:30:29.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.055+0000 7ffb63009640 1 -- 172.21.15.165:0/2496496030 learned_addr learned my addr 172.21.15.165:0/2496496030 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.055+0000 7ffb63009640 1 -- 172.21.15.165:0/2496496030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560077fb3710 con 0x560077eb6d20 2024-04-27T03:30:29.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.055+0000 7ffb63009640 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077fb31d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb54000c00 tx=0x7ffb54002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.055+0000 7ffa4fe07640 1 -- 172.21.15.165:0/2496496030 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5406e070 con 0x560077eb6d20 2024-04-27T03:30:29.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.055+0000 7ffa4fe07640 1 -- 172.21.15.165:0/2496496030 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb540082d0 con 0x560077eb6d20 2024-04-27T03:30:29.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.055+0000 7ffa4fe07640 1 -- 172.21.15.165:0/2496496030 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54065040 con 0x560077eb6d20 2024-04-27T03:30:29.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.055+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560077fb39a0 con 0x560077eb6d20 2024-04-27T03:30:29.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.055+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560077fb6650 con 0x560077eb6d20 2024-04-27T03:30:29.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.056+0000 7ffa4fe07640 1 -- 172.21.15.165:0/2496496030 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffb54071070 con 0x560077eb6d20 2024-04-27T03:30:29.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.056+0000 7ffa4fe07640 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa48036900 0x7ffa48038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.056+0000 7ffa4fe07640 1 -- 172.21.15.165:0/2496496030 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ffb5404ce00 con 0x560077eb6d20 2024-04-27T03:30:29.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.057+0000 7ffb5bfff640 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa48036900 0x7ffa48038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.057+0000 7ffb6550a800 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560077eb7f80 0x560077fba410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.057+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077fba990 con 0x560077eb7f80 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.057+0000 7ffb6380a640 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560077eb7f80 0x560077fba410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.057+0000 7ffb5bfff640 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa48036900 0x7ffa48038dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c000f60 tx=0x7ffb4c0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.057+0000 7ffb6380a640 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560077eb7f80 0x560077fba410 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.057+0000 7ffb6380a640 1 -- 172.21.15.165:0/2496496030 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077eb7f80 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.057+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077fb79e0 con 0x560077eb7f80 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.058+0000 7ffb6380a640 1 -- 172.21.15.165:0/2496496030 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077eb7f80 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.058+0000 7ffb6550a800 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560077fbccf0 0x560077fbf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.058+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077fbf780 con 0x560077fbccf0 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.058+0000 7ffb63009640 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560077fbccf0 0x560077fbf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.058+0000 7ffb63009640 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560077fbccf0 0x560077fbf0e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.058+0000 7ffb63009640 1 -- 172.21.15.165:0/2496496030 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ffb54070030 con 0x560077fbccf0 2024-04-27T03:30:29.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.058+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077fbbca0 con 0x560077eb7f80 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.059+0000 7ffb6380a640 1 -- 172.21.15.165:0/2496496030 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077eb7f80 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.059+0000 7ffb6550a800 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560077fc0c30 0x560077fc3080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.059+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077fc37e0 con 0x560077fc0c30 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.059+0000 7ffb5bfff640 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560077fc0c30 0x560077fc3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.059+0000 7ffb5bfff640 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560077fc0c30 0x560077fc3080 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.059+0000 7ffb5bfff640 1 -- 172.21.15.165:0/2496496030 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ffb4c009050 con 0x560077fc0c30 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.060+0000 7ffb6550a800 1 Processor -- start 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.060+0000 7ffb6550a800 1 -- start start 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.060+0000 7ffb6550a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fd1c40 0x560077fd2060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.060+0000 7ffb6550a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560077da96b0 con 0x560077fd1c40 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.060+0000 7ffb63009640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fd1c40 0x560077fd2060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.060+0000 7ffb63009640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fd1c40 0x560077fd2060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51122/0 (socket says 172.21.15.165:51122) 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.060+0000 7ffb63009640 1 -- 172.21.15.165:0/3330141801 learned_addr learned my addr 172.21.15.165:0/3330141801 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.061+0000 7ffb63009640 1 -- 172.21.15.165:0/3330141801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560077fd25a0 con 0x560077fd1c40 2024-04-27T03:30:29.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.061+0000 7ffb63009640 1 --2- 172.21.15.165:0/3330141801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fd1c40 0x560077fd2060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffb5408b6d0 tx=0x7ffb54060c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.061+0000 7ffa4d602640 1 -- 172.21.15.165:0/3330141801 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54079040 con 0x560077fd1c40 2024-04-27T03:30:29.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.061+0000 7ffa4d602640 1 -- 172.21.15.165:0/3330141801 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb540723b0 con 0x560077fd1c40 2024-04-27T03:30:29.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.061+0000 7ffb6550a800 1 -- 172.21.15.165:0/3330141801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560077fd3010 con 0x560077fd1c40 2024-04-27T03:30:29.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.061+0000 7ffa4d602640 1 -- 172.21.15.165:0/3330141801 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5408a130 con 0x560077fd1c40 2024-04-27T03:30:29.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.061+0000 7ffb6550a800 1 -- 172.21.15.165:0/3330141801 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560077fd3510 con 0x560077fd1c40 2024-04-27T03:30:29.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.062+0000 7ffa4d602640 1 -- 172.21.15.165:0/3330141801 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffb5408bb30 con 0x560077fd1c40 2024-04-27T03:30:29.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.062+0000 7ffa4d602640 1 --2- 172.21.15.165:0/3330141801 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa3c036900 0x7ffa3c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.062+0000 7ffa4d602640 1 -- 172.21.15.165:0/3330141801 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ffb54081ae0 con 0x560077fd1c40 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.062+0000 7ffb5bfff640 1 --2- 172.21.15.165:0/3330141801 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa3c036900 0x7ffa3c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.062+0000 7ffb6550a800 1 Processor -- start 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.062+0000 7ffb6550a800 1 -- start start 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.062+0000 7ffb5bfff640 1 --2- 172.21.15.165:0/3330141801 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa3c036900 0x7ffa3c038dc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00bf50 tx=0x7ffb4c00f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.063+0000 7ffb6550a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 0x560077fdf120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.063+0000 7ffb6550a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560077fa7c50 con 0x560077fded00 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.063+0000 7ffb63009640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 0x560077fdf120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.063+0000 7ffb63009640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 0x560077fdf120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51126/0 (socket says 172.21.15.165:51126) 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.063+0000 7ffb63009640 1 -- 172.21.15.165:0/452079611 learned_addr learned my addr 172.21.15.165:0/452079611 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.063+0000 7ffb63009640 1 -- 172.21.15.165:0/452079611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560077fdf6f0 con 0x560077fded00 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.063+0000 7ffb63009640 1 --2- 172.21.15.165:0/452079611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 0x560077fdf120 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb54085fd0 tx=0x7ffb54085830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe7ff3330a737e94 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.063+0000 7ffa46ffd640 1 -- 172.21.15.165:0/452079611 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb540a4070 con 0x560077fded00 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.063+0000 7ffa46ffd640 1 -- 172.21.15.165:0/452079611 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb54007ca0 con 0x560077fded00 2024-04-27T03:30:29.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.063+0000 7ffa46ffd640 1 -- 172.21.15.165:0/452079611 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb54097070 con 0x560077fded00 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.064+0000 7ffb6550a800 1 -- 172.21.15.165:0/452079611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 msgr2=0x560077fdf120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.064+0000 7ffb6550a800 1 --2- 172.21.15.165:0/452079611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 0x560077fdf120 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ffb54085fd0 tx=0x7ffb54085830 comp rx=0 tx=0).stop 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.064+0000 7ffb6550a800 1 -- 172.21.15.165:0/452079611 shutdown_connections 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.064+0000 7ffb6550a800 1 --2- 172.21.15.165:0/452079611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 0x560077fdf120 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.064+0000 7ffb6550a800 1 -- 172.21.15.165:0/452079611 >> 172.21.15.165:0/452079611 conn(0x560077fdb060 msgr2=0x560077fd90c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.064+0000 7ffb6550a800 1 -- 172.21.15.165:0/452079611 shutdown_connections 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.064+0000 7ffb6550a800 1 -- 172.21.15.165:0/452079611 wait complete. 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.064+0000 7ffb6550a800 1 Processor -- start 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.064+0000 7ffb6550a800 1 -- start start 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.064+0000 7ffb6550a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 0x560077fe60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.064+0000 7ffb6550a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560077fdfda0 con 0x560077fded00 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.065+0000 7ffb63009640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 0x560077fe60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.065+0000 7ffb63009640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 0x560077fe60f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51134/0 (socket says 172.21.15.165:51134) 2024-04-27T03:30:29.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.065+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 learned_addr learned my addr 172.21.15.165:0/3374570988 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.065+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560077fe6630 con 0x560077fded00 2024-04-27T03:30:29.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.065+0000 7ffb63009640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 0x560077fe60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffb5408a520 tx=0x7ffb5407cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.065+0000 7ffa457fa640 1 -- 172.21.15.165:0/3374570988 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb540a4040 con 0x560077fded00 2024-04-27T03:30:29.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.065+0000 7ffa457fa640 1 -- 172.21.15.165:0/3374570988 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb540af020 con 0x560077fded00 2024-04-27T03:30:29.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.065+0000 7ffa457fa640 1 -- 172.21.15.165:0/3374570988 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5409a040 con 0x560077fded00 2024-04-27T03:30:29.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.065+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560077fe6630 con 0x560077fded00 2024-04-27T03:30:29.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.066+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560077fe6630 con 0x560077fded00 2024-04-27T03:30:29.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.066+0000 7ffa457fa640 1 -- 172.21.15.165:0/3374570988 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffb5405d8b0 con 0x560077fded00 2024-04-27T03:30:29.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.066+0000 7ffa457fa640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa2c036900 0x7ffa2c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.066+0000 7ffa457fa640 1 -- 172.21.15.165:0/3374570988 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ffb540b0030 con 0x560077fded00 2024-04-27T03:30:29.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.067+0000 7ffb5bfff640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa2c036900 0x7ffa2c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.067+0000 7ffb6550a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.067+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560077ff18a0 0x560077ff3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.067+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077ff44b0 con 0x560077ff18a0 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.067+0000 7ffb6380a640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560077ff18a0 0x560077ff3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.067+0000 7ffb5bfff640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa2c036900 0x7ffa2c038dc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c001b60 tx=0x7ffb4c010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.067+0000 7ffb6380a640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560077ff18a0 0x560077ff3d60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.067+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077ff18a0 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.067+0000 7ffb6550a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.067+0000 7ffb6550a800 20 realm 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.067+0000 7ffb6550a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.067+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077ff48c0 con 0x560077ff18a0 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077ff18a0 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6550a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6550a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6550a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5190 con 0x560077ff18a0 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077ff18a0 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6550a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6550a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff18a0 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077ff18a0 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6550a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6550a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560077ff6290 0x560077ff8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077ff8c90 con 0x560077ff6290 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.068+0000 7ffb63009640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560077ff6290 0x560077ff8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.069+0000 7ffb63009640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560077ff6290 0x560077ff8680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.069+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.069+0000 7ffb6550a800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.069+0000 7ffb6550a800 20 searching for the correct realm 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.069+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ffa5c0 con 0x560077ff6290 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.069+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.069+0000 7ffb63009640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffb5408f060 0x7ffb5407d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.069+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ffa5c0 con 0x7ffb5408f060 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.069+0000 7ffb5bfff640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffb5408f060 0x7ffb5407d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.070+0000 7ffb5bfff640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffb5408f060 0x7ffb5407d700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.070+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.070+0000 7ffb5bfff640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffb4c037040 0x7ffb4c036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.070+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ffa5c0 con 0x7ffb4c037040 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.070+0000 7ffb6380a640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffb4c037040 0x7ffb4c036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.071+0000 7ffb6380a640 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffb4c037040 0x7ffb4c036410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.071+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x7ffb4c037040 2024-04-27T03:30:29.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.071+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff18a0 2024-04-27T03:30:29.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.071+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077ff18a0 2024-04-27T03:30:29.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.071+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff18a0 2024-04-27T03:30:29.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.072+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077ff18a0 2024-04-27T03:30:29.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.072+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb4c037040 2024-04-27T03:30:29.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.072+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x7ffb4c037040 2024-04-27T03:30:29.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.072+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb4c037040 2024-04-27T03:30:29.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.072+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x7ffb4c037040 2024-04-27T03:30:29.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.072+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb5408f060 2024-04-27T03:30:29.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.072+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.072+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff6290 2024-04-27T03:30:29.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.073+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.073+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff18a0 2024-04-27T03:30:29.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.073+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077ff18a0 2024-04-27T03:30:29.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.073+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb4c037040 2024-04-27T03:30:29.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.073+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x7ffb4c037040 2024-04-27T03:30:29.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.073+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff18a0 2024-04-27T03:30:29.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.073+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077ff18a0 2024-04-27T03:30:29.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.073+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb5408f060 2024-04-27T03:30:29.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.074+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.074+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff6290 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.074+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.074+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff6290 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.074+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.074+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb4c037040 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.075+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x7ffb4c037040 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.075+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff6290 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.075+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v18'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.075+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff6290 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.075+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.075+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff18a0 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.075+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077ff18a0 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.075+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff18a0 2024-04-27T03:30:29.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.075+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ffb5c00a4a0 con 0x560077ff18a0 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.075+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb4c037040 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.076+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c033040 con 0x7ffb4c037040 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.076+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff6290 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.076+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.076+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb5408f060 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.076+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.076+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff18a0 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.077+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.077+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb4c037040 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.077+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.077+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff6290 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.077+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.077+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb4c037040 2024-04-27T03:30:29.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.077+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.077+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb5408f060 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.078+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.078+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff6290 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.078+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.078+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb4c037040 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.078+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.078+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x560077ff18a0 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560077ff5530 con 0x7ffb5408f060 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:29.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077ff9550 con 0x560077ff18a0 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077ff9940 con 0x560077ff18a0 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.079+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077ff9940 con 0x7ffb4c037040 2024-04-27T03:30:29.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6550a800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6550a800 20 zone default found 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6550a800 4 Realm: () 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6550a800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6550a800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6550a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6550a800 20 zonegroup default 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6550a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077ff90d0 con 0x7ffb4c037040 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6550a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6550a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077ff9120 con 0x560077ff6290 2024-04-27T03:30:29.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.080+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.081+0000 7ffb6550a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.081+0000 7ffb6550a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.081+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560077ffc570 con 0x560077ff6290 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.081+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.081+0000 7ffb6550a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.081+0000 7ffb6550a800 20 started sync module instance, tier type = 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.081+0000 7ffb6550a800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.081+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560077ffb8c0 con 0x7ffb5408f060 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.082+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.082+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94560013237472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560077ffce80 con 0x7ffb5408f060 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.082+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560077ffd6d0 con 0x560077ff18a0 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.083+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94560013237472] v25'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.083+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.083+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94560013240784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560077ffd7b0 con 0x560077ff18a0 2024-04-27T03:30:29.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.083+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560077ffe2d0 con 0x560077ff6290 2024-04-27T03:30:29.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.085+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.085+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94560013245856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560077ffeb70 con 0x560077ff6290 2024-04-27T03:30:29.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.085+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560077fff620 con 0x7ffb5408f060 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.085+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94560013240784] v25'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.086+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94560013245856] v25'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.088+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.088+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94560013242912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560077fffa40 con 0x7ffb5408f060 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.088+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078000290 con 0x560077ff18a0 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.091+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.091+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94560013254624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078000b30 con 0x560077ff18a0 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.091+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078001600 con 0x7ffb4c037040 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.094+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94560013242912] v25'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.094+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.094+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94560013258992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5600780015c0 con 0x7ffb4c037040 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.094+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078002040 con 0x7ffb4c037040 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.095+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94560013258992] v25'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.096+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.096+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94560013250864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078002040 con 0x7ffb4c037040 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.096+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078002a00 con 0x7ffb4c037040 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.096+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94560013254624] v25'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.097+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'6 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.097+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94560013256320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078002a00 con 0x7ffb4c037040 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.097+0000 7ffb6550a800 20 add_watcher() i=0 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.097+0000 7ffb6550a800 20 add_watcher() i=1 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.097+0000 7ffb6550a800 20 add_watcher() i=2 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.097+0000 7ffb6550a800 20 add_watcher() i=3 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.097+0000 7ffb6550a800 20 add_watcher() i=4 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.097+0000 7ffb6550a800 20 add_watcher() i=5 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.097+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94560013250864] v25'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.097+0000 7ffb6550a800 20 add_watcher() i=6 2024-04-27T03:30:29.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.098+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94560013256320] v25'7 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.099+0000 7ffb6550a800 20 add_watcher() i=7 2024-04-27T03:30:29.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.099+0000 7ffb6550a800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:29.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.099+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560078002a00 con 0x560077ff18a0 2024-04-27T03:30:29.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.099+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.099+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94560013264208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078002970 con 0x560077ff18a0 2024-04-27T03:30:29.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.100+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94560013264208] v25'32 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.100+0000 7ffa19fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:29.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.100+0000 7ffb6550a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:29.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.100+0000 7ffb6550a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:29.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.100+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56007800e230 con 0x560077ff18a0 2024-04-27T03:30:29.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.102+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'40 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.103+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56007800e880 con 0x7ffb4c037040 2024-04-27T03:30:29.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.104+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'16 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.104+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56007800ece0 con 0x560077ff6290 2024-04-27T03:30:29.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.105+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'16 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.105+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56007800ec20 con 0x560077ff18a0 2024-04-27T03:30:29.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.106+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'33 uv15 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.106+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56007800f310 con 0x7ffb5408f060 2024-04-27T03:30:29.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.107+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'26 uv17 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.107+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56007800fb60 con 0x560077ff6290 2024-04-27T03:30:29.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.108+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'25 uv16 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.108+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56007800fb60 con 0x560077ff18a0 2024-04-27T03:30:29.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.110+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'18 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.110+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56007800fb60 con 0x7ffb4c037040 2024-04-27T03:30:29.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.111+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'19 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.111+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56007800fb60 con 0x7ffb5408f060 2024-04-27T03:30:29.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.112+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'10 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.112+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5600780101b0 con 0x560077ff6290 2024-04-27T03:30:29.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.113+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'16 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.113+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5600780101b0 con 0x7ffb5408f060 2024-04-27T03:30:29.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.114+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'27 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.114+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010660 con 0x7ffb4c037040 2024-04-27T03:30:29.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.116+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'12 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.116+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010660 con 0x7ffb5408f060 2024-04-27T03:30:29.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.117+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'16 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.117+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010660 con 0x7ffb4c037040 2024-04-27T03:30:29.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.118+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'31 uv22 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.118+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010660 con 0x560077ff18a0 2024-04-27T03:30:29.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.119+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'34 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.119+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010660 con 0x560077ff6290 2024-04-27T03:30:29.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.120+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'20 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.120+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010660 con 0x7ffb5408f060 2024-04-27T03:30:29.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.121+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'28 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.121+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010660 con 0x560077ff6290 2024-04-27T03:30:29.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.122+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'22 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.122+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010660 con 0x560077ff18a0 2024-04-27T03:30:29.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.123+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.123+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010660 con 0x560077ff18a0 2024-04-27T03:30:29.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.124+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'35 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.124+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010660 con 0x560077ff6290 2024-04-27T03:30:29.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.125+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.125+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010660 con 0x7ffb5408f060 2024-04-27T03:30:29.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.126+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'26 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.126+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010aa0 con 0x560077ff6290 2024-04-27T03:30:29.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.127+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.127+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010aa0 con 0x560077ff6290 2024-04-27T03:30:29.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.128+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'29 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.128+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010aa0 con 0x560077ff6290 2024-04-27T03:30:29.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.129+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'12 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.129+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010aa0 con 0x560077ff6290 2024-04-27T03:30:29.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.130+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'26 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.130+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010aa0 con 0x7ffb4c037040 2024-04-27T03:30:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.132+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.132+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010aa0 con 0x560077ff6290 2024-04-27T03:30:29.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.133+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'22 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.133+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010aa0 con 0x7ffb4c037040 2024-04-27T03:30:29.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.134+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'20 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.134+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010aa0 con 0x560077ff6290 2024-04-27T03:30:29.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.135+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'22 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.135+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010aa0 con 0x7ffb4c037040 2024-04-27T03:30:29.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.136+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'32 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.136+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078010aa0 con 0x7ffb5408f060 2024-04-27T03:30:29.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.137+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.138+0000 7ffa18fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:29.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.138+0000 7ffb6550a800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:29.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.138+0000 7ffb6550a800 20 Filter name: none 2024-04-27T03:30:29.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.138+0000 7ff9927fc640 20 reqs_thread_entry: start 2024-04-27T03:30:29.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.138+0000 7ffb6550a800 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-04-27T03:30:29.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.138+0000 7ffb6550a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.138+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56007803b5c0 con 0x7ffb5408f060 2024-04-27T03:30:29.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.138+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(94 bud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.138+0000 7ffb6550a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.138+0000 7ffb6550a800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x0 2024-04-27T03:30:29.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.138+0000 7ffb6550a800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T03:30:29.145 INFO:teuthology.orchestra.run.smithi165.stderr:could not remove user: unable to remove user, user does not exist 2024-04-27T03:30:29.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.139+0000 7ffa19fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:29.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.139+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94560013264208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56007800c090 con 0x560077ff18a0 2024-04-27T03:30:29.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.140+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94560013264208] v25'36 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.140+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94560013237472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078002550 con 0x7ffb5408f060 2024-04-27T03:30:29.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.141+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94560013237472] v25'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.141+0000 7ffb6550a800 20 remove_watcher() i=0 2024-04-27T03:30:29.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.141+0000 7ffb6550a800 2 removed watcher, disabling cache 2024-04-27T03:30:29.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.141+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94560013240784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078002550 con 0x560077ff18a0 2024-04-27T03:30:29.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.142+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94560013240784] v25'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.142+0000 7ffb6550a800 20 remove_watcher() i=1 2024-04-27T03:30:29.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.142+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94560013245856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078002550 con 0x560077ff6290 2024-04-27T03:30:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.143+0000 7ffb63009640 1 -- 172.21.15.165:0/3374570988 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94560013245856] v25'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb540a7070 con 0x560077ff6290 2024-04-27T03:30:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.143+0000 7ffb6550a800 20 remove_watcher() i=2 2024-04-27T03:30:29.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.143+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94560013242912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560077ffdda0 con 0x7ffb5408f060 2024-04-27T03:30:29.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.144+0000 7ffb5bfff640 1 -- 172.21.15.165:0/3374570988 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94560013242912] v25'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4c009050 con 0x7ffb5408f060 2024-04-27T03:30:29.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.144+0000 7ffb6550a800 20 remove_watcher() i=3 2024-04-27T03:30:29.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.144+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94560013254624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078002550 con 0x560077ff18a0 2024-04-27T03:30:29.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.147+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94560013254624] v25'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x560077ff18a0 2024-04-27T03:30:29.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.147+0000 7ffb6550a800 20 remove_watcher() i=4 2024-04-27T03:30:29.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.147+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94560013258992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560077ffdda0 con 0x7ffb4c037040 2024-04-27T03:30:29.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.148+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94560013258992] v25'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.148+0000 7ffb6550a800 20 remove_watcher() i=5 2024-04-27T03:30:29.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.148+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94560013250864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560078002550 con 0x7ffb4c037040 2024-04-27T03:30:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.150+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94560013250864] v25'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.150+0000 7ffb6550a800 20 remove_watcher() i=6 2024-04-27T03:30:29.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.150+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94560013256320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5600780010f0 con 0x7ffb4c037040 2024-04-27T03:30:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.151+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94560013256320] v25'8 uv6 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb5c031040 con 0x7ffb4c037040 2024-04-27T03:30:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.151+0000 7ffb6550a800 20 remove_watcher() i=7 2024-04-27T03:30:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffb4c037040 msgr2=0x7ffb4c036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffb4c037040 0x7ffb4c036410 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560077ff18a0 msgr2=0x560077ff3d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560077ff18a0 0x560077ff3d60 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffb5408f060 msgr2=0x7ffb5407d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffb5408f060 0x7ffb5407d700 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560077ff6290 msgr2=0x560077ff8680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560077ff6290 0x560077ff8680 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa2c036900 msgr2=0x7ffa2c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa2c036900 0x7ffa2c038dc0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c001b60 tx=0x7ffb4c010000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 msgr2=0x560077fe60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6380a640 1 -- 172.21.15.165:0/3374570988 reap_dead start 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 0x560077fe60f0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7ffb5408a520 tx=0x7ffb5407cef0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 shutdown_connections 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3374570988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fded00 0x560077fe60f0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 >> 172.21.15.165:0/3374570988 conn(0x560077fdb060 msgr2=0x560077fd86c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 shutdown_connections 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3374570988 wait complete. 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3330141801 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa3c036900 msgr2=0x7ffa3c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3330141801 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa3c036900 0x7ffa3c038dc0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c00bf50 tx=0x7ffb4c00f000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3330141801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fd1c40 msgr2=0x560077fd2060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3330141801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fd1c40 0x560077fd2060 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ffb5408b6d0 tx=0x7ffb54060c60 comp rx=0 tx=0).stop 2024-04-27T03:30:29.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3330141801 shutdown_connections 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3330141801 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa3c036900 0x7ffa3c038dc0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 --2- 172.21.15.165:0/3330141801 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077fd1c40 0x560077fd2060 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3330141801 >> 172.21.15.165:0/3330141801 conn(0x560077fc5ce0 msgr2=0x560077fc60e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3330141801 shutdown_connections 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.152+0000 7ffb6550a800 1 -- 172.21.15.165:0/3330141801 wait complete. 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560077fc0c30 msgr2=0x560077fc3080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560077fc0c30 0x560077fc3080 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560077eb7f80 msgr2=0x560077fba410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560077eb7f80 0x560077fba410 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560077fbccf0 msgr2=0x560077fbf0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560077fbccf0 0x560077fbf0e0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa48036900 msgr2=0x7ffa48038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa48036900 0x7ffa48038dc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ffb4c000f60 tx=0x7ffb4c0012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 msgr2=0x560077fb31d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 --2- 172.21.15.165:0/2496496030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560077eb6d20 0x560077fb31d0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7ffb54000c00 tx=0x7ffb54002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6380a640 1 -- 172.21.15.165:0/2496496030 reap_dead start 2024-04-27T03:30:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 shutdown_connections 2024-04-27T03:30:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 >> 172.21.15.165:0/2496496030 conn(0x560077eb2380 msgr2=0x560077fa1500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 shutdown_connections 2024-04-27T03:30:29.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.153+0000 7ffb6550a800 1 -- 172.21.15.165:0/2496496030 wait complete. 2024-04-27T03:30:29.166 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-27T03:30:29.166 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bar'] 2024-04-27T03:30:29.166 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bar'] 2024-04-27T03:30:29.167 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bar 2024-04-27T03:30:29.249 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:29.249 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:29.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.244+0000 7f39adbdc800 1 Processor -- start 2024-04-27T03:30:29.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.244+0000 7f39adbdc800 1 -- start start 2024-04-27T03:30:29.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.244+0000 7f39adbdc800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5562575432e0 0x55625754bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.244+0000 7f39adbdc800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x556257401d40 con 0x5562575417a0 2024-04-27T03:30:29.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.244+0000 7f39adbdc800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5562573d44d0 con 0x5562575432e0 2024-04-27T03:30:29.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.244+0000 7f39aaeda640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5562575432e0 0x55625754bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.244+0000 7f39ab6db640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5562575417a0 0x556257541bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53538/0 (socket says 172.21.15.165:53538) 2024-04-27T03:30:29.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.244+0000 7f39ab6db640 1 -- 172.21.15.165:0/1112390110 learned_addr learned my addr 172.21.15.165:0/1112390110 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:29.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.245+0000 7f39aaeda640 1 -- 172.21.15.165:0/1112390110 >> v1:172.21.15.89:6789/0 conn(0x5562575417a0 legacy=0x556257541bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.245+0000 7f39aaeda640 1 -- 172.21.15.165:0/1112390110 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x556257400590 con 0x5562575432e0 2024-04-27T03:30:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.247+0000 7f39aaeda640 1 --2- 172.21.15.165:0/1112390110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575432e0 0x55625754bde0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f399400a470 tx=0x7f399401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffc28ddb7cd82c6e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.247+0000 7f39aa6d9640 1 -- 172.21.15.165:0/1112390110 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3994057070 con 0x5562575432e0 2024-04-27T03:30:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.247+0000 7f39aa6d9640 1 -- 172.21.15.165:0/1112390110 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3994052440 con 0x5562575432e0 2024-04-27T03:30:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.247+0000 7f39aa6d9640 1 -- 172.21.15.165:0/1112390110 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3994052740 con 0x5562575432e0 2024-04-27T03:30:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.247+0000 7f39adbdc800 1 -- 172.21.15.165:0/1112390110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575432e0 msgr2=0x55625754bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.247+0000 7f39adbdc800 1 --2- 172.21.15.165:0/1112390110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575432e0 0x55625754bde0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f399400a470 tx=0x7f399401ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.247+0000 7f39adbdc800 1 -- 172.21.15.165:0/1112390110 shutdown_connections 2024-04-27T03:30:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.247+0000 7f39adbdc800 1 --2- 172.21.15.165:0/1112390110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575432e0 0x55625754bde0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.247+0000 7f39adbdc800 1 -- 172.21.15.165:0/1112390110 >> 172.21.15.165:0/1112390110 conn(0x55625753d320 msgr2=0x55625753f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.247+0000 7f39adbdc800 1 -- 172.21.15.165:0/1112390110 shutdown_connections 2024-04-27T03:30:29.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.247+0000 7f39adbdc800 1 -- 172.21.15.165:0/1112390110 wait complete. 2024-04-27T03:30:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.259+0000 7f39adbdc800 1 Processor -- start 2024-04-27T03:30:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.259+0000 7f39adbdc800 1 -- start start 2024-04-27T03:30:29.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.259+0000 7f39adbdc800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 0x556257636500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.259+0000 7f39adbdc800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55625748e3c0 con 0x5562575417a0 2024-04-27T03:30:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.259+0000 7f39ab6db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 0x556257636500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.259+0000 7f39ab6db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 0x556257636500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51150/0 (socket says 172.21.15.165:51150) 2024-04-27T03:30:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.259+0000 7f39ab6db640 1 -- 172.21.15.165:0/2943293229 learned_addr learned my addr 172.21.15.165:0/2943293229 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.260+0000 7f39ab6db640 1 -- 172.21.15.165:0/2943293229 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556257636ad0 con 0x5562575417a0 2024-04-27T03:30:29.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.260+0000 7f39ab6db640 1 --2- 172.21.15.165:0/2943293229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 0x556257636500 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f39a4001de0 tx=0x7f39a400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b8f09d269604ebd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.260+0000 7f389960a640 1 -- 172.21.15.165:0/2943293229 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39a4012020 con 0x5562575417a0 2024-04-27T03:30:29.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.260+0000 7f389960a640 1 -- 172.21.15.165:0/2943293229 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a400e5a0 con 0x5562575417a0 2024-04-27T03:30:29.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.260+0000 7f389960a640 1 -- 172.21.15.165:0/2943293229 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39a400e8c0 con 0x5562575417a0 2024-04-27T03:30:29.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39adbdc800 1 -- 172.21.15.165:0/2943293229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 msgr2=0x556257636500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39adbdc800 1 --2- 172.21.15.165:0/2943293229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 0x556257636500 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f39a4001de0 tx=0x7f39a400b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39adbdc800 1 -- 172.21.15.165:0/2943293229 shutdown_connections 2024-04-27T03:30:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39adbdc800 1 --2- 172.21.15.165:0/2943293229 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 0x556257636500 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39adbdc800 1 -- 172.21.15.165:0/2943293229 >> 172.21.15.165:0/2943293229 conn(0x55625753d320 msgr2=0x556257631e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39adbdc800 1 -- 172.21.15.165:0/2943293229 shutdown_connections 2024-04-27T03:30:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39adbdc800 1 -- 172.21.15.165:0/2943293229 wait complete. 2024-04-27T03:30:29.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39adbdc800 1 Processor -- start 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39adbdc800 1 -- start start 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39adbdc800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 0x55625763e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39adbdc800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55625754e600 con 0x5562575417a0 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39ab6db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 0x55625763e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39ab6db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 0x55625763e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51156/0 (socket says 172.21.15.165:51156) 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.261+0000 7f39ab6db640 1 -- 172.21.15.165:0/1220118913 learned_addr learned my addr 172.21.15.165:0/1220118913 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.262+0000 7f39ab6db640 1 -- 172.21.15.165:0/1220118913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55625763e600 con 0x5562575417a0 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.262+0000 7f39ab6db640 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 0x55625763e0c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c010 tx=0x7f39a4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.262+0000 7f3897e07640 1 -- 172.21.15.165:0/1220118913 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39a4012070 con 0x5562575417a0 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.262+0000 7f3897e07640 1 -- 172.21.15.165:0/1220118913 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a401a7a0 con 0x5562575417a0 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.262+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55625763e890 con 0x5562575417a0 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.262+0000 7f3897e07640 1 -- 172.21.15.165:0/1220118913 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39a401aa80 con 0x5562575417a0 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.262+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5562576414a0 con 0x5562575417a0 2024-04-27T03:30:29.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.263+0000 7f3897e07640 1 -- 172.21.15.165:0/1220118913 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f39a4015020 con 0x5562575417a0 2024-04-27T03:30:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.263+0000 7f3897e07640 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3890036900 0x7f3890038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.263+0000 7f3897e07640 1 -- 172.21.15.165:0/1220118913 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f39a4020070 con 0x5562575417a0 2024-04-27T03:30:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.263+0000 7f39aaeda640 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3890036900 0x7f3890038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.263+0000 7f39adbdc800 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5562575432e0 0x556257645260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.263+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5562576457e0 con 0x5562575432e0 2024-04-27T03:30:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.263+0000 7f39abedc640 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5562575432e0 0x556257645260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.264+0000 7f39aaeda640 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3890036900 0x7f3890038dc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3994054000 tx=0x7f399404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.264+0000 7f39abedc640 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5562575432e0 0x556257645260 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.264+0000 7f39abedc640 1 -- 172.21.15.165:0/1220118913 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f399c00a4a0 con 0x5562575432e0 2024-04-27T03:30:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.264+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556257642810 con 0x5562575432e0 2024-04-27T03:30:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.264+0000 7f39abedc640 1 -- 172.21.15.165:0/1220118913 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f399c00a4a0 con 0x5562575432e0 2024-04-27T03:30:29.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.264+0000 7f39adbdc800 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556257647b40 0x556257649f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.264+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55625764a5d0 con 0x556257647b40 2024-04-27T03:30:29.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.264+0000 7f39ab6db640 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556257647b40 0x556257649f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.265+0000 7f39ab6db640 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556257647b40 0x556257649f30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.265+0000 7f39ab6db640 1 -- 172.21.15.165:0/1220118913 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f39a402b070 con 0x556257647b40 2024-04-27T03:30:29.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.265+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556257646af0 con 0x5562575432e0 2024-04-27T03:30:29.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.265+0000 7f39abedc640 1 -- 172.21.15.165:0/1220118913 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f399c00a4a0 con 0x5562575432e0 2024-04-27T03:30:29.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.265+0000 7f39adbdc800 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55625764ba80 0x55625764ded0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.265+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55625764e630 con 0x55625764ba80 2024-04-27T03:30:29.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.266+0000 7f39aaeda640 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55625764ba80 0x55625764ded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.266+0000 7f39aaeda640 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55625764ba80 0x55625764ded0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.266+0000 7f39aaeda640 1 -- 172.21.15.165:0/1220118913 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3994052910 con 0x55625764ba80 2024-04-27T03:30:29.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.267+0000 7f39adbdc800 1 Processor -- start 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.267+0000 7f39adbdc800 1 -- start start 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.267+0000 7f39adbdc800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55625765ca40 0x55625765ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.267+0000 7f39adbdc800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5562574346b0 con 0x55625765ca40 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.267+0000 7f39ab6db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55625765ca40 0x55625765ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.267+0000 7f39ab6db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55625765ca40 0x55625765ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51164/0 (socket says 172.21.15.165:51164) 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.267+0000 7f39ab6db640 1 -- 172.21.15.165:0/3855042838 learned_addr learned my addr 172.21.15.165:0/3855042838 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.267+0000 7f39ab6db640 1 -- 172.21.15.165:0/3855042838 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55625765d3a0 con 0x55625765ca40 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.268+0000 7f39ab6db640 1 --2- 172.21.15.165:0/3855042838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55625765ca40 0x55625765ce60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f39a40224e0 tx=0x7f39a4016c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.268+0000 7f3895602640 1 -- 172.21.15.165:0/3855042838 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39a4049470 con 0x55625765ca40 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.268+0000 7f39adbdc800 1 -- 172.21.15.165:0/3855042838 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55625765de10 con 0x55625765ca40 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.268+0000 7f3895602640 1 -- 172.21.15.165:0/3855042838 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a4044020 con 0x55625765ca40 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.268+0000 7f3895602640 1 -- 172.21.15.165:0/3855042838 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39a4044300 con 0x55625765ca40 2024-04-27T03:30:29.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.268+0000 7f39adbdc800 1 -- 172.21.15.165:0/3855042838 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55625765e310 con 0x55625765ca40 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.268+0000 7f3895602640 1 -- 172.21.15.165:0/3855042838 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f39a40163b0 con 0x55625765ca40 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.269+0000 7f3895602640 1 --2- 172.21.15.165:0/3855042838 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3884036970 0x7f3884038e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.269+0000 7f3895602640 1 -- 172.21.15.165:0/3855042838 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f39a404bd20 con 0x55625765ca40 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.269+0000 7f39aaeda640 1 --2- 172.21.15.165:0/3855042838 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3884036970 0x7f3884038e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.269+0000 7f39adbdc800 1 Processor -- start 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.269+0000 7f39adbdc800 1 -- start start 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.269+0000 7f39aaeda640 1 --2- 172.21.15.165:0/3855042838 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3884036970 0x7f3884038e30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x556257642750 tx=0x7f3994005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.269+0000 7f39adbdc800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 0x556257669ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.269+0000 7f39adbdc800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556257636e50 con 0x556257669bd0 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.269+0000 7f39ab6db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 0x556257669ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.270+0000 7f39ab6db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 0x556257669ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51180/0 (socket says 172.21.15.165:51180) 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.270+0000 7f39ab6db640 1 -- 172.21.15.165:0/1965634835 learned_addr learned my addr 172.21.15.165:0/1965634835 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.270+0000 7f39ab6db640 1 -- 172.21.15.165:0/1965634835 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55625766a5c0 con 0x556257669bd0 2024-04-27T03:30:29.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.270+0000 7f39ab6db640 1 --2- 172.21.15.165:0/1965634835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 0x556257669ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f39a404a8f0 tx=0x7f39a400cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=583206191795ef4a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.270+0000 7f388effd640 1 -- 172.21.15.165:0/1965634835 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39a405c070 con 0x556257669bd0 2024-04-27T03:30:29.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.270+0000 7f388effd640 1 -- 172.21.15.165:0/1965634835 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a404bb10 con 0x556257669bd0 2024-04-27T03:30:29.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.270+0000 7f388effd640 1 -- 172.21.15.165:0/1965634835 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39a4053040 con 0x556257669bd0 2024-04-27T03:30:29.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39adbdc800 1 -- 172.21.15.165:0/1965634835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 msgr2=0x556257669ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39adbdc800 1 --2- 172.21.15.165:0/1965634835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 0x556257669ff0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f39a404a8f0 tx=0x7f39a400cab0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39adbdc800 1 -- 172.21.15.165:0/1965634835 shutdown_connections 2024-04-27T03:30:29.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39adbdc800 1 --2- 172.21.15.165:0/1965634835 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 0x556257669ff0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39adbdc800 1 -- 172.21.15.165:0/1965634835 >> 172.21.15.165:0/1965634835 conn(0x556257665e50 msgr2=0x556257663eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39adbdc800 1 -- 172.21.15.165:0/1965634835 shutdown_connections 2024-04-27T03:30:29.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39adbdc800 1 -- 172.21.15.165:0/1965634835 wait complete. 2024-04-27T03:30:29.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39adbdc800 1 Processor -- start 2024-04-27T03:30:29.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39adbdc800 1 -- start start 2024-04-27T03:30:29.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39adbdc800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 0x556257670f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39adbdc800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55625766ac30 con 0x556257669bd0 2024-04-27T03:30:29.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39ab6db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 0x556257670f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39ab6db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 0x556257670f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51190/0 (socket says 172.21.15.165:51190) 2024-04-27T03:30:29.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.271+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 learned_addr learned my addr 172.21.15.165:0/441894928 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.272+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556257671470 con 0x556257669bd0 2024-04-27T03:30:29.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.272+0000 7f39ab6db640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 0x556257670f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f39a400cb80 tx=0x7f39a404b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.272+0000 7f388d7fa640 1 -- 172.21.15.165:0/441894928 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39a4065020 con 0x556257669bd0 2024-04-27T03:30:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.272+0000 7f388d7fa640 1 -- 172.21.15.165:0/441894928 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39a40167d0 con 0x556257669bd0 2024-04-27T03:30:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.272+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556257671470 con 0x556257669bd0 2024-04-27T03:30:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.272+0000 7f388d7fa640 1 -- 172.21.15.165:0/441894928 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39a405c040 con 0x556257669bd0 2024-04-27T03:30:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.272+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556257671470 con 0x556257669bd0 2024-04-27T03:30:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.273+0000 7f388d7fa640 1 -- 172.21.15.165:0/441894928 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f39a4068020 con 0x556257669bd0 2024-04-27T03:30:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.273+0000 7f388d7fa640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3874036900 0x7f3874038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.273+0000 7f388d7fa640 1 -- 172.21.15.165:0/441894928 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f39a406bab0 con 0x556257669bd0 2024-04-27T03:30:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.273+0000 7f39aaeda640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3874036900 0x7f3874038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.274+0000 7f39adbdc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.274+0000 7f39adbdc800 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55625767c680 0x55625767eb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.274+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55625767f290 con 0x55625767c680 2024-04-27T03:30:29.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.274+0000 7f39aaeda640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3874036900 0x7f3874038dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f399401ced0 tx=0x7f399404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.274+0000 7f39abedc640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55625767c680 0x55625767eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.274+0000 7f39abedc640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55625767c680 0x55625767eb40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.274+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f399c00a4a0 con 0x55625767c680 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.274+0000 7f39adbdc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.274+0000 7f39adbdc800 20 realm 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.274+0000 7f39adbdc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.274+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55625767f6a0 con 0x55625767c680 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f399c00a4a0 con 0x55625767c680 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39adbdc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39adbdc800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39adbdc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55625767ff70 con 0x55625767c680 2024-04-27T03:30:29.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f399c00a4a0 con 0x55625767c680 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39adbdc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39adbdc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556257680310 con 0x55625767c680 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f399c00a4a0 con 0x55625767c680 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39adbdc800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39adbdc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39adbdc800 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556257681070 0x556257683460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556257683a70 con 0x556257681070 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39ab6db640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556257681070 0x556257683460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.275+0000 7f39ab6db640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556257681070 0x556257683460 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.276+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.276+0000 7f39adbdc800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:29.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.276+0000 7f39adbdc800 20 searching for the correct realm 2024-04-27T03:30:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.276+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5562576853a0 con 0x556257681070 2024-04-27T03:30:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.276+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.276+0000 7f39ab6db640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f39a406a020 0x7f39a40258d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.276+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5562576853a0 con 0x7f39a406a020 2024-04-27T03:30:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.276+0000 7f39aaeda640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f39a406a020 0x7f39a40258d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.277+0000 7f39aaeda640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f39a406a020 0x7f39a40258d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.277+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.277+0000 7f39aaeda640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3994076040 0x7f3994083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.277+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5562576853a0 con 0x7f3994076040 2024-04-27T03:30:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.278+0000 7f39abedc640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3994076040 0x7f3994083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.278+0000 7f39abedc640 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3994076040 0x7f3994083410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.278+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f399c00a4a0 con 0x7f3994076040 2024-04-27T03:30:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.278+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556257680310 con 0x55625767c680 2024-04-27T03:30:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.279+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c00a4a0 con 0x55625767c680 2024-04-27T03:30:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.279+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556257680310 con 0x55625767c680 2024-04-27T03:30:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.279+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c00a4a0 con 0x55625767c680 2024-04-27T03:30:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.279+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556257680310 con 0x7f3994076040 2024-04-27T03:30:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.279+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c00a4a0 con 0x7f3994076040 2024-04-27T03:30:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.279+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556257680310 con 0x7f3994076040 2024-04-27T03:30:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.279+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c00a4a0 con 0x7f3994076040 2024-04-27T03:30:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.279+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556257680310 con 0x7f39a406a020 2024-04-27T03:30:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.280+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.280+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556257680310 con 0x556257681070 2024-04-27T03:30:29.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.280+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.280+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556257680310 con 0x55625767c680 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.280+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c00a4a0 con 0x55625767c680 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.280+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556257680310 con 0x7f3994076040 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.280+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c00a4a0 con 0x7f3994076040 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.280+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556257680310 con 0x55625767c680 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.281+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c00a4a0 con 0x55625767c680 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.281+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556257680310 con 0x7f39a406a020 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.281+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.281+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x556257681070 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.281+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.281+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x556257681070 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.281+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.281+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x7f3994076040 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.282+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c00a4a0 con 0x7f3994076040 2024-04-27T03:30:29.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.282+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x556257681070 2024-04-27T03:30:29.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.282+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v18'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.282+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x556257681070 2024-04-27T03:30:29.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.282+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.282+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x55625767c680 2024-04-27T03:30:29.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.282+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c00a4a0 con 0x55625767c680 2024-04-27T03:30:29.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.282+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x55625767c680 2024-04-27T03:30:29.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.283+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f399c00a4a0 con 0x55625767c680 2024-04-27T03:30:29.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.283+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x7f3994076040 2024-04-27T03:30:29.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.283+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c033040 con 0x7f3994076040 2024-04-27T03:30:29.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.283+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x556257681070 2024-04-27T03:30:29.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.283+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.283+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x7f39a406a020 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.283+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.283+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x55625767c680 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.284+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.284+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x7f3994076040 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.284+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.284+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x556257681070 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.284+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.284+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x7f3994076040 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.284+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.284+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x7f39a406a020 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.285+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.285+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x556257681070 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.285+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.285+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x7f3994076040 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.285+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.285+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x55625767c680 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.285+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.285+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f3994090060 con 0x7f39a406a020 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556257684330 con 0x55625767c680 2024-04-27T03:30:29.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556257684720 con 0x55625767c680 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.286+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556257684720 con 0x7f3994076040 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 20 zone default found 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 4 Realm: () 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:29.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 20 zonegroup default 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556257683eb0 con 0x7f3994076040 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556257683f00 con 0x556257681070 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.287+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556257687330 con 0x556257681070 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.288+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.288+0000 7f39adbdc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.288+0000 7f39adbdc800 20 started sync module instance, tier type = 2024-04-27T03:30:29.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.288+0000 7f39adbdc800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:29.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.288+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5562576866a0 con 0x7f39a406a020 2024-04-27T03:30:29.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.289+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.289+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93880861616784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257687c40 con 0x7f39a406a020 2024-04-27T03:30:29.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.289+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257688490 con 0x55625767c680 2024-04-27T03:30:29.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.292+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.292+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93880861622240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257688a50 con 0x55625767c680 2024-04-27T03:30:29.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.292+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257689550 con 0x556257681070 2024-04-27T03:30:29.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.292+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93880861616784] v25'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.294+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.294+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93880861626432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5562576898f0 con 0x556257681070 2024-04-27T03:30:29.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.294+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625768a3a0 con 0x7f39a406a020 2024-04-27T03:30:29.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.294+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93880861622240] v25'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.297+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93880861626432] v25'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.297+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.297+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93880861620112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5562576884a0 con 0x7f39a406a020 2024-04-27T03:30:29.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.297+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625768adb0 con 0x55625767c680 2024-04-27T03:30:29.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.300+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.300+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93880861633248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625768b510 con 0x55625767c680 2024-04-27T03:30:29.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.300+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625768c030 con 0x7f3994076040 2024-04-27T03:30:29.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.303+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93880861620112] v25'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.303+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.303+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93880861637376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625768bff0 con 0x7f3994076040 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.303+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625768ca40 con 0x7f3994076040 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.306+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93880861633248] v25'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.306+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93880861637376] v25'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.306+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.306+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93880861630080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625768c590 con 0x7f3994076040 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.306+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625768cf50 con 0x7f3994076040 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.307+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93880861630080] v25'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.308+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'9 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.308+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93880861634400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625768cf50 con 0x7f3994076040 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.308+0000 7f39adbdc800 20 add_watcher() i=0 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.308+0000 7f39adbdc800 20 add_watcher() i=1 2024-04-27T03:30:29.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.308+0000 7f39adbdc800 20 add_watcher() i=2 2024-04-27T03:30:29.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.308+0000 7f39adbdc800 20 add_watcher() i=3 2024-04-27T03:30:29.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.308+0000 7f39adbdc800 20 add_watcher() i=4 2024-04-27T03:30:29.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.308+0000 7f39adbdc800 20 add_watcher() i=5 2024-04-27T03:30:29.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.308+0000 7f39adbdc800 20 add_watcher() i=6 2024-04-27T03:30:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.309+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93880861634400] v25'10 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.309+0000 7f39adbdc800 20 add_watcher() i=7 2024-04-27T03:30:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.309+0000 7f39adbdc800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.309+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556257690250 con 0x55625767c680 2024-04-27T03:30:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.309+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.309+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93880861641744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625768d4e0 con 0x55625767c680 2024-04-27T03:30:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.310+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93880861641744] v25'37 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.310+0000 7f3861fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.310+0000 7f39adbdc800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.310+0000 7f39adbdc800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.310+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257698bb0 con 0x55625767c680 2024-04-27T03:30:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.311+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'41 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.311+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257699200 con 0x7f3994076040 2024-04-27T03:30:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.313+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'17 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.313+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257699660 con 0x556257681070 2024-04-27T03:30:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.314+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'17 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.314+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5562576995b0 con 0x55625767c680 2024-04-27T03:30:29.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.316+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'38 uv15 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.316+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257699c00 con 0x7f39a406a020 2024-04-27T03:30:29.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.319+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'29 uv17 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.319+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769a170 con 0x556257681070 2024-04-27T03:30:29.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.320+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'27 uv16 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.320+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769a170 con 0x55625767c680 2024-04-27T03:30:29.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.321+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'19 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.321+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769a620 con 0x7f3994076040 2024-04-27T03:30:29.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.323+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'21 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.323+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769a620 con 0x7f39a406a020 2024-04-27T03:30:29.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.324+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'11 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.324+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769ab90 con 0x556257681070 2024-04-27T03:30:29.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.325+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'17 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.325+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769ab90 con 0x7f39a406a020 2024-04-27T03:30:29.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.326+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'30 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.326+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769ac20 con 0x7f3994076040 2024-04-27T03:30:29.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.327+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'13 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.327+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769ac20 con 0x7f39a406a020 2024-04-27T03:30:29.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.328+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'17 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.328+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769acb0 con 0x7f3994076040 2024-04-27T03:30:29.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.329+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'33 uv22 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.329+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769acb0 con 0x55625767c680 2024-04-27T03:30:29.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.331+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'39 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.331+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769acb0 con 0x556257681070 2024-04-27T03:30:29.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.332+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'23 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.332+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769acb0 con 0x7f39a406a020 2024-04-27T03:30:29.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.333+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'31 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.333+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769acb0 con 0x556257681070 2024-04-27T03:30:29.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.334+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'23 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.334+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769acb0 con 0x55625767c680 2024-04-27T03:30:29.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.336+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.336+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769adf0 con 0x55625767c680 2024-04-27T03:30:29.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.338+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'40 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.338+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769adf0 con 0x556257681070 2024-04-27T03:30:29.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.340+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.341+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769adf0 con 0x7f39a406a020 2024-04-27T03:30:29.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.342+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'27 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.342+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769adf0 con 0x556257681070 2024-04-27T03:30:29.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.343+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.343+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769adf0 con 0x556257681070 2024-04-27T03:30:29.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.346+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'30 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.346+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769adf0 con 0x556257681070 2024-04-27T03:30:29.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.347+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'13 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.347+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769ae80 con 0x556257681070 2024-04-27T03:30:29.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.348+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'28 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.348+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769ae80 con 0x7f3994076040 2024-04-27T03:30:29.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.349+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.349+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769ae80 con 0x556257681070 2024-04-27T03:30:29.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.350+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'23 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.350+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769ae80 con 0x7f3994076040 2024-04-27T03:30:29.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.351+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'22 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.351+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769afc0 con 0x556257681070 2024-04-27T03:30:29.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.353+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'25 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.353+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769afc0 con 0x7f3994076040 2024-04-27T03:30:29.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.354+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'34 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.354+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625769afc0 con 0x7f39a406a020 2024-04-27T03:30:29.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.355+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.355+0000 7f3860fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:29.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.355+0000 7f39adbdc800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:29.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.356+0000 7f39adbdc800 20 Filter name: none 2024-04-27T03:30:29.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.356+0000 7f37da7fc640 20 reqs_thread_entry: start 2024-04-27T03:30:29.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.356+0000 7f39adbdc800 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-04-27T03:30:29.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.356+0000 7f39adbdc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.356+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:94 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5562576c5d00 con 0x7f3994076040 2024-04-27T03:30:29.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.356+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(94 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.356+0000 7f39adbdc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.356+0000 7f39adbdc800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-04-27T03:30:29.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.356+0000 7f39adbdc800 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T03:30:29.362 INFO:teuthology.orchestra.run.smithi165.stderr:could not remove user: unable to remove user, user does not exist 2024-04-27T03:30:29.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.357+0000 7f3861fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:29.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.357+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93880861641744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257696600 con 0x55625767c680 2024-04-27T03:30:29.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.358+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 93880861641744] v25'41 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.358+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93880861616784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257696600 con 0x7f39a406a020 2024-04-27T03:30:29.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.359+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 93880861616784] v25'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.359+0000 7f39adbdc800 20 remove_watcher() i=0 2024-04-27T03:30:29.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.359+0000 7f39adbdc800 2 removed watcher, disabling cache 2024-04-27T03:30:29.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.359+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93880861622240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257686a90 con 0x55625767c680 2024-04-27T03:30:29.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.360+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 93880861622240] v25'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.360+0000 7f39adbdc800 20 remove_watcher() i=1 2024-04-27T03:30:29.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.360+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93880861626432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257686a90 con 0x556257681070 2024-04-27T03:30:29.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.363+0000 7f39ab6db640 1 -- 172.21.15.165:0/441894928 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 93880861626432] v25'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f39a405f040 con 0x556257681070 2024-04-27T03:30:29.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.363+0000 7f39adbdc800 20 remove_watcher() i=2 2024-04-27T03:30:29.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.363+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93880861620112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257689040 con 0x7f39a406a020 2024-04-27T03:30:29.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.364+0000 7f39aaeda640 1 -- 172.21.15.165:0/441894928 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 93880861620112] v25'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3994081070 con 0x7f39a406a020 2024-04-27T03:30:29.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.364+0000 7f39adbdc800 20 remove_watcher() i=3 2024-04-27T03:30:29.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.364+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93880861633248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257686a90 con 0x55625767c680 2024-04-27T03:30:29.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.367+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 93880861633248] v25'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x55625767c680 2024-04-27T03:30:29.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.367+0000 7f39adbdc800 20 remove_watcher() i=4 2024-04-27T03:30:29.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.367+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93880861637376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257687790 con 0x7f3994076040 2024-04-27T03:30:29.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.369+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 93880861637376] v25'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.369+0000 7f39adbdc800 20 remove_watcher() i=5 2024-04-27T03:30:29.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.369+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93880861630080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556257689040 con 0x7f3994076040 2024-04-27T03:30:29.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.374+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 93880861630080] v25'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.374+0000 7f39adbdc800 20 remove_watcher() i=6 2024-04-27T03:30:29.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.374+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93880861634400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55625768aae0 con 0x7f3994076040 2024-04-27T03:30:29.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.375+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 93880861634400] v25'11 uv9 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f399c031040 con 0x7f3994076040 2024-04-27T03:30:29.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.375+0000 7f39adbdc800 20 remove_watcher() i=7 2024-04-27T03:30:29.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3994076040 msgr2=0x7f3994083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3994076040 0x7f3994083410 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55625767c680 msgr2=0x55625767eb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55625767c680 0x55625767eb40 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f39a406a020 msgr2=0x7f39a40258d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f39a406a020 0x7f39a40258d0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556257681070 msgr2=0x556257683460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556257681070 0x556257683460 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3874036900 msgr2=0x7f3874038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3874036900 0x7f3874038dc0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f399401ced0 tx=0x7f399404f000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39abedc640 1 -- 172.21.15.165:0/441894928 reap_dead start 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 msgr2=0x556257670f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 0x556257670f30 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f39a400cb80 tx=0x7f39a404b990 comp rx=0 tx=0).stop 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 shutdown_connections 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 --2- 172.21.15.165:0/441894928 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556257669bd0 0x556257670f30 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 >> 172.21.15.165:0/441894928 conn(0x556257665e50 msgr2=0x5562576634b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 shutdown_connections 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/441894928 wait complete. 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/3855042838 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3884036970 msgr2=0x7f3884038e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 --2- 172.21.15.165:0/3855042838 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3884036970 0x7f3884038e30 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x556257642750 tx=0x7f3994005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/3855042838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55625765ca40 msgr2=0x55625765ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 --2- 172.21.15.165:0/3855042838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55625765ca40 0x55625765ce60 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f39a40224e0 tx=0x7f39a4016c40 comp rx=0 tx=0).stop 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/3855042838 shutdown_connections 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 --2- 172.21.15.165:0/3855042838 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3884036970 0x7f3884038e30 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 --2- 172.21.15.165:0/3855042838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55625765ca40 0x55625765ce60 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/3855042838 >> 172.21.15.165:0/3855042838 conn(0x556257650ae0 msgr2=0x556257650ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/3855042838 shutdown_connections 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.376+0000 7f39adbdc800 1 -- 172.21.15.165:0/3855042838 wait complete. 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55625764ba80 msgr2=0x55625764ded0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55625764ba80 0x55625764ded0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5562575432e0 msgr2=0x556257645260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5562575432e0 0x556257645260 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556257647b40 msgr2=0x556257649f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556257647b40 0x556257649f30 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3890036900 msgr2=0x7f3890038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3890036900 0x7f3890038dc0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f3994054000 tx=0x7f399404e000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 msgr2=0x55625763e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 --2- 172.21.15.165:0/1220118913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562575417a0 0x55625763e0c0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f39a400c010 tx=0x7f39a4000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39abedc640 1 -- 172.21.15.165:0/1220118913 reap_dead start 2024-04-27T03:30:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 shutdown_connections 2024-04-27T03:30:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 >> 172.21.15.165:0/1220118913 conn(0x55625753d320 msgr2=0x556257630620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 shutdown_connections 2024-04-27T03:30:29.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.377+0000 7f39adbdc800 1 -- 172.21.15.165:0/1220118913 wait complete. 2024-04-27T03:30:29.391 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-27T03:30:29.392 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-04-27T03:30:29.392 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-04-27T03:30:29.392 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-04-27T03:30:29.475 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:29.475 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:29.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.470+0000 7f87a8811800 1 Processor -- start 2024-04-27T03:30:29.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.470+0000 7f87a8811800 1 -- start start 2024-04-27T03:30:29.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.470+0000 7f87a8811800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55982cda08b0 0x55982cda2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.470+0000 7f87a8811800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55982cc5ed40 con 0x55982cda31e0 2024-04-27T03:30:29.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.470+0000 7f87a8811800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55982cc314d0 con 0x55982cda08b0 2024-04-27T03:30:29.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.471+0000 7f87a3e21640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55982cda08b0 0x55982cda2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.471+0000 7f879b620640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55982cda31e0 0x55982cda55d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53554/0 (socket says 172.21.15.165:53554) 2024-04-27T03:30:29.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.471+0000 7f879b620640 1 -- 172.21.15.165:0/238028334 learned_addr learned my addr 172.21.15.165:0/238028334 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:29.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.471+0000 7f87a3e21640 1 -- 172.21.15.165:0/238028334 >> v1:172.21.15.89:6789/0 conn(0x55982cda31e0 legacy=0x55982cda55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.471+0000 7f87a3e21640 1 -- 172.21.15.165:0/238028334 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982cc5d590 con 0x55982cda08b0 2024-04-27T03:30:29.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.473+0000 7f87a3e21640 1 --2- 172.21.15.165:0/238028334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982cda2ca0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f879400a470 tx=0x7f879401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e62648159df794ed server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.473+0000 7f87a3620640 1 -- 172.21.15.165:0/238028334 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8794057070 con 0x55982cda08b0 2024-04-27T03:30:29.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.473+0000 7f87a3620640 1 -- 172.21.15.165:0/238028334 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8794052570 con 0x55982cda08b0 2024-04-27T03:30:29.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.473+0000 7f87a3620640 1 -- 172.21.15.165:0/238028334 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8794052870 con 0x55982cda08b0 2024-04-27T03:30:29.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.474+0000 7f87a8811800 1 -- 172.21.15.165:0/238028334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 msgr2=0x55982cda2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.474+0000 7f87a8811800 1 --2- 172.21.15.165:0/238028334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982cda2ca0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f879400a470 tx=0x7f879401ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:29.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.474+0000 7f87a8811800 1 -- 172.21.15.165:0/238028334 shutdown_connections 2024-04-27T03:30:29.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.474+0000 7f87a8811800 1 --2- 172.21.15.165:0/238028334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982cda2ca0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.474+0000 7f87a8811800 1 -- 172.21.15.165:0/238028334 >> 172.21.15.165:0/238028334 conn(0x55982cd9a380 msgr2=0x55982cd9c7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.474+0000 7f87a8811800 1 -- 172.21.15.165:0/238028334 shutdown_connections 2024-04-27T03:30:29.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.474+0000 7f87a8811800 1 -- 172.21.15.165:0/238028334 wait complete. 2024-04-27T03:30:29.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.485+0000 7f87a8811800 1 Processor -- start 2024-04-27T03:30:29.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.486+0000 7f87a8811800 1 -- start start 2024-04-27T03:30:29.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.486+0000 7f87a8811800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982cda01f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.486+0000 7f87a8811800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55982cceb3c0 con 0x55982cda08b0 2024-04-27T03:30:29.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.486+0000 7f87a3e21640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982cda01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.486+0000 7f87a3e21640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982cda01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51206/0 (socket says 172.21.15.165:51206) 2024-04-27T03:30:29.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.486+0000 7f87a3e21640 1 -- 172.21.15.165:0/484451634 learned_addr learned my addr 172.21.15.165:0/484451634 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.486+0000 7f87a3e21640 1 -- 172.21.15.165:0/484451634 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982cd9e7b0 con 0x55982cda08b0 2024-04-27T03:30:29.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.487+0000 7f87a3e21640 1 --2- 172.21.15.165:0/484451634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982cda01f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8794055040 tx=0x7f879404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe56060289f51ce4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.487+0000 7f8691e0b640 1 -- 172.21.15.165:0/484451634 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8794057070 con 0x55982cda08b0 2024-04-27T03:30:29.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.487+0000 7f8691e0b640 1 -- 172.21.15.165:0/484451634 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8794063070 con 0x55982cda08b0 2024-04-27T03:30:29.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.487+0000 7f8691e0b640 1 -- 172.21.15.165:0/484451634 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8794052720 con 0x55982cda08b0 2024-04-27T03:30:29.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.487+0000 7f87a8811800 1 -- 172.21.15.165:0/484451634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 msgr2=0x55982cda01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.487+0000 7f87a8811800 1 --2- 172.21.15.165:0/484451634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982cda01f0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f8794055040 tx=0x7f879404c000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.488+0000 7f87a8811800 1 -- 172.21.15.165:0/484451634 shutdown_connections 2024-04-27T03:30:29.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.488+0000 7f87a8811800 1 --2- 172.21.15.165:0/484451634 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982cda01f0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.488+0000 7f87a8811800 1 -- 172.21.15.165:0/484451634 >> 172.21.15.165:0/484451634 conn(0x55982cd9a380 msgr2=0x55982ce8c9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.488+0000 7f87a8811800 1 -- 172.21.15.165:0/484451634 shutdown_connections 2024-04-27T03:30:29.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.488+0000 7f87a8811800 1 -- 172.21.15.165:0/484451634 wait complete. 2024-04-27T03:30:29.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.488+0000 7f87a8811800 1 Processor -- start 2024-04-27T03:30:29.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.488+0000 7f87a8811800 1 -- start start 2024-04-27T03:30:29.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.488+0000 7f87a8811800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982ce9ade0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.488+0000 7f87a8811800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55982cda6f90 con 0x55982cda08b0 2024-04-27T03:30:29.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.488+0000 7f87a3e21640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982ce9ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.488+0000 7f87a3e21640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982ce9ade0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51212/0 (socket says 172.21.15.165:51212) 2024-04-27T03:30:29.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.488+0000 7f87a3e21640 1 -- 172.21.15.165:0/2664758755 learned_addr learned my addr 172.21.15.165:0/2664758755 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.489+0000 7f87a3e21640 1 -- 172.21.15.165:0/2664758755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982ce9b320 con 0x55982cda08b0 2024-04-27T03:30:29.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.489+0000 7f87a3e21640 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982ce9ade0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8794060040 tx=0x7f8794002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.489+0000 7f868bfff640 1 -- 172.21.15.165:0/2664758755 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8794052e30 con 0x55982cda08b0 2024-04-27T03:30:29.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.489+0000 7f868bfff640 1 -- 172.21.15.165:0/2664758755 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879406c070 con 0x55982cda08b0 2024-04-27T03:30:29.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.489+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55982ce9b5b0 con 0x55982cda08b0 2024-04-27T03:30:29.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.489+0000 7f868bfff640 1 -- 172.21.15.165:0/2664758755 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f879405e5f0 con 0x55982cda08b0 2024-04-27T03:30:29.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.489+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55982ce9e260 con 0x55982cda08b0 2024-04-27T03:30:29.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.490+0000 7f868bfff640 1 -- 172.21.15.165:0/2664758755 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f879405e790 con 0x55982cda08b0 2024-04-27T03:30:29.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.490+0000 7f868bfff640 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f868c036950 0x7f868c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.490+0000 7f868bfff640 1 -- 172.21.15.165:0/2664758755 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f879408e6f0 con 0x55982cda08b0 2024-04-27T03:30:29.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.490+0000 7f879b620640 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f868c036950 0x7f868c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.490+0000 7f87a8811800 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55982cda31e0 0x55982cea2020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.490+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cea2560 con 0x55982cda31e0 2024-04-27T03:30:29.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.490+0000 7f87a4622640 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55982cda31e0 0x55982cea2020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.490+0000 7f879b620640 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f868c036950 0x7f868c038e10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f878c000f60 tx=0x7f878c0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.490+0000 7f87a4622640 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55982cda31e0 0x55982cea2020 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.491+0000 7f87a4622640 1 -- 172.21.15.165:0/2664758755 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982cda31e0 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.491+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982ce9f5f0 con 0x55982cda31e0 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.491+0000 7f87a4622640 1 -- 172.21.15.165:0/2664758755 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982cda31e0 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.491+0000 7f87a8811800 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55982cea4890 0x55982cea6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.491+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cea7370 con 0x55982cea4890 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.491+0000 7f87a3e21640 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55982cea4890 0x55982cea6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.491+0000 7f87a3e21640 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55982cea4890 0x55982cea6c80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.492+0000 7f87a3e21640 1 -- 172.21.15.165:0/2664758755 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f8794071990 con 0x55982cea4890 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.492+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cea3840 con 0x55982cda31e0 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.492+0000 7f87a4622640 1 -- 172.21.15.165:0/2664758755 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982cda31e0 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.492+0000 7f87a8811800 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55982cea8820 0x55982ceaac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.492+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982ceab3d0 con 0x55982cea8820 2024-04-27T03:30:29.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.492+0000 7f879b620640 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55982cea8820 0x55982ceaac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.493+0000 7f879b620640 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55982cea8820 0x55982ceaac70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.493+0000 7f879b620640 1 -- 172.21.15.165:0/2664758755 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f878c009050 con 0x55982cea8820 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.493+0000 7f87a8811800 1 Processor -- start 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.493+0000 7f87a8811800 1 -- start start 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.493+0000 7f87a8811800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982ceb97c0 0x55982ceb9be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.494+0000 7f87a8811800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55982cc916b0 con 0x55982ceb97c0 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.494+0000 7f87a3e21640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982ceb97c0 0x55982ceb9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.494+0000 7f87a3e21640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982ceb97c0 0x55982ceb9be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51224/0 (socket says 172.21.15.165:51224) 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.494+0000 7f87a3e21640 1 -- 172.21.15.165:0/2957073490 learned_addr learned my addr 172.21.15.165:0/2957073490 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.494+0000 7f87a3e21640 1 -- 172.21.15.165:0/2957073490 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982ceba120 con 0x55982ceb97c0 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.494+0000 7f87a3e21640 1 --2- 172.21.15.165:0/2957073490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982ceb97c0 0x55982ceb9be0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8794052520 tx=0x7f8794068e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.494+0000 7f86897fa640 1 -- 172.21.15.165:0/2957073490 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8794092020 con 0x55982ceb97c0 2024-04-27T03:30:29.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.494+0000 7f86897fa640 1 -- 172.21.15.165:0/2957073490 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8794058040 con 0x55982ceb97c0 2024-04-27T03:30:29.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.494+0000 7f87a8811800 1 -- 172.21.15.165:0/2957073490 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55982cebab90 con 0x55982ceb97c0 2024-04-27T03:30:29.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.494+0000 7f86897fa640 1 -- 172.21.15.165:0/2957073490 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8794002a70 con 0x55982ceb97c0 2024-04-27T03:30:29.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.495+0000 7f87a8811800 1 -- 172.21.15.165:0/2957073490 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55982cebab90 con 0x55982ceb97c0 2024-04-27T03:30:29.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.495+0000 7f86897fa640 1 -- 172.21.15.165:0/2957073490 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f8794095020 con 0x55982ceb97c0 2024-04-27T03:30:29.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.495+0000 7f86897fa640 1 --2- 172.21.15.165:0/2957073490 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f867c036900 0x7f867c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.495+0000 7f86897fa640 1 -- 172.21.15.165:0/2957073490 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f879408b050 con 0x55982ceb97c0 2024-04-27T03:30:29.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.495+0000 7f879b620640 1 --2- 172.21.15.165:0/2957073490 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f867c036900 0x7f867c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.496+0000 7f87a8811800 1 Processor -- start 2024-04-27T03:30:29.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.496+0000 7f87a8811800 1 -- start start 2024-04-27T03:30:29.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.496+0000 7f879b620640 1 --2- 172.21.15.165:0/2957073490 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f867c036900 0x7f867c038dc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f878c00bf50 tx=0x7f878c00f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.496+0000 7f87a8811800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 0x55982cec6ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.496+0000 7f87a8811800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55982cd9ea70 con 0x55982cec66b0 2024-04-27T03:30:29.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.496+0000 7f87a3e21640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 0x55982cec6ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.496+0000 7f87a3e21640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 0x55982cec6ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51226/0 (socket says 172.21.15.165:51226) 2024-04-27T03:30:29.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.496+0000 7f87a3e21640 1 -- 172.21.15.165:0/4059915941 learned_addr learned my addr 172.21.15.165:0/4059915941 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.496+0000 7f87a3e21640 1 -- 172.21.15.165:0/4059915941 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982cec70a0 con 0x55982cec66b0 2024-04-27T03:30:29.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.497+0000 7f87a3e21640 1 --2- 172.21.15.165:0/4059915941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 0x55982cec6ad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8794082010 tx=0x7f87940664f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd33352165004f72 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.497+0000 7f8682ffd640 1 -- 172.21.15.165:0/4059915941 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f879408eb80 con 0x55982cec66b0 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.497+0000 7f8682ffd640 1 -- 172.21.15.165:0/4059915941 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8794002c10 con 0x55982cec66b0 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.497+0000 7f8682ffd640 1 -- 172.21.15.165:0/4059915941 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8794061120 con 0x55982cec66b0 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.497+0000 7f87a8811800 1 -- 172.21.15.165:0/4059915941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 msgr2=0x55982cec6ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.497+0000 7f87a8811800 1 --2- 172.21.15.165:0/4059915941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 0x55982cec6ad0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f8794082010 tx=0x7f87940664f0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.497+0000 7f87a8811800 1 -- 172.21.15.165:0/4059915941 shutdown_connections 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.497+0000 7f87a8811800 1 --2- 172.21.15.165:0/4059915941 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 0x55982cec6ad0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.497+0000 7f87a8811800 1 -- 172.21.15.165:0/4059915941 >> 172.21.15.165:0/4059915941 conn(0x55982cec2a10 msgr2=0x55982cec0a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.497+0000 7f87a8811800 1 -- 172.21.15.165:0/4059915941 shutdown_connections 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.497+0000 7f87a8811800 1 -- 172.21.15.165:0/4059915941 wait complete. 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.497+0000 7f87a8811800 1 Processor -- start 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.498+0000 7f87a8811800 1 -- start start 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.498+0000 7f87a8811800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 0x55982cecdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.498+0000 7f87a8811800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55982cec7750 con 0x55982cec66b0 2024-04-27T03:30:29.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.498+0000 7f87a3e21640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 0x55982cecdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.498+0000 7f87a3e21640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 0x55982cecdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51242/0 (socket says 172.21.15.165:51242) 2024-04-27T03:30:29.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.498+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 learned_addr learned my addr 172.21.15.165:0/1420850828 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.498+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55982cece040 con 0x55982cec66b0 2024-04-27T03:30:29.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.498+0000 7f87a3e21640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 0x55982cecdb00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f879409a3d0 tx=0x7f8794081f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.498+0000 7f86817fa640 1 -- 172.21.15.165:0/1420850828 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f879408b470 con 0x55982cec66b0 2024-04-27T03:30:29.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.498+0000 7f86817fa640 1 -- 172.21.15.165:0/1420850828 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f879408b610 con 0x55982cec66b0 2024-04-27T03:30:29.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.498+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55982cece040 con 0x55982cec66b0 2024-04-27T03:30:29.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.498+0000 7f86817fa640 1 -- 172.21.15.165:0/1420850828 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f879408b8f0 con 0x55982cec66b0 2024-04-27T03:30:29.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.499+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55982cece040 con 0x55982cec66b0 2024-04-27T03:30:29.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.499+0000 7f86817fa640 1 -- 172.21.15.165:0/1420850828 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f87940bc030 con 0x55982cec66b0 2024-04-27T03:30:29.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.499+0000 7f86817fa640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f866c036950 0x7f866c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.499+0000 7f86817fa640 1 -- 172.21.15.165:0/1420850828 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f87940ba070 con 0x55982cec66b0 2024-04-27T03:30:29.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.499+0000 7f879b620640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f866c036950 0x7f866c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.500+0000 7f879b620640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f866c036950 0x7f866c038e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f878c001b60 tx=0x7f878c010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.500+0000 7f87a8811800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.500+0000 7f87a8811800 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55982ced91e0 0x55982cedb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.500+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cedbdf0 con 0x55982ced91e0 2024-04-27T03:30:29.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.500+0000 7f87a4622640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55982ced91e0 0x55982cedb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.500+0000 7f87a4622640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55982ced91e0 0x55982cedb6a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.500+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982ced91e0 2024-04-27T03:30:29.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 20 realm 2024-04-27T03:30:29.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cedc200 con 0x55982ced91e0 2024-04-27T03:30:29.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982ced91e0 2024-04-27T03:30:29.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:29.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cedcad0 con 0x55982ced91e0 2024-04-27T03:30:29.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982ced91e0 2024-04-27T03:30:29.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ced91e0 2024-04-27T03:30:29.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982ced91e0 2024-04-27T03:30:29.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:29.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55982ceddbd0 0x55982cedffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cee05d0 con 0x55982ceddbd0 2024-04-27T03:30:29.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.501+0000 7f87a3e21640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55982ceddbd0 0x55982cedffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.502+0000 7f87a3e21640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55982ceddbd0 0x55982cedffc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.502+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.502+0000 7f87a8811800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:29.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.502+0000 7f87a8811800 20 searching for the correct realm 2024-04-27T03:30:29.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.502+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cee1f00 con 0x55982ceddbd0 2024-04-27T03:30:29.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.502+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.502+0000 7f87a3e21640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8794097020 0x7f8794061120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.502+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cee1f00 con 0x7f8794097020 2024-04-27T03:30:29.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.503+0000 7f879b620640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8794097020 0x7f8794061120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.503+0000 7f879b620640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8794097020 0x7f8794061120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.503+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f878c009050 con 0x7f8794097020 2024-04-27T03:30:29.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.503+0000 7f879b620640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f878c038040 0x7f878c01bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.503+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cee1f00 con 0x7f878c038040 2024-04-27T03:30:29.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.503+0000 7f87a4622640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f878c038040 0x7f878c01bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.504+0000 7f87a4622640 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f878c038040 0x7f878c01bab0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.504+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f879c00a4a0 con 0x7f878c038040 2024-04-27T03:30:29.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.504+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ced91e0 2024-04-27T03:30:29.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.504+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982ced91e0 2024-04-27T03:30:29.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.504+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ced91e0 2024-04-27T03:30:29.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.504+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982ced91e0 2024-04-27T03:30:29.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.505+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f878c038040 2024-04-27T03:30:29.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.505+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c00a4a0 con 0x7f878c038040 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.505+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f878c038040 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.505+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c00a4a0 con 0x7f878c038040 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.505+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f8794097020 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.505+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f878c009050 con 0x7f8794097020 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.505+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ceddbd0 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.505+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.506+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ced91e0 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.506+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982ced91e0 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.506+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f878c038040 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.506+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c00a4a0 con 0x7f878c038040 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.506+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ced91e0 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.506+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982ced91e0 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.506+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f8794097020 2024-04-27T03:30:29.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.507+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f878c009050 con 0x7f8794097020 2024-04-27T03:30:29.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.507+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ceddbd0 2024-04-27T03:30:29.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.507+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.507+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ceddbd0 2024-04-27T03:30:29.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.507+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.507+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f878c038040 2024-04-27T03:30:29.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.507+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c00a4a0 con 0x7f878c038040 2024-04-27T03:30:29.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.507+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ceddbd0 2024-04-27T03:30:29.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.508+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v18'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.508+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ceddbd0 2024-04-27T03:30:29.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.508+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.508+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ced91e0 2024-04-27T03:30:29.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.508+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982ced91e0 2024-04-27T03:30:29.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.508+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ced91e0 2024-04-27T03:30:29.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.508+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f879c00a4a0 con 0x55982ced91e0 2024-04-27T03:30:29.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.508+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f878c038040 2024-04-27T03:30:29.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.509+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c033040 con 0x7f878c038040 2024-04-27T03:30:29.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.509+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ceddbd0 2024-04-27T03:30:29.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.509+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.509+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f8794097020 2024-04-27T03:30:29.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.509+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f878c009050 con 0x7f8794097020 2024-04-27T03:30:29.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.509+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ced91e0 2024-04-27T03:30:29.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.509+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.509+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f878c038040 2024-04-27T03:30:29.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.510+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.510+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ceddbd0 2024-04-27T03:30:29.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.510+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.510+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f878c038040 2024-04-27T03:30:29.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.510+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.510+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f8794097020 2024-04-27T03:30:29.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.510+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f878c009050 con 0x7f8794097020 2024-04-27T03:30:29.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.510+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ceddbd0 2024-04-27T03:30:29.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.511+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.511+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f878c038040 2024-04-27T03:30:29.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.511+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.511+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x55982ced91e0 2024-04-27T03:30:29.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.511+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.511+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55982cedce70 con 0x7f8794097020 2024-04-27T03:30:29.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:29.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:29.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:29.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:29.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:29.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:29.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cee0e90 con 0x55982ced91e0 2024-04-27T03:30:29.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cee1280 con 0x55982ced91e0 2024-04-27T03:30:29.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:29.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cee1280 con 0x7f878c038040 2024-04-27T03:30:29.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 zone default found 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 4 Realm: () 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 zonegroup default 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.512+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cee2380 con 0x7f878c038040 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a8811800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a8811800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cee3d60 con 0x55982ceddbd0 2024-04-27T03:30:29.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a8811800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a8811800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cee4100 con 0x55982ceddbd0 2024-04-27T03:30:29.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a8811800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a8811800 20 started sync module instance, tier type = 2024-04-27T03:30:29.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a8811800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:29.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.513+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee3200 con 0x7f8794097020 2024-04-27T03:30:29.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.514+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.514+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94112077196624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee44a0 con 0x7f8794097020 2024-04-27T03:30:29.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.514+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee4cf0 con 0x55982ced91e0 2024-04-27T03:30:29.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.516+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94112077196624] v25'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.516+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.516+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94112077203520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee4c60 con 0x55982ced91e0 2024-04-27T03:30:29.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.516+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee5be0 con 0x55982ceddbd0 2024-04-27T03:30:29.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.517+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.517+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94112077207344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee6430 con 0x55982ceddbd0 2024-04-27T03:30:29.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.517+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee6f00 con 0x7f8794097020 2024-04-27T03:30:29.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.518+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.518+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94112077214448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee7bc0 con 0x7f8794097020 2024-04-27T03:30:29.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.518+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee8610 con 0x55982ced91e0 2024-04-27T03:30:29.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.519+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94112077203520] v25'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.520+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94112077207344] v25'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.520+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.520+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94112077209472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee5110 con 0x55982ced91e0 2024-04-27T03:30:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.520+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee8ee0 con 0x7f878c038040 2024-04-27T03:30:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.521+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.521+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94112077220320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee8e40 con 0x7f878c038040 2024-04-27T03:30:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.521+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee9e60 con 0x7f878c038040 2024-04-27T03:30:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.522+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.522+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94112077224272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982ceea2d0 con 0x7f878c038040 2024-04-27T03:30:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.522+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982ceeada0 con 0x7f878c038040 2024-04-27T03:30:29.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.523+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94112077220320] v25'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.523+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94112077214448] v25'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.526+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94112077209472] v25'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.526+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'12 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.526+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94112077215504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982ceead60 con 0x7f878c038040 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.526+0000 7f87a8811800 20 add_watcher() i=0 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.526+0000 7f87a8811800 20 add_watcher() i=1 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.526+0000 7f87a8811800 20 add_watcher() i=2 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.526+0000 7f87a8811800 20 add_watcher() i=3 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.526+0000 7f87a8811800 20 add_watcher() i=4 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.526+0000 7f87a8811800 20 add_watcher() i=5 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.526+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94112077224272] v25'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.526+0000 7f87a8811800 20 add_watcher() i=6 2024-04-27T03:30:29.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.527+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94112077215504] v25'13 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.527+0000 7f87a8811800 20 add_watcher() i=7 2024-04-27T03:30:29.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.527+0000 7f87a8811800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:29.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.527+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cee9e20 con 0x55982ced91e0 2024-04-27T03:30:29.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.527+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.527+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94112077218048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee9e20 con 0x55982ced91e0 2024-04-27T03:30:29.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.528+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94112077218048] v25'42 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.529+0000 7f8659fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:29.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.529+0000 7f87a8811800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:29.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.529+0000 7f87a8811800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:29.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.529+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef5b80 con 0x55982ced91e0 2024-04-27T03:30:29.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.531+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'42 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.531+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef61d0 con 0x7f878c038040 2024-04-27T03:30:29.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.533+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'18 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.533+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef6630 con 0x55982ceddbd0 2024-04-27T03:30:29.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.534+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'18 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.534+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef6570 con 0x55982ced91e0 2024-04-27T03:30:29.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.537+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'43 uv15 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.537+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef6b60 con 0x7f8794097020 2024-04-27T03:30:29.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.540+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'32 uv17 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.540+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef7580 con 0x55982ceddbd0 2024-04-27T03:30:29.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.542+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'29 uv16 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.542+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef7580 con 0x55982ced91e0 2024-04-27T03:30:29.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.545+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'20 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.545+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef7610 con 0x7f878c038040 2024-04-27T03:30:29.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.546+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'23 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.546+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef7610 con 0x7f8794097020 2024-04-27T03:30:29.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.550+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'12 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.550+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef7b80 con 0x55982ceddbd0 2024-04-27T03:30:29.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.551+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'18 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.551+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef7b80 con 0x7f8794097020 2024-04-27T03:30:29.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.551+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'33 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.552+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef80f0 con 0x7f878c038040 2024-04-27T03:30:29.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.553+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'14 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.553+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef80f0 con 0x7f8794097020 2024-04-27T03:30:29.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.554+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'18 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.554+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x7f878c038040 2024-04-27T03:30:29.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.557+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'35 uv22 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.557+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x55982ced91e0 2024-04-27T03:30:29.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.559+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'44 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.559+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x55982ceddbd0 2024-04-27T03:30:29.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.562+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'26 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.562+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x7f8794097020 2024-04-27T03:30:29.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.565+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'34 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.565+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x55982ceddbd0 2024-04-27T03:30:29.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.567+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'24 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.567+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x55982ced91e0 2024-04-27T03:30:29.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.568+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.568+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x55982ced91e0 2024-04-27T03:30:29.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.571+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'45 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.571+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x55982ceddbd0 2024-04-27T03:30:29.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.573+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'27 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.573+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x7f8794097020 2024-04-27T03:30:29.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.574+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'28 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.574+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x55982ceddbd0 2024-04-27T03:30:29.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.576+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.576+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x55982ceddbd0 2024-04-27T03:30:29.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.579+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'31 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.579+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x55982ceddbd0 2024-04-27T03:30:29.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.582+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'14 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.582+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x55982ceddbd0 2024-04-27T03:30:29.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.584+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'30 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.584+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x7f878c038040 2024-04-27T03:30:29.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.586+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'20 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.586+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x55982ceddbd0 2024-04-27T03:30:29.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.586+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'24 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.587+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x7f878c038040 2024-04-27T03:30:29.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.588+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'24 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.588+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x55982ceddbd0 2024-04-27T03:30:29.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.589+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'28 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.589+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x7f878c038040 2024-04-27T03:30:29.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.590+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'36 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.590+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cef83d0 con 0x7f8794097020 2024-04-27T03:30:29.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.591+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.592+0000 7f8658fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:29.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.592+0000 7f87a8811800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:29.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.592+0000 7f87a8811800 20 Filter name: none 2024-04-27T03:30:29.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.592+0000 7f85d27fc640 20 reqs_thread_entry: start 2024-04-27T03:30:29.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.592+0000 7f87a8811800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T03:30:29.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.592+0000 7f87a8811800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.592+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55982cf230f0 con 0x7f878c038040 2024-04-27T03:30:29.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.592+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(94 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.592+0000 7f87a8811800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.592+0000 7f87a8811800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-27T03:30:29.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.592+0000 7f87a8811800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:30:29.599 INFO:teuthology.orchestra.run.smithi165.stderr:could not remove user: unable to remove user, user does not exist 2024-04-27T03:30:29.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.593+0000 7f8659fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:29.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.593+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94112077218048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982ce88a80 con 0x55982ced91e0 2024-04-27T03:30:29.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.594+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94112077218048] v25'46 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.594+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94112077196624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee8100 con 0x7f8794097020 2024-04-27T03:30:29.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.595+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94112077196624] v25'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.595+0000 7f87a8811800 20 remove_watcher() i=0 2024-04-27T03:30:29.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.595+0000 7f87a8811800 2 removed watcher, disabling cache 2024-04-27T03:30:29.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.595+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94112077203520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee2d50 con 0x55982ced91e0 2024-04-27T03:30:29.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.596+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94112077203520] v25'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.596+0000 7f87a8811800 20 remove_watcher() i=1 2024-04-27T03:30:29.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.596+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94112077207344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee2d50 con 0x55982ceddbd0 2024-04-27T03:30:29.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.598+0000 7f87a3e21640 1 -- 172.21.15.165:0/1420850828 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94112077207344] v25'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8794071990 con 0x55982ceddbd0 2024-04-27T03:30:29.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.598+0000 7f87a8811800 20 remove_watcher() i=2 2024-04-27T03:30:29.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.598+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94112077214448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee2d50 con 0x7f8794097020 2024-04-27T03:30:29.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.599+0000 7f879b620640 1 -- 172.21.15.165:0/1420850828 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94112077214448] v25'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f878c045040 con 0x7f8794097020 2024-04-27T03:30:29.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.599+0000 7f87a8811800 20 remove_watcher() i=3 2024-04-27T03:30:29.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.599+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94112077209472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee72f0 con 0x55982ced91e0 2024-04-27T03:30:29.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.601+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94112077209472] v25'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x55982ced91e0 2024-04-27T03:30:29.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.601+0000 7f87a8811800 20 remove_watcher() i=4 2024-04-27T03:30:29.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.601+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94112077220320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee2d50 con 0x7f878c038040 2024-04-27T03:30:29.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.602+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94112077220320] v25'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.602+0000 7f87a8811800 20 remove_watcher() i=5 2024-04-27T03:30:29.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.602+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94112077224272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee72f0 con 0x7f878c038040 2024-04-27T03:30:29.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.604+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94112077224272] v25'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.604+0000 7f87a8811800 20 remove_watcher() i=6 2024-04-27T03:30:29.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.604+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94112077215504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55982cee89e0 con 0x7f878c038040 2024-04-27T03:30:29.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94112077215504] v25'14 uv12 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f879c031040 con 0x7f878c038040 2024-04-27T03:30:29.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 20 remove_watcher() i=7 2024-04-27T03:30:29.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f878c038040 msgr2=0x7f878c01bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f878c038040 0x7f878c01bab0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55982ced91e0 msgr2=0x55982cedb6a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55982ced91e0 0x55982cedb6a0 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8794097020 msgr2=0x7f8794061120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8794097020 0x7f8794061120 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55982ceddbd0 msgr2=0x55982cedffc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55982ceddbd0 0x55982cedffc0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f866c036950 msgr2=0x7f866c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f866c036950 0x7f866c038e10 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f878c001b60 tx=0x7f878c010000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 msgr2=0x55982cecdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 --2- 172.21.15.165:0/1420850828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cec66b0 0x55982cecdb00 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f879409a3d0 tx=0x7f8794081f40 comp rx=0 tx=0).stop 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 reap_dead start 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a4622640 1 -- 172.21.15.165:0/1420850828 reap_dead start 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 shutdown_connections 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.605+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 >> 172.21.15.165:0/1420850828 conn(0x55982cec2a10 msgr2=0x55982cec0070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 shutdown_connections 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/1420850828 wait complete. 2024-04-27T03:30:29.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2957073490 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f867c036900 msgr2=0x7f867c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 --2- 172.21.15.165:0/2957073490 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f867c036900 0x7f867c038dc0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f878c00bf50 tx=0x7f878c00f000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2957073490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982ceb97c0 msgr2=0x55982ceb9be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 --2- 172.21.15.165:0/2957073490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982ceb97c0 0x55982ceb9be0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f8794052520 tx=0x7f8794068e30 comp rx=0 tx=0).stop 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2957073490 shutdown_connections 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 --2- 172.21.15.165:0/2957073490 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f867c036900 0x7f867c038dc0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 --2- 172.21.15.165:0/2957073490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982ceb97c0 0x55982ceb9be0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2957073490 >> 172.21.15.165:0/2957073490 conn(0x55982cead860 msgr2=0x55982ceadc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2957073490 shutdown_connections 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2957073490 wait complete. 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55982cea8820 msgr2=0x55982ceaac70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55982cea8820 0x55982ceaac70 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55982cda31e0 msgr2=0x55982cea2020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55982cda31e0 0x55982cea2020 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55982cea4890 msgr2=0x55982cea6c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55982cea4890 0x55982cea6c80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f868c036950 msgr2=0x7f868c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f868c036950 0x7f868c038e10 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f878c000f60 tx=0x7f878c0012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 msgr2=0x55982ce9ade0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 --2- 172.21.15.165:0/2664758755 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55982cda08b0 0x55982ce9ade0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f8794060040 tx=0x7f8794002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a4622640 1 -- 172.21.15.165:0/2664758755 reap_dead start 2024-04-27T03:30:29.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 shutdown_connections 2024-04-27T03:30:29.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 >> 172.21.15.165:0/2664758755 conn(0x55982cd9a380 msgr2=0x55982ce8c5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 shutdown_connections 2024-04-27T03:30:29.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.606+0000 7f87a8811800 1 -- 172.21.15.165:0/2664758755 wait complete. 2024-04-27T03:30:29.619 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-27T03:30:29.620 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-04-27T03:30:29.620 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-04-27T03:30:29.620 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-04-27T03:30:29.703 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:29.703 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:29.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.698+0000 7ffae4c0a800 1 Processor -- start 2024-04-27T03:30:29.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.698+0000 7ffae4c0a800 1 -- start start 2024-04-27T03:30:29.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.698+0000 7ffae4c0a800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b2b028df90 0x55b2b028e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.698+0000 7ffae4c0a800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55b2b014cd40 con 0x55b2b028f100 2024-04-27T03:30:29.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.698+0000 7ffae4c0a800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55b2b011f4d0 con 0x55b2b028df90 2024-04-27T03:30:29.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.699+0000 7ffae2709640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b2b028df90 0x55b2b028e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.699+0000 7ffadbfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55b2b028f100 0x55b2b028f580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53566/0 (socket says 172.21.15.165:53566) 2024-04-27T03:30:29.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.699+0000 7ffadbfff640 1 -- 172.21.15.165:0/2126112050 learned_addr learned my addr 172.21.15.165:0/2126112050 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:29.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.699+0000 7ffae2709640 1 -- 172.21.15.165:0/2126112050 >> v1:172.21.15.89:6789/0 conn(0x55b2b028f100 legacy=0x55b2b028f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.699+0000 7ffae2709640 1 -- 172.21.15.165:0/2126112050 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b014b590 con 0x55b2b028df90 2024-04-27T03:30:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.701+0000 7ffae2709640 1 --2- 172.21.15.165:0/2126112050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b028e3b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffad400a470 tx=0x7ffad401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f73634f8348ea5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.701+0000 7ffae1f08640 1 -- 172.21.15.165:0/2126112050 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad4057070 con 0x55b2b028df90 2024-04-27T03:30:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.701+0000 7ffae1f08640 1 -- 172.21.15.165:0/2126112050 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad4052570 con 0x55b2b028df90 2024-04-27T03:30:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.701+0000 7ffae1f08640 1 -- 172.21.15.165:0/2126112050 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad4052870 con 0x55b2b028df90 2024-04-27T03:30:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.701+0000 7ffae4c0a800 1 -- 172.21.15.165:0/2126112050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 msgr2=0x55b2b028e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.701+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/2126112050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b028e3b0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7ffad400a470 tx=0x7ffad401ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.701+0000 7ffae4c0a800 1 -- 172.21.15.165:0/2126112050 shutdown_connections 2024-04-27T03:30:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.701+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/2126112050 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b028e3b0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.701+0000 7ffae4c0a800 1 -- 172.21.15.165:0/2126112050 >> 172.21.15.165:0/2126112050 conn(0x55b2b0289530 msgr2=0x55b2b028b950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.701+0000 7ffae4c0a800 1 -- 172.21.15.165:0/2126112050 shutdown_connections 2024-04-27T03:30:29.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.701+0000 7ffae4c0a800 1 -- 172.21.15.165:0/2126112050 wait complete. 2024-04-27T03:30:29.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.713+0000 7ffae4c0a800 1 Processor -- start 2024-04-27T03:30:29.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.713+0000 7ffae4c0a800 1 -- start start 2024-04-27T03:30:29.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.713+0000 7ffae4c0a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b037f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.713+0000 7ffae4c0a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b2b01d93c0 con 0x55b2b028df90 2024-04-27T03:30:29.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.713+0000 7ffae2709640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b037f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.714+0000 7ffae2709640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b037f580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51256/0 (socket says 172.21.15.165:51256) 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.714+0000 7ffae2709640 1 -- 172.21.15.165:0/2210535189 learned_addr learned my addr 172.21.15.165:0/2210535189 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.714+0000 7ffae2709640 1 -- 172.21.15.165:0/2210535189 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b037fb50 con 0x55b2b028df90 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.714+0000 7ffae2709640 1 --2- 172.21.15.165:0/2210535189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b037f580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffad4055040 tx=0x7ffad404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf3cab43f346c08 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.714+0000 7ff9d0608640 1 -- 172.21.15.165:0/2210535189 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad4057070 con 0x55b2b028df90 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.714+0000 7ff9d0608640 1 -- 172.21.15.165:0/2210535189 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad4063070 con 0x55b2b028df90 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.714+0000 7ff9d0608640 1 -- 172.21.15.165:0/2210535189 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad4052720 con 0x55b2b028df90 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.715+0000 7ffae4c0a800 1 -- 172.21.15.165:0/2210535189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 msgr2=0x55b2b037f580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.715+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/2210535189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b037f580 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7ffad4055040 tx=0x7ffad404c000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.715+0000 7ffae4c0a800 1 -- 172.21.15.165:0/2210535189 shutdown_connections 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.715+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/2210535189 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b037f580 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.715+0000 7ffae4c0a800 1 -- 172.21.15.165:0/2210535189 >> 172.21.15.165:0/2210535189 conn(0x55b2b0289530 msgr2=0x55b2b037aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.715+0000 7ffae4c0a800 1 -- 172.21.15.165:0/2210535189 shutdown_connections 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.715+0000 7ffae4c0a800 1 -- 172.21.15.165:0/2210535189 wait complete. 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.715+0000 7ffae4c0a800 1 Processor -- start 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.715+0000 7ffae4c0a800 1 -- start start 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.715+0000 7ffae4c0a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b0203970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.715+0000 7ffae4c0a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b2b0297570 con 0x55b2b028df90 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.716+0000 7ffae2709640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b0203970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.716+0000 7ffae2709640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b0203970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51260/0 (socket says 172.21.15.165:51260) 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.716+0000 7ffae2709640 1 -- 172.21.15.165:0/422180316 learned_addr learned my addr 172.21.15.165:0/422180316 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.716+0000 7ffae2709640 1 -- 172.21.15.165:0/422180316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b02020b0 con 0x55b2b028df90 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.716+0000 7ffae2709640 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b0203970 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffad4060040 tx=0x7ffad4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.716+0000 7ff9cee05640 1 -- 172.21.15.165:0/422180316 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad4052c70 con 0x55b2b028df90 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.716+0000 7ff9cee05640 1 -- 172.21.15.165:0/422180316 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad406c070 con 0x55b2b028df90 2024-04-27T03:30:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.716+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b2b0202340 con 0x55b2b028df90 2024-04-27T03:30:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.716+0000 7ff9cee05640 1 -- 172.21.15.165:0/422180316 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad405e480 con 0x55b2b028df90 2024-04-27T03:30:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.716+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b02028c0 con 0x55b2b028df90 2024-04-27T03:30:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.717+0000 7ff9cee05640 1 -- 172.21.15.165:0/422180316 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffad405e620 con 0x55b2b028df90 2024-04-27T03:30:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.717+0000 7ff9cee05640 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9c8036900 0x7ff9c8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.717+0000 7ff9cee05640 1 -- 172.21.15.165:0/422180316 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ffad408e5c0 con 0x55b2b028df90 2024-04-27T03:30:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.717+0000 7ffadbfff640 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9c8036900 0x7ff9c8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.717+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b2b028f100 0x55b2b03905a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.717+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b0390ae0 con 0x55b2b028f100 2024-04-27T03:30:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.718+0000 7ffae2f0a640 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b2b028f100 0x55b2b03905a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.718+0000 7ffadbfff640 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9c8036900 0x7ff9c8038dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffacc000f60 tx=0x7ffacc0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.718+0000 7ffae2f0a640 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b2b028f100 0x55b2b03905a0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.718+0000 7ffae2f0a640 1 -- 172.21.15.165:0/422180316 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b028f100 2024-04-27T03:30:29.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.718+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b0202c20 con 0x55b2b028f100 2024-04-27T03:30:29.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.718+0000 7ffae2f0a640 1 -- 172.21.15.165:0/422180316 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b028f100 2024-04-27T03:30:29.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.718+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b2b0392dd0 0x55b2b03951c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.718+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03958b0 con 0x55b2b0392dd0 2024-04-27T03:30:29.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.719+0000 7ffae2709640 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b2b0392dd0 0x55b2b03951c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.719+0000 7ffae2709640 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b2b0392dd0 0x55b2b03951c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.719+0000 7ffae2709640 1 -- 172.21.15.165:0/422180316 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ffad4009d10 con 0x55b2b0392dd0 2024-04-27T03:30:29.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.719+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b0391db0 con 0x55b2b028f100 2024-04-27T03:30:29.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.719+0000 7ffae2f0a640 1 -- 172.21.15.165:0/422180316 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b028f100 2024-04-27T03:30:29.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.719+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b2b0396d10 0x55b2b0399180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.720+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03998e0 con 0x55b2b0396d10 2024-04-27T03:30:29.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.720+0000 7ffadbfff640 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b2b0396d10 0x55b2b0399180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.720+0000 7ffadbfff640 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b2b0396d10 0x55b2b0399180 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.720+0000 7ffadbfff640 1 -- 172.21.15.165:0/422180316 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ffacc009050 con 0x55b2b0396d10 2024-04-27T03:30:29.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.721+0000 7ffae4c0a800 1 Processor -- start 2024-04-27T03:30:29.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.721+0000 7ffae4c0a800 1 -- start start 2024-04-27T03:30:29.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.721+0000 7ffae4c0a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03a7d10 0x55b2b03a8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.721+0000 7ffae4c0a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b2b017f6b0 con 0x55b2b03a7d10 2024-04-27T03:30:29.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.721+0000 7ffae2709640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03a7d10 0x55b2b03a8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.721+0000 7ffae2709640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03a7d10 0x55b2b03a8130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51262/0 (socket says 172.21.15.165:51262) 2024-04-27T03:30:29.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.721+0000 7ffae2709640 1 -- 172.21.15.165:0/1922283580 learned_addr learned my addr 172.21.15.165:0/1922283580 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.721+0000 7ffae2709640 1 -- 172.21.15.165:0/1922283580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b03a8670 con 0x55b2b03a7d10 2024-04-27T03:30:29.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.721+0000 7ffae2709640 1 --2- 172.21.15.165:0/1922283580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03a7d10 0x55b2b03a8130 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffad4001eb0 tx=0x7ffad40793a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.722+0000 7ff9c7fff640 1 -- 172.21.15.165:0/1922283580 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad4081040 con 0x55b2b03a7d10 2024-04-27T03:30:29.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.722+0000 7ff9c7fff640 1 -- 172.21.15.165:0/1922283580 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad4058040 con 0x55b2b03a7d10 2024-04-27T03:30:29.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.722+0000 7ffae4c0a800 1 -- 172.21.15.165:0/1922283580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b2b03a90e0 con 0x55b2b03a7d10 2024-04-27T03:30:29.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.722+0000 7ff9c7fff640 1 -- 172.21.15.165:0/1922283580 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad408e0b0 con 0x55b2b03a7d10 2024-04-27T03:30:29.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.722+0000 7ffae4c0a800 1 -- 172.21.15.165:0/1922283580 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b03a95e0 con 0x55b2b03a7d10 2024-04-27T03:30:29.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.722+0000 7ff9c7fff640 1 -- 172.21.15.165:0/1922283580 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffad4095060 con 0x55b2b03a7d10 2024-04-27T03:30:29.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.723+0000 7ff9c7fff640 1 --2- 172.21.15.165:0/1922283580 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9bc036900 0x7ff9bc038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.723+0000 7ff9c7fff640 1 -- 172.21.15.165:0/1922283580 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ffad404b100 con 0x55b2b03a7d10 2024-04-27T03:30:29.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.723+0000 7ffadbfff640 1 --2- 172.21.15.165:0/1922283580 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9bc036900 0x7ff9bc038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.723+0000 7ffae4c0a800 1 Processor -- start 2024-04-27T03:30:29.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.723+0000 7ffae4c0a800 1 -- start start 2024-04-27T03:30:29.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.723+0000 7ffae4c0a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 0x55b2b03b52c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.723+0000 7ffadbfff640 1 --2- 172.21.15.165:0/1922283580 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9bc036900 0x7ff9bc038dc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00bf50 tx=0x7ffacc00f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.723+0000 7ffae4c0a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b2b037fe10 con 0x55b2b03b4ec0 2024-04-27T03:30:29.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.723+0000 7ffae2709640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 0x55b2b03b52c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.723+0000 7ffae2709640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 0x55b2b03b52c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51264/0 (socket says 172.21.15.165:51264) 2024-04-27T03:30:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.723+0000 7ffae2709640 1 -- 172.21.15.165:0/3230810521 learned_addr learned my addr 172.21.15.165:0/3230810521 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.724+0000 7ffae2709640 1 -- 172.21.15.165:0/3230810521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b03b5890 con 0x55b2b03b4ec0 2024-04-27T03:30:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.724+0000 7ffae2709640 1 --2- 172.21.15.165:0/3230810521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 0x55b2b03b52c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffad407d3f0 tx=0x7ffad408af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20655bce3b479813 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.724+0000 7ff9c5ffb640 1 -- 172.21.15.165:0/3230810521 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad408a0d0 con 0x55b2b03b4ec0 2024-04-27T03:30:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.724+0000 7ff9c5ffb640 1 -- 172.21.15.165:0/3230810521 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad40678a0 con 0x55b2b03b4ec0 2024-04-27T03:30:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.724+0000 7ff9c5ffb640 1 -- 172.21.15.165:0/3230810521 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad408a270 con 0x55b2b03b4ec0 2024-04-27T03:30:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.724+0000 7ffae4c0a800 1 -- 172.21.15.165:0/3230810521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 msgr2=0x55b2b03b52c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.724+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/3230810521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 0x55b2b03b52c0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7ffad407d3f0 tx=0x7ffad408af80 comp rx=0 tx=0).stop 2024-04-27T03:30:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.724+0000 7ffae4c0a800 1 -- 172.21.15.165:0/3230810521 shutdown_connections 2024-04-27T03:30:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.724+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/3230810521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 0x55b2b03b52c0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.724+0000 7ffae4c0a800 1 -- 172.21.15.165:0/3230810521 >> 172.21.15.165:0/3230810521 conn(0x55b2b03b1190 msgr2=0x55b2b03af1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.724+0000 7ffae4c0a800 1 -- 172.21.15.165:0/3230810521 shutdown_connections 2024-04-27T03:30:29.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.724+0000 7ffae4c0a800 1 -- 172.21.15.165:0/3230810521 wait complete. 2024-04-27T03:30:29.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.725+0000 7ffae4c0a800 1 Processor -- start 2024-04-27T03:30:29.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.725+0000 7ffae4c0a800 1 -- start start 2024-04-27T03:30:29.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.725+0000 7ffae4c0a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 0x55b2b03bc1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.725+0000 7ffae4c0a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b2b03b5f80 con 0x55b2b03b4ec0 2024-04-27T03:30:29.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.725+0000 7ffae2709640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 0x55b2b03bc1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.725+0000 7ffae2709640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 0x55b2b03bc1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51274/0 (socket says 172.21.15.165:51274) 2024-04-27T03:30:29.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.725+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 learned_addr learned my addr 172.21.15.165:0/343806808 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.725+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2b03bc720 con 0x55b2b03b4ec0 2024-04-27T03:30:29.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.725+0000 7ffae2709640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 0x55b2b03bc1e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffad407d3f0 tx=0x7ffad4095f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.726+0000 7ff9b3fff640 1 -- 172.21.15.165:0/343806808 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad40b2020 con 0x55b2b03b4ec0 2024-04-27T03:30:29.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.726+0000 7ff9b3fff640 1 -- 172.21.15.165:0/343806808 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffad407adf0 con 0x55b2b03b4ec0 2024-04-27T03:30:29.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.726+0000 7ff9b3fff640 1 -- 172.21.15.165:0/343806808 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffad4009540 con 0x55b2b03b4ec0 2024-04-27T03:30:29.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.726+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b2b03bc720 con 0x55b2b03b4ec0 2024-04-27T03:30:29.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.726+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2b03bc720 con 0x55b2b03b4ec0 2024-04-27T03:30:29.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.726+0000 7ff9b3fff640 1 -- 172.21.15.165:0/343806808 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffad40b5020 con 0x55b2b03b4ec0 2024-04-27T03:30:29.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.727+0000 7ff9b3fff640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9ac036950 0x7ff9ac038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.727+0000 7ff9b3fff640 1 -- 172.21.15.165:0/343806808 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ffad40b8ab0 con 0x55b2b03b4ec0 2024-04-27T03:30:29.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.727+0000 7ffadbfff640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9ac036950 0x7ff9ac038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.727+0000 7ffadbfff640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9ac036950 0x7ff9ac038e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffacc001b60 tx=0x7ffacc010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.727+0000 7ffae4c0a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.727+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b2b03c7950 0x55b2b03c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.727+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03ca560 con 0x55b2b03c7950 2024-04-27T03:30:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.727+0000 7ffae2f0a640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b2b03c7950 0x55b2b03c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae2f0a640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b2b03c7950 0x55b2b03c9e10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b03c7950 2024-04-27T03:30:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 20 realm 2024-04-27T03:30:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03ca970 con 0x55b2b03c7950 2024-04-27T03:30:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b03c7950 2024-04-27T03:30:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb240 con 0x55b2b03c7950 2024-04-27T03:30:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b03c7950 2024-04-27T03:30:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03c7950 2024-04-27T03:30:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b03c7950 2024-04-27T03:30:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.728+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b2b03cc340 0x55b2b03ce730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.729+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03ced40 con 0x55b2b03cc340 2024-04-27T03:30:29.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.729+0000 7ffae2709640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b2b03cc340 0x55b2b03ce730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.729+0000 7ffae2709640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b2b03cc340 0x55b2b03ce730 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.729+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.729+0000 7ffae4c0a800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:29.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.729+0000 7ffae4c0a800 20 searching for the correct realm 2024-04-27T03:30:29.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.729+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d0670 con 0x55b2b03cc340 2024-04-27T03:30:29.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.730+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.730+0000 7ffae2709640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffad40b7020 0x7ffad407d6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.730+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d0670 con 0x7ffad40b7020 2024-04-27T03:30:29.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.730+0000 7ffadbfff640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffad40b7020 0x7ffad407d6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.730+0000 7ffadbfff640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffad40b7020 0x7ffad407d6d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.730+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.730+0000 7ffadbfff640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffacc037040 0x7ffacc036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.730+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d0670 con 0x7ffacc037040 2024-04-27T03:30:29.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.731+0000 7ffae2f0a640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffacc037040 0x7ffacc036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.731+0000 7ffae2f0a640 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffacc037040 0x7ffacc036410 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.731+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ffadc00a4a0 con 0x7ffacc037040 2024-04-27T03:30:29.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.731+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03c7950 2024-04-27T03:30:29.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.731+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b03c7950 2024-04-27T03:30:29.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.731+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03c7950 2024-04-27T03:30:29.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.732+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b03c7950 2024-04-27T03:30:29.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.732+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffacc037040 2024-04-27T03:30:29.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.732+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc00a4a0 con 0x7ffacc037040 2024-04-27T03:30:29.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.732+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffacc037040 2024-04-27T03:30:29.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.732+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc00a4a0 con 0x7ffacc037040 2024-04-27T03:30:29.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.732+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffad40b7020 2024-04-27T03:30:29.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.733+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.733+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03cc340 2024-04-27T03:30:29.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.733+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.733+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03c7950 2024-04-27T03:30:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.733+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b03c7950 2024-04-27T03:30:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.733+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffacc037040 2024-04-27T03:30:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.733+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc00a4a0 con 0x7ffacc037040 2024-04-27T03:30:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.733+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03c7950 2024-04-27T03:30:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.734+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b03c7950 2024-04-27T03:30:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.734+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffad40b7020 2024-04-27T03:30:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.734+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.734+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03cc340 2024-04-27T03:30:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.734+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.734+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03cc340 2024-04-27T03:30:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.734+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.734+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffacc037040 2024-04-27T03:30:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.735+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc00a4a0 con 0x7ffacc037040 2024-04-27T03:30:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.735+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03cc340 2024-04-27T03:30:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.735+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v18'1 uv1 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.735+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03cc340 2024-04-27T03:30:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.735+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.735+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03c7950 2024-04-27T03:30:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.735+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b03c7950 2024-04-27T03:30:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.735+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03c7950 2024-04-27T03:30:29.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.736+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ffadc00a4a0 con 0x55b2b03c7950 2024-04-27T03:30:29.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.736+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffacc037040 2024-04-27T03:30:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.736+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc033040 con 0x7ffacc037040 2024-04-27T03:30:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.736+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03cc340 2024-04-27T03:30:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.736+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.736+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffad40b7020 2024-04-27T03:30:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.737+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.737+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03c7950 2024-04-27T03:30:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.737+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.737+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffacc037040 2024-04-27T03:30:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.737+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.737+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03cc340 2024-04-27T03:30:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.738+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.738+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffacc037040 2024-04-27T03:30:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.738+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.738+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffad40b7020 2024-04-27T03:30:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.738+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.738+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03cc340 2024-04-27T03:30:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffacc037040 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x55b2b03c7950 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cb5e0 con 0x7ffad40b7020 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae4c0a800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae4c0a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae4c0a800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae4c0a800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae4c0a800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae4c0a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae4c0a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.739+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cf600 con 0x55b2b03c7950 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cf9f0 con 0x55b2b03c7950 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03cf9f0 con 0x7ffacc037040 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 20 zone default found 2024-04-27T03:30:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 4 Realm: () 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 20 zonegroup default 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.740+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d0af0 con 0x7ffacc037040 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae4c0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae4c0a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d24d0 con 0x55b2b03cc340 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae4c0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae4c0a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d2870 con 0x55b2b03cc340 2024-04-27T03:30:29.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae4c0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae4c0a800 20 started sync module instance, tier type = 2024-04-27T03:30:29.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae4c0a800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:29.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.741+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d1970 con 0x7ffad40b7020 2024-04-27T03:30:29.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.742+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.742+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94225949332672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d2c10 con 0x7ffad40b7020 2024-04-27T03:30:29.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.742+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d3500 con 0x55b2b03c7950 2024-04-27T03:30:29.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.743+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94225949332672] v25'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.744+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.744+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94225949339568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d38f0 con 0x55b2b03c7950 2024-04-27T03:30:29.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.744+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d4310 con 0x55b2b03cc340 2024-04-27T03:30:29.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.745+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94225949339568] v25'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.745+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.745+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94225949343232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d4730 con 0x55b2b03cc340 2024-04-27T03:30:29.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.745+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d5150 con 0x7ffad40b7020 2024-04-27T03:30:29.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.746+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.746+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94225949343232] v25'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.746+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94225949334928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d5540 con 0x7ffad40b7020 2024-04-27T03:30:29.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.746+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d5fc0 con 0x55b2b03c7950 2024-04-27T03:30:29.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.748+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.748+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94225949350576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d6860 con 0x55b2b03c7950 2024-04-27T03:30:29.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.748+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d7340 con 0x7ffacc037040 2024-04-27T03:30:29.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.748+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94225949334928] v25'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.749+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.749+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94225949355600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d7340 con 0x7ffacc037040 2024-04-27T03:30:29.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.749+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d7d50 con 0x7ffacc037040 2024-04-27T03:30:29.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.749+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94225949350576] v25'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.749+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94225949355600] v25'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.750+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.750+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94225949346880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d78a0 con 0x7ffacc037040 2024-04-27T03:30:29.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.750+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d8260 con 0x7ffacc037040 2024-04-27T03:30:29.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.751+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94225949346880] v25'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.751+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'15 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.751+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94225949352800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d8260 con 0x7ffacc037040 2024-04-27T03:30:29.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.751+0000 7ffae4c0a800 20 add_watcher() i=0 2024-04-27T03:30:29.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.751+0000 7ffae4c0a800 20 add_watcher() i=1 2024-04-27T03:30:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.751+0000 7ffae4c0a800 20 add_watcher() i=2 2024-04-27T03:30:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.751+0000 7ffae4c0a800 20 add_watcher() i=3 2024-04-27T03:30:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.751+0000 7ffae4c0a800 20 add_watcher() i=4 2024-04-27T03:30:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.751+0000 7ffae4c0a800 20 add_watcher() i=5 2024-04-27T03:30:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.751+0000 7ffae4c0a800 20 add_watcher() i=6 2024-04-27T03:30:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.752+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94225949352800] v25'16 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.752+0000 7ffae4c0a800 20 add_watcher() i=7 2024-04-27T03:30:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.752+0000 7ffae4c0a800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.752+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03db560 con 0x55b2b03c7950 2024-04-27T03:30:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.753+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.753+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94225949359904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d87f0 con 0x55b2b03c7950 2024-04-27T03:30:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.754+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94225949359904] v25'47 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.754+0000 7ff998fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.754+0000 7ffae4c0a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.754+0000 7ffae4c0a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.754+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03dad60 con 0x55b2b03c7950 2024-04-27T03:30:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.755+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'43 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.755+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e4620 con 0x7ffacc037040 2024-04-27T03:30:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.757+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'19 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.757+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e4620 con 0x55b2b03cc340 2024-04-27T03:30:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.759+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'19 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.760+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e4ad0 con 0x55b2b03c7950 2024-04-27T03:30:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.762+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'48 uv15 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.762+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e5040 con 0x7ffad40b7020 2024-04-27T03:30:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.765+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'35 uv17 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.765+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e5890 con 0x55b2b03cc340 2024-04-27T03:30:29.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.768+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'31 uv16 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.768+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e5890 con 0x55b2b03c7950 2024-04-27T03:30:29.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.771+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'21 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.771+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e5920 con 0x7ffacc037040 2024-04-27T03:30:29.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.773+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'25 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.773+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e59b0 con 0x7ffad40b7020 2024-04-27T03:30:29.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.774+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'13 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.774+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e5f20 con 0x55b2b03cc340 2024-04-27T03:30:29.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.775+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'19 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.775+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e5fb0 con 0x7ffad40b7020 2024-04-27T03:30:29.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.776+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'36 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.776+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e6460 con 0x7ffacc037040 2024-04-27T03:30:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.777+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'15 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.777+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e6460 con 0x7ffad40b7020 2024-04-27T03:30:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.778+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'19 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.778+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e64f0 con 0x7ffacc037040 2024-04-27T03:30:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.779+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'37 uv22 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.779+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e64f0 con 0x55b2b03c7950 2024-04-27T03:30:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.780+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'49 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.780+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e64f0 con 0x55b2b03cc340 2024-04-27T03:30:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.782+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'29 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.782+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e6580 con 0x7ffad40b7020 2024-04-27T03:30:29.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.783+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'37 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.783+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e6610 con 0x55b2b03cc340 2024-04-27T03:30:29.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.783+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'25 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.783+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e6610 con 0x55b2b03c7950 2024-04-27T03:30:29.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.786+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.786+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e6610 con 0x55b2b03c7950 2024-04-27T03:30:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.789+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'50 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.789+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e6610 con 0x55b2b03cc340 2024-04-27T03:30:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.790+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'30 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.790+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e6610 con 0x7ffad40b7020 2024-04-27T03:30:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.791+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'29 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.791+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e69f0 con 0x55b2b03cc340 2024-04-27T03:30:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.794+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.794+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e69f0 con 0x55b2b03cc340 2024-04-27T03:30:29.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.795+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'32 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.795+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e69f0 con 0x55b2b03cc340 2024-04-27T03:30:29.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.796+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'15 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.796+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e69f0 con 0x55b2b03cc340 2024-04-27T03:30:29.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.797+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'32 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.797+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e69f0 con 0x7ffacc037040 2024-04-27T03:30:29.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.798+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'21 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.798+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e69f0 con 0x55b2b03cc340 2024-04-27T03:30:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.799+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'25 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.799+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e69f0 con 0x7ffacc037040 2024-04-27T03:30:29.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.801+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'26 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.801+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e69f0 con 0x55b2b03cc340 2024-04-27T03:30:29.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.802+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'31 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.802+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e69f0 con 0x7ffacc037040 2024-04-27T03:30:29.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.803+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'38 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.803+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e69f0 con 0x7ffad40b7020 2024-04-27T03:30:29.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.804+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.804+0000 7ff9137fe640 20 reqs_thread_entry: start 2024-04-27T03:30:29.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.804+0000 7ffae4c0a800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:29.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.804+0000 7ffae4c0a800 20 Filter name: none 2024-04-27T03:30:29.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.804+0000 7ff9117fa640 20 reqs_thread_entry: start 2024-04-27T03:30:29.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.804+0000 7ffae4c0a800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:29.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.804+0000 7ffae4c0a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:29.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.804+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b2b0411430 con 0x55b2b03c7950 2024-04-27T03:30:29.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.805+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(94 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.805+0000 7ffae4c0a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:29.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.805+0000 7ffae4c0a800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-04-27T03:30:29.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.805+0000 7ffae4c0a800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:29.811 INFO:teuthology.orchestra.run.smithi165.stderr:could not remove user: unable to remove user, user does not exist 2024-04-27T03:30:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.805+0000 7ff998fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.805+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:95 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94225949359904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03e2b20 con 0x55b2b03c7950 2024-04-27T03:30:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.806+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(95 data_loggenerations_metadata [watch unwatch cookie 94225949359904] v25'51 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.806+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:96 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94225949332672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d7f20 con 0x7ffad40b7020 2024-04-27T03:30:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.807+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(96 notify.0 [watch unwatch cookie 94225949332672] v25'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.807+0000 7ffae4c0a800 20 remove_watcher() i=0 2024-04-27T03:30:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.807+0000 7ffae4c0a800 2 removed watcher, disabling cache 2024-04-27T03:30:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.807+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94225949339568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d7f20 con 0x55b2b03c7950 2024-04-27T03:30:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.809+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(97 notify.1 [watch unwatch cookie 94225949339568] v25'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.809+0000 7ffae4c0a800 20 remove_watcher() i=1 2024-04-27T03:30:29.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.809+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94225949343232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d2fb0 con 0x55b2b03cc340 2024-04-27T03:30:29.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.810+0000 7ffae2709640 1 -- 172.21.15.165:0/343806808 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(98 notify.2 [watch unwatch cookie 94225949343232] v25'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffad40ab040 con 0x55b2b03cc340 2024-04-27T03:30:29.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.810+0000 7ffae4c0a800 20 remove_watcher() i=2 2024-04-27T03:30:29.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.810+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:99 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94225949334928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d7f20 con 0x7ffad40b7020 2024-04-27T03:30:29.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.812+0000 7ffadbfff640 1 -- 172.21.15.165:0/343806808 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(99 notify.3 [watch unwatch cookie 94225949334928] v25'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffacc009050 con 0x7ffad40b7020 2024-04-27T03:30:29.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.812+0000 7ffae4c0a800 20 remove_watcher() i=3 2024-04-27T03:30:29.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.812+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94225949350576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d2fb0 con 0x55b2b03c7950 2024-04-27T03:30:29.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.813+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(100 notify.4 [watch unwatch cookie 94225949350576] v25'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x55b2b03c7950 2024-04-27T03:30:29.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.813+0000 7ffae4c0a800 20 remove_watcher() i=4 2024-04-27T03:30:29.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.813+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94225949355600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d1d90 con 0x7ffacc037040 2024-04-27T03:30:29.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.814+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(101 notify.5 [watch unwatch cookie 94225949355600] v25'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.814+0000 7ffae4c0a800 20 remove_watcher() i=5 2024-04-27T03:30:29.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.814+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94225949346880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d7f20 con 0x7ffacc037040 2024-04-27T03:30:29.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.815+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(102 notify.6 [watch unwatch cookie 94225949346880] v25'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.815+0000 7ffae4c0a800 20 remove_watcher() i=6 2024-04-27T03:30:29.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.815+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:103 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94225949352800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b2b03d2fb0 con 0x7ffacc037040 2024-04-27T03:30:29.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(103 notify.7 [watch unwatch cookie 94225949352800] v25'17 uv15 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffadc031040 con 0x7ffacc037040 2024-04-27T03:30:29.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 20 remove_watcher() i=7 2024-04-27T03:30:29.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffacc037040 msgr2=0x7ffacc036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffacc037040 0x7ffacc036410 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b2b03c7950 msgr2=0x55b2b03c9e10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b2b03c7950 0x55b2b03c9e10 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffad40b7020 msgr2=0x7ffad407d6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffad40b7020 0x7ffad407d6d0 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b2b03cc340 msgr2=0x55b2b03ce730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b2b03cc340 0x55b2b03ce730 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9ac036950 msgr2=0x7ff9ac038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9ac036950 0x7ff9ac038e10 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffacc001b60 tx=0x7ffacc010000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 msgr2=0x55b2b03bc1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae2f0a640 1 -- 172.21.15.165:0/343806808 reap_dead start 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 0x55b2b03bc1e0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ffad407d3f0 tx=0x7ffad4095f60 comp rx=0 tx=0).stop 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 shutdown_connections 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/343806808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03b4ec0 0x55b2b03bc1e0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.816+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 >> 172.21.15.165:0/343806808 conn(0x55b2b03b1190 msgr2=0x55b2b03ae7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 shutdown_connections 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/343806808 wait complete. 2024-04-27T03:30:29.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/1922283580 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9bc036900 msgr2=0x7ff9bc038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/1922283580 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9bc036900 0x7ff9bc038dc0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ffacc00bf50 tx=0x7ffacc00f000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/1922283580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03a7d10 msgr2=0x55b2b03a8130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/1922283580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03a7d10 0x55b2b03a8130 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7ffad4001eb0 tx=0x7ffad40793a0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/1922283580 shutdown_connections 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/1922283580 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9bc036900 0x7ff9bc038dc0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/1922283580 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b03a7d10 0x55b2b03a8130 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/1922283580 >> 172.21.15.165:0/1922283580 conn(0x55b2b039bdb0 msgr2=0x55b2b039c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/1922283580 shutdown_connections 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/1922283580 wait complete. 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b2b0396d10 msgr2=0x55b2b0399180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b2b0396d10 0x55b2b0399180 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b2b028f100 msgr2=0x55b2b03905a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b2b028f100 0x55b2b03905a0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b2b0392dd0 msgr2=0x55b2b03951c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b2b0392dd0 0x55b2b03951c0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9c8036900 msgr2=0x7ff9c8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9c8036900 0x7ff9c8038dc0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ffacc000f60 tx=0x7ffacc0012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 msgr2=0x55b2b0203970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 --2- 172.21.15.165:0/422180316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b2b028df90 0x55b2b0203970 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7ffad4060040 tx=0x7ffad4002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae2f0a640 1 -- 172.21.15.165:0/422180316 reap_dead start 2024-04-27T03:30:29.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 shutdown_connections 2024-04-27T03:30:29.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 >> 172.21.15.165:0/422180316 conn(0x55b2b0289530 msgr2=0x55b2b03796a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 shutdown_connections 2024-04-27T03:30:29.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.817+0000 7ffae4c0a800 1 -- 172.21.15.165:0/422180316 wait complete. 2024-04-27T03:30:29.831 DEBUG:teuthology.orchestra.run:got remote process result: 2 2024-04-27T03:30:29.831 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-27T03:30:29.831 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-27T03:30:29.831 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-04-27T03:30:29.916 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:29.916 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:29.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.911+0000 7fd7c563e800 1 Processor -- start 2024-04-27T03:30:29.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.911+0000 7fd7c563e800 1 -- start start 2024-04-27T03:30:29.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.911+0000 7fd7c563e800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x555a23c3ea50 0x555a23c40e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.911+0000 7fd7c563e800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x555a23afcd40 con 0x555a23c41380 2024-04-27T03:30:29.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.911+0000 7fd7c563e800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x555a23acf4d0 con 0x555a23c3ea50 2024-04-27T03:30:29.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.912+0000 7fd7c02d5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x555a23c3ea50 0x555a23c40e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.912+0000 7fd7bfad4640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x555a23c41380 0x555a23c43770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53580/0 (socket says 172.21.15.165:53580) 2024-04-27T03:30:29.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.912+0000 7fd7bfad4640 1 -- 172.21.15.165:0/1093022346 learned_addr learned my addr 172.21.15.165:0/1093022346 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:29.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.912+0000 7fd7c02d5640 1 -- 172.21.15.165:0/1093022346 >> v1:172.21.15.89:6789/0 conn(0x555a23c41380 legacy=0x555a23c43770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.912+0000 7fd7c02d5640 1 -- 172.21.15.165:0/1093022346 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a23afb590 con 0x555a23c3ea50 2024-04-27T03:30:29.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.912+0000 7fd7bf2d3640 1 -- 172.21.15.165:0/1093022346 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2385978529 0 0) 0x555a23afcd40 con 0x555a23c41380 2024-04-27T03:30:29.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.914+0000 7fd7c02d5640 1 --2- 172.21.15.165:0/1093022346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23c40e40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400a470 tx=0x7fd7b401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57af9028af860765 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.914+0000 7fd7bf2d3640 1 -- 172.21.15.165:0/1093022346 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b4057070 con 0x555a23c3ea50 2024-04-27T03:30:29.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.914+0000 7fd7bf2d3640 1 -- 172.21.15.165:0/1093022346 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b4052570 con 0x555a23c3ea50 2024-04-27T03:30:29.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.914+0000 7fd7bf2d3640 1 -- 172.21.15.165:0/1093022346 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b4052870 con 0x555a23c3ea50 2024-04-27T03:30:29.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.914+0000 7fd7c563e800 1 -- 172.21.15.165:0/1093022346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 msgr2=0x555a23c40e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.914+0000 7fd7c563e800 1 --2- 172.21.15.165:0/1093022346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23c40e40 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fd7b400a470 tx=0x7fd7b401ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:29.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.914+0000 7fd7c563e800 1 -- 172.21.15.165:0/1093022346 shutdown_connections 2024-04-27T03:30:29.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.914+0000 7fd7c563e800 1 --2- 172.21.15.165:0/1093022346 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23c40e40 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.914+0000 7fd7c563e800 1 -- 172.21.15.165:0/1093022346 >> 172.21.15.165:0/1093022346 conn(0x555a23c383e0 msgr2=0x555a23c3a800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.914+0000 7fd7c563e800 1 -- 172.21.15.165:0/1093022346 shutdown_connections 2024-04-27T03:30:29.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.914+0000 7fd7c563e800 1 -- 172.21.15.165:0/1093022346 wait complete. 2024-04-27T03:30:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.926+0000 7fd7c563e800 1 Processor -- start 2024-04-27T03:30:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.926+0000 7fd7c563e800 1 -- start start 2024-04-27T03:30:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.926+0000 7fd7c563e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23c3e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.926+0000 7fd7c563e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555a23b893e0 con 0x555a23c3ea50 2024-04-27T03:30:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.926+0000 7fd7c02d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23c3e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.927+0000 7fd7c02d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23c3e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51296/0 (socket says 172.21.15.165:51296) 2024-04-27T03:30:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.927+0000 7fd7c02d5640 1 -- 172.21.15.165:0/295653689 learned_addr learned my addr 172.21.15.165:0/295653689 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.927+0000 7fd7c02d5640 1 -- 172.21.15.165:0/295653689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a23c3c950 con 0x555a23c3ea50 2024-04-27T03:30:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.927+0000 7fd7c02d5640 1 --2- 172.21.15.165:0/295653689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23c3e390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4055040 tx=0x7fd7b404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7494029771b647 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.927+0000 7fd6ae60c640 1 -- 172.21.15.165:0/295653689 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b4057070 con 0x555a23c3ea50 2024-04-27T03:30:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.927+0000 7fd6ae60c640 1 -- 172.21.15.165:0/295653689 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b4063070 con 0x555a23c3ea50 2024-04-27T03:30:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.927+0000 7fd6ae60c640 1 -- 172.21.15.165:0/295653689 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b4052720 con 0x555a23c3ea50 2024-04-27T03:30:29.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.928+0000 7fd7c563e800 1 -- 172.21.15.165:0/295653689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 msgr2=0x555a23c3e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.928+0000 7fd7c563e800 1 --2- 172.21.15.165:0/295653689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23c3e390 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4055040 tx=0x7fd7b404c000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.928+0000 7fd7c563e800 1 -- 172.21.15.165:0/295653689 shutdown_connections 2024-04-27T03:30:29.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.928+0000 7fd7c563e800 1 --2- 172.21.15.165:0/295653689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23c3e390 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.928+0000 7fd7c563e800 1 -- 172.21.15.165:0/295653689 >> 172.21.15.165:0/295653689 conn(0x555a23c383e0 msgr2=0x555a23d2a950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.928+0000 7fd7c563e800 1 -- 172.21.15.165:0/295653689 shutdown_connections 2024-04-27T03:30:29.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.928+0000 7fd7c563e800 1 -- 172.21.15.165:0/295653689 wait complete. 2024-04-27T03:30:29.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.928+0000 7fd7c563e800 1 Processor -- start 2024-04-27T03:30:29.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.928+0000 7fd7c563e800 1 -- start start 2024-04-27T03:30:29.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.928+0000 7fd7c563e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23d38c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.928+0000 7fd7c563e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555a23c471a0 con 0x555a23c3ea50 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.929+0000 7fd7c02d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23d38c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.929+0000 7fd7c02d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23d38c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51298/0 (socket says 172.21.15.165:51298) 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.929+0000 7fd7c02d5640 1 -- 172.21.15.165:0/3057553952 learned_addr learned my addr 172.21.15.165:0/3057553952 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.929+0000 7fd7c02d5640 1 -- 172.21.15.165:0/3057553952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a23d391a0 con 0x555a23c3ea50 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.929+0000 7fd7c02d5640 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23d38c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4060040 tx=0x7fd7b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.929+0000 7fd6ace09640 1 -- 172.21.15.165:0/3057553952 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b4052e30 con 0x555a23c3ea50 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.929+0000 7fd6ace09640 1 -- 172.21.15.165:0/3057553952 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b406c070 con 0x555a23c3ea50 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.929+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555a23d39430 con 0x555a23c3ea50 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.929+0000 7fd6ace09640 1 -- 172.21.15.165:0/3057553952 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b405e5f0 con 0x555a23c3ea50 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.929+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a23d3c0e0 con 0x555a23c3ea50 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.930+0000 7fd7c563e800 1 Processor -- start 2024-04-27T03:30:29.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.930+0000 7fd7c563e800 1 -- start start 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.930+0000 7fd6ace09640 1 -- 172.21.15.165:0/3057553952 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fd7b405e790 con 0x555a23c3ea50 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.930+0000 7fd7c563e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 0x555a23d46980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.930+0000 7fd7c563e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555a23b2f6b0 con 0x555a23d46560 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.930+0000 7fd6ace09640 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd6a8036950 0x7fd6a8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.930+0000 7fd7c0ad6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 0x555a23d46980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.930+0000 7fd7c0ad6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 0x555a23d46980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51314/0 (socket says 172.21.15.165:51314) 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.930+0000 7fd7c0ad6640 1 -- 172.21.15.165:0/3737523296 learned_addr learned my addr 172.21.15.165:0/3737523296 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.930+0000 7fd6ace09640 1 -- 172.21.15.165:0/3057553952 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fd7b408e6f0 con 0x555a23c3ea50 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.930+0000 7fd7c02d5640 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd6a8036950 0x7fd6a8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.930+0000 7fd7c0ad6640 1 -- 172.21.15.165:0/3737523296 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a23d46ec0 con 0x555a23d46560 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.931+0000 7fd7c02d5640 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd6a8036950 0x7fd6a8038e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4073040 tx=0x7fd7b4071e70 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.931+0000 7fd7c0ad6640 1 --2- 172.21.15.165:0/3737523296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 0x555a23d46980 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800a430 tx=0x7fd7b800a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18fe6e516c03073 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.931+0000 7fd6a67fc640 1 -- 172.21.15.165:0/3737523296 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b8015070 con 0x555a23d46560 2024-04-27T03:30:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.931+0000 7fd6a67fc640 1 -- 172.21.15.165:0/3737523296 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b8001670 con 0x555a23d46560 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.931+0000 7fd6a67fc640 1 -- 172.21.15.165:0/3737523296 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b8001990 con 0x555a23d46560 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.931+0000 7fd7c563e800 1 -- 172.21.15.165:0/3737523296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 msgr2=0x555a23d46980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.931+0000 7fd7c563e800 1 --2- 172.21.15.165:0/3737523296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 0x555a23d46980 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fd7b800a430 tx=0x7fd7b800a8f0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.931+0000 7fd7c563e800 1 -- 172.21.15.165:0/3737523296 shutdown_connections 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.931+0000 7fd7c563e800 1 --2- 172.21.15.165:0/3737523296 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 0x555a23d46980 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.931+0000 7fd7c563e800 1 -- 172.21.15.165:0/3737523296 >> 172.21.15.165:0/3737523296 conn(0x555a23c41380 msgr2=0x555a23d43000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.931+0000 7fd7c563e800 1 -- 172.21.15.165:0/3737523296 shutdown_connections 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.931+0000 7fd7c563e800 1 -- 172.21.15.165:0/3737523296 wait complete. 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.932+0000 7fd7c563e800 1 Processor -- start 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.932+0000 7fd7c563e800 1 -- start start 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.932+0000 7fd7c563e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 0x555a23d4d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.932+0000 7fd7c563e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555a23d47850 con 0x555a23d46560 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.932+0000 7fd7c0ad6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 0x555a23d4d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.932+0000 7fd7c0ad6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 0x555a23d4d9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51326/0 (socket says 172.21.15.165:51326) 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.932+0000 7fd7c0ad6640 1 -- 172.21.15.165:0/4190691310 learned_addr learned my addr 172.21.15.165:0/4190691310 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.932+0000 7fd7c0ad6640 1 -- 172.21.15.165:0/4190691310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a23d4def0 con 0x555a23d46560 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.932+0000 7fd7c0ad6640 1 --2- 172.21.15.165:0/4190691310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 0x555a23d4d9b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8013040 tx=0x7fd7b8003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.932+0000 7fd6a4ff9640 1 -- 172.21.15.165:0/4190691310 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b8020020 con 0x555a23d46560 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.933+0000 7fd6a4ff9640 1 -- 172.21.15.165:0/4190691310 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b80108e0 con 0x555a23d46560 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.933+0000 7fd7c563e800 1 -- 172.21.15.165:0/4190691310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555a23d4e180 con 0x555a23d46560 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.933+0000 7fd6a4ff9640 1 -- 172.21.15.165:0/4190691310 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b8015040 con 0x555a23d46560 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.933+0000 7fd7c563e800 1 -- 172.21.15.165:0/4190691310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a23d50e30 con 0x555a23d46560 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.933+0000 7fd6a4ff9640 1 -- 172.21.15.165:0/4190691310 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fd7b8023020 con 0x555a23d46560 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.933+0000 7fd6a4ff9640 1 --2- 172.21.15.165:0/4190691310 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd698036900 0x7fd698038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.934+0000 7fd6a4ff9640 1 -- 172.21.15.165:0/4190691310 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fd7b804bf50 con 0x555a23d46560 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.934+0000 7fd7c563e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.934+0000 7fd7c563e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.934+0000 7fd7bfad4640 1 --2- 172.21.15.165:0/4190691310 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd698036900 0x7fd698038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.934+0000 7fd7c563e800 1 Processor -- start 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.934+0000 7fd7c563e800 1 -- start start 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.934+0000 7fd7bfad4640 1 --2- 172.21.15.165:0/4190691310 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd698036900 0x7fd698038dc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0000c00 tx=0x7fd7b0001330 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.934+0000 7fd7c563e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d67ff0 0x555a23d68410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.934+0000 7fd7c563e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555a23c3cc10 con 0x555a23d67ff0 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.934+0000 7fd7c02d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d67ff0 0x555a23d68410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.934+0000 7fd7c02d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d67ff0 0x555a23d68410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51342/0 (socket says 172.21.15.165:51342) 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.934+0000 7fd7c02d5640 1 -- 172.21.15.165:0/2028975832 learned_addr learned my addr 172.21.15.165:0/2028975832 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.935+0000 7fd7c02d5640 1 -- 172.21.15.165:0/2028975832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555a23d68950 con 0x555a23d67ff0 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.935+0000 7fd7c02d5640 1 --2- 172.21.15.165:0/2028975832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d67ff0 0x555a23d68410 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4068010 tx=0x7fd7b407a7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.935+0000 7fd684fe1640 1 -- 172.21.15.165:0/2028975832 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b407aa20 con 0x555a23d67ff0 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.935+0000 7fd684fe1640 1 -- 172.21.15.165:0/2028975832 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd7b4058040 con 0x555a23d67ff0 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.935+0000 7fd7c563e800 1 -- 172.21.15.165:0/2028975832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555a23d693c0 con 0x555a23d67ff0 2024-04-27T03:30:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.935+0000 7fd684fe1640 1 -- 172.21.15.165:0/2028975832 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7b4009560 con 0x555a23d67ff0 2024-04-27T03:30:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.935+0000 7fd7c563e800 1 -- 172.21.15.165:0/2028975832 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555a23d698c0 con 0x555a23d67ff0 2024-04-27T03:30:29.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.936+0000 7fd684fe1640 1 -- 172.21.15.165:0/2028975832 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fd7b405ec60 con 0x555a23d67ff0 2024-04-27T03:30:29.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.936+0000 7fd684fe1640 1 --2- 172.21.15.165:0/2028975832 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd6000368b0 0x7fd600038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.936+0000 7fd684fe1640 1 -- 172.21.15.165:0/2028975832 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fd7b4098ae0 con 0x555a23d67ff0 2024-04-27T03:30:29.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.936+0000 7fd7bfad4640 1 --2- 172.21.15.165:0/2028975832 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd6000368b0 0x7fd600038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.936+0000 7fd61e7fc640 20 reqs_thread_entry: start 2024-04-27T03:30:29.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.936+0000 7fd7c563e800 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555a23d75ab0 0x555a23d77f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.936+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555a23d785d0 con 0x555a23d75ab0 2024-04-27T03:30:29.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.936+0000 7fd7bfad4640 1 --2- 172.21.15.165:0/2028975832 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd6000368b0 0x7fd600038d70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0000f60 tx=0x7fd7b0006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.936+0000 7fd7c0ad6640 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555a23d75ab0 0x555a23d77f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.937+0000 7fd7c0ad6640 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555a23d75ab0 0x555a23d77f00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.937+0000 7fd7c0ad6640 1 -- 172.21.15.165:0/3057553952 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd7b800e070 con 0x555a23d75ab0 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.937+0000 7fd7c563e800 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555a23d7a3a0 0x555a23d7c790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.937+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:2 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555a23d7ce80 con 0x555a23d7a3a0 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.937+0000 7fd7c02d5640 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555a23d7a3a0 0x555a23d7c790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.937+0000 7fd7c02d5640 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555a23d7a3a0 0x555a23d7c790 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.938+0000 7fd7c02d5640 1 -- 172.21.15.165:0/3057553952 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(2 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv1 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd7b409d020 con 0x555a23d7a3a0 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.938+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~993 in=993b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555a23d79870 con 0x555a23d7a3a0 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.939+0000 7fd7c02d5640 1 -- 172.21.15.165:0/3057553952 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [stat,call,call,writefull 0~993] v25'2 uv2 ondisk = 0) ==== 316+0+0 (crc 0 0 0) 0x7fd7b409d020 con 0x555a23d7a3a0 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 -- 172.21.15.165:0/4190691310 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd698036900 msgr2=0x7fd698038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 --2- 172.21.15.165:0/4190691310 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd698036900 0x7fd698038dc0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0000c00 tx=0x7fd7b0001330 comp rx=0 tx=0).stop 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 -- 172.21.15.165:0/4190691310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 msgr2=0x555a23d4d9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 --2- 172.21.15.165:0/4190691310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 0x555a23d4d9b0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fd7b8013040 tx=0x7fd7b8003370 comp rx=0 tx=0).stop 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 -- 172.21.15.165:0/4190691310 shutdown_connections 2024-04-27T03:30:29.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 --2- 172.21.15.165:0/4190691310 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd698036900 0x7fd698038dc0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 --2- 172.21.15.165:0/4190691310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d46560 0x555a23d4d9b0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 -- 172.21.15.165:0/4190691310 >> 172.21.15.165:0/4190691310 conn(0x555a23c41380 msgr2=0x555a23d40640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 -- 172.21.15.165:0/4190691310 shutdown_connections 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 -- 172.21.15.165:0/4190691310 wait complete. 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 -- 172.21.15.165:0/2028975832 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd6000368b0 msgr2=0x7fd600038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 --2- 172.21.15.165:0/2028975832 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd6000368b0 0x7fd600038d70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fd7b0000f60 tx=0x7fd7b0006000 comp rx=0 tx=0).stop 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 -- 172.21.15.165:0/2028975832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d67ff0 msgr2=0x555a23d68410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 --2- 172.21.15.165:0/2028975832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d67ff0 0x555a23d68410 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4068010 tx=0x7fd7b407a7c0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 -- 172.21.15.165:0/2028975832 shutdown_connections 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 --2- 172.21.15.165:0/2028975832 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd6000368b0 0x7fd600038d70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 --2- 172.21.15.165:0/2028975832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23d67ff0 0x555a23d68410 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 -- 172.21.15.165:0/2028975832 >> 172.21.15.165:0/2028975832 conn(0x555a23d5cac0 msgr2=0x555a23d5d4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 -- 172.21.15.165:0/2028975832 shutdown_connections 2024-04-27T03:30:29.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.940+0000 7fd7c563e800 1 -- 172.21.15.165:0/2028975832 wait complete. 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555a23d75ab0 msgr2=0x555a23d77f00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555a23d75ab0 0x555a23d77f00 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555a23d7a3a0 msgr2=0x555a23d7c790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555a23d7a3a0 0x555a23d7c790 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd6a8036950 msgr2=0x7fd6a8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd6a8036950 0x7fd6a8038e10 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4073040 tx=0x7fd7b4071e70 comp rx=0 tx=0).stop 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 msgr2=0x555a23d38c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23d38c60 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fd7b4060040 tx=0x7fd7b4002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 shutdown_connections 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555a23d7a3a0 0x555a23d7c790 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555a23d75ab0 0x555a23d77f00 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd6a8036950 0x7fd6a8038e10 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 --2- 172.21.15.165:0/3057553952 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555a23c3ea50 0x555a23d38c60 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:29.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 >> 172.21.15.165:0/3057553952 conn(0x555a23c383e0 msgr2=0x555a23d2a520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:29.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 shutdown_connections 2024-04-27T03:30:29.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.941+0000 7fd7c563e800 1 -- 172.21.15.165:0/3057553952 wait complete. 2024-04-27T03:30:29.956 INFO:teuthology.orchestra.run.smithi165.stdout:{"id":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-04-27T03:30:29.956 DEBUG:tasks.util.rgw: json result: {'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-04-27T03:30:29.957 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-04-27T03:30:29.957 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-04-27T03:30:29.957 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-04-27T03:30:29.998 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:29.998 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:29.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.993+0000 7f3001746800 1 Processor -- start 2024-04-27T03:30:29.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.993+0000 7f3001746800 1 -- start start 2024-04-27T03:30:30.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.993+0000 7f3001746800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55c1abbdb320 0x55c1abbdd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.993+0000 7f3001746800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c1aba95d40 con 0x55c1abbd6f80 2024-04-27T03:30:30.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.993+0000 7f3001746800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55c1aba684d0 con 0x55c1abbdb320 2024-04-27T03:30:30.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.993+0000 7f2ffea44640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55c1abbdb320 0x55c1abbdd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.993+0000 7f2fff245640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55c1abbd6f80 0x55c1abbd73a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53582/0 (socket says 172.21.15.165:53582) 2024-04-27T03:30:30.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.993+0000 7f2ffea44640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55c1abbdb320 0x55c1abbdd710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51356/0 (socket says 172.21.15.165:51356) 2024-04-27T03:30:30.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.993+0000 7f2fff245640 1 -- 172.21.15.165:0/907847654 learned_addr learned my addr 172.21.15.165:0/907847654 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:30.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.994+0000 7f2ffea44640 1 -- 172.21.15.165:0/907847654 >> v1:172.21.15.89:6789/0 conn(0x55c1abbd6f80 legacy=0x55c1abbd73a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.994+0000 7f2ffea44640 1 -- 172.21.15.165:0/907847654 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1aba94590 con 0x55c1abbdb320 2024-04-27T03:30:30.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.996+0000 7f2ffea44640 1 --2- 172.21.15.165:0/907847654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbdb320 0x55c1abbdd710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800a470 tx=0x7f2fe801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3663c1ae44f30308 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.996+0000 7f2ffe243640 1 -- 172.21.15.165:0/907847654 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fe8057070 con 0x55c1abbdb320 2024-04-27T03:30:30.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.996+0000 7f2ffe243640 1 -- 172.21.15.165:0/907847654 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2fe8052570 con 0x55c1abbdb320 2024-04-27T03:30:30.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.996+0000 7f2ffe243640 1 -- 172.21.15.165:0/907847654 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2fe8052870 con 0x55c1abbdb320 2024-04-27T03:30:30.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.996+0000 7f3001746800 1 -- 172.21.15.165:0/907847654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbdb320 msgr2=0x55c1abbdd710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.996+0000 7f3001746800 1 --2- 172.21.15.165:0/907847654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbdb320 0x55c1abbdd710 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800a470 tx=0x7f2fe801ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:30.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.996+0000 7f3001746800 1 -- 172.21.15.165:0/907847654 shutdown_connections 2024-04-27T03:30:30.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.996+0000 7f3001746800 1 --2- 172.21.15.165:0/907847654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbdb320 0x55c1abbdd710 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.996+0000 7f3001746800 1 -- 172.21.15.165:0/907847654 >> 172.21.15.165:0/907847654 conn(0x55c1abbd2520 msgr2=0x55c1abbd4940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.996+0000 7f3001746800 1 -- 172.21.15.165:0/907847654 shutdown_connections 2024-04-27T03:30:30.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:29.996+0000 7f3001746800 1 -- 172.21.15.165:0/907847654 wait complete. 2024-04-27T03:30:30.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.008+0000 7f3001746800 1 Processor -- start 2024-04-27T03:30:30.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.008+0000 7f3001746800 1 -- start start 2024-04-27T03:30:30.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.008+0000 7f3001746800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 0x55c1abb4c990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.008+0000 7f3001746800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c1abb223b0 con 0x55c1abbd6f80 2024-04-27T03:30:30.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.008+0000 7f2fff245640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 0x55c1abb4c990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.008+0000 7f2fff245640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 0x55c1abb4c990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51362/0 (socket says 172.21.15.165:51362) 2024-04-27T03:30:30.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.008+0000 7f2fff245640 1 -- 172.21.15.165:0/3277490705 learned_addr learned my addr 172.21.15.165:0/3277490705 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.009+0000 7f2fff245640 1 -- 172.21.15.165:0/3277490705 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1abb4af50 con 0x55c1abbd6f80 2024-04-27T03:30:30.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.009+0000 7f2fff245640 1 --2- 172.21.15.165:0/3277490705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 0x55c1abb4c990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8001e10 tx=0x7f2ff800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab11e11d195fa60c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.009+0000 7f2eed60a640 1 -- 172.21.15.165:0/3277490705 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ff8012020 con 0x55c1abbd6f80 2024-04-27T03:30:30.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.009+0000 7f2eed60a640 1 -- 172.21.15.165:0/3277490705 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff800e5a0 con 0x55c1abbd6f80 2024-04-27T03:30:30.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.009+0000 7f2eed60a640 1 -- 172.21.15.165:0/3277490705 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ff800e8e0 con 0x55c1abbd6f80 2024-04-27T03:30:30.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f3001746800 1 -- 172.21.15.165:0/3277490705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 msgr2=0x55c1abb4c990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f3001746800 1 --2- 172.21.15.165:0/3277490705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 0x55c1abb4c990 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2ff8001e10 tx=0x7f2ff800b780 comp rx=0 tx=0).stop 2024-04-27T03:30:30.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f3001746800 1 -- 172.21.15.165:0/3277490705 shutdown_connections 2024-04-27T03:30:30.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f3001746800 1 --2- 172.21.15.165:0/3277490705 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 0x55c1abb4c990 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f3001746800 1 -- 172.21.15.165:0/3277490705 >> 172.21.15.165:0/3277490705 conn(0x55c1abbd2520 msgr2=0x55c1abcc3a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f3001746800 1 -- 172.21.15.165:0/3277490705 shutdown_connections 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f3001746800 1 -- 172.21.15.165:0/3277490705 wait complete. 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f3001746800 1 Processor -- start 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f3001746800 1 -- start start 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f3001746800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 0x55c1abcd1e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f3001746800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c1abbddfe0 con 0x55c1abbd6f80 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f2fff245640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 0x55c1abcd1e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f2fff245640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 0x55c1abcd1e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51378/0 (socket says 172.21.15.165:51378) 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.010+0000 7f2fff245640 1 -- 172.21.15.165:0/3359259474 learned_addr learned my addr 172.21.15.165:0/3359259474 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.011+0000 7f2fff245640 1 -- 172.21.15.165:0/3359259474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1abcd23d0 con 0x55c1abbd6f80 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.011+0000 7f2fff245640 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 0x55c1abcd1e90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800b7c0 tx=0x7f2ff8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.011+0000 7f2eebe07640 1 -- 172.21.15.165:0/3359259474 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ff8012030 con 0x55c1abbd6f80 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.011+0000 7f2eebe07640 1 -- 172.21.15.165:0/3359259474 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8001470 con 0x55c1abbd6f80 2024-04-27T03:30:30.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.011+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c1abcd2660 con 0x55c1abbd6f80 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.011+0000 7f2eebe07640 1 -- 172.21.15.165:0/3359259474 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ff801a8d0 con 0x55c1abbd6f80 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.011+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1abcd52c0 con 0x55c1abbd6f80 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.012+0000 7f2eebe07640 1 -- 172.21.15.165:0/3359259474 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2ff8015020 con 0x55c1abbd6f80 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.012+0000 7f2eebe07640 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ee4036950 0x7f2ee4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.012+0000 7f2eebe07640 1 -- 172.21.15.165:0/3359259474 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2ff8020030 con 0x55c1abbd6f80 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.012+0000 7f2ffea44640 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ee4036950 0x7f2ee4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.012+0000 7f3001746800 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c1abbdb320 0x55c1abcd9080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.012+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abcd95c0 con 0x55c1abbdb320 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.012+0000 7f2fffa46640 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c1abbdb320 0x55c1abcd9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.012+0000 7f2ffea44640 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ee4036950 0x7f2ee4038e10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8054000 tx=0x7f2fe804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.013+0000 7f2fffa46640 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c1abbdb320 0x55c1abcd9080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.013+0000 7f2fffa46640 1 -- 172.21.15.165:0/3359259474 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abbdb320 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.013+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abcd6650 con 0x55c1abbdb320 2024-04-27T03:30:30.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.013+0000 7f2fffa46640 1 -- 172.21.15.165:0/3359259474 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abbdb320 2024-04-27T03:30:30.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.013+0000 7f3001746800 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c1abcdb910 0x55c1abcddd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.013+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abcde3f0 con 0x55c1abcdb910 2024-04-27T03:30:30.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.013+0000 7f2fff245640 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c1abcdb910 0x55c1abcddd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.014+0000 7f2fff245640 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c1abcdb910 0x55c1abcddd00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.014+0000 7f2fff245640 1 -- 172.21.15.165:0/3359259474 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abcdb910 2024-04-27T03:30:30.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.014+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abcda8c0 con 0x55c1abbdb320 2024-04-27T03:30:30.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.014+0000 7f2fffa46640 1 -- 172.21.15.165:0/3359259474 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abbdb320 2024-04-27T03:30:30.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.014+0000 7f3001746800 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c1abcdf8a0 0x55c1abce1cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.014+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abce2450 con 0x55c1abcdf8a0 2024-04-27T03:30:30.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.015+0000 7f2ffea44640 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c1abcdf8a0 0x55c1abce1cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.015+0000 7f2ffea44640 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c1abcdf8a0 0x55c1abce1cf0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.015+0000 7f2ffea44640 1 -- 172.21.15.165:0/3359259474 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2fe8052a40 con 0x55c1abcdf8a0 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.015+0000 7f3001746800 1 Processor -- start 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.015+0000 7f3001746800 1 -- start start 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.016+0000 7f3001746800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcf0870 0x55c1abcf0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.016+0000 7f3001746800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c1abac86b0 con 0x55c1abcf0870 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.016+0000 7f2fff245640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcf0870 0x55c1abcf0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.016+0000 7f2fff245640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcf0870 0x55c1abcf0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51394/0 (socket says 172.21.15.165:51394) 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.016+0000 7f2fff245640 1 -- 172.21.15.165:0/1931125944 learned_addr learned my addr 172.21.15.165:0/1931125944 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.016+0000 7f2fff245640 1 -- 172.21.15.165:0/1931125944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1abcf11d0 con 0x55c1abcf0870 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.016+0000 7f2fff245640 1 --2- 172.21.15.165:0/1931125944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcf0870 0x55c1abcf0c90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2ff801a6f0 tx=0x7f2ff800db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.016+0000 7f2ee9602640 1 -- 172.21.15.165:0/1931125944 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ff801ad80 con 0x55c1abcf0870 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.016+0000 7f3001746800 1 -- 172.21.15.165:0/1931125944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c1abcf1c40 con 0x55c1abcf0870 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.016+0000 7f2ee9602640 1 -- 172.21.15.165:0/1931125944 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8044960 con 0x55c1abcf0870 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.017+0000 7f2ee9602640 1 -- 172.21.15.165:0/1931125944 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ff8044c40 con 0x55c1abcf0870 2024-04-27T03:30:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.017+0000 7f3001746800 1 -- 172.21.15.165:0/1931125944 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1abcf2140 con 0x55c1abcf0870 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.017+0000 7f2ee9602640 1 -- 172.21.15.165:0/1931125944 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2ff803d020 con 0x55c1abcf0870 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.017+0000 7f2ee9602640 1 --2- 172.21.15.165:0/1931125944 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ed8036900 0x7f2ed8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.017+0000 7f2ee9602640 1 -- 172.21.15.165:0/1931125944 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2ff803b020 con 0x55c1abcf0870 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.017+0000 7f2ffea44640 1 --2- 172.21.15.165:0/1931125944 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ed8036900 0x7f2ed8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.018+0000 7f3001746800 1 Processor -- start 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.018+0000 7f3001746800 1 -- start start 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.018+0000 7f2ffea44640 1 --2- 172.21.15.165:0/1931125944 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ed8036900 0x7f2ed8038dc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55c1abcd9ec0 tx=0x7f2fe8005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.018+0000 7f3001746800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 0x55c1abcfde80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.018+0000 7f3001746800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c1abb4b2d0 con 0x55c1abcfda60 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.018+0000 7f2fff245640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 0x55c1abcfde80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.018+0000 7f2fff245640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 0x55c1abcfde80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51406/0 (socket says 172.21.15.165:51406) 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.018+0000 7f2fff245640 1 -- 172.21.15.165:0/3348473480 learned_addr learned my addr 172.21.15.165:0/3348473480 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.018+0000 7f2fff245640 1 -- 172.21.15.165:0/3348473480 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1abcfe450 con 0x55c1abcfda60 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.018+0000 7f2fff245640 1 --2- 172.21.15.165:0/3348473480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 0x55c1abcfde80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2ff801b140 tx=0x7f2ff8036bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a48abdf98f544c97 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.019+0000 7f2ee2ffd640 1 -- 172.21.15.165:0/3348473480 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ff805f020 con 0x55c1abcfda60 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.019+0000 7f2ee2ffd640 1 -- 172.21.15.165:0/3348473480 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff80353c0 con 0x55c1abcfda60 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.019+0000 7f2ee2ffd640 1 -- 172.21.15.165:0/3348473480 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ff802d810 con 0x55c1abcfda60 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.019+0000 7f3001746800 1 -- 172.21.15.165:0/3348473480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 msgr2=0x55c1abcfde80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.019+0000 7f3001746800 1 --2- 172.21.15.165:0/3348473480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 0x55c1abcfde80 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f2ff801b140 tx=0x7f2ff8036bb0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.019+0000 7f3001746800 1 -- 172.21.15.165:0/3348473480 shutdown_connections 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.019+0000 7f3001746800 1 --2- 172.21.15.165:0/3348473480 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 0x55c1abcfde80 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.019+0000 7f3001746800 1 -- 172.21.15.165:0/3348473480 >> 172.21.15.165:0/3348473480 conn(0x55c1abcf9d30 msgr2=0x55c1abcf7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.019+0000 7f3001746800 1 -- 172.21.15.165:0/3348473480 shutdown_connections 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.019+0000 7f3001746800 1 -- 172.21.15.165:0/3348473480 wait complete. 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.019+0000 7f3001746800 1 Processor -- start 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.019+0000 7f3001746800 1 -- start start 2024-04-27T03:30:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.020+0000 7f3001746800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 0x55c1abd04d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.020+0000 7f3001746800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c1abcfeb20 con 0x55c1abcfda60 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.020+0000 7f2fff245640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 0x55c1abd04d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.020+0000 7f2fff245640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 0x55c1abd04d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51412/0 (socket says 172.21.15.165:51412) 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.020+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 learned_addr learned my addr 172.21.15.165:0/2589250038 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.020+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1abd05280 con 0x55c1abcfda60 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.020+0000 7f2fff245640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 0x55c1abd04d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80499a0 tx=0x7f2ff801b0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.020+0000 7f2ee17fa640 1 -- 172.21.15.165:0/2589250038 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ff8074070 con 0x55c1abcfda60 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.020+0000 7f2ee17fa640 1 -- 172.21.15.165:0/2589250038 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2ff8044200 con 0x55c1abcfda60 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.020+0000 7f2ee17fa640 1 -- 172.21.15.165:0/2589250038 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2ff8024b30 con 0x55c1abcfda60 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.020+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c1abd05280 con 0x55c1abcfda60 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.021+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1abd05280 con 0x55c1abcfda60 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.021+0000 7f2ee17fa640 1 -- 172.21.15.165:0/2589250038 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2ff8062020 con 0x55c1abcfda60 2024-04-27T03:30:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.021+0000 7f2ee17fa640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ec8036950 0x7f2ec8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.021+0000 7f2ee17fa640 1 -- 172.21.15.165:0/2589250038 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2ff806a030 con 0x55c1abcfda60 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.021+0000 7f2ffea44640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ec8036950 0x7f2ec8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f3001746800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f3001746800 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c1abd10410 0x55c1abd128d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd13020 con 0x55c1abd10410 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f2fffa46640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c1abd10410 0x55c1abd128d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f2ffea44640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ec8036950 0x7f2ec8038e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8052fd0 tx=0x7f2fe804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f2fffa46640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c1abd10410 0x55c1abd128d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f3001746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f3001746800 20 realm 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f3001746800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd13430 con 0x55c1abd10410 2024-04-27T03:30:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f3001746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f3001746800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.022+0000 7f3001746800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.023+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd13d00 con 0x55c1abd10410 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.023+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.023+0000 7f3001746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.023+0000 7f3001746800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.023+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd140a0 con 0x55c1abd10410 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.023+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.023+0000 7f3001746800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.023+0000 7f3001746800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.023+0000 7f3001746800 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c1abd14b90 0x55c1abd16f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.023+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd176e0 con 0x55c1abd14b90 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.023+0000 7f2fff245640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c1abd14b90 0x55c1abd16f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.023+0000 7f2fff245640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c1abd14b90 0x55c1abd16f80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.024+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.024+0000 7f3001746800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.024+0000 7f3001746800 20 searching for the correct realm 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.024+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd190c0 con 0x55c1abd14b90 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.024+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.024+0000 7f2fff245640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2ff8064040 0x7f2ff8035870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.024+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd190c0 con 0x7f2ff8064040 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.024+0000 7f2ffea44640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2ff8064040 0x7f2ff8035870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.025+0000 7f2ffea44640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2ff8064040 0x7f2ff8035870 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.025+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.025+0000 7f2ffea44640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2fe8076040 0x7f2fe8083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.025+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd190c0 con 0x7f2fe8076040 2024-04-27T03:30:30.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.025+0000 7f2fffa46640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2fe8076040 0x7f2fe8083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.026+0000 7f2fffa46640 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2fe8076040 0x7f2fe8083410 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.026+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.026+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd140a0 con 0x55c1abd10410 2024-04-27T03:30:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.026+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.026+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd140a0 con 0x55c1abd10410 2024-04-27T03:30:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.026+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.026+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd140a0 con 0x7f2fe8076040 2024-04-27T03:30:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.027+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.027+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd140a0 con 0x7f2fe8076040 2024-04-27T03:30:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.027+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.027+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd140a0 con 0x7f2ff8064040 2024-04-27T03:30:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.027+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.027+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd140a0 con 0x55c1abd14b90 2024-04-27T03:30:30.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.027+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.028+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd140a0 con 0x55c1abd10410 2024-04-27T03:30:30.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.028+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.028+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd140a0 con 0x7f2fe8076040 2024-04-27T03:30:30.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.028+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.028+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd140a0 con 0x55c1abd10410 2024-04-27T03:30:30.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.028+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.028+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd140a0 con 0x7f2ff8064040 2024-04-27T03:30:30.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.029+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.029+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x55c1abd14b90 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.029+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.029+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x55c1abd14b90 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.029+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.029+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x7f2fe8076040 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.030+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.030+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x55c1abd14b90 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.030+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.030+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x55c1abd14b90 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.030+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.030+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x55c1abd10410 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.030+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.030+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x55c1abd10410 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.031+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.031+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x7f2fe8076040 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.031+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.031+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x55c1abd14b90 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.031+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.031+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x7f2ff8064040 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.032+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.032+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x55c1abd10410 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.032+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.032+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x7f2fe8076040 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.032+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.032+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x55c1abd14b90 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.032+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.032+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x7f2fe8076040 2024-04-27T03:30:30.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.033+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.033+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x7f2ff8064040 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.033+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.033+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x55c1abd14b90 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.033+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x7f2fe8076040 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2fe8090060 con 0x55c1abd10410 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2ff0038040 con 0x7f2ff8064040 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f3001746800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f3001746800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f3001746800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:30.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f3001746800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f3001746800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f3001746800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f3001746800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.034+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd18090 con 0x55c1abd10410 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd18480 con 0x55c1abd10410 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd18480 con 0x7f2fe8076040 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 20 zone default found 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 4 Realm: () 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 20 zonegroup default 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd188a0 con 0x7f2fe8076040 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.035+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd18cc0 con 0x55c1abd14b90 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.036+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.036+0000 7f3001746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.036+0000 7f3001746800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.036+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1af70 con 0x55c1abd14b90 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.036+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.036+0000 7f3001746800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.036+0000 7f3001746800 20 started sync module instance, tier type = 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.036+0000 7f3001746800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.036+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1a410 con 0x7f2ff8064040 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.037+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.037+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94290299693104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1b7c0 con 0x7f2ff8064040 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.037+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1c010 con 0x55c1abd10410 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.038+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94290299693104] v25'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.038+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.038+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94290299695888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1c010 con 0x55c1abd10410 2024-04-27T03:30:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.038+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1cac0 con 0x55c1abd14b90 2024-04-27T03:30:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.041+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.041+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94290299700640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1d360 con 0x55c1abd14b90 2024-04-27T03:30:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.041+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1de60 con 0x7f2ff8064040 2024-04-27T03:30:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.041+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94290299695888] v25'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.043+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94290299700640] v25'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.043+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.044+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94290299705680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1e250 con 0x7f2ff8064040 2024-04-27T03:30:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.044+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1eae0 con 0x55c1abd10410 2024-04-27T03:30:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.045+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.045+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94290299708912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1f380 con 0x55c1abd10410 2024-04-27T03:30:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.045+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1fe20 con 0x7f2fe8076040 2024-04-27T03:30:30.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.047+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94290299705680] v25'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.047+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.047+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94290299713776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd1fde0 con 0x7f2fe8076040 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.047+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd20830 con 0x7f2fe8076040 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.050+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94290299708912] v25'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.050+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94290299713776] v25'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.050+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.050+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94290299698016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd20380 con 0x7f2fe8076040 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.050+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd20c50 con 0x7f2fe8076040 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.051+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94290299698016] v25'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.051+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'18 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.051+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94290299711184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd20830 con 0x7f2fe8076040 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.051+0000 7f3001746800 20 add_watcher() i=0 2024-04-27T03:30:30.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.051+0000 7f3001746800 20 add_watcher() i=1 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.051+0000 7f3001746800 20 add_watcher() i=2 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.051+0000 7f3001746800 20 add_watcher() i=3 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.051+0000 7f3001746800 20 add_watcher() i=4 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.051+0000 7f3001746800 20 add_watcher() i=5 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.051+0000 7f3001746800 20 add_watcher() i=6 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.052+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94290299711184] v25'19 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.052+0000 7f3001746800 20 add_watcher() i=7 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.052+0000 7f3001746800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.052+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd24070 con 0x55c1abd10410 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.052+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.052+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94290299716944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd21570 con 0x55c1abd10410 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.053+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94290299716944] v25'52 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.054+0000 7f2eb5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.054+0000 7f3001746800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.054+0000 7f3001746800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.054+0000 7f3001746800 5 note: GC not initialized 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.054+0000 7f2eb4fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.054+0000 7f3001746800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.054+0000 7f3001746800 20 Filter name: none 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.054+0000 7f2e2e7fc640 20 reqs_thread_entry: start 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.054+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57940 con 0x55c1abd10410 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.054+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.054+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57940 con 0x7f2ff8064040 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.055+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.055+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57960 con 0x55c1abd10410 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.055+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.055+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57920 con 0x55c1abd14b90 2024-04-27T03:30:30.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.055+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.055+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57ec0 con 0x55c1abd14b90 2024-04-27T03:30:30.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.055+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.055+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57ec0 con 0x55c1abd14b90 2024-04-27T03:30:30.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.055+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.056+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57e70 con 0x55c1abd10410 2024-04-27T03:30:30.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.056+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.056+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57e70 con 0x55c1abd10410 2024-04-27T03:30:30.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.056+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.056+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57de0 con 0x55c1abd14b90 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.056+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.056+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57de0 con 0x55c1abd14b90 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.056+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.056+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57de0 con 0x7f2fe8076040 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.057+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 usage.10 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.057+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57de0 con 0x55c1abd10410 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.057+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.057+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57de0 con 0x7f2ff8064040 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.057+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.057+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57de0 con 0x55c1abd14b90 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.058+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.058+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x7f2fe8076040 2024-04-27T03:30:30.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.058+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.058+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x7f2fe8076040 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.058+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.058+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57e00 con 0x55c1abd14b90 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.058+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.058+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x7f2fe8076040 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.058+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(79 usage.17 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.059+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd10410 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.059+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.059+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd14b90 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.059+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.059+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x7f2fe8076040 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.059+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.059+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x7f2ff8064040 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.060+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.060+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x7f2fe8076040 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.060+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.060+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd10410 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.060+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.060+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd10410 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.060+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.060+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd14b90 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.061+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.061+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd10410 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.061+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.061+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd10410 2024-04-27T03:30:30.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.061+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.061+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd14b90 2024-04-27T03:30:30.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.061+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.061+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd14b90 2024-04-27T03:30:30.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.061+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.061+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd14b90 2024-04-27T03:30:30.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.062+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.062+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x7f2fe8076040 2024-04-27T03:30:30.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.062+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.063+0000 7f2eb5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.063+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94290299716944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd10410 2024-04-27T03:30:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.064+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94290299716944] v25'53 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.064+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94290299693104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x7f2ff8064040 2024-04-27T03:30:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.066+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94290299693104] v25'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.066+0000 7f3001746800 20 remove_watcher() i=0 2024-04-27T03:30:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.066+0000 7f3001746800 2 removed watcher, disabling cache 2024-04-27T03:30:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.066+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94290299695888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd10410 2024-04-27T03:30:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.068+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94290299695888] v25'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.068+0000 7f3001746800 20 remove_watcher() i=1 2024-04-27T03:30:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.068+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94290299700640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd14b90 2024-04-27T03:30:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.069+0000 7f2fff245640 1 -- 172.21.15.165:0/2589250038 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94290299700640] v25'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff8018950 con 0x55c1abd14b90 2024-04-27T03:30:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.069+0000 7f3001746800 20 remove_watcher() i=2 2024-04-27T03:30:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.069+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94290299705680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x7f2ff8064040 2024-04-27T03:30:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.070+0000 7f2ffea44640 1 -- 172.21.15.165:0/2589250038 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94290299705680] v25'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2fe8081070 con 0x7f2ff8064040 2024-04-27T03:30:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.070+0000 7f3001746800 20 remove_watcher() i=3 2024-04-27T03:30:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.070+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94290299708912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x55c1abd10410 2024-04-27T03:30:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.073+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94290299708912] v25'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x55c1abd10410 2024-04-27T03:30:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.073+0000 7f3001746800 20 remove_watcher() i=4 2024-04-27T03:30:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.073+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94290299713776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x7f2fe8076040 2024-04-27T03:30:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.074+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94290299713776] v25'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.074+0000 7f3001746800 20 remove_watcher() i=5 2024-04-27T03:30:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.074+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94290299698016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x7f2fe8076040 2024-04-27T03:30:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.075+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94290299698016] v25'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.076+0000 7f3001746800 20 remove_watcher() i=6 2024-04-27T03:30:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.076+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94290299711184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c1abd57dc0 con 0x7f2fe8076040 2024-04-27T03:30:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94290299711184] v25'20 uv18 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2ff000a4a0 con 0x7f2fe8076040 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 20 remove_watcher() i=7 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2fe8076040 msgr2=0x7f2fe8083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2fe8076040 0x7f2fe8083410 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c1abd10410 msgr2=0x55c1abd128d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c1abd10410 0x55c1abd128d0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2ff8064040 msgr2=0x7f2ff8035870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2ff8064040 0x7f2ff8035870 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c1abd14b90 msgr2=0x55c1abd16f80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c1abd14b90 0x55c1abd16f80 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ec8036950 msgr2=0x7f2ec8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ec8036950 0x7f2ec8038e10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8052fd0 tx=0x7f2fe804f000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 msgr2=0x55c1abd04d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 0x55c1abd04d40 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f2ff80499a0 tx=0x7f2ff801b0a0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f2fffa46640 1 -- 172.21.15.165:0/2589250038 reap_dead start 2024-04-27T03:30:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 shutdown_connections 2024-04-27T03:30:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 --2- 172.21.15.165:0/2589250038 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcfda60 0x55c1abd04d40 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 >> 172.21.15.165:0/2589250038 conn(0x55c1abcf9d30 msgr2=0x55c1abcf7390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 shutdown_connections 2024-04-27T03:30:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/2589250038 wait complete. 2024-04-27T03:30:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/1931125944 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ed8036900 msgr2=0x7f2ed8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 --2- 172.21.15.165:0/1931125944 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ed8036900 0x7f2ed8038dc0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x55c1abcd9ec0 tx=0x7f2fe8005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/1931125944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcf0870 msgr2=0x55c1abcf0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 --2- 172.21.15.165:0/1931125944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcf0870 0x55c1abcf0c90 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2ff801a6f0 tx=0x7f2ff800db20 comp rx=0 tx=0).stop 2024-04-27T03:30:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/1931125944 shutdown_connections 2024-04-27T03:30:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 --2- 172.21.15.165:0/1931125944 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ed8036900 0x7f2ed8038dc0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 --2- 172.21.15.165:0/1931125944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abcf0870 0x55c1abcf0c90 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/1931125944 >> 172.21.15.165:0/1931125944 conn(0x55c1abce4910 msgr2=0x55c1abce4d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/1931125944 shutdown_connections 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.077+0000 7f3001746800 1 -- 172.21.15.165:0/1931125944 wait complete. 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c1abcdf8a0 msgr2=0x55c1abce1cf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c1abcdf8a0 0x55c1abce1cf0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c1abbdb320 msgr2=0x55c1abcd9080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c1abbdb320 0x55c1abcd9080 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c1abcdb910 msgr2=0x55c1abcddd00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c1abcdb910 0x55c1abcddd00 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ee4036950 msgr2=0x7f2ee4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ee4036950 0x7f2ee4038e10 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f2fe8054000 tx=0x7f2fe804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 msgr2=0x55c1abcd1e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 --2- 172.21.15.165:0/3359259474 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c1abbd6f80 0x55c1abcd1e90 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f2ff800b7c0 tx=0x7f2ff8000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f2fffa46640 1 -- 172.21.15.165:0/3359259474 reap_dead start 2024-04-27T03:30:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 shutdown_connections 2024-04-27T03:30:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 >> 172.21.15.165:0/3359259474 conn(0x55c1abbd2520 msgr2=0x55c1abb4b830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 shutdown_connections 2024-04-27T03:30:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.078+0000 7f3001746800 1 -- 172.21.15.165:0/3359259474 wait complete. 2024-04-27T03:30:30.091 INFO:teuthology.orchestra.run.smithi165.stdout:{"entries":[],"summary":[]} 2024-04-27T03:30:30.091 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-04-27T03:30:30.092 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T03:30:30.092 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T03:30:30.092 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T03:30:30.174 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:30.174 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.169+0000 7f8556e2f800 1 Processor -- start 2024-04-27T03:30:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.169+0000 7f8556e2f800 1 -- start start 2024-04-27T03:30:30.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.169+0000 7f8556e2f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d6727512e0 0x55d672759de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.169+0000 7f8556e2f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d67260fd40 con 0x55d67274f7a0 2024-04-27T03:30:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.169+0000 7f8556e2f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55d6725e24d0 con 0x55d6727512e0 2024-04-27T03:30:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.170+0000 7f85499bf640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d6727512e0 0x55d672759de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.170+0000 7f85521c0640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55d67274f7a0 0x55d67274fbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53586/0 (socket says 172.21.15.165:53586) 2024-04-27T03:30:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.170+0000 7f85521c0640 1 -- 172.21.15.165:0/2361642184 learned_addr learned my addr 172.21.15.165:0/2361642184 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.170+0000 7f85499bf640 1 -- 172.21.15.165:0/2361642184 >> v1:172.21.15.89:6789/0 conn(0x55d67274f7a0 legacy=0x55d67274fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.170+0000 7f85499bf640 1 -- 172.21.15.165:0/2361642184 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d67260e590 con 0x55d6727512e0 2024-04-27T03:30:30.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.170+0000 7f85519bf640 1 -- 172.21.15.165:0/2361642184 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1542031788 0 0) 0x55d67260fd40 con 0x55d67274f7a0 2024-04-27T03:30:30.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.172+0000 7f85499bf640 1 --2- 172.21.15.165:0/2361642184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6727512e0 0x55d672759de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f853c00a470 tx=0x7f853c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcaaccaab01e1d45 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.172+0000 7f85519bf640 1 -- 172.21.15.165:0/2361642184 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f853c057070 con 0x55d6727512e0 2024-04-27T03:30:30.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.172+0000 7f85519bf640 1 -- 172.21.15.165:0/2361642184 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f853c052440 con 0x55d6727512e0 2024-04-27T03:30:30.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.172+0000 7f85519bf640 1 -- 172.21.15.165:0/2361642184 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f853c052740 con 0x55d6727512e0 2024-04-27T03:30:30.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.173+0000 7f8556e2f800 1 -- 172.21.15.165:0/2361642184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6727512e0 msgr2=0x55d672759de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.173+0000 7f8556e2f800 1 --2- 172.21.15.165:0/2361642184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6727512e0 0x55d672759de0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f853c00a470 tx=0x7f853c01ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:30.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.173+0000 7f8556e2f800 1 -- 172.21.15.165:0/2361642184 shutdown_connections 2024-04-27T03:30:30.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.173+0000 7f8556e2f800 1 --2- 172.21.15.165:0/2361642184 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d6727512e0 0x55d672759de0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.173+0000 7f8556e2f800 1 -- 172.21.15.165:0/2361642184 >> 172.21.15.165:0/2361642184 conn(0x55d67274b320 msgr2=0x55d67274d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.173+0000 7f8556e2f800 1 -- 172.21.15.165:0/2361642184 shutdown_connections 2024-04-27T03:30:30.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.173+0000 7f8556e2f800 1 -- 172.21.15.165:0/2361642184 wait complete. 2024-04-27T03:30:30.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.184+0000 7f8556e2f800 1 Processor -- start 2024-04-27T03:30:30.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.184+0000 7f8556e2f800 1 -- start start 2024-04-27T03:30:30.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.185+0000 7f8556e2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 0x55d672755230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.185+0000 7f8556e2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d67269c3c0 con 0x55d67274f7a0 2024-04-27T03:30:30.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.185+0000 7f85521c0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 0x55d672755230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.185+0000 7f85521c0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 0x55d672755230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51426/0 (socket says 172.21.15.165:51426) 2024-04-27T03:30:30.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.185+0000 7f85521c0640 1 -- 172.21.15.165:0/180143779 learned_addr learned my addr 172.21.15.165:0/180143779 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.185+0000 7f85521c0640 1 -- 172.21.15.165:0/180143779 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d672755800 con 0x55d67274f7a0 2024-04-27T03:30:30.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.185+0000 7f85521c0640 1 --2- 172.21.15.165:0/180143779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 0x55d672755230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f854400b210 tx=0x7f854400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c87941fdffd56b8c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.186+0000 7f8440608640 1 -- 172.21.15.165:0/180143779 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8544008070 con 0x55d67274f7a0 2024-04-27T03:30:30.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.186+0000 7f8440608640 1 -- 172.21.15.165:0/180143779 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854400d500 con 0x55d67274f7a0 2024-04-27T03:30:30.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.186+0000 7f8440608640 1 -- 172.21.15.165:0/180143779 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f854400d840 con 0x55d67274f7a0 2024-04-27T03:30:30.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.186+0000 7f8556e2f800 1 -- 172.21.15.165:0/180143779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 msgr2=0x55d672755230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.186+0000 7f8556e2f800 1 --2- 172.21.15.165:0/180143779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 0x55d672755230 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f854400b210 tx=0x7f854400b6d0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.186+0000 7f8556e2f800 1 -- 172.21.15.165:0/180143779 shutdown_connections 2024-04-27T03:30:30.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.186+0000 7f8556e2f800 1 --2- 172.21.15.165:0/180143779 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 0x55d672755230 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.186+0000 7f8556e2f800 1 -- 172.21.15.165:0/180143779 >> 172.21.15.165:0/180143779 conn(0x55d67274b320 msgr2=0x55d67283fc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.186+0000 7f8556e2f800 1 -- 172.21.15.165:0/180143779 shutdown_connections 2024-04-27T03:30:30.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.186+0000 7f8556e2f800 1 -- 172.21.15.165:0/180143779 wait complete. 2024-04-27T03:30:30.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.187+0000 7f8556e2f800 1 Processor -- start 2024-04-27T03:30:30.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.187+0000 7f8556e2f800 1 -- start start 2024-04-27T03:30:30.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.187+0000 7f8556e2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 0x55d6728502d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.187+0000 7f8556e2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d67275c600 con 0x55d67274f7a0 2024-04-27T03:30:30.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.187+0000 7f85521c0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 0x55d6728502d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.187+0000 7f85521c0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 0x55d6728502d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51434/0 (socket says 172.21.15.165:51434) 2024-04-27T03:30:30.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.187+0000 7f85521c0640 1 -- 172.21.15.165:0/751311487 learned_addr learned my addr 172.21.15.165:0/751311487 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.187+0000 7f85521c0640 1 -- 172.21.15.165:0/751311487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d672850810 con 0x55d67274f7a0 2024-04-27T03:30:30.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.187+0000 7f85521c0640 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 0x55d6728502d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f854400b210 tx=0x7f8544000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.188+0000 7f843ee05640 1 -- 172.21.15.165:0/751311487 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f854400f950 con 0x55d67274f7a0 2024-04-27T03:30:30.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.188+0000 7f843ee05640 1 -- 172.21.15.165:0/751311487 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854400faf0 con 0x55d67274f7a0 2024-04-27T03:30:30.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.188+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d672850aa0 con 0x55d67274f7a0 2024-04-27T03:30:30.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.188+0000 7f843ee05640 1 -- 172.21.15.165:0/751311487 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f854400fdd0 con 0x55d67274f7a0 2024-04-27T03:30:30.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.188+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d672853750 con 0x55d67274f7a0 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.188+0000 7f843ee05640 1 -- 172.21.15.165:0/751311487 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f8544017040 con 0x55d67274f7a0 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.189+0000 7f843ee05640 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8438036900 0x7f8438038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.189+0000 7f843ee05640 1 -- 172.21.15.165:0/751311487 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f8544015020 con 0x55d67274f7a0 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.189+0000 7f85499bf640 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8438036900 0x7f8438038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.189+0000 7f8556e2f800 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d6727512e0 0x55d672857510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.189+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d672857a50 con 0x55d6727512e0 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.189+0000 7f85529c1640 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d6727512e0 0x55d672857510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.189+0000 7f85499bf640 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8438036900 0x7f8438038dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f853c054000 tx=0x7f853c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.189+0000 7f85529c1640 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d6727512e0 0x55d672857510 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.190+0000 7f85529c1640 1 -- 172.21.15.165:0/751311487 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d6727512e0 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.190+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d672854ae0 con 0x55d6727512e0 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.190+0000 7f85529c1640 1 -- 172.21.15.165:0/751311487 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d6727512e0 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.190+0000 7f8556e2f800 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d672859d60 0x55d67285c150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.190+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d67285c840 con 0x55d672859d60 2024-04-27T03:30:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.190+0000 7f85521c0640 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d672859d60 0x55d67285c150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.190+0000 7f85521c0640 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d672859d60 0x55d67285c150 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.190+0000 7f85521c0640 1 -- 172.21.15.165:0/751311487 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f854402d070 con 0x55d672859d60 2024-04-27T03:30:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.191+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d672858d10 con 0x55d6727512e0 2024-04-27T03:30:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.191+0000 7f85529c1640 1 -- 172.21.15.165:0/751311487 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d6727512e0 2024-04-27T03:30:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.191+0000 7f8556e2f800 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d67285dcf0 0x55d672860140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.191+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d6728608a0 con 0x55d67285dcf0 2024-04-27T03:30:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.191+0000 7f85499bf640 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d67285dcf0 0x55d672860140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.192+0000 7f85499bf640 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d67285dcf0 0x55d672860140 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.192+0000 7f85499bf640 1 -- 172.21.15.165:0/751311487 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f853c0525f0 con 0x55d67285dcf0 2024-04-27T03:30:30.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.192+0000 7f8556e2f800 1 Processor -- start 2024-04-27T03:30:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.192+0000 7f8556e2f800 1 -- start start 2024-04-27T03:30:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.192+0000 7f8556e2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67286ed00 0x55d67286f120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.192+0000 7f8556e2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d6726426b0 con 0x55d67286ed00 2024-04-27T03:30:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.193+0000 7f85521c0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67286ed00 0x55d67286f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.193+0000 7f85521c0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67286ed00 0x55d67286f120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51448/0 (socket says 172.21.15.165:51448) 2024-04-27T03:30:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.193+0000 7f85521c0640 1 -- 172.21.15.165:0/3653262037 learned_addr learned my addr 172.21.15.165:0/3653262037 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.193+0000 7f85521c0640 1 -- 172.21.15.165:0/3653262037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d67286f660 con 0x55d67286ed00 2024-04-27T03:30:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.193+0000 7f85521c0640 1 --2- 172.21.15.165:0/3653262037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67286ed00 0x55d67286f120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f854404b4b0 tx=0x7f854401aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.193+0000 7f8437fff640 1 -- 172.21.15.165:0/3653262037 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8544019470 con 0x55d67286ed00 2024-04-27T03:30:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.193+0000 7f8437fff640 1 -- 172.21.15.165:0/3653262037 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8544019610 con 0x55d67286ed00 2024-04-27T03:30:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.193+0000 7f8556e2f800 1 -- 172.21.15.165:0/3653262037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d6728700d0 con 0x55d67286ed00 2024-04-27T03:30:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.193+0000 7f8437fff640 1 -- 172.21.15.165:0/3653262037 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f85440198f0 con 0x55d67286ed00 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.193+0000 7f8556e2f800 1 -- 172.21.15.165:0/3653262037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d6728705d0 con 0x55d67286ed00 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.194+0000 7f8437fff640 1 -- 172.21.15.165:0/3653262037 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f854403f1b0 con 0x55d67286ed00 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.194+0000 7f8437fff640 1 --2- 172.21.15.165:0/3653262037 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f842c0368b0 0x7f842c038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.194+0000 7f8437fff640 1 -- 172.21.15.165:0/3653262037 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f854403caa0 con 0x55d67286ed00 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.194+0000 7f85499bf640 1 --2- 172.21.15.165:0/3653262037 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f842c0368b0 0x7f842c038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.194+0000 7f8556e2f800 1 Processor -- start 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.194+0000 7f8556e2f800 1 -- start start 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.194+0000 7f85499bf640 1 --2- 172.21.15.165:0/3653262037 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f842c0368b0 0x7f842c038d70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55d672858310 tx=0x7f853c005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.195+0000 7f8556e2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 0x55d67287c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.195+0000 7f8556e2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d672755b80 con 0x55d67287c020 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.195+0000 7f85521c0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 0x55d67287c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.195+0000 7f85521c0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 0x55d67287c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51452/0 (socket says 172.21.15.165:51452) 2024-04-27T03:30:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.195+0000 7f85521c0640 1 -- 172.21.15.165:0/1090593366 learned_addr learned my addr 172.21.15.165:0/1090593366 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.195+0000 7f85521c0640 1 -- 172.21.15.165:0/1090593366 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d67287ca10 con 0x55d67287c020 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.195+0000 7f85521c0640 1 --2- 172.21.15.165:0/1090593366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 0x55d67287c440 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8544026750 tx=0x7f85440264f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6350b8fc04001af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.195+0000 7f8435ffb640 1 -- 172.21.15.165:0/1090593366 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f854405d070 con 0x55d67287c020 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.195+0000 7f8435ffb640 1 -- 172.21.15.165:0/1090593366 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f854403ce00 con 0x55d67287c020 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.195+0000 7f8435ffb640 1 -- 172.21.15.165:0/1090593366 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8544030d50 con 0x55d67287c020 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.196+0000 7f8556e2f800 1 -- 172.21.15.165:0/1090593366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 msgr2=0x55d67287c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.196+0000 7f8556e2f800 1 --2- 172.21.15.165:0/1090593366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 0x55d67287c440 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f8544026750 tx=0x7f85440264f0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.196+0000 7f8556e2f800 1 -- 172.21.15.165:0/1090593366 shutdown_connections 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.196+0000 7f8556e2f800 1 --2- 172.21.15.165:0/1090593366 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 0x55d67287c440 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.196+0000 7f8556e2f800 1 -- 172.21.15.165:0/1090593366 >> 172.21.15.165:0/1090593366 conn(0x55d672878380 msgr2=0x55d6728763e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.196+0000 7f8556e2f800 1 -- 172.21.15.165:0/1090593366 shutdown_connections 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.196+0000 7f8556e2f800 1 -- 172.21.15.165:0/1090593366 wait complete. 2024-04-27T03:30:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.196+0000 7f8556e2f800 1 Processor -- start 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.196+0000 7f8556e2f800 1 -- start start 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.196+0000 7f8556e2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 0x55d672883450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.196+0000 7f8556e2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d67287d0c0 con 0x55d67287c020 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.197+0000 7f85521c0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 0x55d672883450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.197+0000 7f85521c0640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 0x55d672883450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51460/0 (socket says 172.21.15.165:51460) 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.197+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 learned_addr learned my addr 172.21.15.165:0/2553326179 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.197+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d672883990 con 0x55d67287c020 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.197+0000 7f85521c0640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 0x55d672883450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f854405b040 tx=0x7f8544041f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.197+0000 7f8423fff640 1 -- 172.21.15.165:0/2553326179 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8544067070 con 0x55d67287c020 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.197+0000 7f8423fff640 1 -- 172.21.15.165:0/2553326179 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8544026be0 con 0x55d67287c020 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.197+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d672883990 con 0x55d67287c020 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.197+0000 7f8423fff640 1 -- 172.21.15.165:0/2553326179 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f854405d040 con 0x55d67287c020 2024-04-27T03:30:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.197+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d672883990 con 0x55d67287c020 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.198+0000 7f8423fff640 1 -- 172.21.15.165:0/2553326179 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f854406a070 con 0x55d67287c020 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.198+0000 7f8423fff640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f841c036900 0x7f841c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.198+0000 7f8423fff640 1 -- 172.21.15.165:0/2553326179 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f8544053040 con 0x55d67287c020 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.198+0000 7f85499bf640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f841c036900 0x7f841c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.198+0000 7f8556e2f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.198+0000 7f8556e2f800 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d67288ebf0 0x55d6728910b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.198+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d672891800 con 0x55d67288ebf0 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f85529c1640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d67288ebf0 0x55d6728910b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f85499bf640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f841c036900 0x7f841c038dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f853c000c00 tx=0x7f853c04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f85529c1640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d67288ebf0 0x55d6728910b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f8556e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f8556e2f800 20 realm 2024-04-27T03:30:30.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f8556e2f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d672891c10 con 0x55d67288ebf0 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f8556e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f8556e2f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f8556e2f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d6728924e0 con 0x55d67288ebf0 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.199+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f8556e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f8556e2f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d672892880 con 0x55d67288ebf0 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f8556e2f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f8556e2f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f8556e2f800 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d672893370 0x55d672895760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d672895ec0 con 0x55d672893370 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f85521c0640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d672893370 0x55d672895760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f85521c0640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d672893370 0x55d672895760 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f8556e2f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f8556e2f800 20 searching for the correct realm 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.200+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d6728978a0 con 0x55d672893370 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.201+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.201+0000 7f85521c0640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f854406c070 0x7f8544015980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.201+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d6728978a0 con 0x7f854406c070 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.201+0000 7f85499bf640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f854406c070 0x7f8544015980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.201+0000 7f85499bf640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f854406c070 0x7f8544015980 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.202+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f853c081070 con 0x7f854406c070 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.202+0000 7f85499bf640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f853c076040 0x7f853c083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.202+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d6728978a0 con 0x7f853c076040 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.202+0000 7f85529c1640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f853c076040 0x7f853c083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.202+0000 7f85529c1640 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f853c076040 0x7f853c083410 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.203+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.203+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d672892880 con 0x55d67288ebf0 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.203+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.203+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d672892880 con 0x55d67288ebf0 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.203+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.203+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d672892880 con 0x7f853c076040 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.203+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.203+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d672892880 con 0x7f853c076040 2024-04-27T03:30:30.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.204+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.204+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d672892880 con 0x7f854406c070 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.204+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f853c081070 con 0x7f854406c070 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.204+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d672892880 con 0x55d672893370 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.204+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.204+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d672892880 con 0x55d67288ebf0 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.204+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.204+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d672892880 con 0x7f853c076040 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.205+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.205+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d672892880 con 0x55d67288ebf0 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.205+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.205+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d672892880 con 0x7f854406c070 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.205+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f853c081070 con 0x7f854406c070 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.205+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x55d672893370 2024-04-27T03:30:30.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.205+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.205+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x55d672893370 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.206+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.206+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x7f853c076040 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.206+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.206+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x55d672893370 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.206+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.206+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x55d672893370 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.206+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.206+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x55d67288ebf0 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.206+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.206+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x55d67288ebf0 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.207+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.207+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x7f853c076040 2024-04-27T03:30:30.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.207+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.207+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x55d672893370 2024-04-27T03:30:30.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.207+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.207+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x7f854406c070 2024-04-27T03:30:30.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.207+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f853c081070 con 0x7f854406c070 2024-04-27T03:30:30.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.207+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x55d67288ebf0 2024-04-27T03:30:30.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.208+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.208+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x7f853c076040 2024-04-27T03:30:30.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.208+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.208+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x55d672893370 2024-04-27T03:30:30.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.208+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.208+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x7f853c076040 2024-04-27T03:30:30.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.208+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.208+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x7f854406c070 2024-04-27T03:30:30.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.209+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f853c081070 con 0x7f854406c070 2024-04-27T03:30:30.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.209+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x55d672893370 2024-04-27T03:30:30.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.209+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.209+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x7f853c076040 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.209+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.209+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f853c090060 con 0x55d67288ebf0 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f854c038040 con 0x7f854406c070 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f853c081070 con 0x7f854406c070 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:30.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d672896870 con 0x55d67288ebf0 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d672896c60 con 0x55d67288ebf0 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.210+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d672896c60 con 0x7f853c076040 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 20 zone default found 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 4 Realm: () 2024-04-27T03:30:30.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 20 zonegroup default 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d672897080 con 0x7f853c076040 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d6728974a0 con 0x55d672893370 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.211+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d672899750 con 0x55d672893370 2024-04-27T03:30:30.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.212+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.212+0000 7f8556e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.212+0000 7f8556e2f800 20 started sync module instance, tier type = 2024-04-27T03:30:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.212+0000 7f8556e2f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.212+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d672898bf0 con 0x7f854406c070 2024-04-27T03:30:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.213+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f853c081070 con 0x7f854406c070 2024-04-27T03:30:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.213+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94379532980240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d672898850 con 0x7f854406c070 2024-04-27T03:30:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.213+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289a5c0 con 0x55d67288ebf0 2024-04-27T03:30:30.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.215+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.215+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94379532985072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289ad40 con 0x55d67288ebf0 2024-04-27T03:30:30.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.215+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289b860 con 0x55d672893370 2024-04-27T03:30:30.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.216+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94379532980240] v25'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f853c081070 con 0x7f854406c070 2024-04-27T03:30:30.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.218+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.218+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94379532989232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289bc50 con 0x55d672893370 2024-04-27T03:30:30.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.218+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289c680 con 0x7f854406c070 2024-04-27T03:30:30.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.218+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94379532985072] v25'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.219+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94379532989232] v25'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.222+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f853c081070 con 0x7f854406c070 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.222+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94379532986256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289caa0 con 0x7f854406c070 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.222+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289d350 con 0x55d67288ebf0 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.224+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.224+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94379532996160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289dbf0 con 0x55d67288ebf0 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.225+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289e6d0 con 0x7f853c076040 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.225+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94379532986256] v25'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f853c081070 con 0x7f854406c070 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.228+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94379532996160] v25'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.228+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.228+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94379533001184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289d670 con 0x7f853c076040 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.228+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289ef70 con 0x7f853c076040 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.230+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.230+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94379533003232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289f2b0 con 0x7f853c076040 2024-04-27T03:30:30.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.230+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289fb00 con 0x7f853c076040 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.230+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94379533001184] v25'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.231+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94379533003232] v25'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.231+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'21 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.231+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94379533004288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289f6d0 con 0x7f853c076040 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.231+0000 7f8556e2f800 20 add_watcher() i=0 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.231+0000 7f8556e2f800 20 add_watcher() i=1 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.231+0000 7f8556e2f800 20 add_watcher() i=2 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.231+0000 7f8556e2f800 20 add_watcher() i=3 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.231+0000 7f8556e2f800 20 add_watcher() i=4 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.231+0000 7f8556e2f800 20 add_watcher() i=5 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.231+0000 7f8556e2f800 20 add_watcher() i=6 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.232+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94379533004288] v25'22 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.232+0000 7f8556e2f800 20 add_watcher() i=7 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.232+0000 7f8556e2f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.232+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d6728a2890 con 0x55d67288ebf0 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.233+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.233+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94379533005344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289faf0 con 0x55d67288ebf0 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.233+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94379533005344] v25'54 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.234+0000 7f8408fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.234+0000 7f8556e2f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.234+0000 7f8556e2f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.234+0000 7f8556e2f800 5 note: GC not initialized 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.234+0000 7f83837fe640 20 reqs_thread_entry: start 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.234+0000 7f8556e2f800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.234+0000 7f8556e2f800 20 Filter name: none 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.234+0000 7f83817fa640 20 reqs_thread_entry: start 2024-04-27T03:30:30.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.234+0000 7f8556e2f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.234+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d6728d5770 con 0x55d67288ebf0 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.234+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.235+0000 7f8556e2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:could not fetch user info: no user info saved 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.235+0000 7f8408fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.235+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94379533005344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d6728d5730 con 0x55d67288ebf0 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.236+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94379533005344] v25'55 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.236+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94379532980240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d6728d5730 con 0x7f854406c070 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.237+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94379532980240] v25'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f853c081070 con 0x7f854406c070 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.237+0000 7f8556e2f800 20 remove_watcher() i=0 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.237+0000 7f8556e2f800 2 removed watcher, disabling cache 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.237+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94379532985072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d6728d5730 con 0x55d67288ebf0 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.239+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94379532985072] v25'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.239+0000 7f8556e2f800 20 remove_watcher() i=1 2024-04-27T03:30:30.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.239+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94379532989232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d6728d5730 con 0x55d672893370 2024-04-27T03:30:30.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.240+0000 7f85521c0640 1 -- 172.21.15.165:0/2553326179 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94379532989232] v25'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8544073070 con 0x55d672893370 2024-04-27T03:30:30.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.240+0000 7f8556e2f800 20 remove_watcher() i=2 2024-04-27T03:30:30.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.240+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94379532986256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d672899010 con 0x7f854406c070 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.241+0000 7f85499bf640 1 -- 172.21.15.165:0/2553326179 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94379532986256] v25'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f853c081070 con 0x7f854406c070 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.241+0000 7f8556e2f800 20 remove_watcher() i=3 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.241+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94379532996160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289a2f0 con 0x55d67288ebf0 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.242+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94379532996160] v25'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x55d67288ebf0 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.242+0000 7f8556e2f800 20 remove_watcher() i=4 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.242+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94379533001184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289b330 con 0x7f853c076040 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.243+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94379533001184] v25'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.243+0000 7f8556e2f800 20 remove_watcher() i=5 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.243+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94379533003232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d672899010 con 0x7f853c076040 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.244+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94379533003232] v25'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.244+0000 7f8556e2f800 20 remove_watcher() i=6 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.244+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94379533004288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d67289a2f0 con 0x7f853c076040 2024-04-27T03:30:30.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.245+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94379533004288] v25'23 uv21 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f854c00a4a0 con 0x7f853c076040 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.245+0000 7f8556e2f800 20 remove_watcher() i=7 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.245+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f853c076040 msgr2=0x7f853c083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.245+0000 7f8556e2f800 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f853c076040 0x7f853c083410 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.245+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d67288ebf0 msgr2=0x55d6728910b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.245+0000 7f8556e2f800 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d67288ebf0 0x55d6728910b0 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.245+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f854406c070 msgr2=0x7f8544015980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.245+0000 7f8556e2f800 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f854406c070 0x7f8544015980 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.245+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d672893370 msgr2=0x55d672895760 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.245+0000 7f8556e2f800 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d672893370 0x55d672895760 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f841c036900 msgr2=0x7f841c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f841c036900 0x7f841c038dc0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f853c000c00 tx=0x7f853c04f000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 msgr2=0x55d672883450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 --2- 172.21.15.165:0/2553326179 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67287c020 0x55d672883450 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f854405b040 tx=0x7f8544041f70 comp rx=0 tx=0).stop 2024-04-27T03:30:30.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 reap_dead start 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f85529c1640 1 -- 172.21.15.165:0/2553326179 reap_dead start 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 shutdown_connections 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 >> 172.21.15.165:0/2553326179 conn(0x55d672878380 msgr2=0x55d6728759e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 shutdown_connections 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/2553326179 wait complete. 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/3653262037 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f842c0368b0 msgr2=0x7f842c038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 --2- 172.21.15.165:0/3653262037 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f842c0368b0 0x7f842c038d70 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x55d672858310 tx=0x7f853c005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/3653262037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67286ed00 msgr2=0x55d67286f120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 --2- 172.21.15.165:0/3653262037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67286ed00 0x55d67286f120 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f854404b4b0 tx=0x7f854401aaf0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/3653262037 shutdown_connections 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 --2- 172.21.15.165:0/3653262037 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f842c0368b0 0x7f842c038d70 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 --2- 172.21.15.165:0/3653262037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67286ed00 0x55d67286f120 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/3653262037 >> 172.21.15.165:0/3653262037 conn(0x55d672862da0 msgr2=0x55d6728631a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/3653262037 shutdown_connections 2024-04-27T03:30:30.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/3653262037 wait complete. 2024-04-27T03:30:30.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d67285dcf0 msgr2=0x55d672860140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d67285dcf0 0x55d672860140 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d6727512e0 msgr2=0x55d672857510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d6727512e0 0x55d672857510 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d672859d60 msgr2=0x55d67285c150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d672859d60 0x55d67285c150 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8438036900 msgr2=0x7f8438038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8438036900 0x7f8438038dc0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f853c054000 tx=0x7f853c04e000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.246+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 msgr2=0x55d6728502d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.247+0000 7f8556e2f800 1 --2- 172.21.15.165:0/751311487 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d67274f7a0 0x55d6728502d0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f854400b210 tx=0x7f8544000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.247+0000 7f85529c1640 1 -- 172.21.15.165:0/751311487 reap_dead start 2024-04-27T03:30:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.247+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 shutdown_connections 2024-04-27T03:30:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.247+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 >> 172.21.15.165:0/751311487 conn(0x55d67274b320 msgr2=0x55d6727523b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.247+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 shutdown_connections 2024-04-27T03:30:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.247+0000 7f8556e2f800 1 -- 172.21.15.165:0/751311487 wait complete. 2024-04-27T03:30:30.260 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-27T03:30:30.260 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-04-27T03:30:30.261 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'foo', '--display-name', 'Foo', '--email', 'foo@foo.com', '--access-key', '9te6NH5mcdcq0Tc5i8i1', '--secret', 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', '--max-buckets', '4'] 2024-04-27T03:30:30.261 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid foo --display-name Foo --email foo@foo.com --access-key 9te6NH5mcdcq0Tc5i8i1 --secret Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu --max-buckets 4 2024-04-27T03:30:30.353 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:30.353 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:30.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.348+0000 7ffb63b2e800 1 Processor -- start 2024-04-27T03:30:30.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.348+0000 7ffb63b2e800 1 -- start start 2024-04-27T03:30:30.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.348+0000 7ffb63b2e800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5613b0cf9420 0x5613b0cfb810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.348+0000 7ffb63b2e800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5613b0bb3d40 con 0x5613b0cf5910 2024-04-27T03:30:30.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.348+0000 7ffb63b2e800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5613b0b864d0 con 0x5613b0cf9420 2024-04-27T03:30:30.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.349+0000 7ffb57066640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5613b0cf9420 0x5613b0cfb810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.349+0000 7ffb57066640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5613b0cf9420 0x5613b0cfb810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51476/0 (socket says 172.21.15.165:51476) 2024-04-27T03:30:30.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.349+0000 7ffb5f867640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5613b0cf5910 0x5613b0cf7d00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53600/0 (socket says 172.21.15.165:53600) 2024-04-27T03:30:30.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.349+0000 7ffb57066640 1 -- 172.21.15.165:0/162603732 learned_addr learned my addr 172.21.15.165:0/162603732 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.349+0000 7ffb57066640 1 -- 172.21.15.165:0/162603732 >> v1:172.21.15.89:6789/0 conn(0x5613b0cf5910 legacy=0x5613b0cf7d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.349+0000 7ffb57066640 1 -- 172.21.15.165:0/162603732 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b0bb2590 con 0x5613b0cf9420 2024-04-27T03:30:30.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.351+0000 7ffb57066640 1 --2- 172.21.15.165:0/162603732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf9420 0x5613b0cfb810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800a520 tx=0x7ffb4801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a482950760045ef5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.351+0000 7ffb5f066640 1 -- 172.21.15.165:0/162603732 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb48057070 con 0x5613b0cf9420 2024-04-27T03:30:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.351+0000 7ffb5f066640 1 -- 172.21.15.165:0/162603732 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb48052570 con 0x5613b0cf9420 2024-04-27T03:30:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.351+0000 7ffb5f066640 1 -- 172.21.15.165:0/162603732 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb4804ae60 con 0x5613b0cf9420 2024-04-27T03:30:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.351+0000 7ffb63b2e800 1 -- 172.21.15.165:0/162603732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf9420 msgr2=0x5613b0cfb810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.351+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/162603732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf9420 0x5613b0cfb810 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7ffb4800a520 tx=0x7ffb4801d010 comp rx=0 tx=0).stop 2024-04-27T03:30:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.351+0000 7ffb63b2e800 1 -- 172.21.15.165:0/162603732 shutdown_connections 2024-04-27T03:30:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.351+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/162603732 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf9420 0x5613b0cfb810 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.351+0000 7ffb63b2e800 1 -- 172.21.15.165:0/162603732 >> 172.21.15.165:0/162603732 conn(0x5613b0cef2c0 msgr2=0x5613b0cf16e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.352+0000 7ffb63b2e800 1 -- 172.21.15.165:0/162603732 shutdown_connections 2024-04-27T03:30:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.352+0000 7ffb63b2e800 1 -- 172.21.15.165:0/162603732 wait complete. 2024-04-27T03:30:30.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.363+0000 7ffb63b2e800 1 Processor -- start 2024-04-27T03:30:30.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.363+0000 7ffb63b2e800 1 -- start start 2024-04-27T03:30:30.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.363+0000 7ffb63b2e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 0x5613b0cf5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.363+0000 7ffb63b2e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5613b0c40360 con 0x5613b0cf5910 2024-04-27T03:30:30.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.364+0000 7ffb5f867640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 0x5613b0cf5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.364+0000 7ffb5f867640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 0x5613b0cf5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51490/0 (socket says 172.21.15.165:51490) 2024-04-27T03:30:30.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.364+0000 7ffb5f867640 1 -- 172.21.15.165:0/3858521719 learned_addr learned my addr 172.21.15.165:0/3858521719 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.364+0000 7ffb5f867640 1 -- 172.21.15.165:0/3858521719 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b0cf3810 con 0x5613b0cf5910 2024-04-27T03:30:30.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.364+0000 7ffb5f867640 1 --2- 172.21.15.165:0/3858521719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 0x5613b0cf5250 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffb50001e90 tx=0x7ffb5000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3771c07b3fc729 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.364+0000 7ffa4de0b640 1 -- 172.21.15.165:0/3858521719 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5000e5c0 con 0x5613b0cf5910 2024-04-27T03:30:30.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.364+0000 7ffa4de0b640 1 -- 172.21.15.165:0/3858521719 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb50006070 con 0x5613b0cf5910 2024-04-27T03:30:30.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.364+0000 7ffa4de0b640 1 -- 172.21.15.165:0/3858521719 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5000e890 con 0x5613b0cf5910 2024-04-27T03:30:30.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.365+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3858521719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 msgr2=0x5613b0cf5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.365+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/3858521719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 0x5613b0cf5250 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7ffb50001e90 tx=0x7ffb5000b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.365+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3858521719 shutdown_connections 2024-04-27T03:30:30.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.365+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/3858521719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 0x5613b0cf5250 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.365+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3858521719 >> 172.21.15.165:0/3858521719 conn(0x5613b0cef2c0 msgr2=0x5613b0de1a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.365+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3858521719 shutdown_connections 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.365+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3858521719 wait complete. 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.365+0000 7ffb63b2e800 1 Processor -- start 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.365+0000 7ffb63b2e800 1 -- start start 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.365+0000 7ffb63b2e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 0x5613b0defef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.365+0000 7ffb63b2e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5613b0cfe060 con 0x5613b0cf5910 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.366+0000 7ffb5f867640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 0x5613b0defef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.366+0000 7ffb5f867640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 0x5613b0defef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51500/0 (socket says 172.21.15.165:51500) 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.366+0000 7ffb5f867640 1 -- 172.21.15.165:0/3038071949 learned_addr learned my addr 172.21.15.165:0/3038071949 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.366+0000 7ffb5f867640 1 -- 172.21.15.165:0/3038071949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b0df0430 con 0x5613b0cf5910 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.366+0000 7ffb5f867640 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 0x5613b0defef0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000c010 tx=0x7ffb50000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.366+0000 7ffa47fff640 1 -- 172.21.15.165:0/3038071949 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5000f8d0 con 0x5613b0cf5910 2024-04-27T03:30:30.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.366+0000 7ffa47fff640 1 -- 172.21.15.165:0/3038071949 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5000fa90 con 0x5613b0cf5910 2024-04-27T03:30:30.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.366+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5613b0df06c0 con 0x5613b0cf5910 2024-04-27T03:30:30.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.366+0000 7ffa47fff640 1 -- 172.21.15.165:0/3038071949 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5000fdb0 con 0x5613b0cf5910 2024-04-27T03:30:30.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.366+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613b0df3370 con 0x5613b0cf5910 2024-04-27T03:30:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.367+0000 7ffa47fff640 1 -- 172.21.15.165:0/3038071949 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffb50017070 con 0x5613b0cf5910 2024-04-27T03:30:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.367+0000 7ffa47fff640 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa48036900 0x7ffa48038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.367+0000 7ffa47fff640 1 -- 172.21.15.165:0/3038071949 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ffb50015020 con 0x5613b0cf5910 2024-04-27T03:30:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.367+0000 7ffb57066640 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa48036900 0x7ffa48038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.367+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5613b0cf9420 0x5613b0df7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.367+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0df7700 con 0x5613b0cf9420 2024-04-27T03:30:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.368+0000 7ffb60068640 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5613b0cf9420 0x5613b0df7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.368+0000 7ffb57066640 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa48036900 0x7ffa48038dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffb48054000 tx=0x7ffb4804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.368+0000 7ffb60068640 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5613b0cf9420 0x5613b0df7130 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.368+0000 7ffb60068640 1 -- 172.21.15.165:0/3038071949 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0cf9420 2024-04-27T03:30:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.368+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0df4700 con 0x5613b0cf9420 2024-04-27T03:30:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.368+0000 7ffb60068640 1 -- 172.21.15.165:0/3038071949 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0cf9420 2024-04-27T03:30:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.368+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5613b0df9a10 0x5613b0dfbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.368+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0dfc4a0 con 0x5613b0df9a10 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.369+0000 7ffb5f867640 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5613b0df9a10 0x5613b0dfbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.369+0000 7ffb5f867640 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5613b0df9a10 0x5613b0dfbe00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.369+0000 7ffb5f867640 1 -- 172.21.15.165:0/3038071949 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ffb5002e070 con 0x5613b0df9a10 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.369+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0df89c0 con 0x5613b0cf9420 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.370+0000 7ffb60068640 1 -- 172.21.15.165:0/3038071949 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0cf9420 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.370+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5613b0dfd950 0x5613b0dffda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.370+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e00500 con 0x5613b0dfd950 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.370+0000 7ffb57066640 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5613b0dfd950 0x5613b0dffda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.370+0000 7ffb57066640 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5613b0dfd950 0x5613b0dffda0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.370+0000 7ffb57066640 1 -- 172.21.15.165:0/3038071949 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ffb4805a040 con 0x5613b0dfd950 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.371+0000 7ffb63b2e800 1 Processor -- start 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.371+0000 7ffb63b2e800 1 -- start start 2024-04-27T03:30:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.371+0000 7ffb63b2e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e0e960 0x5613b0e0ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.371+0000 7ffb63b2e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5613b0be66b0 con 0x5613b0e0e960 2024-04-27T03:30:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.371+0000 7ffb5f867640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e0e960 0x5613b0e0ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.371+0000 7ffb5f867640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e0e960 0x5613b0e0ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51516/0 (socket says 172.21.15.165:51516) 2024-04-27T03:30:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.371+0000 7ffb5f867640 1 -- 172.21.15.165:0/2930513977 learned_addr learned my addr 172.21.15.165:0/2930513977 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.371+0000 7ffb5f867640 1 -- 172.21.15.165:0/2930513977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b0e0f2c0 con 0x5613b0e0e960 2024-04-27T03:30:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.372+0000 7ffb5f867640 1 --2- 172.21.15.165:0/2930513977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e0e960 0x5613b0e0ed80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffb5001cfd0 tx=0x7ffb50041d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.372+0000 7ffa457fa640 1 -- 172.21.15.165:0/2930513977 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5001a690 con 0x5613b0e0e960 2024-04-27T03:30:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.372+0000 7ffa457fa640 1 -- 172.21.15.165:0/2930513977 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5001a830 con 0x5613b0e0e960 2024-04-27T03:30:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.372+0000 7ffb63b2e800 1 -- 172.21.15.165:0/2930513977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5613b0e0fd30 con 0x5613b0e0e960 2024-04-27T03:30:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.372+0000 7ffa457fa640 1 -- 172.21.15.165:0/2930513977 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb50044b10 con 0x5613b0e0e960 2024-04-27T03:30:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.372+0000 7ffb63b2e800 1 -- 172.21.15.165:0/2930513977 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613b0e10230 con 0x5613b0e0e960 2024-04-27T03:30:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.372+0000 7ffa457fa640 1 -- 172.21.15.165:0/2930513977 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffb50044cb0 con 0x5613b0e0e960 2024-04-27T03:30:30.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.373+0000 7ffa457fa640 1 --2- 172.21.15.165:0/2930513977 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa38036950 0x7ffa38038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.373+0000 7ffa457fa640 1 -- 172.21.15.165:0/2930513977 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ffb5003cae0 con 0x5613b0e0e960 2024-04-27T03:30:30.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.373+0000 7ffb57066640 1 --2- 172.21.15.165:0/2930513977 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa38036950 0x7ffa38038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.373+0000 7ffb63b2e800 1 Processor -- start 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.373+0000 7ffb63b2e800 1 -- start start 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.373+0000 7ffb57066640 1 --2- 172.21.15.165:0/2930513977 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa38036950 0x7ffa38038e10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x5613b0df4620 tx=0x7ffb48005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.373+0000 7ffb63b2e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 0x5613b0e1bf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.373+0000 7ffb63b2e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5613b0cf3b90 con 0x5613b0e1bb10 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.373+0000 7ffb5f867640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 0x5613b0e1bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.373+0000 7ffb5f867640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 0x5613b0e1bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51532/0 (socket says 172.21.15.165:51532) 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.373+0000 7ffb5f867640 1 -- 172.21.15.165:0/3267003879 learned_addr learned my addr 172.21.15.165:0/3267003879 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.374+0000 7ffb5f867640 1 -- 172.21.15.165:0/3267003879 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b0e1c500 con 0x5613b0e1bb10 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.374+0000 7ffb5f867640 1 --2- 172.21.15.165:0/3267003879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 0x5613b0e1bf30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb5004b7a0 tx=0x7ffb50019f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b51fac6191c9d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.374+0000 7ffa3effd640 1 -- 172.21.15.165:0/3267003879 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5003cd80 con 0x5613b0e1bb10 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.374+0000 7ffa3effd640 1 -- 172.21.15.165:0/3267003879 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb5005b030 con 0x5613b0e1bb10 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.374+0000 7ffa3effd640 1 -- 172.21.15.165:0/3267003879 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb50044fa0 con 0x5613b0e1bb10 2024-04-27T03:30:30.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.374+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3267003879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 msgr2=0x5613b0e1bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.374+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/3267003879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 0x5613b0e1bf30 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7ffb5004b7a0 tx=0x7ffb50019f80 comp rx=0 tx=0).stop 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.374+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3267003879 shutdown_connections 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.374+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/3267003879 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 0x5613b0e1bf30 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.374+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3267003879 >> 172.21.15.165:0/3267003879 conn(0x5613b0e17de0 msgr2=0x5613b0e15e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.374+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3267003879 shutdown_connections 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.374+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3267003879 wait complete. 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.375+0000 7ffb63b2e800 1 Processor -- start 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.375+0000 7ffb63b2e800 1 -- start start 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.375+0000 7ffb63b2e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 0x5613b0e22ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.375+0000 7ffb63b2e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5613b0e1cb60 con 0x5613b0e1bb10 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.375+0000 7ffb5f867640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 0x5613b0e22ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.375+0000 7ffb5f867640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 0x5613b0e22ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51546/0 (socket says 172.21.15.165:51546) 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.375+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 learned_addr learned my addr 172.21.15.165:0/580118052 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.375+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5613b0e233e0 con 0x5613b0e1bb10 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.375+0000 7ffb5f867640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 0x5613b0e22ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb50026bb0 tx=0x7ffb50045f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.376+0000 7ffa3d7fa640 1 -- 172.21.15.165:0/580118052 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb50070070 con 0x5613b0e1bb10 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.376+0000 7ffa3d7fa640 1 -- 172.21.15.165:0/580118052 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffb50044fa0 con 0x5613b0e1bb10 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.376+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5613b0e233e0 con 0x5613b0e1bb10 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.376+0000 7ffa3d7fa640 1 -- 172.21.15.165:0/580118052 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffb5004b850 con 0x5613b0e1bb10 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.376+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5613b0e233e0 con 0x5613b0e1bb10 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.377+0000 7ffa3d7fa640 1 -- 172.21.15.165:0/580118052 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffb50073070 con 0x5613b0e1bb10 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.377+0000 7ffa3d7fa640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa28036950 0x7ffa28038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.377+0000 7ffa3d7fa640 1 -- 172.21.15.165:0/580118052 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ffb50076af0 con 0x5613b0e1bb10 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.377+0000 7ffb57066640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa28036950 0x7ffa28038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.377+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.377+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5613b0e2e490 0x5613b0e30950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.377+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e310a0 con 0x5613b0e2e490 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.377+0000 7ffb57066640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa28036950 0x7ffa28038e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffb48000c00 tx=0x7ffb4804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.377+0000 7ffb60068640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5613b0e2e490 0x5613b0e30950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb60068640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5613b0e2e490 0x5613b0e30950 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0e2e490 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb63b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb63b2e800 20 realm 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e314b0 con 0x5613b0e2e490 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0e2e490 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb63b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb63b2e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e31d80 con 0x5613b0e2e490 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0e2e490 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb63b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.378+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e32120 con 0x5613b0e2e490 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.379+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0e2e490 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.379+0000 7ffb63b2e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.379+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.379+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5613b0e32e80 0x5613b0e35270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.379+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e35880 con 0x5613b0e32e80 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.379+0000 7ffb5f867640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5613b0e32e80 0x5613b0e35270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.379+0000 7ffb5f867640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5613b0e32e80 0x5613b0e35270 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.379+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.379+0000 7ffb63b2e800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.379+0000 7ffb63b2e800 20 searching for the correct realm 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.379+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e371b0 con 0x5613b0e32e80 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.380+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.380+0000 7ffb5f867640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffb50075060 0x7ffb50005860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.380+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e371b0 con 0x7ffb50075060 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.380+0000 7ffb57066640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffb50075060 0x7ffb50005860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.380+0000 7ffb57066640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffb50075060 0x7ffb50005860 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.380+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.380+0000 7ffb57066640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffb48074040 0x7ffb48081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.380+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e371b0 con 0x7ffb48074040 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.381+0000 7ffb60068640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffb48074040 0x7ffb48081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.381+0000 7ffb60068640 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffb48074040 0x7ffb48081410 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.381+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ffb5800a4a0 con 0x7ffb48074040 2024-04-27T03:30:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.381+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e32120 con 0x5613b0e2e490 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.382+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0e2e490 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.382+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e32120 con 0x5613b0e2e490 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.382+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0e2e490 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.382+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e32120 con 0x7ffb48074040 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.382+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5800a4a0 con 0x7ffb48074040 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.382+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e32120 con 0x7ffb48074040 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.382+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5800a4a0 con 0x7ffb48074040 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.382+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e32120 con 0x7ffb50075060 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.383+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.383+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e32120 con 0x5613b0e32e80 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.383+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.383+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e32120 con 0x5613b0e2e490 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.383+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0e2e490 2024-04-27T03:30:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.383+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e32120 con 0x7ffb48074040 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.383+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5800a4a0 con 0x7ffb48074040 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.383+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e32120 con 0x5613b0e2e490 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.384+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0e2e490 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.384+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e32120 con 0x7ffb50075060 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.384+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.384+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x5613b0e32e80 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.384+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.384+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x5613b0e32e80 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.384+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.384+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x7ffb48074040 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.385+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5800a4a0 con 0x7ffb48074040 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.385+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x5613b0e32e80 2024-04-27T03:30:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.385+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.385+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x5613b0e32e80 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.385+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.385+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x5613b0e2e490 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.385+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0e2e490 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.385+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x5613b0e2e490 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.385+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ffb5800a4a0 con 0x5613b0e2e490 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.385+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x7ffb48074040 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.386+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb58033040 con 0x7ffb48074040 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.386+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x5613b0e32e80 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.386+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.386+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x7ffb50075060 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.386+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.386+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x5613b0e2e490 2024-04-27T03:30:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.386+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.386+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x7ffb48074040 2024-04-27T03:30:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.387+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.387+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x5613b0e32e80 2024-04-27T03:30:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.387+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.387+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x7ffb48074040 2024-04-27T03:30:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.387+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.387+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x7ffb50075060 2024-04-27T03:30:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.388+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.388+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x5613b0e32e80 2024-04-27T03:30:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.388+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.388+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x7ffb48074040 2024-04-27T03:30:30.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.388+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.388+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x5613b0e2e490 2024-04-27T03:30:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffb4808e060 con 0x7ffb50075060 2024-04-27T03:30:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e36140 con 0x5613b0e2e490 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e36530 con 0x5613b0e2e490 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.389+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e36530 con 0x7ffb48074040 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb63b2e800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb63b2e800 20 zone default found 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb63b2e800 4 Realm: () 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb63b2e800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:30.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb63b2e800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb63b2e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb63b2e800 20 zonegroup default 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e35da0 con 0x7ffb48074040 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb63b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.390+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e35da0 con 0x5613b0e32e80 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.391+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.391+0000 7ffb63b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.391+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.391+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e390b0 con 0x5613b0e32e80 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.391+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.391+0000 7ffb63b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.391+0000 7ffb63b2e800 20 started sync module instance, tier type = 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.391+0000 7ffb63b2e800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.391+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e384b0 con 0x7ffb50075060 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.392+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.392+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94642572069024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e399c0 con 0x7ffb50075060 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.392+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3a210 con 0x5613b0e2e490 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.393+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94642572069024] v25'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.393+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.393+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94642572072208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3a290 con 0x5613b0e2e490 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.393+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3adb0 con 0x5613b0e32e80 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.394+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94642572072208] v25'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.395+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.395+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94642572077184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3b1a0 con 0x5613b0e32e80 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.395+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3bbe0 con 0x7ffb50075060 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.396+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.396+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94642572077184] v25'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.396+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94642572083152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3c3f0 con 0x7ffb50075060 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.396+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3cdb0 con 0x5613b0e2e490 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.397+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.397+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94642572086016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3d650 con 0x5613b0e2e490 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.397+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3e140 con 0x7ffb48074040 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.397+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94642572083152] v25'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.398+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94642572086016] v25'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.398+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.398+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94642572090384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3d0e0 con 0x7ffb48074040 2024-04-27T03:30:30.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.398+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3e9e0 con 0x7ffb48074040 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.399+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.399+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94642572092720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3d0e0 con 0x7ffb48074040 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.399+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3f500 con 0x7ffb48074040 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.399+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94642572090384] v25'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.400+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94642572092720] v25'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.400+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'24 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.400+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94642572095440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3e950 con 0x7ffb48074040 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.400+0000 7ffb63b2e800 20 add_watcher() i=0 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.400+0000 7ffb63b2e800 20 add_watcher() i=1 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.400+0000 7ffb63b2e800 20 add_watcher() i=2 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.400+0000 7ffb63b2e800 20 add_watcher() i=3 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.400+0000 7ffb63b2e800 20 add_watcher() i=4 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.400+0000 7ffb63b2e800 20 add_watcher() i=5 2024-04-27T03:30:30.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.400+0000 7ffb63b2e800 20 add_watcher() i=6 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.401+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94642572095440] v25'25 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.401+0000 7ffb63b2e800 20 add_watcher() i=7 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.401+0000 7ffb63b2e800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.401+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e424f0 con 0x5613b0e2e490 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.401+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.401+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94642572074336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e3f970 con 0x5613b0e2e490 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.402+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94642572074336] v25'56 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.402+0000 7ffa15fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.402+0000 7ffb63b2e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.402+0000 7ffb63b2e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.402+0000 7ffb63b2e800 5 note: GC not initialized 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.402+0000 7ffa14fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.402+0000 7ffb63b2e800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.402+0000 7ffb63b2e800 20 Filter name: none 2024-04-27T03:30:30.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ff98e7fc640 20 reqs_thread_entry: start 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e75750 con 0x5613b0e2e490 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x0 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e73ef0 con 0x5613b0e2e490 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo@foo.com [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 197+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x0 2024-04-27T03:30:30.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : miss 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:64 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e742e0 con 0x7ffb48074040 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(64 9te6NH5mcdcq0Tc5i8i1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x0 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 10 adding default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 10 cache get: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 : hit (negative entry) 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.403+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e75230 con 0x5613b0e2e490 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 foo [delete,create,call,writefull 0~391] v25'1 uv1 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffb63b2e800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:30.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffb63b2e800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffb63b2e800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify cookie 94642572314400 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e79290 con 0x5613b0e2e490 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify (1) cookie 94642572086016 notify 107374182400 ret 0) ==== 609+0+0 (crc 0 0 0) 0x5613b0e74d30 con 0x5613b0e2e490 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffa3effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182400 cookie 94642572086016 notifier 4280 bl.length()=567 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffa3effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 notify.4 [notify cookie 94642572314400 out=8b] v0'0 uv24 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffa3effd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffa3effd640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7ffa24003ea0 con 0x5613b0e2e490 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(67 notify.4 [notify-ack] v0'0 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== watch-notify(notify_complete (2) cookie 94642572314400 notify 107374182400 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5613b0e744d0 con 0x5613b0e2e490 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.405+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e79210 con 0x5613b0e2e490 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.406+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(68 foo@foo.com [delete,create,writefull 0~7] v25'1 uv1 ondisk = 0) ==== 239+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.406+0000 7ffb63b2e800 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.406+0000 7ffb63b2e800 10 moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.406+0000 7ffb63b2e800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.406+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [notify cookie 94642572328160 in=181b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e796c0 con 0x7ffb50075060 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.407+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== watch-notify(notify (1) cookie 94642572083152 notify 107374182400 ret 0) ==== 211+0+0 (crc 0 0 0) 0x7ffb48088040 con 0x7ffb50075060 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.407+0000 7ffa3effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182400 cookie 94642572083152 notifier 4280 bl.length()=169 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.407+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(69 notify.3 [notify cookie 94642572328160 out=8b] v0'0 uv24 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.407+0000 7ffa3effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x7 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.407+0000 7ffa3effd640 10 rgw watcher librados: moving default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.407+0000 7ffa3effd640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7ffa24003ea0 con 0x7ffb50075060 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.407+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== watch-notify(notify_complete (2) cookie 94642572328160 notify 107374182400 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5613b0bb2590 con 0x7ffb50075060 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.407+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(70 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.407+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e77180 con 0x7ffb48074040 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.408+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(71 9te6NH5mcdcq0Tc5i8i1 [delete,create,writefull 0~7] v25'1 uv1 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.408+0000 7ffb63b2e800 10 cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-04-27T03:30:30.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.408+0000 7ffb63b2e800 10 moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-04-27T03:30:30.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.408+0000 7ffb63b2e800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-04-27T03:30:30.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.408+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 7.e 7:7759931f:::notify.3:head [notify cookie 94642572324048 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e786e0 con 0x7ffb50075060 2024-04-27T03:30:30.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.408+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== watch-notify(notify (1) cookie 94642572083152 notify 107374182401 ret 0) ==== 219+0+0 (crc 0 0 0) 0x5613b0bb2590 con 0x7ffb50075060 2024-04-27T03:30:30.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.408+0000 7ffa3effd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182401 cookie 94642572083152 notifier 4280 bl.length()=177 2024-04-27T03:30:30.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.408+0000 7ffa3effd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 info.flags=0x7 2024-04-27T03:30:30.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.408+0000 7ffa3effd640 10 rgw watcher librados: moving default.rgw.meta+users.keys+9te6NH5mcdcq0Tc5i8i1 to cache LRU end 2024-04-27T03:30:30.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.408+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(72 notify.3 [notify cookie 94642572324048 out=8b] v0'0 uv24 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.408+0000 7ffa3effd640 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7ffa24003ea0 con 0x7ffb50075060 2024-04-27T03:30:30.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.409+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== watch-notify(notify_complete (2) cookie 94642572324048 notify 107374182401 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7ffb4805d440 con 0x7ffb50075060 2024-04-27T03:30:30.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.409+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(73 notify.3 [notify-ack] v0'0 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.427 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:30.431 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.410+0000 7ffa15fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.410+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:74 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94642572074336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e786e0 con 0x5613b0e2e490 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.411+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(74 data_loggenerations_metadata [watch unwatch cookie 94642572074336] v25'57 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.411+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:75 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94642572069024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e786e0 con 0x7ffb50075060 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.412+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(75 notify.0 [watch unwatch cookie 94642572069024] v25'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.412+0000 7ffb63b2e800 20 remove_watcher() i=0 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.412+0000 7ffb63b2e800 2 removed watcher, disabling cache 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.412+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94642572072208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e786e0 con 0x5613b0e2e490 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.413+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(76 notify.1 [watch unwatch cookie 94642572072208] v25'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.413+0000 7ffb63b2e800 20 remove_watcher() i=1 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.413+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94642572077184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e786e0 con 0x5613b0e32e80 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.414+0000 7ffb5f867640 1 -- 172.21.15.165:0/580118052 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(77 notify.2 [watch unwatch cookie 94642572077184] v25'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb50072030 con 0x5613b0e32e80 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.414+0000 7ffb63b2e800 20 remove_watcher() i=2 2024-04-27T03:30:30.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.414+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94642572083152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e786e0 con 0x7ffb50075060 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.414+0000 7ffb57066640 1 -- 172.21.15.165:0/580118052 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(78 notify.3 [watch unwatch cookie 94642572083152] v25'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb4807f070 con 0x7ffb50075060 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.415+0000 7ffb63b2e800 20 remove_watcher() i=3 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.415+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:79 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94642572086016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e786e0 con 0x5613b0e2e490 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.416+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(79 notify.4 [watch unwatch cookie 94642572086016] v25'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x5613b0e2e490 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.416+0000 7ffb63b2e800 20 remove_watcher() i=4 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.416+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:80 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94642572090384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e786e0 con 0x7ffb48074040 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.416+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(80 notify.5 [watch unwatch cookie 94642572090384] v25'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.416+0000 7ffb63b2e800 20 remove_watcher() i=5 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.416+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:81 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94642572092720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e786e0 con 0x7ffb48074040 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.417+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(81 notify.6 [watch unwatch cookie 94642572092720] v25'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.417+0000 7ffb63b2e800 20 remove_watcher() i=6 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.417+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:82 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94642572095440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5613b0e786e0 con 0x7ffb48074040 2024-04-27T03:30:30.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.418+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(82 notify.7 [watch unwatch cookie 94642572095440] v25'26 uv24 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffb58031040 con 0x7ffb48074040 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.418+0000 7ffb63b2e800 20 remove_watcher() i=7 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.418+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffb48074040 msgr2=0x7ffb48081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.418+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffb48074040 0x7ffb48081410 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.418+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5613b0e2e490 msgr2=0x5613b0e30950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.418+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5613b0e2e490 0x5613b0e30950 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.418+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffb50075060 msgr2=0x7ffb50005860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.418+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffb50075060 0x7ffb50005860 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.418+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5613b0e32e80 msgr2=0x5613b0e35270 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.418+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5613b0e32e80 0x5613b0e35270 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa28036950 msgr2=0x7ffa28038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa28036950 0x7ffa28038e10 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ffb48000c00 tx=0x7ffb4804f000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 msgr2=0x5613b0e22ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb60068640 1 -- 172.21.15.165:0/580118052 reap_dead start 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 0x5613b0e22ea0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffb50026bb0 tx=0x7ffb50045f50 comp rx=0 tx=0).stop 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 shutdown_connections 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/580118052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e1bb10 0x5613b0e22ea0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 >> 172.21.15.165:0/580118052 conn(0x5613b0e17de0 msgr2=0x5613b0e15440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 shutdown_connections 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/580118052 wait complete. 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/2930513977 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa38036950 msgr2=0x7ffa38038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/2930513977 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa38036950 0x7ffa38038e10 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x5613b0df4620 tx=0x7ffb48005ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/2930513977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e0e960 msgr2=0x5613b0e0ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/2930513977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e0e960 0x5613b0e0ed80 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7ffb5001cfd0 tx=0x7ffb50041d70 comp rx=0 tx=0).stop 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/2930513977 shutdown_connections 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/2930513977 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa38036950 0x7ffa38038e10 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/2930513977 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0e0e960 0x5613b0e0ed80 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/2930513977 >> 172.21.15.165:0/2930513977 conn(0x5613b0e02a00 msgr2=0x5613b0e02e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/2930513977 shutdown_connections 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/2930513977 wait complete. 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5613b0dfd950 msgr2=0x5613b0dffda0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5613b0dfd950 0x5613b0dffda0 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5613b0cf9420 msgr2=0x5613b0df7130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5613b0cf9420 0x5613b0df7130 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5613b0df9a10 msgr2=0x5613b0dfbe00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5613b0df9a10 0x5613b0dfbe00 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa48036900 msgr2=0x7ffa48038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ffa48036900 0x7ffa48038dc0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7ffb48054000 tx=0x7ffb4804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 msgr2=0x5613b0defef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.419+0000 7ffb63b2e800 1 --2- 172.21.15.165:0/3038071949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5613b0cf5910 0x5613b0defef0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ffb5000c010 tx=0x7ffb50000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.420+0000 7ffb60068640 1 -- 172.21.15.165:0/3038071949 reap_dead start 2024-04-27T03:30:30.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.420+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 shutdown_connections 2024-04-27T03:30:30.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.420+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 >> 172.21.15.165:0/3038071949 conn(0x5613b0cef2c0 msgr2=0x5613b0cf40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.420+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 shutdown_connections 2024-04-27T03:30:30.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.420+0000 7ffb63b2e800 1 -- 172.21.15.165:0/3038071949 wait complete. 2024-04-27T03:30:30.438 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:30.438 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-04-27T03:30:30.438 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--email', 'foo@foo.com'] 2024-04-27T03:30:30.438 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --email foo@foo.com 2024-04-27T03:30:30.485 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:30.485 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:30.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.480+0000 7ff48a498800 1 Processor -- start 2024-04-27T03:30:30.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.480+0000 7ff48a498800 1 -- start start 2024-04-27T03:30:30.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.480+0000 7ff48a498800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d3eb1784a0 0x55d3eb17a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.480+0000 7ff48a498800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d3eb032d40 con 0x55d3eb174990 2024-04-27T03:30:30.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.480+0000 7ff48a498800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55d3eb0054d0 con 0x55d3eb1784a0 2024-04-27T03:30:30.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.480+0000 7ff487796640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d3eb1784a0 0x55d3eb17a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.480+0000 7ff487796640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d3eb1784a0 0x55d3eb17a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51552/0 (socket says 172.21.15.165:51552) 2024-04-27T03:30:30.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.480+0000 7ff487796640 1 -- 172.21.15.165:0/3167752781 learned_addr learned my addr 172.21.15.165:0/3167752781 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.480+0000 7ff487f97640 1 --1- 172.21.15.165:0/3167752781 >> v1:172.21.15.89:6789/0 conn(0x55d3eb174990 0x55d3eb176d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53608/0 (socket says 172.21.15.165:53608) 2024-04-27T03:30:30.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.481+0000 7ff487796640 1 -- 172.21.15.165:0/3167752781 >> v1:172.21.15.89:6789/0 conn(0x55d3eb174990 legacy=0x55d3eb176d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.481+0000 7ff487796640 1 -- 172.21.15.165:0/3167752781 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3eb031590 con 0x55d3eb1784a0 2024-04-27T03:30:30.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.483+0000 7ff487796640 1 --2- 172.21.15.165:0/3167752781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb1784a0 0x55d3eb17a890 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00a520 tx=0x7ff47c01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7bb01684dc9ab5c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.483+0000 7ff486f95640 1 -- 172.21.15.165:0/3167752781 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff47c057070 con 0x55d3eb1784a0 2024-04-27T03:30:30.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.483+0000 7ff486f95640 1 -- 172.21.15.165:0/3167752781 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47c052570 con 0x55d3eb1784a0 2024-04-27T03:30:30.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.483+0000 7ff486f95640 1 -- 172.21.15.165:0/3167752781 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff47c04ae60 con 0x55d3eb1784a0 2024-04-27T03:30:30.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.484+0000 7ff48a498800 1 -- 172.21.15.165:0/3167752781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb1784a0 msgr2=0x55d3eb17a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.484+0000 7ff48a498800 1 --2- 172.21.15.165:0/3167752781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb1784a0 0x55d3eb17a890 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff47c00a520 tx=0x7ff47c01d010 comp rx=0 tx=0).stop 2024-04-27T03:30:30.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.484+0000 7ff48a498800 1 -- 172.21.15.165:0/3167752781 shutdown_connections 2024-04-27T03:30:30.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.484+0000 7ff48a498800 1 --2- 172.21.15.165:0/3167752781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb1784a0 0x55d3eb17a890 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.484+0000 7ff48a498800 1 -- 172.21.15.165:0/3167752781 >> 172.21.15.165:0/3167752781 conn(0x55d3eb16e340 msgr2=0x55d3eb170760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.484+0000 7ff48a498800 1 -- 172.21.15.165:0/3167752781 shutdown_connections 2024-04-27T03:30:30.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.484+0000 7ff48a498800 1 -- 172.21.15.165:0/3167752781 wait complete. 2024-04-27T03:30:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.495+0000 7ff48a498800 1 Processor -- start 2024-04-27T03:30:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.495+0000 7ff48a498800 1 -- start start 2024-04-27T03:30:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.495+0000 7ff48a498800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 0x55d3eb1742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.496+0000 7ff48a498800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d3eb0bf3e0 con 0x55d3eb174990 2024-04-27T03:30:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.496+0000 7ff487f97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 0x55d3eb1742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.496+0000 7ff487f97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 0x55d3eb1742d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51568/0 (socket says 172.21.15.165:51568) 2024-04-27T03:30:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.496+0000 7ff487f97640 1 -- 172.21.15.165:0/2599543751 learned_addr learned my addr 172.21.15.165:0/2599543751 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.496+0000 7ff487f97640 1 -- 172.21.15.165:0/2599543751 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3eb172890 con 0x55d3eb174990 2024-04-27T03:30:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.496+0000 7ff487f97640 1 --2- 172.21.15.165:0/2599543751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 0x55d3eb1742d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff478001e10 tx=0x7ff47800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7694bdfee43a2a97 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.496+0000 7ff375e0b640 1 -- 172.21.15.165:0/2599543751 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff47800e5c0 con 0x55d3eb174990 2024-04-27T03:30:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.496+0000 7ff375e0b640 1 -- 172.21.15.165:0/2599543751 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff478006070 con 0x55d3eb174990 2024-04-27T03:30:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.496+0000 7ff375e0b640 1 -- 172.21.15.165:0/2599543751 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff47800e930 con 0x55d3eb174990 2024-04-27T03:30:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.497+0000 7ff48a498800 1 -- 172.21.15.165:0/2599543751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 msgr2=0x55d3eb1742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.497+0000 7ff48a498800 1 --2- 172.21.15.165:0/2599543751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 0x55d3eb1742d0 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7ff478001e10 tx=0x7ff47800b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.497+0000 7ff48a498800 1 -- 172.21.15.165:0/2599543751 shutdown_connections 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.497+0000 7ff48a498800 1 --2- 172.21.15.165:0/2599543751 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 0x55d3eb1742d0 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.497+0000 7ff48a498800 1 -- 172.21.15.165:0/2599543751 >> 172.21.15.165:0/2599543751 conn(0x55d3eb16e340 msgr2=0x55d3eb2609e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.497+0000 7ff48a498800 1 -- 172.21.15.165:0/2599543751 shutdown_connections 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.497+0000 7ff48a498800 1 -- 172.21.15.165:0/2599543751 wait complete. 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.497+0000 7ff48a498800 1 Processor -- start 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.497+0000 7ff48a498800 1 -- start start 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.498+0000 7ff48a498800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 0x55d3eb26ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.498+0000 7ff48a498800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d3eb17d0e0 con 0x55d3eb174990 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.498+0000 7ff487f97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 0x55d3eb26ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.498+0000 7ff487f97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 0x55d3eb26ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51576/0 (socket says 172.21.15.165:51576) 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.498+0000 7ff487f97640 1 -- 172.21.15.165:0/2101070109 learned_addr learned my addr 172.21.15.165:0/2101070109 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.498+0000 7ff487f97640 1 -- 172.21.15.165:0/2101070109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3eb26f390 con 0x55d3eb174990 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.498+0000 7ff487f97640 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 0x55d3eb26ee50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff47800b700 tx=0x7ff478000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.498+0000 7ff36ffff640 1 -- 172.21.15.165:0/2101070109 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff47800f8d0 con 0x55d3eb174990 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.498+0000 7ff36ffff640 1 -- 172.21.15.165:0/2101070109 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47800fa70 con 0x55d3eb174990 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.499+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d3eb26f620 con 0x55d3eb174990 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.499+0000 7ff36ffff640 1 -- 172.21.15.165:0/2101070109 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff47800fd50 con 0x55d3eb174990 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.499+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3eb2722d0 con 0x55d3eb174990 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.499+0000 7ff36ffff640 1 -- 172.21.15.165:0/2101070109 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ff478017070 con 0x55d3eb174990 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.499+0000 7ff36ffff640 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff370036900 0x7ff370038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.499+0000 7ff36ffff640 1 -- 172.21.15.165:0/2101070109 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ff478015020 con 0x55d3eb174990 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.500+0000 7ff487796640 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff370036900 0x7ff370038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.500+0000 7ff48a498800 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d3eb1784a0 0x55d3eb276090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.500+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb276660 con 0x55d3eb1784a0 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.500+0000 7ff488798640 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d3eb1784a0 0x55d3eb276090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.500+0000 7ff487796640 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff370036900 0x7ff370038dc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff47c054000 tx=0x7ff47c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.500+0000 7ff488798640 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d3eb1784a0 0x55d3eb276090 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.500+0000 7ff488798640 1 -- 172.21.15.165:0/2101070109 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb1784a0 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.500+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb273660 con 0x55d3eb1784a0 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.501+0000 7ff488798640 1 -- 172.21.15.165:0/2101070109 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb1784a0 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.501+0000 7ff48a498800 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d3eb278970 0x55d3eb27ad60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.501+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb27b400 con 0x55d3eb278970 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.501+0000 7ff487f97640 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d3eb278970 0x55d3eb27ad60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.501+0000 7ff487f97640 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d3eb278970 0x55d3eb27ad60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.501+0000 7ff487f97640 1 -- 172.21.15.165:0/2101070109 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff478046040 con 0x55d3eb278970 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.501+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb277920 con 0x55d3eb1784a0 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.501+0000 7ff488798640 1 -- 172.21.15.165:0/2101070109 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb1784a0 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.502+0000 7ff48a498800 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d3eb27c8b0 0x55d3eb27ed00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.502+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb27f460 con 0x55d3eb27c8b0 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.502+0000 7ff487796640 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d3eb27c8b0 0x55d3eb27ed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.502+0000 7ff487796640 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d3eb27c8b0 0x55d3eb27ed00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.502+0000 7ff487796640 1 -- 172.21.15.165:0/2101070109 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff47c05a040 con 0x55d3eb27c8b0 2024-04-27T03:30:30.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.503+0000 7ff48a498800 1 Processor -- start 2024-04-27T03:30:30.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.503+0000 7ff48a498800 1 -- start start 2024-04-27T03:30:30.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.503+0000 7ff48a498800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb28d8c0 0x55d3eb28dce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.503+0000 7ff48a498800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d3eb0656b0 con 0x55d3eb28d8c0 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.503+0000 7ff487f97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb28d8c0 0x55d3eb28dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.503+0000 7ff487f97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb28d8c0 0x55d3eb28dce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51578/0 (socket says 172.21.15.165:51578) 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.503+0000 7ff487f97640 1 -- 172.21.15.165:0/759436208 learned_addr learned my addr 172.21.15.165:0/759436208 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.503+0000 7ff487f97640 1 -- 172.21.15.165:0/759436208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3eb28e220 con 0x55d3eb28d8c0 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.504+0000 7ff487f97640 1 --2- 172.21.15.165:0/759436208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb28d8c0 0x55d3eb28dce0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff47804b4d0 tx=0x7ff478017d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.504+0000 7ff36d7fa640 1 -- 172.21.15.165:0/759436208 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff47801ae10 con 0x55d3eb28d8c0 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.504+0000 7ff36d7fa640 1 -- 172.21.15.165:0/759436208 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47801a690 con 0x55d3eb28d8c0 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.504+0000 7ff36d7fa640 1 -- 172.21.15.165:0/759436208 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff478044b00 con 0x55d3eb28d8c0 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.504+0000 7ff48a498800 1 -- 172.21.15.165:0/759436208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d3eb28ec90 con 0x55d3eb28d8c0 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.504+0000 7ff48a498800 1 -- 172.21.15.165:0/759436208 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3eb28f190 con 0x55d3eb28d8c0 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.504+0000 7ff36d7fa640 1 -- 172.21.15.165:0/759436208 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ff478044ca0 con 0x55d3eb28d8c0 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.505+0000 7ff36d7fa640 1 --2- 172.21.15.165:0/759436208 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff360036950 0x7ff360038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.505+0000 7ff36d7fa640 1 -- 172.21.15.165:0/759436208 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ff47803cae0 con 0x55d3eb28d8c0 2024-04-27T03:30:30.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.505+0000 7ff487796640 1 --2- 172.21.15.165:0/759436208 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff360036950 0x7ff360038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.505+0000 7ff48a498800 1 Processor -- start 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.505+0000 7ff48a498800 1 -- start start 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.505+0000 7ff487796640 1 --2- 172.21.15.165:0/759436208 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff360036950 0x7ff360038e10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55d3eb273580 tx=0x7ff47c005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.505+0000 7ff48a498800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 0x55d3eb29ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.505+0000 7ff48a498800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d3eb172c10 con 0x55d3eb29aa20 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.506+0000 7ff487f97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 0x55d3eb29ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.506+0000 7ff487f97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 0x55d3eb29ae40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51582/0 (socket says 172.21.15.165:51582) 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.506+0000 7ff487f97640 1 -- 172.21.15.165:0/2311467756 learned_addr learned my addr 172.21.15.165:0/2311467756 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.506+0000 7ff487f97640 1 -- 172.21.15.165:0/2311467756 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3eb29b410 con 0x55d3eb29aa20 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.506+0000 7ff487f97640 1 --2- 172.21.15.165:0/2311467756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 0x55d3eb29ae40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff47801afd0 tx=0x7ff478026ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=253d3a74189ceb07 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.506+0000 7ff366ffd640 1 -- 172.21.15.165:0/2311467756 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff478060070 con 0x55d3eb29aa20 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.506+0000 7ff366ffd640 1 -- 172.21.15.165:0/2311467756 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47803cdc0 con 0x55d3eb29aa20 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.506+0000 7ff366ffd640 1 -- 172.21.15.165:0/2311467756 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff478055040 con 0x55d3eb29aa20 2024-04-27T03:30:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff48a498800 1 -- 172.21.15.165:0/2311467756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 msgr2=0x55d3eb29ae40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff48a498800 1 --2- 172.21.15.165:0/2311467756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 0x55d3eb29ae40 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7ff47801afd0 tx=0x7ff478026ae0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff48a498800 1 -- 172.21.15.165:0/2311467756 shutdown_connections 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff48a498800 1 --2- 172.21.15.165:0/2311467756 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 0x55d3eb29ae40 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff48a498800 1 -- 172.21.15.165:0/2311467756 >> 172.21.15.165:0/2311467756 conn(0x55d3eb296d40 msgr2=0x55d3eb294da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff48a498800 1 -- 172.21.15.165:0/2311467756 shutdown_connections 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff48a498800 1 -- 172.21.15.165:0/2311467756 wait complete. 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff48a498800 1 Processor -- start 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff48a498800 1 -- start start 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff48a498800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 0x55d3eb2a1e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff48a498800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d3eb29bb30 con 0x55d3eb29aa20 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff487f97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 0x55d3eb2a1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff487f97640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 0x55d3eb2a1e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51584/0 (socket says 172.21.15.165:51584) 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.507+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 learned_addr learned my addr 172.21.15.165:0/2109212302 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.508+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3eb2a2350 con 0x55d3eb29aa20 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.508+0000 7ff487f97640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 0x55d3eb2a1e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff478026b20 tx=0x7ff47801a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.508+0000 7ff3657fa640 1 -- 172.21.15.165:0/2109212302 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff478060030 con 0x55d3eb29aa20 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.508+0000 7ff3657fa640 1 -- 172.21.15.165:0/2109212302 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff47806c070 con 0x55d3eb29aa20 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.508+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d3eb2a2350 con 0x55d3eb29aa20 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.508+0000 7ff3657fa640 1 -- 172.21.15.165:0/2109212302 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff47804bb60 con 0x55d3eb29aa20 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.508+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3eb2a2350 con 0x55d3eb29aa20 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.509+0000 7ff3657fa640 1 -- 172.21.15.165:0/2109212302 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ff478026390 con 0x55d3eb29aa20 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.509+0000 7ff3657fa640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff350036950 0x7ff350038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.509+0000 7ff3657fa640 1 -- 172.21.15.165:0/2109212302 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ff478061070 con 0x55d3eb29aa20 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.509+0000 7ff487796640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff350036950 0x7ff350038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.509+0000 7ff487796640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff350036950 0x7ff350038e10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff47c055040 tx=0x7ff47c04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff48a498800 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d3eb2ad2b0 0x55d3eb2af770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2afec0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff488798640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d3eb2ad2b0 0x55d3eb2af770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff488798640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d3eb2ad2b0 0x55d3eb2af770 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff48a498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff48a498800 20 realm 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b02d0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff48a498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff48a498800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.510+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b0ba0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff48a498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b0f40 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff48a498800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff48a498800 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d3eb2b1ca0 0x55d3eb2b4090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b46a0 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff487f97640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d3eb2b1ca0 0x55d3eb2b4090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff487f97640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d3eb2b1ca0 0x55d3eb2b4090 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.511+0000 7ff48a498800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.512+0000 7ff48a498800 20 searching for the correct realm 2024-04-27T03:30:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.512+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b5fd0 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.512+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.512+0000 7ff487f97640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ff478071c20 0x7ff4780318d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.512+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b5fd0 con 0x7ff478071c20 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.512+0000 7ff487796640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ff478071c20 0x7ff4780318d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.512+0000 7ff487796640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ff478071c20 0x7ff4780318d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.513+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff47c07f070 con 0x7ff478071c20 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.513+0000 7ff487796640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ff47c074040 0x7ff47c081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.513+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b5fd0 con 0x7ff47c074040 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.513+0000 7ff488798640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ff47c074040 0x7ff47c081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.513+0000 7ff488798640 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ff47c074040 0x7ff47c081410 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.514+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff48000a4a0 con 0x7ff47c074040 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.514+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b0f40 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.514+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.514+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b0f40 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.514+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.514+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b0f40 con 0x7ff47c074040 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.514+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff48000a4a0 con 0x7ff47c074040 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.514+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b0f40 con 0x7ff47c074040 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.515+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff48000a4a0 con 0x7ff47c074040 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.515+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b0f40 con 0x7ff478071c20 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.515+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff47c07f070 con 0x7ff478071c20 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.515+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b0f40 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.515+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.515+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b0f40 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.515+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.515+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b0f40 con 0x7ff47c074040 2024-04-27T03:30:30.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.516+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff48000a4a0 con 0x7ff47c074040 2024-04-27T03:30:30.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.516+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b0f40 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.516+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.516+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b0f40 con 0x7ff478071c20 2024-04-27T03:30:30.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.516+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff47c07f070 con 0x7ff478071c20 2024-04-27T03:30:30.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.516+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.516+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.516+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.517+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.517+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x7ff47c074040 2024-04-27T03:30:30.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.517+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff48000a4a0 con 0x7ff47c074040 2024-04-27T03:30:30.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.517+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.517+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.517+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.518+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.518+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.518+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.518+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.518+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff48000a4a0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.518+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x7ff47c074040 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.518+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff480033040 con 0x7ff47c074040 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.518+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.519+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.519+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x7ff478071c20 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.519+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff47c07f070 con 0x7ff478071c20 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.519+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.519+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.519+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x7ff47c074040 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.519+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.519+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.520+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.520+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x7ff47c074040 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.520+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.520+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x7ff478071c20 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.520+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff47c07f070 con 0x7ff478071c20 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.520+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.521+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.521+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x7ff47c074040 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.521+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.521+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.521+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.521+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ff47c08e060 con 0x7ff478071c20 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff47c07f070 con 0x7ff478071c20 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:30.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b4f60 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b5350 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.522+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b5350 con 0x7ff47c074040 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 20 zone default found 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 4 Realm: () 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 20 zonegroup default 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b4bc0 con 0x7ff47c074040 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b4bc0 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b7f30 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 20 started sync module instance, tier type = 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.523+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b72d0 con 0x7ff478071c20 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.524+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff47c07f070 con 0x7ff478071c20 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.524+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94368671954624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b6f30 con 0x7ff478071c20 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.524+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b8e60 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.525+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94368671954624] v25'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff47c07f070 con 0x7ff478071c20 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.526+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.526+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94368671959472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b9280 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.526+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b9c40 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.527+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94368671959472] v25'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.527+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.527+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94368671960528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b91f0 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.527+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2baab0 con 0x7ff478071c20 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.528+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94368671960528] v25'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.528+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff47c07f070 con 0x7ff478071c20 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.528+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94368671965152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bb350 con 0x7ff478071c20 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.528+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bbc00 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.529+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.529+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94368671971056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bc4d0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.529+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bcf80 con 0x7ff47c074040 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.529+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94368671965152] v25'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff47c07f070 con 0x7ff478071c20 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.530+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.530+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94368671971056] v25'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.530+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94368671968928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bcf40 con 0x7ff47c074040 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.530+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bc040 con 0x7ff47c074040 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.531+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94368671968928] v25'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.531+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.531+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94368671976080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bceb0 con 0x7ff47c074040 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.531+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bdfd0 con 0x7ff47c074040 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.532+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'27 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.532+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94368671980320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2be3f0 con 0x7ff47c074040 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.532+0000 7ff48a498800 20 add_watcher() i=0 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.532+0000 7ff48a498800 20 add_watcher() i=1 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.532+0000 7ff48a498800 20 add_watcher() i=2 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.532+0000 7ff48a498800 20 add_watcher() i=3 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.532+0000 7ff48a498800 20 add_watcher() i=4 2024-04-27T03:30:30.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.532+0000 7ff48a498800 20 add_watcher() i=5 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.532+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94368671976080] v25'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.532+0000 7ff48a498800 20 add_watcher() i=6 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.533+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94368671980320] v25'28 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.533+0000 7ff48a498800 20 add_watcher() i=7 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.533+0000 7ff48a498800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.533+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2be3f0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.533+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.533+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94368671981376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2be360 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.534+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94368671981376] v25'58 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.534+0000 7ff33dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.534+0000 7ff48a498800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.534+0000 7ff48a498800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff48a498800 5 note: GC not initialized 2024-04-27T03:30:30.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff33cfe1640 20 reqs_thread_entry: start 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff48a498800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff48a498800 20 Filter name: none 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff2b67fc640 20 reqs_thread_entry: start 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff48a498800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2f4620 con 0x7ff47c074040 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff48a498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff48a498800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff48a498800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff48a498800 10 cache get: name=default.rgw.meta+users.email+foo@foo.com : miss 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.535+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.3 8:c162d5d1:users.email::foo@foo.com:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2f2dc0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(63 foo@foo.com [stat out=16b,read 0~7 out=7b] v0'0 uv1 ondisk = 0) ==== 197+0+23 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff48a498800 20 rados_obj.operate() r=0 bl.length=7 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff48a498800 10 cache put: name=default.rgw.meta+users.email+foo@foo.com info.flags=0x5 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff48a498800 10 adding default.rgw.meta+users.email+foo@foo.com to cache LRU end 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff48a498800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff48a498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2f31b0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff48a498800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff48a498800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff48a498800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff48a498800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff48a498800 10 chain_cache_entry: cache_locator=default.rgw.meta+users.uid+foo 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.536+0000 7ff48a498800 20 get_user_info_from_index found 0 xattrs 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:could not create user: unable to parse parameters, email: foo@foo.com is the email address of an existing user 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.537+0000 7ff33dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.537+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94368671981376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2a94b0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.538+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94368671981376] v25'59 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.538+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94368671954624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bdf40 con 0x7ff478071c20 2024-04-27T03:30:30.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.539+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94368671954624] v25'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff47c07f070 con 0x7ff478071c20 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.539+0000 7ff48a498800 20 remove_watcher() i=0 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.539+0000 7ff48a498800 2 removed watcher, disabling cache 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.539+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94368671959472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bdf40 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.540+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94368671959472] v25'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.540+0000 7ff48a498800 20 remove_watcher() i=1 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.540+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94368671960528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b89b0 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.541+0000 7ff487f97640 1 -- 172.21.15.165:0/2109212302 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94368671960528] v25'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff478058040 con 0x55d3eb2b1ca0 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.541+0000 7ff48a498800 20 remove_watcher() i=2 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.541+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94368671965152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bdf40 con 0x7ff478071c20 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.542+0000 7ff487796640 1 -- 172.21.15.165:0/2109212302 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94368671965152] v25'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff47c07f070 con 0x7ff478071c20 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.542+0000 7ff48a498800 20 remove_watcher() i=3 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.542+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94368671971056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2b9fe0 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.543+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94368671971056] v25'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x55d3eb2ad2b0 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.543+0000 7ff48a498800 20 remove_watcher() i=4 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.543+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94368671968928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bdf40 con 0x7ff47c074040 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.544+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94368671968928] v25'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.544+0000 7ff48a498800 20 remove_watcher() i=5 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.544+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94368671976080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bdf40 con 0x7ff47c074040 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.545+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94368671976080] v25'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.545+0000 7ff48a498800 20 remove_watcher() i=6 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.545+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94368671980320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d3eb2bca90 con 0x7ff47c074040 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94368671980320] v25'29 uv27 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff480031040 con 0x7ff47c074040 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 20 remove_watcher() i=7 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ff47c074040 msgr2=0x7ff47c081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ff47c074040 0x7ff47c081410 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d3eb2ad2b0 msgr2=0x55d3eb2af770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d3eb2ad2b0 0x55d3eb2af770 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ff478071c20 msgr2=0x7ff4780318d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ff478071c20 0x7ff4780318d0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d3eb2b1ca0 msgr2=0x55d3eb2b4090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d3eb2b1ca0 0x55d3eb2b4090 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff350036950 msgr2=0x7ff350038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff350036950 0x7ff350038e10 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff47c055040 tx=0x7ff47c04f000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 msgr2=0x55d3eb2a1e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 --2- 172.21.15.165:0/2109212302 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb29aa20 0x55d3eb2a1e10 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7ff478026b20 tx=0x7ff47801a970 comp rx=0 tx=0).stop 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 reap_dead start 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff488798640 1 -- 172.21.15.165:0/2109212302 reap_dead start 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 shutdown_connections 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 >> 172.21.15.165:0/2109212302 conn(0x55d3eb296d40 msgr2=0x55d3eb2943a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 shutdown_connections 2024-04-27T03:30:30.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.546+0000 7ff48a498800 1 -- 172.21.15.165:0/2109212302 wait complete. 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/759436208 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff360036950 msgr2=0x7ff360038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 --2- 172.21.15.165:0/759436208 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff360036950 0x7ff360038e10 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x55d3eb273580 tx=0x7ff47c005ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/759436208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb28d8c0 msgr2=0x55d3eb28dce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 --2- 172.21.15.165:0/759436208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb28d8c0 0x55d3eb28dce0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7ff47804b4d0 tx=0x7ff478017d50 comp rx=0 tx=0).stop 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/759436208 shutdown_connections 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 --2- 172.21.15.165:0/759436208 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff360036950 0x7ff360038e10 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 --2- 172.21.15.165:0/759436208 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb28d8c0 0x55d3eb28dce0 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/759436208 >> 172.21.15.165:0/759436208 conn(0x55d3eb281960 msgr2=0x55d3eb281d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/759436208 shutdown_connections 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/759436208 wait complete. 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d3eb27c8b0 msgr2=0x55d3eb27ed00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d3eb27c8b0 0x55d3eb27ed00 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d3eb1784a0 msgr2=0x55d3eb276090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d3eb1784a0 0x55d3eb276090 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d3eb278970 msgr2=0x55d3eb27ad60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d3eb278970 0x55d3eb27ad60 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff370036900 msgr2=0x7ff370038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff370036900 0x7ff370038dc0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff47c054000 tx=0x7ff47c04e000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 msgr2=0x55d3eb26ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 --2- 172.21.15.165:0/2101070109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d3eb174990 0x55d3eb26ee50 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7ff47800b700 tx=0x7ff478000bc0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff488798640 1 -- 172.21.15.165:0/2101070109 reap_dead start 2024-04-27T03:30:30.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 shutdown_connections 2024-04-27T03:30:30.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 >> 172.21.15.165:0/2101070109 conn(0x55d3eb16e340 msgr2=0x55d3eb173170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 shutdown_connections 2024-04-27T03:30:30.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.547+0000 7ff48a498800 1 -- 172.21.15.165:0/2101070109 wait complete. 2024-04-27T03:30:30.564 DEBUG:teuthology.orchestra.run:got remote process result: 240 2024-04-27T03:30:30.564 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T03:30:30.564 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T03:30:30.565 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T03:30:30.607 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:30.607 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.602+0000 7f161ed2e800 1 Processor -- start 2024-04-27T03:30:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.602+0000 7f161ed2e800 1 -- start start 2024-04-27T03:30:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.602+0000 7f161ed2e800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5631a8dfe130 0x5631a8e005f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.602+0000 7f161ed2e800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5631a8cb8d40 con 0x5631a8d6d1c0 2024-04-27T03:30:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.602+0000 7f161ed2e800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5631a8c8b4d0 con 0x5631a8dfe130 2024-04-27T03:30:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.603+0000 7f161a26d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5631a8dfe130 0x5631a8e005f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.603+0000 7f161a26d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5631a8dfe130 0x5631a8e005f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51594/0 (socket says 172.21.15.165:51594) 2024-04-27T03:30:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.603+0000 7f161a26d640 1 -- 172.21.15.165:0/1243666640 learned_addr learned my addr 172.21.15.165:0/1243666640 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.603+0000 7f161aa6e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5631a8d6d1c0 0x5631a8dfc9e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53620/0 (socket says 172.21.15.165:53620) 2024-04-27T03:30:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.603+0000 7f161a26d640 1 -- 172.21.15.165:0/1243666640 >> v1:172.21.15.89:6789/0 conn(0x5631a8d6d1c0 legacy=0x5631a8dfc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.603+0000 7f161a26d640 1 -- 172.21.15.165:0/1243666640 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a8cb7590 con 0x5631a8dfe130 2024-04-27T03:30:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.605+0000 7f161a26d640 1 --2- 172.21.15.165:0/1243666640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8dfe130 0x5631a8e005f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f161000a520 tx=0x7f161001d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fd71395a33388cd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.605+0000 7f1619a6c640 1 -- 172.21.15.165:0/1243666640 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1610057070 con 0x5631a8dfe130 2024-04-27T03:30:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.605+0000 7f1619a6c640 1 -- 172.21.15.165:0/1243666640 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1610052570 con 0x5631a8dfe130 2024-04-27T03:30:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.605+0000 7f1619a6c640 1 -- 172.21.15.165:0/1243666640 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f161004ae60 con 0x5631a8dfe130 2024-04-27T03:30:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.605+0000 7f161ed2e800 1 -- 172.21.15.165:0/1243666640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8dfe130 msgr2=0x5631a8e005f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.605+0000 7f161ed2e800 1 --2- 172.21.15.165:0/1243666640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8dfe130 0x5631a8e005f0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f161000a520 tx=0x7f161001d010 comp rx=0 tx=0).stop 2024-04-27T03:30:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.606+0000 7f161ed2e800 1 -- 172.21.15.165:0/1243666640 shutdown_connections 2024-04-27T03:30:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.606+0000 7f161ed2e800 1 --2- 172.21.15.165:0/1243666640 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8dfe130 0x5631a8e005f0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.606+0000 7f161ed2e800 1 -- 172.21.15.165:0/1243666640 >> 172.21.15.165:0/1243666640 conn(0x5631a8df4380 msgr2=0x5631a8df67a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.606+0000 7f161ed2e800 1 -- 172.21.15.165:0/1243666640 shutdown_connections 2024-04-27T03:30:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.606+0000 7f161ed2e800 1 -- 172.21.15.165:0/1243666640 wait complete. 2024-04-27T03:30:30.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.617+0000 7f161ed2e800 1 Processor -- start 2024-04-27T03:30:30.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.617+0000 7f161ed2e800 1 -- start start 2024-04-27T03:30:30.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.617+0000 7f161ed2e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 0x5631a8eeb060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.617+0000 7f161ed2e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5631a8d453c0 con 0x5631a8d6d1c0 2024-04-27T03:30:30.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.618+0000 7f161aa6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 0x5631a8eeb060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.618+0000 7f161aa6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 0x5631a8eeb060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51610/0 (socket says 172.21.15.165:51610) 2024-04-27T03:30:30.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.618+0000 7f161aa6e640 1 -- 172.21.15.165:0/3670487118 learned_addr learned my addr 172.21.15.165:0/3670487118 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.618+0000 7f161aa6e640 1 -- 172.21.15.165:0/3670487118 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a8eeb630 con 0x5631a8d6d1c0 2024-04-27T03:30:30.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.618+0000 7f161aa6e640 1 --2- 172.21.15.165:0/3670487118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 0x5631a8eeb060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f160c001e10 tx=0x7f160c00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9da99c2d5df4daeb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.618+0000 7f1508e09640 1 -- 172.21.15.165:0/3670487118 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f160c00e5c0 con 0x5631a8d6d1c0 2024-04-27T03:30:30.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.618+0000 7f1508e09640 1 -- 172.21.15.165:0/3670487118 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160c006070 con 0x5631a8d6d1c0 2024-04-27T03:30:30.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.618+0000 7f1508e09640 1 -- 172.21.15.165:0/3670487118 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f160c00e930 con 0x5631a8d6d1c0 2024-04-27T03:30:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.619+0000 7f161ed2e800 1 -- 172.21.15.165:0/3670487118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 msgr2=0x5631a8eeb060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.619+0000 7f161ed2e800 1 --2- 172.21.15.165:0/3670487118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 0x5631a8eeb060 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f160c001e10 tx=0x7f160c00b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.619+0000 7f161ed2e800 1 -- 172.21.15.165:0/3670487118 shutdown_connections 2024-04-27T03:30:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.619+0000 7f161ed2e800 1 --2- 172.21.15.165:0/3670487118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 0x5631a8eeb060 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.619+0000 7f161ed2e800 1 -- 172.21.15.165:0/3670487118 >> 172.21.15.165:0/3670487118 conn(0x5631a8df4380 msgr2=0x5631a8ee6980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.619+0000 7f161ed2e800 1 -- 172.21.15.165:0/3670487118 shutdown_connections 2024-04-27T03:30:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.619+0000 7f161ed2e800 1 -- 172.21.15.165:0/3670487118 wait complete. 2024-04-27T03:30:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.619+0000 7f161ed2e800 1 Processor -- start 2024-04-27T03:30:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.619+0000 7f161ed2e800 1 -- start start 2024-04-27T03:30:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.620+0000 7f161ed2e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 0x5631a8ef2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.620+0000 7f161ed2e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5631a8e00fa0 con 0x5631a8d6d1c0 2024-04-27T03:30:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.620+0000 7f161aa6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 0x5631a8ef2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.620+0000 7f161aa6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 0x5631a8ef2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51614/0 (socket says 172.21.15.165:51614) 2024-04-27T03:30:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.620+0000 7f161aa6e640 1 -- 172.21.15.165:0/2404906149 learned_addr learned my addr 172.21.15.165:0/2404906149 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.620+0000 7f161aa6e640 1 -- 172.21.15.165:0/2404906149 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a8ef30f0 con 0x5631a8d6d1c0 2024-04-27T03:30:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.620+0000 7f161aa6e640 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 0x5631a8ef2bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f160c00b700 tx=0x7f160c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.620+0000 7f1507606640 1 -- 172.21.15.165:0/2404906149 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f160c00f8d0 con 0x5631a8d6d1c0 2024-04-27T03:30:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.620+0000 7f1507606640 1 -- 172.21.15.165:0/2404906149 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160c00fa70 con 0x5631a8d6d1c0 2024-04-27T03:30:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.620+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5631a8ef3380 con 0x5631a8d6d1c0 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.620+0000 7f1507606640 1 -- 172.21.15.165:0/2404906149 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f160c00fd50 con 0x5631a8d6d1c0 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.620+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a8ef6030 con 0x5631a8d6d1c0 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.621+0000 7f1507606640 1 -- 172.21.15.165:0/2404906149 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f160c017070 con 0x5631a8d6d1c0 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.621+0000 7f1507606640 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1500036900 0x7f1500038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.621+0000 7f1507606640 1 -- 172.21.15.165:0/2404906149 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f160c015020 con 0x5631a8d6d1c0 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.621+0000 7f161a26d640 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1500036900 0x7f1500038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.621+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631a8dfe130 0x5631a8ef9df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.621+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8efa3c0 con 0x5631a8dfe130 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.622+0000 7f161b26f640 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631a8dfe130 0x5631a8ef9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.622+0000 7f161a26d640 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1500036900 0x7f1500038dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1610054000 tx=0x7f161004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.622+0000 7f161b26f640 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631a8dfe130 0x5631a8ef9df0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.622+0000 7f161b26f640 1 -- 172.21.15.165:0/2404906149 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8dfe130 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.622+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8ef73c0 con 0x5631a8dfe130 2024-04-27T03:30:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.622+0000 7f161b26f640 1 -- 172.21.15.165:0/2404906149 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8dfe130 2024-04-27T03:30:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.622+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631a8efc6d0 0x5631a8efeac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.622+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8eff160 con 0x5631a8efc6d0 2024-04-27T03:30:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.623+0000 7f161aa6e640 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631a8efc6d0 0x5631a8efeac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.623+0000 7f161aa6e640 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631a8efc6d0 0x5631a8efeac0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.623+0000 7f161aa6e640 1 -- 172.21.15.165:0/2404906149 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f160c046040 con 0x5631a8efc6d0 2024-04-27T03:30:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.623+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8efb680 con 0x5631a8dfe130 2024-04-27T03:30:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.624+0000 7f161b26f640 1 -- 172.21.15.165:0/2404906149 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8dfe130 2024-04-27T03:30:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.624+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5631a8f00610 0x5631a8f02a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.624+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f031c0 con 0x5631a8f00610 2024-04-27T03:30:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.624+0000 7f161a26d640 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5631a8f00610 0x5631a8f02a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.624+0000 7f161a26d640 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5631a8f00610 0x5631a8f02a60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.625+0000 7f161a26d640 1 -- 172.21.15.165:0/2404906149 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f161005a040 con 0x5631a8f00610 2024-04-27T03:30:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.625+0000 7f161ed2e800 1 Processor -- start 2024-04-27T03:30:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.625+0000 7f161ed2e800 1 -- start start 2024-04-27T03:30:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.625+0000 7f161ed2e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8dfa570 0x5631a8df89e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.625+0000 7f161ed2e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5631a8ceb6b0 con 0x5631a8dfa570 2024-04-27T03:30:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.625+0000 7f161aa6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8dfa570 0x5631a8df89e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.625+0000 7f161aa6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8dfa570 0x5631a8df89e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51618/0 (socket says 172.21.15.165:51618) 2024-04-27T03:30:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.625+0000 7f161aa6e640 1 -- 172.21.15.165:0/2689300317 learned_addr learned my addr 172.21.15.165:0/2689300317 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.626+0000 7f161aa6e640 1 -- 172.21.15.165:0/2689300317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a8df8f20 con 0x5631a8dfa570 2024-04-27T03:30:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.626+0000 7f161aa6e640 1 --2- 172.21.15.165:0/2689300317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8dfa570 0x5631a8df89e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f160c04b4d0 tx=0x7f160c017d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.626+0000 7f1504e01640 1 -- 172.21.15.165:0/2689300317 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f160c01ae10 con 0x5631a8dfa570 2024-04-27T03:30:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.626+0000 7f1504e01640 1 -- 172.21.15.165:0/2689300317 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160c01a690 con 0x5631a8dfa570 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.626+0000 7f1504e01640 1 -- 172.21.15.165:0/2689300317 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f160c044b00 con 0x5631a8dfa570 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.626+0000 7f161ed2e800 1 -- 172.21.15.165:0/2689300317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5631a8df91b0 con 0x5631a8dfa570 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.626+0000 7f161ed2e800 1 -- 172.21.15.165:0/2689300317 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a8df96b0 con 0x5631a8dfa570 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.627+0000 7f1504e01640 1 -- 172.21.15.165:0/2689300317 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f160c044ca0 con 0x5631a8dfa570 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.627+0000 7f1504e01640 1 --2- 172.21.15.165:0/2689300317 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f14f4036950 0x7f14f4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.627+0000 7f1504e01640 1 -- 172.21.15.165:0/2689300317 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f160c03cae0 con 0x5631a8dfa570 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.627+0000 7f161a26d640 1 --2- 172.21.15.165:0/2689300317 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f14f4036950 0x7f14f4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.627+0000 7f161ed2e800 1 Processor -- start 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.627+0000 7f161ed2e800 1 -- start start 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.627+0000 7f161a26d640 1 --2- 172.21.15.165:0/2689300317 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f14f4036950 0x7f14f4038e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x5631a8ef72e0 tx=0x7f1610005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.627+0000 7f161ed2e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 0x5631a8f20dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.627+0000 7f161ed2e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5631a8eeb9b0 con 0x5631a8f209a0 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.628+0000 7f161aa6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 0x5631a8f20dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.628+0000 7f161aa6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 0x5631a8f20dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51624/0 (socket says 172.21.15.165:51624) 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.628+0000 7f161aa6e640 1 -- 172.21.15.165:0/369174273 learned_addr learned my addr 172.21.15.165:0/369174273 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.628+0000 7f161aa6e640 1 -- 172.21.15.165:0/369174273 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a8f213b0 con 0x5631a8f209a0 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.628+0000 7f161aa6e640 1 --2- 172.21.15.165:0/369174273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 0x5631a8f20dc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f160c00eb60 tx=0x7f160c026ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee91552154970a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.628+0000 7f14fe7fc640 1 -- 172.21.15.165:0/369174273 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f160c03cdc0 con 0x5631a8f209a0 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.628+0000 7f14fe7fc640 1 -- 172.21.15.165:0/369174273 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160c05e070 con 0x5631a8f209a0 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.628+0000 7f14fe7fc640 1 -- 172.21.15.165:0/369174273 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f160c044f90 con 0x5631a8f209a0 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161ed2e800 1 -- 172.21.15.165:0/369174273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 msgr2=0x5631a8f20dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161ed2e800 1 --2- 172.21.15.165:0/369174273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 0x5631a8f20dc0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f160c00eb60 tx=0x7f160c026ae0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161ed2e800 1 -- 172.21.15.165:0/369174273 shutdown_connections 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161ed2e800 1 --2- 172.21.15.165:0/369174273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 0x5631a8f20dc0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161ed2e800 1 -- 172.21.15.165:0/369174273 >> 172.21.15.165:0/369174273 conn(0x5631a8f1cce0 msgr2=0x5631a8f1ad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161ed2e800 1 -- 172.21.15.165:0/369174273 shutdown_connections 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161ed2e800 1 -- 172.21.15.165:0/369174273 wait complete. 2024-04-27T03:30:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161ed2e800 1 Processor -- start 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161ed2e800 1 -- start start 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161ed2e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 0x5631a8f27d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161ed2e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5631a8f21ad0 con 0x5631a8f209a0 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161aa6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 0x5631a8f27d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161aa6e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 0x5631a8f27d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51636/0 (socket says 172.21.15.165:51636) 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 learned_addr learned my addr 172.21.15.165:0/2343150881 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.629+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631a8f28240 con 0x5631a8f209a0 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.630+0000 7f161aa6e640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 0x5631a8f27d00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f160c02c650 tx=0x7f160c03ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.630+0000 7f14fcff9640 1 -- 172.21.15.165:0/2343150881 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f160c06c070 con 0x5631a8f209a0 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.630+0000 7f14fcff9640 1 -- 172.21.15.165:0/2343150881 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f160c03ce60 con 0x5631a8f209a0 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.630+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5631a8f28240 con 0x5631a8f209a0 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.630+0000 7f14fcff9640 1 -- 172.21.15.165:0/2343150881 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f160c03f2a0 con 0x5631a8f209a0 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.630+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631a8f28240 con 0x5631a8f209a0 2024-04-27T03:30:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.631+0000 7f14fcff9640 1 -- 172.21.15.165:0/2343150881 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f160c06f070 con 0x5631a8f209a0 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.631+0000 7f14fcff9640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f14e40368b0 0x7f14e4038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.631+0000 7f14fcff9640 1 -- 172.21.15.165:0/2343150881 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f160c055070 con 0x5631a8f209a0 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.631+0000 7f161ed2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.631+0000 7f161a26d640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f14e40368b0 0x7f14e4038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.631+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631a8f331e0 0x5631a8f356a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.631+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f35df0 con 0x5631a8f331e0 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.631+0000 7f161b26f640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631a8f331e0 0x5631a8f356a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.631+0000 7f161a26d640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f14e40368b0 0x7f14e4038d70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1610055040 tx=0x7f161004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.631+0000 7f161b26f640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631a8f331e0 0x5631a8f356a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 20 realm 2024-04-27T03:30:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36200 con 0x5631a8f331e0 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36ad0 con 0x5631a8f331e0 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36e70 con 0x5631a8f331e0 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631a8f37960 0x5631a8f39d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.632+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3a4b0 con 0x5631a8f37960 2024-04-27T03:30:30.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.633+0000 7f161aa6e640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631a8f37960 0x5631a8f39d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.633+0000 7f161aa6e640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631a8f37960 0x5631a8f39d50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.633+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.633+0000 7f161ed2e800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:30.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.633+0000 7f161ed2e800 20 searching for the correct realm 2024-04-27T03:30:30.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.633+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3be90 con 0x5631a8f37960 2024-04-27T03:30:30.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.634+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.634+0000 7f161aa6e640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f160c071070 0x7f160c031450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.634+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3be90 con 0x7f160c071070 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.634+0000 7f161a26d640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f160c071070 0x7f160c031450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.634+0000 7f161a26d640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f160c071070 0x7f160c031450 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.634+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161007f070 con 0x7f160c071070 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.634+0000 7f161a26d640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1610074040 0x7f1610081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.634+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3be90 con 0x7f1610074040 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.635+0000 7f161b26f640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1610074040 0x7f1610081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.635+0000 7f161b26f640 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1610074040 0x7f1610081410 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.635+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.635+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36e70 con 0x5631a8f331e0 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.635+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.635+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36e70 con 0x5631a8f331e0 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.636+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.636+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36e70 con 0x7f1610074040 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.636+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.636+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36e70 con 0x7f1610074040 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.636+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.636+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36e70 con 0x7f160c071070 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.636+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161007f070 con 0x7f160c071070 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.636+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36e70 con 0x5631a8f37960 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.637+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.637+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36e70 con 0x5631a8f331e0 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.637+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.637+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36e70 con 0x7f1610074040 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.637+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.637+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36e70 con 0x5631a8f331e0 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.637+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.637+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f36e70 con 0x7f160c071070 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.637+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f161007f070 con 0x7f160c071070 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.637+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x5631a8f37960 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.638+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.638+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x5631a8f37960 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.638+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.638+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x7f1610074040 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.638+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.638+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x5631a8f37960 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.638+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.638+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x5631a8f37960 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.639+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.639+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x5631a8f331e0 2024-04-27T03:30:30.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.639+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.639+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x5631a8f331e0 2024-04-27T03:30:30.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.639+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.639+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x7f1610074040 2024-04-27T03:30:30.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.639+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.639+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x5631a8f37960 2024-04-27T03:30:30.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.640+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.640+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x7f160c071070 2024-04-27T03:30:30.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.640+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161007f070 con 0x7f160c071070 2024-04-27T03:30:30.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.640+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x5631a8f331e0 2024-04-27T03:30:30.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.640+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.640+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x7f1610074040 2024-04-27T03:30:30.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.640+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.640+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x5631a8f37960 2024-04-27T03:30:30.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.641+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.641+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x7f1610074040 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.641+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.641+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x7f160c071070 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.641+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161007f070 con 0x7f160c071070 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.641+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x5631a8f37960 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.642+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.642+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x7f1610074040 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.642+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.642+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f161008e060 con 0x5631a8f331e0 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.642+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.642+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1614038040 con 0x7f160c071070 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.642+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f161007f070 con 0x7f160c071070 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:30.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3ae60 con 0x5631a8f331e0 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3b250 con 0x5631a8f331e0 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3b250 con 0x7f1610074040 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 zone default found 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 4 Realm: () 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 zonegroup default 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.643+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3b670 con 0x7f1610074040 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161ed2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3b670 con 0x5631a8f37960 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161ed2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3dd40 con 0x5631a8f37960 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161ed2e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161ed2e800 20 started sync module instance, tier type = 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161ed2e800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.644+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3d1e0 con 0x7f160c071070 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.645+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161007f070 con 0x7f160c071070 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.645+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94771287938512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3e590 con 0x7f160c071070 2024-04-27T03:30:30.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.645+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3ede0 con 0x5631a8f331e0 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.646+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94771287938512] v25'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161007f070 con 0x7f160c071070 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.646+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.646+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94771287941344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3ee80 con 0x5631a8f331e0 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.646+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3f9a0 con 0x5631a8f37960 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.647+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94771287941344] v25'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.647+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.647+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94771287946352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3fd90 con 0x5631a8f37960 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.647+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f407b0 con 0x7f160c071070 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.648+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94771287946352] v25'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.649+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161007f070 con 0x7f160c071070 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.649+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94771287943472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f41080 con 0x7f160c071070 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.649+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f41a40 con 0x5631a8f331e0 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.649+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.650+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94771287955312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f42200 con 0x5631a8f331e0 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.650+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f42d00 con 0x7f1610074040 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.650+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94771287943472] v25'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161007f070 con 0x7f160c071070 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.650+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.651+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94771287959536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f42cc0 con 0x7f1610074040 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.651+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94771287955312] v25'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.651+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f43740 con 0x7f1610074040 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.652+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94771287959536] v25'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.652+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.652+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94771287956704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f43290 con 0x7f1610074040 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.652+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f43b80 con 0x7f1610074040 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.653+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'30 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.653+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94771287962256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f44210 con 0x7f1610074040 2024-04-27T03:30:30.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.653+0000 7f161ed2e800 20 add_watcher() i=0 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.653+0000 7f161ed2e800 20 add_watcher() i=1 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.653+0000 7f161ed2e800 20 add_watcher() i=2 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.653+0000 7f161ed2e800 20 add_watcher() i=3 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.653+0000 7f161ed2e800 20 add_watcher() i=4 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.653+0000 7f161ed2e800 20 add_watcher() i=5 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.655+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94771287956704] v25'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.655+0000 7f161ed2e800 20 add_watcher() i=6 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.655+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94771287962256] v25'31 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.655+0000 7f161ed2e800 20 add_watcher() i=7 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.655+0000 7f161ed2e800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.655+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f47130 con 0x5631a8f331e0 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.655+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.655+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94771287964752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f44520 con 0x5631a8f331e0 2024-04-27T03:30:30.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.656+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94771287964752] v25'60 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.656+0000 7f14d17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.656+0000 7f161ed2e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:30.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.656+0000 7f161ed2e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:30.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.656+0000 7f161ed2e800 5 note: GC not initialized 2024-04-27T03:30:30.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.656+0000 7f144bfff640 20 reqs_thread_entry: start 2024-04-27T03:30:30.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.656+0000 7f161ed2e800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:30.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.657+0000 7f161ed2e800 20 Filter name: none 2024-04-27T03:30:30.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.657+0000 7f1449ffb640 20 reqs_thread_entry: start 2024-04-27T03:30:30.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.657+0000 7f161ed2e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.657+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f79df0 con 0x5631a8f331e0 2024-04-27T03:30:30.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.657+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.657+0000 7f161ed2e800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:30.675 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:30.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.658+0000 7f14d17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.658+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94771287964752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f78d60 con 0x5631a8f331e0 2024-04-27T03:30:30.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.659+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94771287964752] v25'61 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.659+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94771287938512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f78d60 con 0x7f160c071070 2024-04-27T03:30:30.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.660+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94771287938512] v25'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161007f070 con 0x7f160c071070 2024-04-27T03:30:30.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.660+0000 7f161ed2e800 20 remove_watcher() i=0 2024-04-27T03:30:30.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.660+0000 7f161ed2e800 2 removed watcher, disabling cache 2024-04-27T03:30:30.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.660+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94771287941344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3d5d0 con 0x5631a8f331e0 2024-04-27T03:30:30.680 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.661+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94771287941344] v25'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.661+0000 7f161ed2e800 20 remove_watcher() i=1 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.661+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94771287946352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3d5d0 con 0x5631a8f37960 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.662+0000 7f161aa6e640 1 -- 172.21.15.165:0/2343150881 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94771287946352] v25'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f160c078070 con 0x5631a8f37960 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.662+0000 7f161ed2e800 20 remove_watcher() i=2 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.662+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94771287943472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3e0e0 con 0x7f160c071070 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.663+0000 7f161a26d640 1 -- 172.21.15.165:0/2343150881 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94771287943472] v25'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161007f070 con 0x7f160c071070 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.663+0000 7f161ed2e800 20 remove_watcher() i=3 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.663+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94771287955312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3e0e0 con 0x5631a8f331e0 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.664+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94771287955312] v25'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x5631a8f331e0 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.664+0000 7f161ed2e800 20 remove_watcher() i=4 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.664+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94771287959536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f41770 con 0x7f1610074040 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.665+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94771287959536] v25'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.665+0000 7f161ed2e800 20 remove_watcher() i=5 2024-04-27T03:30:30.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.665+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94771287956704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3e0e0 con 0x7f1610074040 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.665+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94771287956704] v25'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.665+0000 7f161ed2e800 20 remove_watcher() i=6 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.665+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94771287962256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5631a8f3e0e0 con 0x7f1610074040 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.666+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94771287962256] v25'32 uv30 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f161400a4a0 con 0x7f1610074040 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.666+0000 7f161ed2e800 20 remove_watcher() i=7 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.666+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1610074040 msgr2=0x7f1610081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.666+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1610074040 0x7f1610081410 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.666+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631a8f331e0 msgr2=0x5631a8f356a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.666+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631a8f331e0 0x5631a8f356a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.666+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f160c071070 msgr2=0x7f160c031450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.666+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f160c071070 0x7f160c031450 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.666+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631a8f37960 msgr2=0x5631a8f39d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.666+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631a8f37960 0x5631a8f39d50 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f14e40368b0 msgr2=0x7f14e4038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f14e40368b0 0x7f14e4038d70 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1610055040 tx=0x7f161004f000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161b26f640 1 -- 172.21.15.165:0/2343150881 reap_dead start 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 msgr2=0x5631a8f27d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 0x5631a8f27d00 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f160c02c650 tx=0x7f160c03ef50 comp rx=0 tx=0).stop 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 shutdown_connections 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2343150881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8f209a0 0x5631a8f27d00 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 >> 172.21.15.165:0/2343150881 conn(0x5631a8f1cce0 msgr2=0x5631a8f1a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 shutdown_connections 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2343150881 wait complete. 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2689300317 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f14f4036950 msgr2=0x7f14f4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2689300317 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f14f4036950 0x7f14f4038e10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x5631a8ef72e0 tx=0x7f1610005ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2689300317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8dfa570 msgr2=0x5631a8df89e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2689300317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8dfa570 0x5631a8df89e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f160c04b4d0 tx=0x7f160c017d50 comp rx=0 tx=0).stop 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2689300317 shutdown_connections 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2689300317 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f14f4036950 0x7f14f4038e10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2689300317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8dfa570 0x5631a8df89e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2689300317 >> 172.21.15.165:0/2689300317 conn(0x5631a8f056c0 msgr2=0x5631a8f05ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2689300317 shutdown_connections 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2689300317 wait complete. 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5631a8f00610 msgr2=0x5631a8f02a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5631a8f00610 0x5631a8f02a60 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631a8dfe130 msgr2=0x5631a8ef9df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631a8dfe130 0x5631a8ef9df0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631a8efc6d0 msgr2=0x5631a8efeac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631a8efc6d0 0x5631a8efeac0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1500036900 msgr2=0x7f1500038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.667+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1500036900 0x7f1500038dc0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1610054000 tx=0x7f161004e000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.668+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 msgr2=0x5631a8ef2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.668+0000 7f161ed2e800 1 --2- 172.21.15.165:0/2404906149 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631a8d6d1c0 0x5631a8ef2bb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f160c00b700 tx=0x7f160c000bc0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.668+0000 7f161b26f640 1 -- 172.21.15.165:0/2404906149 reap_dead start 2024-04-27T03:30:30.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.668+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 shutdown_connections 2024-04-27T03:30:30.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.668+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 >> 172.21.15.165:0/2404906149 conn(0x5631a8df4380 msgr2=0x5631a8ee5260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.668+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 shutdown_connections 2024-04-27T03:30:30.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.668+0000 7f161ed2e800 1 -- 172.21.15.165:0/2404906149 wait complete. 2024-04-27T03:30:30.686 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:30.686 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-04-27T03:30:30.686 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-04-27T03:30:30.686 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-04-27T03:30:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.764+0000 7f0feea3f800 1 Processor -- start 2024-04-27T03:30:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.764+0000 7f0feea3f800 1 -- start start 2024-04-27T03:30:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.764+0000 7f0feea3f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x56401ba43970 0x56401ba45d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.764+0000 7f0feea3f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56401b901d40 con 0x56401ba462a0 2024-04-27T03:30:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.764+0000 7f0feea3f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x56401b8d44d0 con 0x56401ba43970 2024-04-27T03:30:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.765+0000 7f0fec545640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x56401ba43970 0x56401ba45d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.765+0000 7f0fe3d44640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x56401ba462a0 0x56401ba48690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53628/0 (socket says 172.21.15.165:53628) 2024-04-27T03:30:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.765+0000 7f0fec545640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x56401ba43970 0x56401ba45d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51650/0 (socket says 172.21.15.165:51650) 2024-04-27T03:30:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.765+0000 7f0fe3d44640 1 -- 172.21.15.165:0/4162616136 learned_addr learned my addr 172.21.15.165:0/4162616136 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.765+0000 7f0fec545640 1 -- 172.21.15.165:0/4162616136 >> v1:172.21.15.89:6789/0 conn(0x56401ba462a0 legacy=0x56401ba48690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.765+0000 7f0fec545640 1 -- 172.21.15.165:0/4162616136 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56401b900590 con 0x56401ba43970 2024-04-27T03:30:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.767+0000 7f0fec545640 1 --2- 172.21.15.165:0/4162616136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401ba45d60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00a470 tx=0x7f0fdc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=975b527d7b9e7793 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.767+0000 7f0febd44640 1 -- 172.21.15.165:0/4162616136 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc057070 con 0x56401ba43970 2024-04-27T03:30:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.767+0000 7f0febd44640 1 -- 172.21.15.165:0/4162616136 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fdc052570 con 0x56401ba43970 2024-04-27T03:30:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.767+0000 7f0febd44640 1 -- 172.21.15.165:0/4162616136 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc052870 con 0x56401ba43970 2024-04-27T03:30:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.767+0000 7f0feea3f800 1 -- 172.21.15.165:0/4162616136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 msgr2=0x56401ba45d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.767+0000 7f0feea3f800 1 --2- 172.21.15.165:0/4162616136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401ba45d60 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc00a470 tx=0x7f0fdc01ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.767+0000 7f0feea3f800 1 -- 172.21.15.165:0/4162616136 shutdown_connections 2024-04-27T03:30:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.767+0000 7f0feea3f800 1 --2- 172.21.15.165:0/4162616136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401ba45d60 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.767+0000 7f0feea3f800 1 -- 172.21.15.165:0/4162616136 >> 172.21.15.165:0/4162616136 conn(0x56401ba3d320 msgr2=0x56401ba3f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.767+0000 7f0feea3f800 1 -- 172.21.15.165:0/4162616136 shutdown_connections 2024-04-27T03:30:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.767+0000 7f0feea3f800 1 -- 172.21.15.165:0/4162616136 wait complete. 2024-04-27T03:30:30.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.779+0000 7f0feea3f800 1 Processor -- start 2024-04-27T03:30:30.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.779+0000 7f0feea3f800 1 -- start start 2024-04-27T03:30:30.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.779+0000 7f0feea3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401bb34110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.779+0000 7f0feea3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56401b98e3c0 con 0x56401ba43970 2024-04-27T03:30:30.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.780+0000 7f0fec545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401bb34110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.780+0000 7f0fec545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401bb34110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51654/0 (socket says 172.21.15.165:51654) 2024-04-27T03:30:30.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.780+0000 7f0fec545640 1 -- 172.21.15.165:0/1257860136 learned_addr learned my addr 172.21.15.165:0/1257860136 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.780+0000 7f0fec545640 1 -- 172.21.15.165:0/1257860136 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56401bb346e0 con 0x56401ba43970 2024-04-27T03:30:30.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.780+0000 7f0fec545640 1 --2- 172.21.15.165:0/1257860136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401bb34110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc055040 tx=0x7f0fdc04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc463cecdd87b22 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.780+0000 7f0eda60c640 1 -- 172.21.15.165:0/1257860136 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc057070 con 0x56401ba43970 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.780+0000 7f0eda60c640 1 -- 172.21.15.165:0/1257860136 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fdc063070 con 0x56401ba43970 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.780+0000 7f0eda60c640 1 -- 172.21.15.165:0/1257860136 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc052720 con 0x56401ba43970 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.781+0000 7f0feea3f800 1 -- 172.21.15.165:0/1257860136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 msgr2=0x56401bb34110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.781+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1257860136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401bb34110 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc055040 tx=0x7f0fdc04c000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.781+0000 7f0feea3f800 1 -- 172.21.15.165:0/1257860136 shutdown_connections 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.781+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1257860136 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401bb34110 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.781+0000 7f0feea3f800 1 -- 172.21.15.165:0/1257860136 >> 172.21.15.165:0/1257860136 conn(0x56401ba3d320 msgr2=0x56401bb2fa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.781+0000 7f0feea3f800 1 -- 172.21.15.165:0/1257860136 shutdown_connections 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.781+0000 7f0feea3f800 1 -- 172.21.15.165:0/1257860136 wait complete. 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.781+0000 7f0feea3f800 1 Processor -- start 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.781+0000 7f0feea3f800 1 -- start start 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.782+0000 7f0feea3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401ba43210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.782+0000 7f0feea3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56401ba4c0c0 con 0x56401ba43970 2024-04-27T03:30:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.782+0000 7f0fec545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401ba43210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.782+0000 7f0fec545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401ba43210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51670/0 (socket says 172.21.15.165:51670) 2024-04-27T03:30:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.782+0000 7f0fec545640 1 -- 172.21.15.165:0/1203754155 learned_addr learned my addr 172.21.15.165:0/1203754155 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.782+0000 7f0fec545640 1 -- 172.21.15.165:0/1203754155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56401ba43750 con 0x56401ba43970 2024-04-27T03:30:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.782+0000 7f0fec545640 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401ba43210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc060040 tx=0x7f0fdc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.783+0000 7f0ed8e09640 1 -- 172.21.15.165:0/1203754155 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc00a2d0 con 0x56401ba43970 2024-04-27T03:30:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.783+0000 7f0ed8e09640 1 -- 172.21.15.165:0/1203754155 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fdc06c070 con 0x56401ba43970 2024-04-27T03:30:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.783+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56401ba419a0 con 0x56401ba43970 2024-04-27T03:30:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.783+0000 7f0ed8e09640 1 -- 172.21.15.165:0/1203754155 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc05e5d0 con 0x56401ba43970 2024-04-27T03:30:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.783+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56401ba41f20 con 0x56401ba43970 2024-04-27T03:30:30.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.783+0000 7f0ed8e09640 1 -- 172.21.15.165:0/1203754155 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0fdc05e770 con 0x56401ba43970 2024-04-27T03:30:30.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.784+0000 7f0ed8e09640 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ed4036950 0x7f0ed4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.784+0000 7f0ed8e09640 1 -- 172.21.15.165:0/1203754155 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f0fdc08e8f0 con 0x56401ba43970 2024-04-27T03:30:30.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.784+0000 7f0fe3d44640 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ed4036950 0x7f0ed4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.784+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56401ba462a0 0x56401bb44f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.784+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb45490 con 0x56401ba462a0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.784+0000 7f0fecd46640 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56401ba462a0 0x56401bb44f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.784+0000 7f0fe3d44640 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ed4036950 0x7f0ed4038e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4000f60 tx=0x7f0fd40012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.784+0000 7f0fecd46640 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56401ba462a0 0x56401bb44f50 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.784+0000 7f0fecd46640 1 -- 172.21.15.165:0/1203754155 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401ba462a0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.784+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401ba423d0 con 0x56401ba462a0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.785+0000 7f0fecd46640 1 -- 172.21.15.165:0/1203754155 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401ba462a0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.785+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56401bb477a0 0x56401bb49b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.785+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb4a230 con 0x56401bb477a0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.785+0000 7f0fec545640 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56401bb477a0 0x56401bb49b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.785+0000 7f0fec545640 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56401bb477a0 0x56401bb49b90 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.785+0000 7f0fec545640 1 -- 172.21.15.165:0/1203754155 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0fdc071990 con 0x56401bb477a0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.785+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb46750 con 0x56401ba462a0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.786+0000 7f0fecd46640 1 -- 172.21.15.165:0/1203754155 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401ba462a0 2024-04-27T03:30:30.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.786+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x56401bb4b6e0 0x56401bb4db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.786+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb4e290 con 0x56401bb4b6e0 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.786+0000 7f0fe3d44640 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x56401bb4b6e0 0x56401bb4db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.786+0000 7f0fe3d44640 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x56401bb4b6e0 0x56401bb4db30 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.787+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1203754155 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0fd4009050 con 0x56401bb4b6e0 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.787+0000 7f0feea3f800 1 Processor -- start 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.787+0000 7f0feea3f800 1 -- start start 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.787+0000 7f0feea3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb5c6f0 0x56401bb5cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.787+0000 7f0feea3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56401b9346b0 con 0x56401bb5c6f0 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.787+0000 7f0fec545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb5c6f0 0x56401bb5cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.787+0000 7f0fec545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb5c6f0 0x56401bb5cb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51682/0 (socket says 172.21.15.165:51682) 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.787+0000 7f0fec545640 1 -- 172.21.15.165:0/3291875113 learned_addr learned my addr 172.21.15.165:0/3291875113 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.788+0000 7f0fec545640 1 -- 172.21.15.165:0/3291875113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56401bb5d050 con 0x56401bb5c6f0 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.788+0000 7f0fec545640 1 --2- 172.21.15.165:0/3291875113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb5c6f0 0x56401bb5cb10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc052b60 tx=0x7f0fdc067420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.788+0000 7f0ed1ffb640 1 -- 172.21.15.165:0/3291875113 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc092070 con 0x56401bb5c6f0 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.788+0000 7f0ed1ffb640 1 -- 172.21.15.165:0/3291875113 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fdc058040 con 0x56401bb5c6f0 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.788+0000 7f0feea3f800 1 -- 172.21.15.165:0/3291875113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56401bb5dac0 con 0x56401bb5c6f0 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.788+0000 7f0ed1ffb640 1 -- 172.21.15.165:0/3291875113 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc05ec40 con 0x56401bb5c6f0 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.788+0000 7f0feea3f800 1 -- 172.21.15.165:0/3291875113 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56401bb5dfc0 con 0x56401bb5c6f0 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.789+0000 7f0ed1ffb640 1 -- 172.21.15.165:0/3291875113 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0fdc095070 con 0x56401bb5c6f0 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.789+0000 7f0ed1ffb640 1 --2- 172.21.15.165:0/3291875113 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ec4036950 0x7f0ec4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.789+0000 7f0ed1ffb640 1 -- 172.21.15.165:0/3291875113 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f0fdc098b00 con 0x56401bb5c6f0 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.789+0000 7f0fe3d44640 1 --2- 172.21.15.165:0/3291875113 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ec4036950 0x7f0ec4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.789+0000 7f0feea3f800 1 Processor -- start 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.789+0000 7f0feea3f800 1 -- start start 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.789+0000 7f0fe3d44640 1 --2- 172.21.15.165:0/3291875113 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ec4036950 0x7f0ec4038e10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400bf50 tx=0x7f0fd400f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.789+0000 7f0feea3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 0x56401bb69c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.789+0000 7f0feea3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56401bb349a0 con 0x56401bb69810 2024-04-27T03:30:30.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.790+0000 7f0fec545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 0x56401bb69c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.790+0000 7f0fec545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 0x56401bb69c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51696/0 (socket says 172.21.15.165:51696) 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.790+0000 7f0fec545640 1 -- 172.21.15.165:0/2399513300 learned_addr learned my addr 172.21.15.165:0/2399513300 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.790+0000 7f0fec545640 1 -- 172.21.15.165:0/2399513300 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56401bb6a200 con 0x56401bb69810 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.790+0000 7f0fec545640 1 --2- 172.21.15.165:0/2399513300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 0x56401bb69c30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc071780 tx=0x7f0fdc067770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f52984449cb316e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.790+0000 7f0ecb7fe640 1 -- 172.21.15.165:0/2399513300 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc098da0 con 0x56401bb69810 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.790+0000 7f0ecb7fe640 1 -- 172.21.15.165:0/2399513300 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fdc08e250 con 0x56401bb69810 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.790+0000 7f0ecb7fe640 1 -- 172.21.15.165:0/2399513300 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc08eb90 con 0x56401bb69810 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0feea3f800 1 -- 172.21.15.165:0/2399513300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 msgr2=0x56401bb69c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0feea3f800 1 --2- 172.21.15.165:0/2399513300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 0x56401bb69c30 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc071780 tx=0x7f0fdc067770 comp rx=0 tx=0).stop 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0feea3f800 1 -- 172.21.15.165:0/2399513300 shutdown_connections 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0feea3f800 1 --2- 172.21.15.165:0/2399513300 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 0x56401bb69c30 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0feea3f800 1 -- 172.21.15.165:0/2399513300 >> 172.21.15.165:0/2399513300 conn(0x56401bb65b70 msgr2=0x56401bb63bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0feea3f800 1 -- 172.21.15.165:0/2399513300 shutdown_connections 2024-04-27T03:30:30.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0feea3f800 1 -- 172.21.15.165:0/2399513300 wait complete. 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0feea3f800 1 Processor -- start 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0feea3f800 1 -- start start 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0feea3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 0x56401bb70c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0feea3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56401bb6a8b0 con 0x56401bb69810 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0fec545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 0x56401bb70c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0fec545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 0x56401bb70c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51706/0 (socket says 172.21.15.165:51706) 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.791+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 learned_addr learned my addr 172.21.15.165:0/1244022057 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.792+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56401bb711c0 con 0x56401bb69810 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.792+0000 7f0fec545640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 0x56401bb70c80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc052520 tx=0x7f0fdc082f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.792+0000 7f0ec9ffb640 1 -- 172.21.15.165:0/1244022057 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc085af0 con 0x56401bb69810 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.792+0000 7f0ec9ffb640 1 -- 172.21.15.165:0/1244022057 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0fdc085c90 con 0x56401bb69810 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.792+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56401bb711c0 con 0x56401bb69810 2024-04-27T03:30:30.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.792+0000 7f0ec9ffb640 1 -- 172.21.15.165:0/1244022057 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fdc066580 con 0x56401bb69810 2024-04-27T03:30:30.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.792+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56401bb711c0 con 0x56401bb69810 2024-04-27T03:30:30.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.793+0000 7f0ec9ffb640 1 -- 172.21.15.165:0/1244022057 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0fdc066720 con 0x56401bb69810 2024-04-27T03:30:30.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.793+0000 7f0ec9ffb640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ebc036950 0x7f0ebc038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.793+0000 7f0ec9ffb640 1 -- 172.21.15.165:0/1244022057 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f0fdc0b4070 con 0x56401bb69810 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.793+0000 7f0fe3d44640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ebc036950 0x7f0ebc038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.793+0000 7f0feea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.793+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56401bb7c3d0 0x56401bb7e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.793+0000 7f0fe3d44640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ebc036950 0x7f0ebc038e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4001b60 tx=0x7f0fd4010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.793+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb7efe0 con 0x56401bb7c3d0 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0fecd46640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56401bb7c3d0 0x56401bb7e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0fecd46640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56401bb7c3d0 0x56401bb7e890 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401bb7c3d0 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0feea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0feea3f800 20 realm 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0feea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb7f3f0 con 0x56401bb7c3d0 2024-04-27T03:30:30.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401bb7c3d0 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0feea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0feea3f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0feea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb7fcc0 con 0x56401bb7c3d0 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401bb7c3d0 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0feea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0feea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.794+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb7c3d0 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.795+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401bb7c3d0 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.795+0000 7f0feea3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.795+0000 7f0feea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.795+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56401bb80dc0 0x56401bb831b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.795+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb837c0 con 0x56401bb80dc0 2024-04-27T03:30:30.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.795+0000 7f0fec545640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56401bb80dc0 0x56401bb831b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.795+0000 7f0fec545640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56401bb80dc0 0x56401bb831b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.795+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.795+0000 7f0feea3f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.796+0000 7f0feea3f800 20 searching for the correct realm 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.796+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb850f0 con 0x56401bb80dc0 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.796+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.796+0000 7f0fec545640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0fdc0b7030 0x7f0fdc0978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.796+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb850f0 con 0x7f0fdc0b7030 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.796+0000 7f0fe3d44640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0fdc0b7030 0x7f0fdc0978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.796+0000 7f0fe3d44640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0fdc0b7030 0x7f0fdc0978b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.797+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fd4009050 con 0x7f0fdc0b7030 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.797+0000 7f0fe3d44640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0fd4038040 0x7f0fd401bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.797+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb850f0 con 0x7f0fd4038040 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.797+0000 7f0fecd46640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0fd4038040 0x7f0fd401bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.797+0000 7f0fecd46640 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0fd4038040 0x7f0fd401bab0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.797+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f0fe400a4a0 con 0x7f0fd4038040 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.797+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb7c3d0 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.798+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401bb7c3d0 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.798+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb7c3d0 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.798+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401bb7c3d0 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.798+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fd4038040 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.798+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe400a4a0 con 0x7f0fd4038040 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.798+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fd4038040 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.798+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe400a4a0 con 0x7f0fd4038040 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.798+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fdc0b7030 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.799+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fd4009050 con 0x7f0fdc0b7030 2024-04-27T03:30:30.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.799+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb80dc0 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.799+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.799+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb7c3d0 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.799+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401bb7c3d0 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.799+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fd4038040 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.799+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe400a4a0 con 0x7f0fd4038040 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.799+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb7c3d0 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.800+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401bb7c3d0 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.800+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fdc0b7030 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.800+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0fd4009050 con 0x7f0fdc0b7030 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.800+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb80dc0 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.800+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.800+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb80dc0 2024-04-27T03:30:30.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.800+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.800+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fd4038040 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.801+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe400a4a0 con 0x7f0fd4038040 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.801+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb80dc0 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.801+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.801+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb80dc0 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.801+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.801+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb7c3d0 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.801+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401bb7c3d0 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.801+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb7c3d0 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.802+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0fe400a4a0 con 0x56401bb7c3d0 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.802+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fd4038040 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.802+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe4033040 con 0x7f0fd4038040 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.802+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb80dc0 2024-04-27T03:30:30.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.802+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.802+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fdc0b7030 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.802+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fd4009050 con 0x7f0fdc0b7030 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.802+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb7c3d0 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.803+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.803+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fd4038040 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.803+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.803+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb80dc0 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.803+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.803+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fd4038040 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.803+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.803+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fdc0b7030 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.804+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fd4009050 con 0x7f0fdc0b7030 2024-04-27T03:30:30.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.804+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb80dc0 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.804+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.804+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fd4038040 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.804+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.804+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x56401bb7c3d0 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56401bb80060 con 0x7f0fdc0b7030 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0fd4045040 con 0x7f0fdc0b7030 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:30.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb84080 con 0x56401bb7c3d0 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb84470 con 0x56401bb7c3d0 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.805+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb84470 con 0x7f0fd4038040 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0feea3f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0feea3f800 20 zone default found 2024-04-27T03:30:30.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0feea3f800 4 Realm: () 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0feea3f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0feea3f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0feea3f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0feea3f800 20 zonegroup default 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0feea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb83c00 con 0x7f0fd4038040 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0feea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0feea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.806+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb86f50 con 0x56401bb80dc0 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.807+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.807+0000 7f0feea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.807+0000 7f0feea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.807+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb872f0 con 0x56401bb80dc0 2024-04-27T03:30:30.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.807+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.807+0000 7f0feea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.807+0000 7f0feea3f800 20 started sync module instance, tier type = 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.807+0000 7f0feea3f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.807+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb863f0 con 0x7f0fdc0b7030 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.808+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fd4045040 con 0x7f0fdc0b7030 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.808+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94833342963520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb87690 con 0x7f0fdc0b7030 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.808+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb87f80 con 0x56401bb7c3d0 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.809+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94833342963520] v25'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fd4045040 con 0x7f0fdc0b7030 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.809+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.809+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94833342970416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb88370 con 0x56401bb7c3d0 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.809+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb88dd0 con 0x56401bb80dc0 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.810+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94833342970416] v25'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.810+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.810+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94833342965776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb89170 con 0x56401bb80dc0 2024-04-27T03:30:30.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.810+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb89c20 con 0x7f0fdc0b7030 2024-04-27T03:30:30.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.811+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94833342965776] v25'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.811+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fd4045040 con 0x7f0fdc0b7030 2024-04-27T03:30:30.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.811+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94833342974128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb8a4c0 con 0x7f0fdc0b7030 2024-04-27T03:30:30.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.811+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb8ae80 con 0x56401bb7c3d0 2024-04-27T03:30:30.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.812+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.812+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94833342983088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb8b640 con 0x56401bb7c3d0 2024-04-27T03:30:30.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.812+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb8c160 con 0x7f0fd4038040 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.813+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94833342974128] v25'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fd4045040 con 0x7f0fdc0b7030 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.813+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.813+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94833342987312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb8c120 con 0x7f0fd4038040 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.813+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb8cb90 con 0x7f0fd4038040 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.813+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94833342983088] v25'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.814+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.814+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94833342990016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb8cb90 con 0x7f0fd4038040 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.814+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb8d550 con 0x7f0fd4038040 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.815+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94833342987312] v25'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.815+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94833342990016] v25'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.815+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'33 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.815+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94833342984336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb8d0a0 con 0x7f0fd4038040 2024-04-27T03:30:30.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.815+0000 7f0feea3f800 20 add_watcher() i=0 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.815+0000 7f0feea3f800 20 add_watcher() i=1 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.815+0000 7f0feea3f800 20 add_watcher() i=2 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.815+0000 7f0feea3f800 20 add_watcher() i=3 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.815+0000 7f0feea3f800 20 add_watcher() i=4 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.815+0000 7f0feea3f800 20 add_watcher() i=5 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.815+0000 7f0feea3f800 20 add_watcher() i=6 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.816+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94833342984336] v25'34 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.816+0000 7f0feea3f800 20 add_watcher() i=7 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.816+0000 7f0feea3f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.816+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bb90580 con 0x56401bb7c3d0 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.817+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.817+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94833342992544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb8d9a0 con 0x56401bb7c3d0 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.818+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94833342992544] v25'62 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.818+0000 7f0ea27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.818+0000 7f0feea3f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.818+0000 7f0feea3f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.818+0000 7f0feea3f800 5 note: GC not initialized 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0ea17e2640 20 reqs_thread_entry: start 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0feea3f800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0feea3f800 20 Filter name: none 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0e12ffd640 20 reqs_thread_entry: start 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0feea3f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0feea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bbc3650 con 0x56401bb7c3d0 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0feea3f800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0feea3f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0feea3f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:30.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0feea3f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bbc3130 con 0x7f0fdc0b7030 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.819+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7f0fd4045040 con 0x7f0fdc0b7030 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.820+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bbc9c20 con 0x56401bb7c3d0 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~391] v25'2 uv2 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0feea3f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0feea3f800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0feea3f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94833343239888 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56401bbc5270 con 0x56401bb7c3d0 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify (1) cookie 94833342983088 notify 107374182401 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7f0fe401e040 con 0x56401bb7c3d0 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0ecb7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182401 cookie 94833342983088 notifier 4301 bl.length()=567 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(65 notify.4 [notify cookie 94833343239888 out=8b] v0'0 uv33 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0ecb7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:30.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0ecb7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:30.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0ecb7fe640 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f0eb0003ea0 con 0x56401bb7c3d0 2024-04-27T03:30:30.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.821+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== watch-notify(notify_complete (2) cookie 94833343239888 notify 107374182401 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f0fe4018dd0 con 0x56401bb7c3d0 2024-04-27T03:30:30.838 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.822+0000 7f0ea27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.822+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94833342992544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bbc59a0 con 0x56401bb7c3d0 2024-04-27T03:30:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.823+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94833342992544] v25'63 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.823+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94833342963520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bbc59a0 con 0x7f0fdc0b7030 2024-04-27T03:30:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.826+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94833342963520] v25'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fd4045040 con 0x7f0fdc0b7030 2024-04-27T03:30:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.826+0000 7f0feea3f800 20 remove_watcher() i=0 2024-04-27T03:30:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.826+0000 7f0feea3f800 2 removed watcher, disabling cache 2024-04-27T03:30:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.826+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94833342970416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb85f40 con 0x56401bb7c3d0 2024-04-27T03:30:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.827+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94833342970416] v25'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.827+0000 7f0feea3f800 20 remove_watcher() i=1 2024-04-27T03:30:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.827+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94833342965776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb85f40 con 0x56401bb80dc0 2024-04-27T03:30:30.844 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.828+0000 7f0fec545640 1 -- 172.21.15.165:0/1244022057 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94833342965776] v25'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fdc0bd020 con 0x56401bb80dc0 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.828+0000 7f0feea3f800 20 remove_watcher() i=2 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.828+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94833342974128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb86810 con 0x7f0fdc0b7030 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.829+0000 7f0fe3d44640 1 -- 172.21.15.165:0/1244022057 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94833342974128] v25'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fd4045040 con 0x7f0fdc0b7030 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.829+0000 7f0feea3f800 20 remove_watcher() i=3 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.829+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94833342983088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb85f40 con 0x56401bb7c3d0 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.830+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94833342983088] v25'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x56401bb7c3d0 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.830+0000 7f0feea3f800 20 remove_watcher() i=4 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.830+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94833342987312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb888b0 con 0x7f0fd4038040 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.831+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94833342987312] v25'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.831+0000 7f0feea3f800 20 remove_watcher() i=5 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.831+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94833342990016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb86810 con 0x7f0fd4038040 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.832+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94833342990016] v25'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.832+0000 7f0feea3f800 20 remove_watcher() i=6 2024-04-27T03:30:30.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.832+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94833342984336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56401bb8abb0 con 0x7f0fd4038040 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94833342984336] v25'35 uv33 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0fe4031040 con 0x7f0fd4038040 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 20 remove_watcher() i=7 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0fd4038040 msgr2=0x7f0fd401bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0fd4038040 0x7f0fd401bab0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56401bb7c3d0 msgr2=0x56401bb7e890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56401bb7c3d0 0x56401bb7e890 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0fdc0b7030 msgr2=0x7f0fdc0978b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0fdc0b7030 0x7f0fdc0978b0 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56401bb80dc0 msgr2=0x56401bb831b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56401bb80dc0 0x56401bb831b0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ebc036950 msgr2=0x7f0ebc038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ebc036950 0x7f0ebc038e10 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4001b60 tx=0x7f0fd4010000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 msgr2=0x56401bb70c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1244022057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb69810 0x56401bb70c80 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc052520 tx=0x7f0fdc082f50 comp rx=0 tx=0).stop 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 reap_dead start 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.833+0000 7f0fecd46640 1 -- 172.21.15.165:0/1244022057 reap_dead start 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 shutdown_connections 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 >> 172.21.15.165:0/1244022057 conn(0x56401bb65b70 msgr2=0x56401bb631d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 shutdown_connections 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/1244022057 wait complete. 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/3291875113 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ec4036950 msgr2=0x7f0ec4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 --2- 172.21.15.165:0/3291875113 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ec4036950 0x7f0ec4038e10 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f0fd400bf50 tx=0x7f0fd400f000 comp rx=0 tx=0).stop 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/3291875113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb5c6f0 msgr2=0x56401bb5cb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 --2- 172.21.15.165:0/3291875113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb5c6f0 0x56401bb5cb10 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc052b60 tx=0x7f0fdc067420 comp rx=0 tx=0).stop 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/3291875113 shutdown_connections 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 --2- 172.21.15.165:0/3291875113 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ec4036950 0x7f0ec4038e10 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 --2- 172.21.15.165:0/3291875113 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401bb5c6f0 0x56401bb5cb10 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/3291875113 >> 172.21.15.165:0/3291875113 conn(0x56401bb50790 msgr2=0x56401bb50b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/3291875113 shutdown_connections 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/3291875113 wait complete. 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x56401bb4b6e0 msgr2=0x56401bb4db30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x56401bb4b6e0 0x56401bb4db30 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56401ba462a0 msgr2=0x56401bb44f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56401ba462a0 0x56401bb44f50 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56401bb477a0 msgr2=0x56401bb49b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56401bb477a0 0x56401bb49b90 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ed4036950 msgr2=0x7f0ed4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ed4036950 0x7f0ed4038e10 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0fd4000f60 tx=0x7f0fd40012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 msgr2=0x56401ba43210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 --2- 172.21.15.165:0/1203754155 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56401ba43970 0x56401ba43210 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f0fdc060040 tx=0x7f0fdc002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0fecd46640 1 -- 172.21.15.165:0/1203754155 reap_dead start 2024-04-27T03:30:30.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 shutdown_connections 2024-04-27T03:30:30.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.834+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 >> 172.21.15.165:0/1203754155 conn(0x56401ba3d320 msgr2=0x56401bb2e230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.835+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 shutdown_connections 2024-04-27T03:30:30.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.835+0000 7f0feea3f800 1 -- 172.21.15.165:0/1203754155 wait complete. 2024-04-27T03:30:30.850 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:30.850 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T03:30:30.850 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T03:30:30.850 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T03:30:30.933 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:30.934 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.929+0000 7f6a6bf1f800 1 Processor -- start 2024-04-27T03:30:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.929+0000 7f6a6bf1f800 1 -- start start 2024-04-27T03:30:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.929+0000 7f6a6bf1f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55c928ea42e0 0x55c928eacde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.929+0000 7f6a6bf1f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c928d62d40 con 0x55c928ea27a0 2024-04-27T03:30:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.929+0000 7f6a6bf1f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55c928d354d0 con 0x55c928ea42e0 2024-04-27T03:30:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.929+0000 7f6a5ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55c928ea42e0 0x55c928eacde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.929+0000 7f6a5ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55c928ea42e0 0x55c928eacde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51718/0 (socket says 172.21.15.165:51718) 2024-04-27T03:30:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.929+0000 7f6a5ffff640 1 -- 172.21.15.165:0/838909211 learned_addr learned my addr 172.21.15.165:0/838909211 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.929+0000 7f6a67be7640 1 --1- 172.21.15.165:0/838909211 >> v1:172.21.15.89:6789/0 conn(0x55c928ea27a0 0x55c928ea2bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53642/0 (socket says 172.21.15.165:53642) 2024-04-27T03:30:30.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.930+0000 7f6a5ffff640 1 -- 172.21.15.165:0/838909211 >> v1:172.21.15.89:6789/0 conn(0x55c928ea27a0 legacy=0x55c928ea2bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.930+0000 7f6a5ffff640 1 -- 172.21.15.165:0/838909211 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c928d61590 con 0x55c928ea42e0 2024-04-27T03:30:30.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.931+0000 7f6a5ffff640 1 --2- 172.21.15.165:0/838909211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea42e0 0x55c928eacde0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000a520 tx=0x7f6a5001d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73bde382a0514a30 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.932+0000 7f6a673e6640 1 -- 172.21.15.165:0/838909211 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a50057070 con 0x55c928ea42e0 2024-04-27T03:30:30.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.932+0000 7f6a673e6640 1 -- 172.21.15.165:0/838909211 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a50052440 con 0x55c928ea42e0 2024-04-27T03:30:30.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.932+0000 7f6a673e6640 1 -- 172.21.15.165:0/838909211 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a5004ae60 con 0x55c928ea42e0 2024-04-27T03:30:30.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.932+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/838909211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea42e0 msgr2=0x55c928eacde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.932+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/838909211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea42e0 0x55c928eacde0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f6a5000a520 tx=0x7f6a5001d010 comp rx=0 tx=0).stop 2024-04-27T03:30:30.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.932+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/838909211 shutdown_connections 2024-04-27T03:30:30.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.932+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/838909211 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea42e0 0x55c928eacde0 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.932+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/838909211 >> 172.21.15.165:0/838909211 conn(0x55c928e9e320 msgr2=0x55c928ea0740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.932+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/838909211 shutdown_connections 2024-04-27T03:30:30.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.932+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/838909211 wait complete. 2024-04-27T03:30:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.944+0000 7f6a6bf1f800 1 Processor -- start 2024-04-27T03:30:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.944+0000 7f6a6bf1f800 1 -- start start 2024-04-27T03:30:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.944+0000 7f6a6bf1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 0x55c928f974b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.944+0000 7f6a6bf1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c928def3c0 con 0x55c928ea27a0 2024-04-27T03:30:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.944+0000 7f6a67be7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 0x55c928f974b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.944+0000 7f6a67be7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 0x55c928f974b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51734/0 (socket says 172.21.15.165:51734) 2024-04-27T03:30:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.944+0000 7f6a67be7640 1 -- 172.21.15.165:0/3013392323 learned_addr learned my addr 172.21.15.165:0/3013392323 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.945+0000 7f6a67be7640 1 -- 172.21.15.165:0/3013392323 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c928f97a80 con 0x55c928ea27a0 2024-04-27T03:30:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.945+0000 7f6a67be7640 1 --2- 172.21.15.165:0/3013392323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 0x55c928f974b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6a58001e10 tx=0x7f6a5800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c78f23ceeeb6a406 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.945+0000 7f6955e0b640 1 -- 172.21.15.165:0/3013392323 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a5800e5c0 con 0x55c928ea27a0 2024-04-27T03:30:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.945+0000 7f6955e0b640 1 -- 172.21.15.165:0/3013392323 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a58006070 con 0x55c928ea27a0 2024-04-27T03:30:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.945+0000 7f6955e0b640 1 -- 172.21.15.165:0/3013392323 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a5800e930 con 0x55c928ea27a0 2024-04-27T03:30:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3013392323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 msgr2=0x55c928f974b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/3013392323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 0x55c928f974b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f6a58001e10 tx=0x7f6a5800b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3013392323 shutdown_connections 2024-04-27T03:30:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/3013392323 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 0x55c928f974b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3013392323 >> 172.21.15.165:0/3013392323 conn(0x55c928e9e320 msgr2=0x55c928f92dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3013392323 shutdown_connections 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3013392323 wait complete. 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a6bf1f800 1 Processor -- start 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a6bf1f800 1 -- start start 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a6bf1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 0x55c928f9f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a6bf1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c928eaf600 con 0x55c928ea27a0 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a67be7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 0x55c928f9f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a67be7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 0x55c928f9f050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51746/0 (socket says 172.21.15.165:51746) 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.946+0000 7f6a67be7640 1 -- 172.21.15.165:0/3407675811 learned_addr learned my addr 172.21.15.165:0/3407675811 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.947+0000 7f6a67be7640 1 -- 172.21.15.165:0/3407675811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c928f9f590 con 0x55c928ea27a0 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.947+0000 7f6a67be7640 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 0x55c928f9f050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800b700 tx=0x7f6a58000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.947+0000 7f694ffff640 1 -- 172.21.15.165:0/3407675811 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a5800f8d0 con 0x55c928ea27a0 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.947+0000 7f694ffff640 1 -- 172.21.15.165:0/3407675811 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a5800fa70 con 0x55c928ea27a0 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.947+0000 7f694ffff640 1 -- 172.21.15.165:0/3407675811 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a5800fd50 con 0x55c928ea27a0 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.947+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c928f9f820 con 0x55c928ea27a0 2024-04-27T03:30:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.947+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c928fa24d0 con 0x55c928ea27a0 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.948+0000 7f694ffff640 1 -- 172.21.15.165:0/3407675811 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6a58017070 con 0x55c928ea27a0 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.948+0000 7f694ffff640 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6950036900 0x7f6950038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.948+0000 7f694ffff640 1 -- 172.21.15.165:0/3407675811 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f6a58015020 con 0x55c928ea27a0 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.948+0000 7f6a5ffff640 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6950036900 0x7f6950038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.948+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c928ea42e0 0x55c928fa6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.948+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fa6860 con 0x55c928ea42e0 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.948+0000 7f6a683e8640 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c928ea42e0 0x55c928fa6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.948+0000 7f6a5ffff640 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6950036900 0x7f6950038dc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6a50054000 tx=0x7f6a5004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.949+0000 7f6a683e8640 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c928ea42e0 0x55c928fa6290 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.949+0000 7f6a683e8640 1 -- 172.21.15.165:0/3407675811 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928ea42e0 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.949+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fa3860 con 0x55c928ea42e0 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.949+0000 7f6a683e8640 1 -- 172.21.15.165:0/3407675811 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928ea42e0 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.949+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c928fa8b70 0x55c928faaf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.949+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fab600 con 0x55c928fa8b70 2024-04-27T03:30:30.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.949+0000 7f6a67be7640 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c928fa8b70 0x55c928faaf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.950+0000 7f6a67be7640 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c928fa8b70 0x55c928faaf60 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.950+0000 7f6a67be7640 1 -- 172.21.15.165:0/3407675811 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fa8b70 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.950+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fa7b20 con 0x55c928ea42e0 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.950+0000 7f6a683e8640 1 -- 172.21.15.165:0/3407675811 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928ea42e0 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.950+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c928facab0 0x55c928faef00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.950+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928faf660 con 0x55c928facab0 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.950+0000 7f6a5ffff640 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c928facab0 0x55c928faef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.951+0000 7f6a5ffff640 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c928facab0 0x55c928faef00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.951+0000 7f6a5ffff640 1 -- 172.21.15.165:0/3407675811 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f6a5005a040 con 0x55c928facab0 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.951+0000 7f6a6bf1f800 1 Processor -- start 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.951+0000 7f6a6bf1f800 1 -- start start 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.952+0000 7f6a6bf1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fbdac0 0x55c928fbdee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.952+0000 7f6a6bf1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c928d956b0 con 0x55c928fbdac0 2024-04-27T03:30:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.952+0000 7f6a67be7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fbdac0 0x55c928fbdee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.952+0000 7f6a67be7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fbdac0 0x55c928fbdee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51750/0 (socket says 172.21.15.165:51750) 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.952+0000 7f6a67be7640 1 -- 172.21.15.165:0/1755866170 learned_addr learned my addr 172.21.15.165:0/1755866170 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.952+0000 7f6a67be7640 1 -- 172.21.15.165:0/1755866170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c928fbe420 con 0x55c928fbdac0 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.952+0000 7f6a67be7640 1 --2- 172.21.15.165:0/1755866170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fbdac0 0x55c928fbdee0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6a5804b4d0 tx=0x7f6a58017d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.952+0000 7f694d7fa640 1 -- 172.21.15.165:0/1755866170 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a5801ae10 con 0x55c928fbdac0 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.952+0000 7f694d7fa640 1 -- 172.21.15.165:0/1755866170 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a5801a690 con 0x55c928fbdac0 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.952+0000 7f694d7fa640 1 -- 172.21.15.165:0/1755866170 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a58044b00 con 0x55c928fbdac0 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.952+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1755866170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c928fbee90 con 0x55c928fbdac0 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.952+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1755866170 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c928fbf390 con 0x55c928fbdac0 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.953+0000 7f694d7fa640 1 -- 172.21.15.165:0/1755866170 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6a58044ca0 con 0x55c928fbdac0 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.953+0000 7f694d7fa640 1 --2- 172.21.15.165:0/1755866170 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6940036950 0x7f6940038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.953+0000 7f694d7fa640 1 -- 172.21.15.165:0/1755866170 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f6a5803cae0 con 0x55c928fbdac0 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.953+0000 7f6a5ffff640 1 --2- 172.21.15.165:0/1755866170 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6940036950 0x7f6940038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.954+0000 7f6a6bf1f800 1 Processor -- start 2024-04-27T03:30:30.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.954+0000 7f6a6bf1f800 1 -- start start 2024-04-27T03:30:30.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.954+0000 7f6a5ffff640 1 --2- 172.21.15.165:0/1755866170 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6940036950 0x7f6940038e10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55c928fa3780 tx=0x7f6a50005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.954+0000 7f6a6bf1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 0x55c928fcb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.954+0000 7f6a6bf1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c928f97e00 con 0x55c928fcac70 2024-04-27T03:30:30.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.954+0000 7f6a67be7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 0x55c928fcb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.954+0000 7f6a67be7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 0x55c928fcb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51754/0 (socket says 172.21.15.165:51754) 2024-04-27T03:30:30.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.954+0000 7f6a67be7640 1 -- 172.21.15.165:0/1850016671 learned_addr learned my addr 172.21.15.165:0/1850016671 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.954+0000 7f6a67be7640 1 -- 172.21.15.165:0/1850016671 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c928fcb660 con 0x55c928fcac70 2024-04-27T03:30:30.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.954+0000 7f6a67be7640 1 --2- 172.21.15.165:0/1850016671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 0x55c928fcb090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800eb60 tx=0x7f6a5801aa80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ded37c4ca1c0ca50 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6946ffd640 1 -- 172.21.15.165:0/1850016671 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a5801dc30 con 0x55c928fcac70 2024-04-27T03:30:30.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6946ffd640 1 -- 172.21.15.165:0/1850016671 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a5805e070 con 0x55c928fcac70 2024-04-27T03:30:30.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6946ffd640 1 -- 172.21.15.165:0/1850016671 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a5803cd80 con 0x55c928fcac70 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1850016671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 msgr2=0x55c928fcb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/1850016671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 0x55c928fcb090 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800eb60 tx=0x7f6a5801aa80 comp rx=0 tx=0).stop 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1850016671 shutdown_connections 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/1850016671 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 0x55c928fcb090 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1850016671 >> 172.21.15.165:0/1850016671 conn(0x55c928fc6f40 msgr2=0x55c928fc4fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1850016671 shutdown_connections 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1850016671 wait complete. 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6a6bf1f800 1 Processor -- start 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6a6bf1f800 1 -- start start 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.955+0000 7f6a6bf1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 0x55c928fd1fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.956+0000 7f6a6bf1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c928fcbcc0 con 0x55c928fcac70 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.956+0000 7f6a67be7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 0x55c928fd1fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.956+0000 7f6a67be7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 0x55c928fd1fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51770/0 (socket says 172.21.15.165:51770) 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.956+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 learned_addr learned my addr 172.21.15.165:0/2675608173 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:30.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.956+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c928fd24f0 con 0x55c928fcac70 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.956+0000 7f6a67be7640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 0x55c928fd1fb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6a5802c650 tx=0x7f6a5803ef50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.956+0000 7f69457fa640 1 -- 172.21.15.165:0/2675608173 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a5806c020 con 0x55c928fcac70 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.956+0000 7f69457fa640 1 -- 172.21.15.165:0/2675608173 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6a5803cd30 con 0x55c928fcac70 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.956+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c928fd24f0 con 0x55c928fcac70 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.956+0000 7f69457fa640 1 -- 172.21.15.165:0/2675608173 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6a5803f170 con 0x55c928fcac70 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.956+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c928fd24f0 con 0x55c928fcac70 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.957+0000 7f69457fa640 1 -- 172.21.15.165:0/2675608173 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6a5806f020 con 0x55c928fcac70 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.957+0000 7f69457fa640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6930036950 0x7f6930038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.957+0000 7f6a6bf1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.957+0000 7f69457fa640 1 -- 172.21.15.165:0/2675608173 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f6a58055070 con 0x55c928fcac70 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.957+0000 7f6a5ffff640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6930036950 0x7f6930038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.957+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c928fdd450 0x55c928fdf910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.957+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe0060 con 0x55c928fdd450 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a683e8640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c928fdd450 0x55c928fdf910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a5ffff640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6930036950 0x7f6930038e10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55c928fafa60 tx=0x7f6a5004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a683e8640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c928fdd450 0x55c928fdf910 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a6bf1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a6bf1f800 20 realm 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a6bf1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe0470 con 0x55c928fdd450 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a6bf1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a6bf1f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a6bf1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.958+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe0d10 con 0x55c928fdd450 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.959+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.959+0000 7f6a6bf1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.959+0000 7f6a6bf1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.959+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe10b0 con 0x55c928fdd450 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.959+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.959+0000 7f6a6bf1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.959+0000 7f6a6bf1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.959+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c928fe1e10 0x55c928fe4200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.959+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe4740 con 0x55c928fe1e10 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.959+0000 7f6a67be7640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c928fe1e10 0x55c928fe4200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.959+0000 7f6a67be7640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c928fe1e10 0x55c928fe4200 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.960+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.960+0000 7f6a6bf1f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.960+0000 7f6a6bf1f800 20 searching for the correct realm 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.960+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe60e0 con 0x55c928fe1e10 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.960+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.960+0000 7f6a67be7640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6a5807c040 0x7f6a58021ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.960+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe60e0 con 0x7f6a5807c040 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.960+0000 7f6a5ffff640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6a5807c040 0x7f6a58021ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.961+0000 7f6a5ffff640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6a5807c040 0x7f6a58021ab0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.961+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a5007f070 con 0x7f6a5807c040 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.961+0000 7f6a5ffff640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6a50074040 0x7f6a50081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.961+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe60e0 con 0x7f6a50074040 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.961+0000 7f6a683e8640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6a50074040 0x7f6a50081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.961+0000 7f6a683e8640 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6a50074040 0x7f6a50081410 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.962+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.962+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe10b0 con 0x55c928fdd450 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.962+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.962+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe10b0 con 0x55c928fdd450 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.962+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.962+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe10b0 con 0x7f6a50074040 2024-04-27T03:30:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.962+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.962+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe10b0 con 0x7f6a50074040 2024-04-27T03:30:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.963+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.963+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe10b0 con 0x7f6a5807c040 2024-04-27T03:30:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.963+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a5007f070 con 0x7f6a5807c040 2024-04-27T03:30:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.963+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe10b0 con 0x55c928fe1e10 2024-04-27T03:30:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.963+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.963+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe10b0 con 0x55c928fdd450 2024-04-27T03:30:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.963+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.963+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe10b0 con 0x7f6a50074040 2024-04-27T03:30:30.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.964+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.964+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe10b0 con 0x55c928fdd450 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.964+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.964+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe10b0 con 0x7f6a5807c040 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.964+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f6a5007f070 con 0x7f6a5807c040 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.964+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x55c928fe1e10 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.964+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.964+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x55c928fe1e10 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.965+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.965+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x7f6a50074040 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.965+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.965+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x55c928fe1e10 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.965+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.965+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x55c928fe1e10 2024-04-27T03:30:30.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.965+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.965+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x55c928fdd450 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.966+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.966+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x55c928fdd450 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.966+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.966+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x7f6a50074040 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.966+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.966+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x55c928fe1e10 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.966+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.966+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x7f6a5807c040 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.967+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a5007f070 con 0x7f6a5807c040 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.967+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x55c928fdd450 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.967+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.967+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x7f6a50074040 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.967+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.967+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x55c928fe1e10 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.968+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.968+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x7f6a50074040 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.968+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.968+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x7f6a5807c040 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.968+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a5007f070 con 0x7f6a5807c040 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.968+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x55c928fe1e10 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.968+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.968+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x7f6a50074040 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a5008e060 con 0x55c928fdd450 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f6a60038040 con 0x7f6a5807c040 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6a5007f070 con 0x7f6a5807c040 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a6bf1f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a6bf1f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a6bf1f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a6bf1f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a6bf1f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a6bf1f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a6bf1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe50b0 con 0x55c928fdd450 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a6bf1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a6bf1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.969+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe54a0 con 0x55c928fdd450 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a6bf1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a6bf1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe54a0 con 0x7f6a50074040 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a6bf1f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a6bf1f800 20 zone default found 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a6bf1f800 4 Realm: () 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a6bf1f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a6bf1f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a6bf1f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a6bf1f800 20 zonegroup default 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a6bf1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.970+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe58c0 con 0x7f6a50074040 2024-04-27T03:30:30.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a6bf1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a6bf1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe58c0 con 0x55c928fe1e10 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a6bf1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a6bf1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe7f90 con 0x55c928fe1e10 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a6bf1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a6bf1f800 20 started sync module instance, tier type = 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a6bf1f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.971+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe7430 con 0x7f6a5807c040 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.972+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a5007f070 con 0x7f6a5807c040 2024-04-27T03:30:30.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.972+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94322464552992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe87e0 con 0x7f6a5807c040 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.972+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe9030 con 0x55c928fdd450 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.973+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94322464552992] v25'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a5007f070 con 0x7f6a5807c040 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.973+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.973+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94322464555824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe90b0 con 0x55c928fdd450 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.973+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe9bd0 con 0x55c928fe1e10 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.975+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94322464555824] v25'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.975+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.975+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94322464560800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe9fc0 con 0x55c928fe1e10 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.975+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fea9c0 con 0x7f6a5807c040 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.978+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94322464560800] v25'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.978+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a5007f070 con 0x7f6a5807c040 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.978+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94322464557952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928feb260 con 0x7f6a5807c040 2024-04-27T03:30:30.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.978+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928febc20 con 0x55c928fdd450 2024-04-27T03:30:30.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.979+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.979+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94322464569680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fec400 con 0x55c928fdd450 2024-04-27T03:30:30.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.979+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94322464557952] v25'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a5007f070 con 0x7f6a5807c040 2024-04-27T03:30:30.993 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.979+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928feadf0 con 0x7f6a50074040 2024-04-27T03:30:30.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.980+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.980+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94322464573936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928feadf0 con 0x7f6a50074040 2024-04-27T03:30:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.980+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94322464569680] v25'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.980+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928febeb0 con 0x7f6a50074040 2024-04-27T03:30:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.981+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.981+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94322464571056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fed8e0 con 0x7f6a50074040 2024-04-27T03:30:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.981+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fee320 con 0x7f6a50074040 2024-04-27T03:30:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.981+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94322464573936] v25'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.982+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'36 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.982+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94322464579056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fee2e0 con 0x7f6a50074040 2024-04-27T03:30:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.982+0000 7f6a6bf1f800 20 add_watcher() i=0 2024-04-27T03:30:30.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.982+0000 7f6a6bf1f800 20 add_watcher() i=1 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.982+0000 7f6a6bf1f800 20 add_watcher() i=2 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.982+0000 7f6a6bf1f800 20 add_watcher() i=3 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.982+0000 7f6a6bf1f800 20 add_watcher() i=4 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.982+0000 7f6a6bf1f800 20 add_watcher() i=5 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.983+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94322464571056] v25'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.983+0000 7f6a6bf1f800 20 add_watcher() i=6 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.983+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94322464579056] v25'37 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.983+0000 7f6a6bf1f800 20 add_watcher() i=7 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.983+0000 7f6a6bf1f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.983+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c928fed430 con 0x55c928fdd450 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.983+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.983+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94322464574992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fed430 con 0x55c928fdd450 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.984+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94322464574992] v25'64 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.984+0000 7f691dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.984+0000 7f6a6bf1f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.984+0000 7f6a6bf1f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.984+0000 7f6a6bf1f800 5 note: GC not initialized 2024-04-27T03:30:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.985+0000 7f691cfe1640 20 reqs_thread_entry: start 2024-04-27T03:30:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.985+0000 7f6a6bf1f800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.985+0000 7f6a6bf1f800 20 Filter name: none 2024-04-27T03:30:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.985+0000 7f68967fc640 20 reqs_thread_entry: start 2024-04-27T03:30:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.985+0000 7f6a6bf1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.985+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c929024100 con 0x55c928fdd450 2024-04-27T03:30:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.985+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:30.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.985+0000 7f6a6bf1f800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:31.000 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:31.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.986+0000 7f691dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.986+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94322464574992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c929023090 con 0x55c928fdd450 2024-04-27T03:30:31.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.987+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94322464574992] v25'65 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:31.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.987+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94322464552992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fece10 con 0x7f6a5807c040 2024-04-27T03:30:31.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.988+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94322464552992] v25'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a5007f070 con 0x7f6a5807c040 2024-04-27T03:30:31.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.988+0000 7f6a6bf1f800 20 remove_watcher() i=0 2024-04-27T03:30:31.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.988+0000 7f6a6bf1f800 2 removed watcher, disabling cache 2024-04-27T03:30:31.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.988+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94322464555824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fece10 con 0x55c928fdd450 2024-04-27T03:30:31.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.989+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94322464555824] v25'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.989+0000 7f6a6bf1f800 20 remove_watcher() i=1 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.989+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94322464560800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe8330 con 0x55c928fe1e10 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.990+0000 7f6a67be7640 1 -- 172.21.15.165:0/2675608173 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94322464560800] v25'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a58046040 con 0x55c928fe1e10 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.990+0000 7f6a6bf1f800 20 remove_watcher() i=2 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.990+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94322464557952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fece10 con 0x7f6a5807c040 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.991+0000 7f6a5ffff640 1 -- 172.21.15.165:0/2675608173 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94322464557952] v25'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a5007f070 con 0x7f6a5807c040 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.991+0000 7f6a6bf1f800 20 remove_watcher() i=3 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.991+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94322464569680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe8b80 con 0x55c928fdd450 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.992+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94322464569680] v25'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x55c928fdd450 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.992+0000 7f6a6bf1f800 20 remove_watcher() i=4 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.993+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94322464573936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fece10 con 0x7f6a50074040 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.993+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94322464573936] v25'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:31.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.993+0000 7f6a6bf1f800 20 remove_watcher() i=5 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.993+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94322464571056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928fe8b80 con 0x7f6a50074040 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.994+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94322464571056] v25'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.994+0000 7f6a6bf1f800 20 remove_watcher() i=6 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.994+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94322464579056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c928febeb0 con 0x7f6a50074040 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94322464579056] v25'38 uv36 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6a6000a4a0 con 0x7f6a50074040 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 20 remove_watcher() i=7 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6a50074040 msgr2=0x7f6a50081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6a50074040 0x7f6a50081410 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c928fdd450 msgr2=0x55c928fdf910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c928fdd450 0x55c928fdf910 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6a5807c040 msgr2=0x7f6a58021ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6a5807c040 0x7f6a58021ab0 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c928fe1e10 msgr2=0x55c928fe4200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c928fe1e10 0x55c928fe4200 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6930036950 msgr2=0x7f6930038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6930036950 0x7f6930038e10 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x55c928fafa60 tx=0x7f6a5004f000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a683e8640 1 -- 172.21.15.165:0/2675608173 reap_dead start 2024-04-27T03:30:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 msgr2=0x55c928fd1fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.995+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 0x55c928fd1fb0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f6a5802c650 tx=0x7f6a5803ef50 comp rx=0 tx=0).stop 2024-04-27T03:30:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 shutdown_connections 2024-04-27T03:30:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/2675608173 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fcac70 0x55c928fd1fb0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 >> 172.21.15.165:0/2675608173 conn(0x55c928fc6f40 msgr2=0x55c928fc45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 shutdown_connections 2024-04-27T03:30:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/2675608173 wait complete. 2024-04-27T03:30:31.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1755866170 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6940036950 msgr2=0x7f6940038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/1755866170 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6940036950 0x7f6940038e10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x55c928fa3780 tx=0x7f6a50005ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1755866170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fbdac0 msgr2=0x55c928fbdee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/1755866170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fbdac0 0x55c928fbdee0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f6a5804b4d0 tx=0x7f6a58017d50 comp rx=0 tx=0).stop 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1755866170 shutdown_connections 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/1755866170 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6940036950 0x7f6940038e10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/1755866170 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928fbdac0 0x55c928fbdee0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1755866170 >> 172.21.15.165:0/1755866170 conn(0x55c928fb1b60 msgr2=0x55c928fb1f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1755866170 shutdown_connections 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/1755866170 wait complete. 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c928facab0 msgr2=0x55c928faef00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c928facab0 0x55c928faef00 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c928ea42e0 msgr2=0x55c928fa6290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c928ea42e0 0x55c928fa6290 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c928fa8b70 msgr2=0x55c928faaf60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c928fa8b70 0x55c928faaf60 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6950036900 msgr2=0x7f6950038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6950036900 0x7f6950038dc0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6a50054000 tx=0x7f6a5004e000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 msgr2=0x55c928f9f050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 --2- 172.21.15.165:0/3407675811 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c928ea27a0 0x55c928f9f050 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f6a5800b700 tx=0x7f6a58000bc0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a683e8640 1 -- 172.21.15.165:0/3407675811 reap_dead start 2024-04-27T03:30:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 shutdown_connections 2024-04-27T03:30:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.996+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 >> 172.21.15.165:0/3407675811 conn(0x55c928e9e320 msgr2=0x55c928f915d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.997+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 shutdown_connections 2024-04-27T03:30:31.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:30.997+0000 7f6a6bf1f800 1 -- 172.21.15.165:0/3407675811 wait complete. 2024-04-27T03:30:31.011 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:31.011 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-04-27T03:30:31.012 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-04-27T03:30:31.012 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-04-27T03:30:31.094 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:31.094 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.089+0000 7f2776b37800 1 Processor -- start 2024-04-27T03:30:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.089+0000 7f2776b37800 1 -- start start 2024-04-27T03:30:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.089+0000 7f2776b37800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b7ef86a1b0 0x55b7ef86c5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.089+0000 7f2776b37800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55b7ef725d40 con 0x55b7ef865d20 2024-04-27T03:30:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.089+0000 7f2776b37800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55b7ef6f84d0 con 0x55b7ef86a1b0 2024-04-27T03:30:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.089+0000 7f27720f8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b7ef86a1b0 0x55b7ef86c5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.089+0000 7f27728f9640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55b7ef865d20 0x55b7ef866140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53654/0 (socket says 172.21.15.165:53654) 2024-04-27T03:30:31.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.089+0000 7f27728f9640 1 -- 172.21.15.165:0/1793514054 learned_addr learned my addr 172.21.15.165:0/1793514054 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:31.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.089+0000 7f27720f8640 1 -- 172.21.15.165:0/1793514054 >> v1:172.21.15.89:6789/0 conn(0x55b7ef865d20 legacy=0x55b7ef866140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.089+0000 7f27720f8640 1 -- 172.21.15.165:0/1793514054 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ef724590 con 0x55b7ef86a1b0 2024-04-27T03:30:31.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.091+0000 7f27720f8640 1 --2- 172.21.15.165:0/1793514054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef86a1b0 0x55b7ef86c5a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f276800a470 tx=0x7f276801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf3987dd372a6fde server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.091+0000 7f27718f7640 1 -- 172.21.15.165:0/1793514054 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2768057070 con 0x55b7ef86a1b0 2024-04-27T03:30:31.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.091+0000 7f27718f7640 1 -- 172.21.15.165:0/1793514054 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2768052570 con 0x55b7ef86a1b0 2024-04-27T03:30:31.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.091+0000 7f27718f7640 1 -- 172.21.15.165:0/1793514054 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2768052870 con 0x55b7ef86a1b0 2024-04-27T03:30:31.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.092+0000 7f2776b37800 1 -- 172.21.15.165:0/1793514054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef86a1b0 msgr2=0x55b7ef86c5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.092+0000 7f2776b37800 1 --2- 172.21.15.165:0/1793514054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef86a1b0 0x55b7ef86c5a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f276800a470 tx=0x7f276801ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:31.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.092+0000 7f2776b37800 1 -- 172.21.15.165:0/1793514054 shutdown_connections 2024-04-27T03:30:31.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.092+0000 7f2776b37800 1 --2- 172.21.15.165:0/1793514054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef86a1b0 0x55b7ef86c5a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.092+0000 7f2776b37800 1 -- 172.21.15.165:0/1793514054 >> 172.21.15.165:0/1793514054 conn(0x55b7ef861380 msgr2=0x55b7ef8637a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.092+0000 7f2776b37800 1 -- 172.21.15.165:0/1793514054 shutdown_connections 2024-04-27T03:30:31.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.092+0000 7f2776b37800 1 -- 172.21.15.165:0/1793514054 wait complete. 2024-04-27T03:30:31.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.104+0000 7f2776b37800 1 Processor -- start 2024-04-27T03:30:31.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.104+0000 7f2776b37800 1 -- start start 2024-04-27T03:30:31.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.104+0000 7f2776b37800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 0x55b7ef7d9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.104+0000 7f2776b37800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b7ef7b23c0 con 0x55b7ef865d20 2024-04-27T03:30:31.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.104+0000 7f27728f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 0x55b7ef7d9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.104+0000 7f27728f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 0x55b7ef7d9ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51786/0 (socket says 172.21.15.165:51786) 2024-04-27T03:30:31.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.104+0000 7f27728f9640 1 -- 172.21.15.165:0/788500715 learned_addr learned my addr 172.21.15.165:0/788500715 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.105+0000 7f27728f9640 1 -- 172.21.15.165:0/788500715 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ef95ad80 con 0x55b7ef865d20 2024-04-27T03:30:31.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.105+0000 7f27728f9640 1 --2- 172.21.15.165:0/788500715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 0x55b7ef7d9ff0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2764001d80 tx=0x7f2764001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18349085e54e72c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.105+0000 7f2660e09640 1 -- 172.21.15.165:0/788500715 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2764013030 con 0x55b7ef865d20 2024-04-27T03:30:31.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.105+0000 7f2660e09640 1 -- 172.21.15.165:0/788500715 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f276400e470 con 0x55b7ef865d20 2024-04-27T03:30:31.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.105+0000 7f2660e09640 1 -- 172.21.15.165:0/788500715 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f276400e810 con 0x55b7ef865d20 2024-04-27T03:30:31.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f2776b37800 1 -- 172.21.15.165:0/788500715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 msgr2=0x55b7ef7d9ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f2776b37800 1 --2- 172.21.15.165:0/788500715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 0x55b7ef7d9ff0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2764001d80 tx=0x7f2764001e70 comp rx=0 tx=0).stop 2024-04-27T03:30:31.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f2776b37800 1 -- 172.21.15.165:0/788500715 shutdown_connections 2024-04-27T03:30:31.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f2776b37800 1 --2- 172.21.15.165:0/788500715 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 0x55b7ef7d9ff0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f2776b37800 1 -- 172.21.15.165:0/788500715 >> 172.21.15.165:0/788500715 conn(0x55b7ef861380 msgr2=0x55b7ef952870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f2776b37800 1 -- 172.21.15.165:0/788500715 shutdown_connections 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f2776b37800 1 -- 172.21.15.165:0/788500715 wait complete. 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f2776b37800 1 Processor -- start 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f2776b37800 1 -- start start 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f2776b37800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 0x55b7ef95fc60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f2776b37800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b7ef86ce70 con 0x55b7ef865d20 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f27728f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 0x55b7ef95fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f27728f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 0x55b7ef95fc60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51794/0 (socket says 172.21.15.165:51794) 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.106+0000 7f27728f9640 1 -- 172.21.15.165:0/3074051460 learned_addr learned my addr 172.21.15.165:0/3074051460 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.107+0000 7f27728f9640 1 -- 172.21.15.165:0/3074051460 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ef9601a0 con 0x55b7ef865d20 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.107+0000 7f27728f9640 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 0x55b7ef95fc60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2764004000 tx=0x7f2764000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.107+0000 7f265f606640 1 -- 172.21.15.165:0/3074051460 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2764027070 con 0x55b7ef865d20 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.107+0000 7f265f606640 1 -- 172.21.15.165:0/3074051460 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2764019900 con 0x55b7ef865d20 2024-04-27T03:30:31.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.107+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7ef960430 con 0x55b7ef865d20 2024-04-27T03:30:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.107+0000 7f265f606640 1 -- 172.21.15.165:0/3074051460 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2764019c60 con 0x55b7ef865d20 2024-04-27T03:30:31.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.107+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7ef9630e0 con 0x55b7ef865d20 2024-04-27T03:30:31.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.108+0000 7f265f606640 1 -- 172.21.15.165:0/3074051460 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f276402a070 con 0x55b7ef865d20 2024-04-27T03:30:31.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.108+0000 7f265f606640 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2658036900 0x7f2658038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.108+0000 7f265f606640 1 -- 172.21.15.165:0/3074051460 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f276401f070 con 0x55b7ef865d20 2024-04-27T03:30:31.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.108+0000 7f27720f8640 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2658036900 0x7f2658038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.108+0000 7f2776b37800 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b7ef86a1b0 0x55b7ef966ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.108+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef967470 con 0x55b7ef86a1b0 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.108+0000 7f27730fa640 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b7ef86a1b0 0x55b7ef966ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.108+0000 7f27720f8640 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2658036900 0x7f2658038dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2768054000 tx=0x7f276804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.109+0000 7f27730fa640 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b7ef86a1b0 0x55b7ef966ea0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.109+0000 7f27730fa640 1 -- 172.21.15.165:0/3074051460 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef86a1b0 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.109+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef964470 con 0x55b7ef86a1b0 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.109+0000 7f27730fa640 1 -- 172.21.15.165:0/3074051460 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef86a1b0 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.109+0000 7f2776b37800 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b7ef969780 0x55b7ef96bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.109+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef96c210 con 0x55b7ef969780 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.109+0000 7f27728f9640 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b7ef969780 0x55b7ef96bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.110+0000 7f27728f9640 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b7ef969780 0x55b7ef96bb70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.110+0000 7f27728f9640 1 -- 172.21.15.165:0/3074051460 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2764029030 con 0x55b7ef969780 2024-04-27T03:30:31.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.110+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef968730 con 0x55b7ef86a1b0 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.110+0000 7f27730fa640 1 -- 172.21.15.165:0/3074051460 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef86a1b0 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.110+0000 7f2776b37800 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b7ef96d6c0 0x55b7ef96fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.110+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef970270 con 0x55b7ef96d6c0 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.110+0000 7f27720f8640 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b7ef96d6c0 0x55b7ef96fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.111+0000 7f27720f8640 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b7ef96d6c0 0x55b7ef96fb10 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.111+0000 7f27720f8640 1 -- 172.21.15.165:0/3074051460 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2768052980 con 0x55b7ef96d6c0 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.111+0000 7f2776b37800 1 Processor -- start 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.111+0000 7f2776b37800 1 -- start start 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.112+0000 7f2776b37800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef97e660 0x55b7ef97ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.112+0000 7f2776b37800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b7ef7586b0 con 0x55b7ef97e660 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.112+0000 7f27728f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef97e660 0x55b7ef97ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.112+0000 7f27728f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef97e660 0x55b7ef97ea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51802/0 (socket says 172.21.15.165:51802) 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.112+0000 7f27728f9640 1 -- 172.21.15.165:0/3362430582 learned_addr learned my addr 172.21.15.165:0/3362430582 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.112+0000 7f27728f9640 1 -- 172.21.15.165:0/3362430582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ef97efc0 con 0x55b7ef97e660 2024-04-27T03:30:31.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.112+0000 7f27728f9640 1 --2- 172.21.15.165:0/3362430582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef97e660 0x55b7ef97ea80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2764016fd0 tx=0x7f276404c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.112+0000 7f265ce01640 1 -- 172.21.15.165:0/3362430582 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2764044c20 con 0x55b7ef97e660 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.112+0000 7f265ce01640 1 -- 172.21.15.165:0/3362430582 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2764044dc0 con 0x55b7ef97e660 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.112+0000 7f2776b37800 1 -- 172.21.15.165:0/3362430582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7ef97fa30 con 0x55b7ef97e660 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.112+0000 7f265ce01640 1 -- 172.21.15.165:0/3362430582 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f27640442b0 con 0x55b7ef97e660 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.112+0000 7f2776b37800 1 -- 172.21.15.165:0/3362430582 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7ef97ff30 con 0x55b7ef97e660 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.113+0000 7f265ce01640 1 -- 172.21.15.165:0/3362430582 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2764033070 con 0x55b7ef97e660 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.113+0000 7f265ce01640 1 --2- 172.21.15.165:0/3362430582 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f264c036950 0x7f264c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.113+0000 7f265ce01640 1 -- 172.21.15.165:0/3362430582 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2764014070 con 0x55b7ef97e660 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.113+0000 7f27720f8640 1 --2- 172.21.15.165:0/3362430582 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f264c036950 0x7f264c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.113+0000 7f2776b37800 1 Processor -- start 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.114+0000 7f2776b37800 1 -- start start 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.114+0000 7f27720f8640 1 --2- 172.21.15.165:0/3362430582 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f264c036950 0x7f264c038e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55b7ef964390 tx=0x7f2768005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.114+0000 7f2776b37800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 0x55b7ef958820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.114+0000 7f2776b37800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b7ef95b0d0 con 0x55b7ef958400 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.114+0000 7f27728f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 0x55b7ef958820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.114+0000 7f27728f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 0x55b7ef958820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51816/0 (socket says 172.21.15.165:51816) 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.114+0000 7f27728f9640 1 -- 172.21.15.165:0/2155487589 learned_addr learned my addr 172.21.15.165:0/2155487589 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.114+0000 7f27728f9640 1 -- 172.21.15.165:0/2155487589 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ef956bf0 con 0x55b7ef958400 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.114+0000 7f27728f9640 1 --2- 172.21.15.165:0/2155487589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 0x55b7ef958820 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2764019fd0 tx=0x7f276402cd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36a85eedf57cfe8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.114+0000 7f26567fc640 1 -- 172.21.15.165:0/2155487589 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f276405f070 con 0x55b7ef958400 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.114+0000 7f26567fc640 1 -- 172.21.15.165:0/2155487589 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2764049650 con 0x55b7ef958400 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.114+0000 7f26567fc640 1 -- 172.21.15.165:0/2155487589 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2764021d90 con 0x55b7ef958400 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.115+0000 7f2776b37800 1 -- 172.21.15.165:0/2155487589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 msgr2=0x55b7ef958820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.115+0000 7f2776b37800 1 --2- 172.21.15.165:0/2155487589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 0x55b7ef958820 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f2764019fd0 tx=0x7f276402cd10 comp rx=0 tx=0).stop 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.115+0000 7f2776b37800 1 -- 172.21.15.165:0/2155487589 shutdown_connections 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.115+0000 7f2776b37800 1 --2- 172.21.15.165:0/2155487589 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 0x55b7ef958820 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.115+0000 7f2776b37800 1 -- 172.21.15.165:0/2155487589 >> 172.21.15.165:0/2155487589 conn(0x55b7ef987ab0 msgr2=0x55b7ef985b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.115+0000 7f2776b37800 1 -- 172.21.15.165:0/2155487589 shutdown_connections 2024-04-27T03:30:31.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.115+0000 7f2776b37800 1 -- 172.21.15.165:0/2155487589 wait complete. 2024-04-27T03:30:31.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.115+0000 7f2776b37800 1 Processor -- start 2024-04-27T03:30:31.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.115+0000 7f2776b37800 1 -- start start 2024-04-27T03:30:31.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.115+0000 7f2776b37800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 0x55b7ef994e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.115+0000 7f2776b37800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b7ef957000 con 0x55b7ef958400 2024-04-27T03:30:31.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.116+0000 7f27728f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 0x55b7ef994e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.116+0000 7f27728f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 0x55b7ef994e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51822/0 (socket says 172.21.15.165:51822) 2024-04-27T03:30:31.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.116+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 learned_addr learned my addr 172.21.15.165:0/3570255385 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.116+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b7ef995380 con 0x55b7ef958400 2024-04-27T03:30:31.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.116+0000 7f27728f9640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 0x55b7ef994e40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2764019fd0 tx=0x7f2764017a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.116+0000 7f2654ff9640 1 -- 172.21.15.165:0/3570255385 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2764069030 con 0x55b7ef958400 2024-04-27T03:30:31.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.116+0000 7f2654ff9640 1 -- 172.21.15.165:0/3570255385 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f276402c620 con 0x55b7ef958400 2024-04-27T03:30:31.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.116+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b7ef995380 con 0x55b7ef958400 2024-04-27T03:30:31.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.116+0000 7f2654ff9640 1 -- 172.21.15.165:0/3570255385 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f276405f040 con 0x55b7ef958400 2024-04-27T03:30:31.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.116+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b7ef995380 con 0x55b7ef958400 2024-04-27T03:30:31.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.117+0000 7f2654ff9640 1 -- 172.21.15.165:0/3570255385 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f276406c030 con 0x55b7ef958400 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.117+0000 7f2654ff9640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f263c036950 0x7f263c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.117+0000 7f2654ff9640 1 -- 172.21.15.165:0/3570255385 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f276406a070 con 0x55b7ef958400 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.117+0000 7f27720f8640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f263c036950 0x7f263c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f2776b37800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f2776b37800 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b7ef9a03c0 0x55b7ef9a2880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a2fd0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f27720f8640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f263c036950 0x7f263c038e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f276801ced0 tx=0x7f276804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f27730fa640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b7ef9a03c0 0x55b7ef9a2880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f27730fa640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b7ef9a03c0 0x55b7ef9a2880 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f2776b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f2776b37800 20 realm 2024-04-27T03:30:31.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f2776b37800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a33e0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f2776b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.118+0000 7f2776b37800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f2776b37800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a3cb0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f2776b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f2776b37800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a4050 con 0x55b7ef9a03c0 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f2776b37800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f2776b37800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f2776b37800 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b7ef9a4db0 0x55b7ef9a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a77b0 con 0x55b7ef9a4db0 2024-04-27T03:30:31.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f27728f9640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b7ef9a4db0 0x55b7ef9a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.119+0000 7f27728f9640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b7ef9a4db0 0x55b7ef9a71a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.120+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.120+0000 7f2776b37800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:31.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.120+0000 7f2776b37800 20 searching for the correct realm 2024-04-27T03:30:31.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.120+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a90e0 con 0x55b7ef9a4db0 2024-04-27T03:30:31.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.120+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.120+0000 7f27728f9640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f276405a030 0x7f276402b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.120+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a90e0 con 0x7f276405a030 2024-04-27T03:30:31.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.120+0000 7f27720f8640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f276405a030 0x7f276402b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.121+0000 7f27720f8640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f276405a030 0x7f276402b9a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.121+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.121+0000 7f27720f8640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2768076040 0x7f2768083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.121+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a90e0 con 0x7f2768076040 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.121+0000 7f27730fa640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2768076040 0x7f2768083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.121+0000 7f27730fa640 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2768076040 0x7f2768083410 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.122+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f276c00a4a0 con 0x7f2768076040 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.122+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a4050 con 0x55b7ef9a03c0 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.122+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.122+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a4050 con 0x55b7ef9a03c0 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.122+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.122+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a4050 con 0x7f2768076040 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.122+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c00a4a0 con 0x7f2768076040 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.122+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a4050 con 0x7f2768076040 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.122+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c00a4a0 con 0x7f2768076040 2024-04-27T03:30:31.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.122+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a4050 con 0x7f276405a030 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.123+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.123+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a4050 con 0x55b7ef9a4db0 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.123+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.123+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a4050 con 0x55b7ef9a03c0 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.123+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.123+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a4050 con 0x7f2768076040 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.124+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c00a4a0 con 0x7f2768076040 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.124+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a4050 con 0x55b7ef9a03c0 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.124+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.124+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a4050 con 0x7f276405a030 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.124+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.124+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x55b7ef9a4db0 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.124+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.124+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x55b7ef9a4db0 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.125+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.125+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x7f2768076040 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.125+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c00a4a0 con 0x7f2768076040 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.125+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x55b7ef9a4db0 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.125+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.125+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x55b7ef9a4db0 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.126+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.126+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x55b7ef9a03c0 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.126+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.126+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x55b7ef9a03c0 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.126+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f276c00a4a0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.126+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x7f2768076040 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.126+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c033040 con 0x7f2768076040 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.126+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x55b7ef9a4db0 2024-04-27T03:30:31.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.126+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.126+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x7f276405a030 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.127+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.127+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x55b7ef9a03c0 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.127+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.127+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x7f2768076040 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.127+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.127+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x55b7ef9a4db0 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.128+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.128+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x7f2768076040 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.128+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.128+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x7f276405a030 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.128+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.128+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x55b7ef9a4db0 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.128+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.128+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x7f2768076040 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x55b7ef9a03c0 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2768090060 con 0x7f276405a030 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f2776b37800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f2776b37800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f2776b37800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f2776b37800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f2776b37800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f2776b37800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:31.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f2776b37800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a8070 con 0x55b7ef9a03c0 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.129+0000 7f2776b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a8460 con 0x55b7ef9a03c0 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a8460 con 0x7f2768076040 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 20 zone default found 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 4 Realm: () 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:31.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:31.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:31.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 20 zonegroup default 2024-04-27T03:30:31.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9a7bf0 con 0x7f2768076040 2024-04-27T03:30:31.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.130+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.131+0000 7f2776b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.131+0000 7f2776b37800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.131+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9aaf40 con 0x55b7ef9a4db0 2024-04-27T03:30:31.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.131+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.131+0000 7f2776b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.131+0000 7f2776b37800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.131+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9ab2e0 con 0x55b7ef9a4db0 2024-04-27T03:30:31.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.131+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.131+0000 7f2776b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.131+0000 7f2776b37800 20 started sync module instance, tier type = 2024-04-27T03:30:31.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.131+0000 7f2776b37800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:31.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.131+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9aa3e0 con 0x7f276405a030 2024-04-27T03:30:31.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.132+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.132+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94248487264048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9ab680 con 0x7f276405a030 2024-04-27T03:30:31.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.132+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9abf10 con 0x55b7ef9a03c0 2024-04-27T03:30:31.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.133+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.133+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94248487270944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9ac7b0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.133+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94248487264048] v25'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.133+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9aa830 con 0x55b7ef9a4db0 2024-04-27T03:30:31.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.134+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94248487270944] v25'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.135+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.135+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94248487273696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9ad1f0 con 0x55b7ef9a4db0 2024-04-27T03:30:31.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.135+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9adbb0 con 0x7f276405a030 2024-04-27T03:30:31.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.136+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94248487273696] v25'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.136+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.136+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94248487275840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9ae400 con 0x7f276405a030 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.136+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9aedc0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.137+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.137+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94248487283440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9af560 con 0x55b7ef9a03c0 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.137+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b0080 con 0x7f2768076040 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.137+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94248487275840] v25'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.138+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.138+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94248487287632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b0040 con 0x7f2768076040 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.138+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b0ad0 con 0x7f2768076040 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.138+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94248487283440] v25'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.139+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94248487287632] v25'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.139+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.139+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94248487280464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9aefb0 con 0x7f2768076040 2024-04-27T03:30:31.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.139+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b0f80 con 0x7f2768076040 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.140+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'39 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.140+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94248487280464] v25'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.140+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94248487291600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b13a0 con 0x7f2768076040 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.140+0000 7f2776b37800 20 add_watcher() i=0 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.140+0000 7f2776b37800 20 add_watcher() i=1 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.140+0000 7f2776b37800 20 add_watcher() i=2 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.140+0000 7f2776b37800 20 add_watcher() i=3 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.140+0000 7f2776b37800 20 add_watcher() i=4 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.140+0000 7f2776b37800 20 add_watcher() i=5 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.140+0000 7f2776b37800 20 add_watcher() i=6 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.141+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94248487291600] v25'40 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.141+0000 7f2776b37800 20 add_watcher() i=7 2024-04-27T03:30:31.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.141+0000 7f2776b37800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.141+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b13a0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.141+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.141+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94248487292656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b1c40 con 0x55b7ef9a03c0 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.142+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94248487292656] v25'66 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.142+0000 7f26297e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.142+0000 7f2776b37800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.142+0000 7f2776b37800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.142+0000 7f2776b37800 5 note: GC not initialized 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.142+0000 7f25a3fff640 20 reqs_thread_entry: start 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.142+0000 7f2776b37800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.142+0000 7f2776b37800 20 Filter name: none 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.143+0000 7f25a1ffb640 20 reqs_thread_entry: start 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.143+0000 7f2776b37800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:31.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.143+0000 7f2776b37800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.143+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9e7680 con 0x55b7ef9a03c0 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.143+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.143+0000 7f2776b37800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.143+0000 7f2776b37800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.143+0000 7f2776b37800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.143+0000 7f2776b37800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.143+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9ea110 con 0x7f276405a030 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.143+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(63 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.143+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9ed8c0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.144+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~391] v25'3 uv3 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.144+0000 7f2776b37800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.144+0000 7f2776b37800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.144+0000 7f2776b37800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:31.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.144+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94248487539472 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9e9100 con 0x55b7ef9a03c0 2024-04-27T03:30:31.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.145+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify (1) cookie 94248487283440 notify 107374182402 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7f276c01e040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.145+0000 7f26567fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182402 cookie 94248487283440 notifier 4315 bl.length()=567 2024-04-27T03:30:31.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.145+0000 7f26567fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:31.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.145+0000 7f26567fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:31.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.145+0000 7f26567fc640 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f2638003ea0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.145+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(65 notify.4 [notify cookie 94248487539472 out=8b] v0'0 uv39 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.145+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.145+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== watch-notify(notify_complete (2) cookie 94248487539472 notify 107374182402 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f276c018db0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.161 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:31.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.146+0000 7f26297e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.146+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94248487292656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b3e70 con 0x55b7ef9a03c0 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.147+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(67 data_loggenerations_metadata [watch unwatch cookie 94248487292656] v25'67 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.147+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94248487264048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b0ef0 con 0x7f276405a030 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.148+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(68 notify.0 [watch unwatch cookie 94248487264048] v25'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.148+0000 7f2776b37800 20 remove_watcher() i=0 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.148+0000 7f2776b37800 2 removed watcher, disabling cache 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.148+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94248487270944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b0ef0 con 0x55b7ef9a03c0 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.149+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(69 notify.1 [watch unwatch cookie 94248487270944] v25'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.149+0000 7f2776b37800 20 remove_watcher() i=1 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.149+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:70 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94248487273696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b0ef0 con 0x55b7ef9a4db0 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.150+0000 7f27728f9640 1 -- 172.21.15.165:0/3570255385 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(70 notify.2 [watch unwatch cookie 94248487273696] v25'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2764075070 con 0x55b7ef9a4db0 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.150+0000 7f2776b37800 20 remove_watcher() i=2 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.150+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94248487275840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b0ef0 con 0x7f276405a030 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.151+0000 7f27720f8640 1 -- 172.21.15.165:0/3570255385 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(71 notify.3 [watch unwatch cookie 94248487275840] v25'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2768081070 con 0x7f276405a030 2024-04-27T03:30:31.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.151+0000 7f2776b37800 20 remove_watcher() i=3 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.151+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94248487283440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9acd40 con 0x55b7ef9a03c0 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.152+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(72 notify.4 [watch unwatch cookie 94248487283440] v25'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x55b7ef9a03c0 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.152+0000 7f2776b37800 20 remove_watcher() i=4 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.152+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94248487287632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9b0ef0 con 0x7f2768076040 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.153+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 notify.5 [watch unwatch cookie 94248487287632] v25'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.153+0000 7f2776b37800 20 remove_watcher() i=5 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.153+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94248487280464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9aeaf0 con 0x7f2768076040 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.154+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 notify.6 [watch unwatch cookie 94248487280464] v25'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.154+0000 7f2776b37800 20 remove_watcher() i=6 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.154+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94248487291600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b7ef9afb50 con 0x7f2768076040 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(75 notify.7 [watch unwatch cookie 94248487291600] v25'41 uv39 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f276c031040 con 0x7f2768076040 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 20 remove_watcher() i=7 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2768076040 msgr2=0x7f2768083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2768076040 0x7f2768083410 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b7ef9a03c0 msgr2=0x55b7ef9a2880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b7ef9a03c0 0x55b7ef9a2880 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f276405a030 msgr2=0x7f276402b9a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f276405a030 0x7f276402b9a0 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b7ef9a4db0 msgr2=0x55b7ef9a71a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b7ef9a4db0 0x55b7ef9a71a0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f263c036950 msgr2=0x7f263c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f263c036950 0x7f263c038e10 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f276801ced0 tx=0x7f276804f000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 msgr2=0x55b7ef994e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f27730fa640 1 -- 172.21.15.165:0/3570255385 reap_dead start 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 0x55b7ef994e40 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f2764019fd0 tx=0x7f2764017a20 comp rx=0 tx=0).stop 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 shutdown_connections 2024-04-27T03:30:31.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 --2- 172.21.15.165:0/3570255385 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef958400 0x55b7ef994e40 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 >> 172.21.15.165:0/3570255385 conn(0x55b7ef987ab0 msgr2=0x55b7ef985110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 shutdown_connections 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3570255385 wait complete. 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3362430582 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f264c036950 msgr2=0x7f264c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 --2- 172.21.15.165:0/3362430582 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f264c036950 0x7f264c038e10 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x55b7ef964390 tx=0x7f2768005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3362430582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef97e660 msgr2=0x55b7ef97ea80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 --2- 172.21.15.165:0/3362430582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef97e660 0x55b7ef97ea80 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f2764016fd0 tx=0x7f276404c480 comp rx=0 tx=0).stop 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3362430582 shutdown_connections 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 --2- 172.21.15.165:0/3362430582 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f264c036950 0x7f264c038e10 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 --2- 172.21.15.165:0/3362430582 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef97e660 0x55b7ef97ea80 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3362430582 >> 172.21.15.165:0/3362430582 conn(0x55b7ef972700 msgr2=0x55b7ef972b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3362430582 shutdown_connections 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.155+0000 7f2776b37800 1 -- 172.21.15.165:0/3362430582 wait complete. 2024-04-27T03:30:31.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b7ef96d6c0 msgr2=0x55b7ef96fb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b7ef96d6c0 0x55b7ef96fb10 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b7ef86a1b0 msgr2=0x55b7ef966ea0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b7ef86a1b0 0x55b7ef966ea0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b7ef969780 msgr2=0x55b7ef96bb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b7ef969780 0x55b7ef96bb70 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2658036900 msgr2=0x7f2658038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2658036900 0x7f2658038dc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f2768054000 tx=0x7f276804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 msgr2=0x55b7ef95fc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 --2- 172.21.15.165:0/3074051460 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b7ef865d20 0x55b7ef95fc60 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2764004000 tx=0x7f2764000bc0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f27730fa640 1 -- 172.21.15.165:0/3074051460 reap_dead start 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 shutdown_connections 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 >> 172.21.15.165:0/3074051460 conn(0x55b7ef861380 msgr2=0x55b7ef951110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 shutdown_connections 2024-04-27T03:30:31.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.156+0000 7f2776b37800 1 -- 172.21.15.165:0/3074051460 wait complete. 2024-04-27T03:30:31.172 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:31.172 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T03:30:31.173 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T03:30:31.173 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T03:30:31.255 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:31.256 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:31.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.251+0000 7f8e8e23f800 1 Processor -- start 2024-04-27T03:30:31.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.251+0000 7f8e8e23f800 1 -- start start 2024-04-27T03:30:31.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.251+0000 7f8e8e23f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ec7e496130 0x55ec7e4985f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.251+0000 7f8e8e23f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ec7e350d40 con 0x55ec7e4051c0 2024-04-27T03:30:31.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.251+0000 7f8e8e23f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55ec7e3234d0 con 0x55ec7e496130 2024-04-27T03:30:31.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.251+0000 7f8e8b53d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ec7e496130 0x55ec7e4985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.251+0000 7f8e8bd3e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55ec7e4051c0 0x55ec7e4949e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53656/0 (socket says 172.21.15.165:53656) 2024-04-27T03:30:31.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.251+0000 7f8e8b53d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ec7e496130 0x55ec7e4985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51836/0 (socket says 172.21.15.165:51836) 2024-04-27T03:30:31.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.251+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/2024780953 learned_addr learned my addr 172.21.15.165:0/2024780953 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:31.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.251+0000 7f8e8b53d640 1 -- 172.21.15.165:0/2024780953 >> v1:172.21.15.89:6789/0 conn(0x55ec7e4051c0 legacy=0x55ec7e4949e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.251+0000 7f8e8b53d640 1 -- 172.21.15.165:0/2024780953 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7e34f590 con 0x55ec7e496130 2024-04-27T03:30:31.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.253+0000 7f8e8b53d640 1 --2- 172.21.15.165:0/2024780953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e496130 0x55ec7e4985f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000a470 tx=0x7f8e8001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae73e535bbcd6219 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.253+0000 7f8e8ad3c640 1 -- 172.21.15.165:0/2024780953 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e80057070 con 0x55ec7e496130 2024-04-27T03:30:31.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.253+0000 7f8e8ad3c640 1 -- 172.21.15.165:0/2024780953 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e80052570 con 0x55ec7e496130 2024-04-27T03:30:31.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.253+0000 7f8e8ad3c640 1 -- 172.21.15.165:0/2024780953 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e80052870 con 0x55ec7e496130 2024-04-27T03:30:31.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.254+0000 7f8e8e23f800 1 -- 172.21.15.165:0/2024780953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e496130 msgr2=0x55ec7e4985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.254+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/2024780953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e496130 0x55ec7e4985f0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8e8000a470 tx=0x7f8e8001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:31.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.254+0000 7f8e8e23f800 1 -- 172.21.15.165:0/2024780953 shutdown_connections 2024-04-27T03:30:31.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.254+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/2024780953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e496130 0x55ec7e4985f0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.254+0000 7f8e8e23f800 1 -- 172.21.15.165:0/2024780953 >> 172.21.15.165:0/2024780953 conn(0x55ec7e48c380 msgr2=0x55ec7e48e7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.254+0000 7f8e8e23f800 1 -- 172.21.15.165:0/2024780953 shutdown_connections 2024-04-27T03:30:31.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.254+0000 7f8e8e23f800 1 -- 172.21.15.165:0/2024780953 wait complete. 2024-04-27T03:30:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.266+0000 7f8e8e23f800 1 Processor -- start 2024-04-27T03:30:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.266+0000 7f8e8e23f800 1 -- start start 2024-04-27T03:30:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.266+0000 7f8e8e23f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 0x55ec7e492310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.266+0000 7f8e8e23f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ec7e3dd3c0 con 0x55ec7e4051c0 2024-04-27T03:30:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.267+0000 7f8e8bd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 0x55ec7e492310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.267+0000 7f8e8bd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 0x55ec7e492310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51844/0 (socket says 172.21.15.165:51844) 2024-04-27T03:30:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.267+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/68400789 learned_addr learned my addr 172.21.15.165:0/68400789 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.267+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/68400789 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7e4908d0 con 0x55ec7e4051c0 2024-04-27T03:30:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.267+0000 7f8e8bd3e640 1 --2- 172.21.15.165:0/68400789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 0x55ec7e492310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c001de0 tx=0x7f8e7c00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3fe2319936ffe48 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.267+0000 7f8d79e0b640 1 -- 172.21.15.165:0/68400789 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e7c012020 con 0x55ec7e4051c0 2024-04-27T03:30:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.267+0000 7f8d79e0b640 1 -- 172.21.15.165:0/68400789 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7c00e5a0 con 0x55ec7e4051c0 2024-04-27T03:30:31.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.267+0000 7f8d79e0b640 1 -- 172.21.15.165:0/68400789 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e7c00e8c0 con 0x55ec7e4051c0 2024-04-27T03:30:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.268+0000 7f8e8e23f800 1 -- 172.21.15.165:0/68400789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 msgr2=0x55ec7e492310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.268+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/68400789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 0x55ec7e492310 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c001de0 tx=0x7f8e7c00b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.268+0000 7f8e8e23f800 1 -- 172.21.15.165:0/68400789 shutdown_connections 2024-04-27T03:30:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.268+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/68400789 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 0x55ec7e492310 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.268+0000 7f8e8e23f800 1 -- 172.21.15.165:0/68400789 >> 172.21.15.165:0/68400789 conn(0x55ec7e48c380 msgr2=0x55ec7e57ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.268+0000 7f8e8e23f800 1 -- 172.21.15.165:0/68400789 shutdown_connections 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.268+0000 7f8e8e23f800 1 -- 172.21.15.165:0/68400789 wait complete. 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.268+0000 7f8e8e23f800 1 Processor -- start 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.268+0000 7f8e8e23f800 1 -- start start 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.268+0000 7f8e8e23f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 0x55ec7e58ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.269+0000 7f8e8e23f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ec7e498fa0 con 0x55ec7e4051c0 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.269+0000 7f8e8bd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 0x55ec7e58ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.269+0000 7f8e8bd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 0x55ec7e58ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51854/0 (socket says 172.21.15.165:51854) 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.269+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3601561052 learned_addr learned my addr 172.21.15.165:0/3601561052 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.269+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3601561052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7e58d340 con 0x55ec7e4051c0 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.269+0000 7f8e8bd3e640 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 0x55ec7e58ce00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00c010 tx=0x7f8e7c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.269+0000 7f8d73fff640 1 -- 172.21.15.165:0/3601561052 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e7c012070 con 0x55ec7e4051c0 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.269+0000 7f8d73fff640 1 -- 172.21.15.165:0/3601561052 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7c01a7a0 con 0x55ec7e4051c0 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.269+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec7e58d5d0 con 0x55ec7e4051c0 2024-04-27T03:30:31.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.269+0000 7f8d73fff640 1 -- 172.21.15.165:0/3601561052 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e7c01aa80 con 0x55ec7e4051c0 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.269+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec7e590280 con 0x55ec7e4051c0 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.270+0000 7f8d73fff640 1 -- 172.21.15.165:0/3601561052 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f8e7c015020 con 0x55ec7e4051c0 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.270+0000 7f8d73fff640 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d74036900 0x7f8d74038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.270+0000 7f8d73fff640 1 -- 172.21.15.165:0/3601561052 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f8e7c020070 con 0x55ec7e4051c0 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.270+0000 7f8e8b53d640 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d74036900 0x7f8d74038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.271+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ec7e496130 0x55ec7e594040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.271+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e594580 con 0x55ec7e496130 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.271+0000 7f8e8c53f640 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ec7e496130 0x55ec7e594040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.271+0000 7f8e8b53d640 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d74036900 0x7f8d74038dc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e80054000 tx=0x7f8e8004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.271+0000 7f8e8c53f640 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ec7e496130 0x55ec7e594040 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.271+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3601561052 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e496130 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.271+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e591610 con 0x55ec7e496130 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.271+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3601561052 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e496130 2024-04-27T03:30:31.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.271+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ec7e5968d0 0x55ec7e598cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.271+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5993b0 con 0x55ec7e5968d0 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.272+0000 7f8e8bd3e640 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ec7e5968d0 0x55ec7e598cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.272+0000 7f8e8bd3e640 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ec7e5968d0 0x55ec7e598cc0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.272+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3601561052 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5968d0 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.272+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e595880 con 0x55ec7e496130 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.272+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3601561052 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e496130 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.272+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ec7e59a860 0x55ec7e59ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.272+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e59d410 con 0x55ec7e59a860 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.273+0000 7f8e8b53d640 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ec7e59a860 0x55ec7e59ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.273+0000 7f8e8b53d640 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ec7e59a860 0x55ec7e59ccb0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.273+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3601561052 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f8e80052a40 con 0x55ec7e59a860 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.274+0000 7f8e8e23f800 1 Processor -- start 2024-04-27T03:30:31.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.274+0000 7f8e8e23f800 1 -- start start 2024-04-27T03:30:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.274+0000 7f8e8e23f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5ab830 0x55ec7e5abc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.274+0000 7f8e8e23f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ec7e3836b0 con 0x55ec7e5ab830 2024-04-27T03:30:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.274+0000 7f8e8bd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5ab830 0x55ec7e5abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.274+0000 7f8e8bd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5ab830 0x55ec7e5abc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51870/0 (socket says 172.21.15.165:51870) 2024-04-27T03:30:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.274+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3843747221 learned_addr learned my addr 172.21.15.165:0/3843747221 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.274+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3843747221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7e5ac190 con 0x55ec7e5ab830 2024-04-27T03:30:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.274+0000 7f8e8bd3e640 1 --2- 172.21.15.165:0/3843747221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5ab830 0x55ec7e5abc50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c04acf0 tx=0x7f8e7c01af10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.274+0000 7f8d717fa640 1 -- 172.21.15.165:0/3843747221 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e7c047260 con 0x55ec7e5ab830 2024-04-27T03:30:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.274+0000 7f8d717fa640 1 -- 172.21.15.165:0/3843747221 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7c0490e0 con 0x55ec7e5ab830 2024-04-27T03:30:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.275+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3843747221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec7e5acc00 con 0x55ec7e5ab830 2024-04-27T03:30:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.275+0000 7f8d717fa640 1 -- 172.21.15.165:0/3843747221 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e7c0493e0 con 0x55ec7e5ab830 2024-04-27T03:30:31.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.275+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3843747221 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec7e5ad100 con 0x55ec7e5ab830 2024-04-27T03:30:31.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.275+0000 7f8d717fa640 1 -- 172.21.15.165:0/3843747221 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f8e7c044060 con 0x55ec7e5ab830 2024-04-27T03:30:31.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.275+0000 7f8d717fa640 1 --2- 172.21.15.165:0/3843747221 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d64036950 0x7f8d64038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.275+0000 7f8d717fa640 1 -- 172.21.15.165:0/3843747221 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f8e7c04dcc0 con 0x55ec7e5ab830 2024-04-27T03:30:31.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.275+0000 7f8e8b53d640 1 --2- 172.21.15.165:0/3843747221 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d64036950 0x7f8d64038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.276+0000 7f8e8e23f800 1 Processor -- start 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.276+0000 7f8e8e23f800 1 -- start start 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.276+0000 7f8e8b53d640 1 --2- 172.21.15.165:0/3843747221 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d64036950 0x7f8d64038e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55ec7e594e80 tx=0x7f8e80005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.276+0000 7f8e8e23f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 0x55ec7e5b8e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.276+0000 7f8e8e23f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ec7e490c50 con 0x55ec7e5b89e0 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.276+0000 7f8e8bd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 0x55ec7e5b8e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.276+0000 7f8e8bd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 0x55ec7e5b8e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51876/0 (socket says 172.21.15.165:51876) 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.276+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/103745036 learned_addr learned my addr 172.21.15.165:0/103745036 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.276+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/103745036 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7e5b93d0 con 0x55ec7e5b89e0 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.277+0000 7f8e8bd3e640 1 --2- 172.21.15.165:0/103745036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 0x55ec7e5b8e00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c047910 tx=0x7f8e7c047ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3e9f9011c113935 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.277+0000 7f8d6affd640 1 -- 172.21.15.165:0/103745036 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e7c03d070 con 0x55ec7e5b89e0 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.277+0000 7f8d6affd640 1 -- 172.21.15.165:0/103745036 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7c05b070 con 0x55ec7e5b89e0 2024-04-27T03:30:31.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.277+0000 7f8d6affd640 1 -- 172.21.15.165:0/103745036 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e7c044860 con 0x55ec7e5b89e0 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.277+0000 7f8e8e23f800 1 -- 172.21.15.165:0/103745036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 msgr2=0x55ec7e5b8e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.277+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/103745036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 0x55ec7e5b8e00 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c047910 tx=0x7f8e7c047ca0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.277+0000 7f8e8e23f800 1 -- 172.21.15.165:0/103745036 shutdown_connections 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.277+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/103745036 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 0x55ec7e5b8e00 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.277+0000 7f8e8e23f800 1 -- 172.21.15.165:0/103745036 >> 172.21.15.165:0/103745036 conn(0x55ec7e5b4cb0 msgr2=0x55ec7e5b2d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.277+0000 7f8e8e23f800 1 -- 172.21.15.165:0/103745036 shutdown_connections 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.277+0000 7f8e8e23f800 1 -- 172.21.15.165:0/103745036 wait complete. 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.277+0000 7f8e8e23f800 1 Processor -- start 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.278+0000 7f8e8e23f800 1 -- start start 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.278+0000 7f8e8e23f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 0x55ec7e5bfe40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.278+0000 7f8e8e23f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ec7e5b9aa0 con 0x55ec7e5b89e0 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.278+0000 7f8e8bd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 0x55ec7e5bfe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.278+0000 7f8e8bd3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 0x55ec7e5bfe40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51884/0 (socket says 172.21.15.165:51884) 2024-04-27T03:30:31.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.278+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 learned_addr learned my addr 172.21.15.165:0/3070989475 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.278+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec7e5c0380 con 0x55ec7e5b89e0 2024-04-27T03:30:31.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.278+0000 7f8e8bd3e640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 0x55ec7e5bfe40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c01a6a0 tx=0x7f8e7c03f060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.278+0000 7f8d697fa640 1 -- 172.21.15.165:0/3070989475 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e7c016810 con 0x55ec7e5b89e0 2024-04-27T03:30:31.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.279+0000 7f8d697fa640 1 -- 172.21.15.165:0/3070989475 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8e7c0169d0 con 0x55ec7e5b89e0 2024-04-27T03:30:31.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.279+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ec7e5c0380 con 0x55ec7e5b89e0 2024-04-27T03:30:31.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.279+0000 7f8d697fa640 1 -- 172.21.15.165:0/3070989475 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8e7c016d30 con 0x55ec7e5b89e0 2024-04-27T03:30:31.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.279+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec7e5c0380 con 0x55ec7e5b89e0 2024-04-27T03:30:31.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.279+0000 7f8d697fa640 1 -- 172.21.15.165:0/3070989475 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f8e7c055070 con 0x55ec7e5b89e0 2024-04-27T03:30:31.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.279+0000 7f8d697fa640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d54036860 0x7f8d54038d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.279+0000 7f8d697fa640 1 -- 172.21.15.165:0/3070989475 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f8e7c052070 con 0x55ec7e5b89e0 2024-04-27T03:30:31.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8e23f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8b53d640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d54036860 0x7f8d54038d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ec7e5cb320 0x55ec7e5cd7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cdf30 con 0x55ec7e5cb320 2024-04-27T03:30:31.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8c53f640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ec7e5cb320 0x55ec7e5cd7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8c53f640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ec7e5cb320 0x55ec7e5cd7e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8b53d640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d54036860 0x7f8d54038d20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e80052fd0 tx=0x7f8e8004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8e23f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8e23f800 20 realm 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8e23f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5ce340 con 0x55ec7e5cb320 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.280+0000 7f8e8e23f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8e23f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8e23f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cebe0 con 0x55ec7e5cb320 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8e23f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8e23f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cef80 con 0x55ec7e5cb320 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8e23f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8e23f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ec7e5cfce0 0x55ec7e5d20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d2610 con 0x55ec7e5cfce0 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8bd3e640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ec7e5cfce0 0x55ec7e5d20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.281+0000 7f8e8bd3e640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ec7e5cfce0 0x55ec7e5d20d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.282+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.282+0000 7f8e8e23f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.282+0000 7f8e8e23f800 20 searching for the correct realm 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.282+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d3fb0 con 0x55ec7e5cfce0 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.282+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.282+0000 7f8e8bd3e640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8e7c017050 0x7f8e7c06b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.282+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d3fb0 con 0x7f8e7c017050 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.282+0000 7f8e8b53d640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8e7c017050 0x7f8e7c06b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.283+0000 7f8e8b53d640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8e7c017050 0x7f8e7c06b440 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.283+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e80081070 con 0x7f8e7c017050 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.283+0000 7f8e8b53d640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f8e80076040 0x7f8e80083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.283+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d3fb0 con 0x7f8e80076040 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.283+0000 7f8e8c53f640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f8e80076040 0x7f8e80083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.283+0000 7f8e8c53f640 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f8e80076040 0x7f8e80083410 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.284+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.284+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cef80 con 0x55ec7e5cb320 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.284+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.284+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cef80 con 0x55ec7e5cb320 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.284+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.284+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cef80 con 0x7f8e80076040 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.284+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.284+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cef80 con 0x7f8e80076040 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.285+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.285+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cef80 con 0x7f8e7c017050 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.285+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e80081070 con 0x7f8e7c017050 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.285+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cef80 con 0x55ec7e5cfce0 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.285+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.285+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cef80 con 0x55ec7e5cb320 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.285+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.285+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cef80 con 0x7f8e80076040 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.286+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.286+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cef80 con 0x55ec7e5cb320 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.286+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.286+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5cef80 con 0x7f8e7c017050 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.286+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f8e80081070 con 0x7f8e7c017050 2024-04-27T03:30:31.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.286+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x55ec7e5cfce0 2024-04-27T03:30:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.286+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.286+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x55ec7e5cfce0 2024-04-27T03:30:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.287+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.287+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x7f8e80076040 2024-04-27T03:30:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.287+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.287+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x55ec7e5cfce0 2024-04-27T03:30:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.287+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.287+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x55ec7e5cfce0 2024-04-27T03:30:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.287+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.287+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x55ec7e5cb320 2024-04-27T03:30:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.287+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.287+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x55ec7e5cb320 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.288+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.288+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x7f8e80076040 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.288+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.288+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x55ec7e5cfce0 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.288+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.288+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x7f8e7c017050 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.288+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e80081070 con 0x7f8e7c017050 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.288+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x55ec7e5cb320 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.289+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.289+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x7f8e80076040 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.289+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.289+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x55ec7e5cfce0 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.289+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.289+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x7f8e80076040 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.289+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.289+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x7f8e7c017050 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.290+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e80081070 con 0x7f8e7c017050 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.290+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x55ec7e5cfce0 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.290+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.290+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x7f8e80076040 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.290+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.290+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e80090060 con 0x55ec7e5cb320 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8e84038040 con 0x7f8e7c017050 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8e80081070 con 0x7f8e7c017050 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:31.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d2f80 con 0x55ec7e5cb320 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d3370 con 0x55ec7e5cb320 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.291+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d3370 con 0x7f8e80076040 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8e23f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8e23f800 20 zone default found 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8e23f800 4 Realm: () 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8e23f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8e23f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8e23f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8e23f800 20 zonegroup default 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8e23f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d3790 con 0x7f8e80076040 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8e23f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8e23f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d3790 con 0x55ec7e5cfce0 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.292+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.293+0000 7f8e8e23f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.293+0000 7f8e8e23f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.293+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d5e60 con 0x55ec7e5cfce0 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.293+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.293+0000 7f8e8e23f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.293+0000 7f8e8e23f800 20 started sync module instance, tier type = 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.293+0000 7f8e8e23f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.293+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d2b20 con 0x7f8e7c017050 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.294+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e80081070 con 0x7f8e7c017050 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.294+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94474220688336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d6240 con 0x7f8e7c017050 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.294+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d6d50 con 0x55ec7e5cb320 2024-04-27T03:30:31.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.295+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94474220688336] v25'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e80081070 con 0x7f8e7c017050 2024-04-27T03:30:31.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.295+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.295+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94474220693504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d7140 con 0x55ec7e5cb320 2024-04-27T03:30:31.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.295+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d7ba0 con 0x55ec7e5cfce0 2024-04-27T03:30:31.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.296+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94474220693504] v25'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.296+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.296+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94474220689632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d7f90 con 0x55ec7e5cfce0 2024-04-27T03:30:31.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.296+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d8990 con 0x7f8e7c017050 2024-04-27T03:30:31.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.297+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94474220689632] v25'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.297+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e80081070 con 0x7f8e7c017050 2024-04-27T03:30:31.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.297+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94474220697216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d9230 con 0x7f8e7c017050 2024-04-27T03:30:31.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.297+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d9bf0 con 0x55ec7e5cb320 2024-04-27T03:30:31.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.298+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.298+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94474220706080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5da370 con 0x55ec7e5cb320 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.298+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5dae90 con 0x7f8e80076040 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.299+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94474220697216] v25'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e80081070 con 0x7f8e7c017050 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.299+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.299+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94474220710240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5dae50 con 0x7f8e80076040 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.299+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5db8c0 con 0x7f8e80076040 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.299+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94474220706080] v25'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.300+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.300+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94474220712944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5db8c0 con 0x7f8e80076040 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.300+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5dc280 con 0x7f8e80076040 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.301+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94474220710240] v25'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.301+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'42 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.301+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94474220703104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5dc280 con 0x7f8e80076040 2024-04-27T03:30:31.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.301+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94474220712944] v25'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.301+0000 7f8e8e23f800 20 add_watcher() i=0 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.301+0000 7f8e8e23f800 20 add_watcher() i=1 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.301+0000 7f8e8e23f800 20 add_watcher() i=2 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.301+0000 7f8e8e23f800 20 add_watcher() i=3 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.301+0000 7f8e8e23f800 20 add_watcher() i=4 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.301+0000 7f8e8e23f800 20 add_watcher() i=5 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.301+0000 7f8e8e23f800 20 add_watcher() i=6 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.302+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94474220703104] v25'43 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.302+0000 7f8e8e23f800 20 add_watcher() i=7 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.302+0000 7f8e8e23f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.302+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5df2b0 con 0x55ec7e5cb320 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.303+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.303+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94474220715472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5dc6a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.304+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94474220715472] v25'68 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.304+0000 7f8d41fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.304+0000 7f8e8e23f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:31.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.304+0000 7f8e8e23f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:31.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.304+0000 7f8e8e23f800 5 note: GC not initialized 2024-04-27T03:30:31.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.304+0000 7f8d40fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:31.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.304+0000 7f8e8e23f800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:31.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.304+0000 7f8e8e23f800 20 Filter name: none 2024-04-27T03:30:31.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.304+0000 7f8cba7fc640 20 reqs_thread_entry: start 2024-04-27T03:30:31.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.304+0000 7f8e8e23f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.304+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e611f30 con 0x55ec7e5cb320 2024-04-27T03:30:31.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.305+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.305+0000 7f8e8e23f800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:31.324 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:31.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.305+0000 7f8d41fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.305+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94474220715472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e610ea0 con 0x55ec7e5cb320 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.306+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94474220715472] v25'69 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.306+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94474220688336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e610ea0 con 0x7f8e7c017050 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.307+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94474220688336] v25'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e80081070 con 0x7f8e7c017050 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.307+0000 7f8e8e23f800 20 remove_watcher() i=0 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.307+0000 7f8e8e23f800 2 removed watcher, disabling cache 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.307+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94474220693504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d53d0 con 0x55ec7e5cb320 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.308+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94474220693504] v25'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.308+0000 7f8e8e23f800 20 remove_watcher() i=1 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.309+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94474220689632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d53d0 con 0x55ec7e5cfce0 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.309+0000 7f8e8bd3e640 1 -- 172.21.15.165:0/3070989475 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94474220689632] v25'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e7c02b070 con 0x55ec7e5cfce0 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.309+0000 7f8e8e23f800 20 remove_watcher() i=2 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.309+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94474220697216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d53d0 con 0x7f8e7c017050 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.310+0000 7f8e8b53d640 1 -- 172.21.15.165:0/3070989475 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94474220697216] v25'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e80081070 con 0x7f8e7c017050 2024-04-27T03:30:31.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.310+0000 7f8e8e23f800 20 remove_watcher() i=3 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.310+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94474220706080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d7680 con 0x55ec7e5cb320 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.311+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94474220706080] v25'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x55ec7e5cb320 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.311+0000 7f8e8e23f800 20 remove_watcher() i=4 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.312+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94474220710240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d53d0 con 0x7f8e80076040 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.312+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94474220710240] v25'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.312+0000 7f8e8e23f800 20 remove_watcher() i=5 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.312+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94474220712944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d9920 con 0x7f8e80076040 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.313+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94474220712944] v25'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.313+0000 7f8e8e23f800 20 remove_watcher() i=6 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.313+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94474220703104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ec7e5d7680 con 0x7f8e80076040 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94474220703104] v25'44 uv42 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8e8400a4a0 con 0x7f8e80076040 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 20 remove_watcher() i=7 2024-04-27T03:30:31.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f8e80076040 msgr2=0x7f8e80083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f8e80076040 0x7f8e80083410 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ec7e5cb320 msgr2=0x55ec7e5cd7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ec7e5cb320 0x55ec7e5cd7e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8e7c017050 msgr2=0x7f8e7c06b440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8e7c017050 0x7f8e7c06b440 crc :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ec7e5cfce0 msgr2=0x55ec7e5d20d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ec7e5cfce0 0x55ec7e5d20d0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d54036860 msgr2=0x7f8d54038d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d54036860 0x7f8d54038d20 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f8e80052fd0 tx=0x7f8e8004f000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 msgr2=0x55ec7e5bfe40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3070989475 reap_dead start 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.314+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 0x55ec7e5bfe40 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c01a6a0 tx=0x7f8e7c03f060 comp rx=0 tx=0).stop 2024-04-27T03:30:31.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 shutdown_connections 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3070989475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5b89e0 0x55ec7e5bfe40 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 >> 172.21.15.165:0/3070989475 conn(0x55ec7e5b4cb0 msgr2=0x55ec7e5b2310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 shutdown_connections 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3070989475 wait complete. 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3843747221 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d64036950 msgr2=0x7f8d64038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3843747221 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d64036950 0x7f8d64038e10 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x55ec7e594e80 tx=0x7f8e80005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3843747221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5ab830 msgr2=0x55ec7e5abc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3843747221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5ab830 0x55ec7e5abc50 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c04acf0 tx=0x7f8e7c01af10 comp rx=0 tx=0).stop 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3843747221 shutdown_connections 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3843747221 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d64036950 0x7f8d64038e10 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3843747221 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e5ab830 0x55ec7e5abc50 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3843747221 >> 172.21.15.165:0/3843747221 conn(0x55ec7e59f8d0 msgr2=0x55ec7e59fcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3843747221 shutdown_connections 2024-04-27T03:30:31.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3843747221 wait complete. 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ec7e59a860 msgr2=0x55ec7e59ccb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ec7e59a860 0x55ec7e59ccb0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ec7e496130 msgr2=0x55ec7e594040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ec7e496130 0x55ec7e594040 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ec7e5968d0 msgr2=0x55ec7e598cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ec7e5968d0 0x55ec7e598cc0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d74036900 msgr2=0x7f8d74038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8d74036900 0x7f8d74038dc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8e80054000 tx=0x7f8e8004e000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 msgr2=0x55ec7e58ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 --2- 172.21.15.165:0/3601561052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ec7e4051c0 0x55ec7e58ce00 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f8e7c00c010 tx=0x7f8e7c000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8c53f640 1 -- 172.21.15.165:0/3601561052 reap_dead start 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 shutdown_connections 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.315+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 >> 172.21.15.165:0/3601561052 conn(0x55ec7e48c380 msgr2=0x55ec7e4911b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.316+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 shutdown_connections 2024-04-27T03:30:31.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.316+0000 7f8e8e23f800 1 -- 172.21.15.165:0/3601561052 wait complete. 2024-04-27T03:30:31.334 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:31.335 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-04-27T03:30:31.335 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'create', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'] 2024-04-27T03:30:31.335 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key create --uid foo --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh 2024-04-27T03:30:31.425 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:31.425 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:31.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.420+0000 7efc7fe1f800 1 Processor -- start 2024-04-27T03:30:31.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.420+0000 7efc7fe1f800 1 -- start start 2024-04-27T03:30:31.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.420+0000 7efc7fe1f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d00dbd6fb0 0x55d00dbd73d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.420+0000 7efc7fe1f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d00da95d40 con 0x55d00dbd8120 2024-04-27T03:30:31.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.420+0000 7efc7fe1f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55d00da684d0 con 0x55d00dbd6fb0 2024-04-27T03:30:31.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.421+0000 7efc7d925640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d00dbd6fb0 0x55d00dbd73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.421+0000 7efc7d925640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d00dbd6fb0 0x55d00dbd73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51896/0 (socket says 172.21.15.165:51896) 2024-04-27T03:30:31.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.421+0000 7efc7d925640 1 -- 172.21.15.165:0/2845797193 learned_addr learned my addr 172.21.15.165:0/2845797193 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.421+0000 7efc7d925640 1 -- 172.21.15.165:0/2845797193 >> v1:172.21.15.89:6789/0 conn(0x55d00dbd8120 legacy=0x55d00dbd85a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.421+0000 7efc7d925640 1 -- 172.21.15.165:0/2845797193 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00da94590 con 0x55d00dbd6fb0 2024-04-27T03:30:31.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.423+0000 7efc7d925640 1 --2- 172.21.15.165:0/2845797193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dbd73d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc7000a520 tx=0x7efc7001d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73290c9b4b52f5d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.423+0000 7efc7d124640 1 -- 172.21.15.165:0/2845797193 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc70057070 con 0x55d00dbd6fb0 2024-04-27T03:30:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.423+0000 7efc7d124640 1 -- 172.21.15.165:0/2845797193 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc70052570 con 0x55d00dbd6fb0 2024-04-27T03:30:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.423+0000 7efc7d124640 1 -- 172.21.15.165:0/2845797193 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc7004ae60 con 0x55d00dbd6fb0 2024-04-27T03:30:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.424+0000 7efc7fe1f800 1 -- 172.21.15.165:0/2845797193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 msgr2=0x55d00dbd73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.424+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/2845797193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dbd73d0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7efc7000a520 tx=0x7efc7001d010 comp rx=0 tx=0).stop 2024-04-27T03:30:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.424+0000 7efc7fe1f800 1 -- 172.21.15.165:0/2845797193 shutdown_connections 2024-04-27T03:30:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.424+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/2845797193 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dbd73d0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.424+0000 7efc7fe1f800 1 -- 172.21.15.165:0/2845797193 >> 172.21.15.165:0/2845797193 conn(0x55d00dbd2550 msgr2=0x55d00dbd4970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.424+0000 7efc7fe1f800 1 -- 172.21.15.165:0/2845797193 shutdown_connections 2024-04-27T03:30:31.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.424+0000 7efc7fe1f800 1 -- 172.21.15.165:0/2845797193 wait complete. 2024-04-27T03:30:31.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.435+0000 7efc7fe1f800 1 Processor -- start 2024-04-27T03:30:31.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.435+0000 7efc7fe1f800 1 -- start start 2024-04-27T03:30:31.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.436+0000 7efc7fe1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dcc85e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.436+0000 7efc7fe1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d00db223e0 con 0x55d00dbd6fb0 2024-04-27T03:30:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.436+0000 7efc7d925640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dcc85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.436+0000 7efc7d925640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dcc85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51902/0 (socket says 172.21.15.165:51902) 2024-04-27T03:30:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.436+0000 7efc7d925640 1 -- 172.21.15.165:0/325588673 learned_addr learned my addr 172.21.15.165:0/325588673 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.436+0000 7efc7d925640 1 -- 172.21.15.165:0/325588673 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00dcc8bb0 con 0x55d00dbd6fb0 2024-04-27T03:30:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.437+0000 7efc7d925640 1 --2- 172.21.15.165:0/325588673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dcc85e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efc7001d050 tx=0x7efc7004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9bb8cee69583845 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.437+0000 7efb6be07640 1 -- 172.21.15.165:0/325588673 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc70057040 con 0x55d00dbd6fb0 2024-04-27T03:30:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.437+0000 7efb6be07640 1 -- 172.21.15.165:0/325588673 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc70052720 con 0x55d00dbd6fb0 2024-04-27T03:30:31.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.437+0000 7efb6be07640 1 -- 172.21.15.165:0/325588673 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc70052a40 con 0x55d00dbd6fb0 2024-04-27T03:30:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.437+0000 7efc7fe1f800 1 -- 172.21.15.165:0/325588673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 msgr2=0x55d00dcc85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.437+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/325588673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dcc85e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7efc7001d050 tx=0x7efc7004c000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.437+0000 7efc7fe1f800 1 -- 172.21.15.165:0/325588673 shutdown_connections 2024-04-27T03:30:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.437+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/325588673 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dcc85e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.437+0000 7efc7fe1f800 1 -- 172.21.15.165:0/325588673 >> 172.21.15.165:0/325588673 conn(0x55d00dbd2550 msgr2=0x55d00dcc3f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.437+0000 7efc7fe1f800 1 -- 172.21.15.165:0/325588673 shutdown_connections 2024-04-27T03:30:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.437+0000 7efc7fe1f800 1 -- 172.21.15.165:0/325588673 wait complete. 2024-04-27T03:30:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.438+0000 7efc7fe1f800 1 Processor -- start 2024-04-27T03:30:31.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.438+0000 7efc7fe1f800 1 -- start start 2024-04-27T03:30:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.438+0000 7efc7fe1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dccfff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.438+0000 7efc7fe1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d00dbe0590 con 0x55d00dbd6fb0 2024-04-27T03:30:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.438+0000 7efc7d925640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dccfff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.438+0000 7efc7d925640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dccfff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51910/0 (socket says 172.21.15.165:51910) 2024-04-27T03:30:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.438+0000 7efc7d925640 1 -- 172.21.15.165:0/1677978569 learned_addr learned my addr 172.21.15.165:0/1677978569 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.438+0000 7efc7d925640 1 -- 172.21.15.165:0/1677978569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00dcd0530 con 0x55d00dbd6fb0 2024-04-27T03:30:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.439+0000 7efc7d925640 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dccfff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efc7001cfe0 tx=0x7efc70003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.439+0000 7efb6a604640 1 -- 172.21.15.165:0/1677978569 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc70073070 con 0x55d00dbd6fb0 2024-04-27T03:30:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.439+0000 7efb6a604640 1 -- 172.21.15.165:0/1677978569 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc70052da0 con 0x55d00dbd6fb0 2024-04-27T03:30:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.439+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d00dcd07c0 con 0x55d00dbd6fb0 2024-04-27T03:30:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.439+0000 7efb6a604640 1 -- 172.21.15.165:0/1677978569 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc70057070 con 0x55d00dbd6fb0 2024-04-27T03:30:31.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.439+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d00dcd3470 con 0x55d00dbd6fb0 2024-04-27T03:30:31.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.440+0000 7efb6a604640 1 -- 172.21.15.165:0/1677978569 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7efc70076070 con 0x55d00dbd6fb0 2024-04-27T03:30:31.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.440+0000 7efb6a604640 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb64036900 0x7efb64038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.440+0000 7efb6a604640 1 -- 172.21.15.165:0/1677978569 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7efc7008d9e0 con 0x55d00dbd6fb0 2024-04-27T03:30:31.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.440+0000 7efc75124640 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb64036900 0x7efb64038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.440+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d00dbd8120 0x55d00dcd7230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.440+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dcd7800 con 0x55d00dbd8120 2024-04-27T03:30:31.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.440+0000 7efc7e126640 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d00dbd8120 0x55d00dcd7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.440+0000 7efc75124640 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb64036900 0x7efb64038dc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc68001e10 tx=0x7efc68001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.440+0000 7efc7e126640 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d00dbd8120 0x55d00dcd7230 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.441+0000 7efc7e126640 1 -- 172.21.15.165:0/1677978569 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dbd8120 2024-04-27T03:30:31.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.441+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dcd4800 con 0x55d00dbd8120 2024-04-27T03:30:31.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.441+0000 7efc7e126640 1 -- 172.21.15.165:0/1677978569 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dbd8120 2024-04-27T03:30:31.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.441+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d00dcd9b30 0x55d00dcdbf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.441+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dcdc5c0 con 0x55d00dcd9b30 2024-04-27T03:30:31.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.441+0000 7efc7d925640 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d00dcd9b30 0x55d00dcdbf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.441+0000 7efc7d925640 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d00dcd9b30 0x55d00dcdbf20 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.442+0000 7efc7d925640 1 -- 172.21.15.165:0/1677978569 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7efc70075030 con 0x55d00dcd9b30 2024-04-27T03:30:31.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.442+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dcd8ae0 con 0x55d00dbd8120 2024-04-27T03:30:31.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.442+0000 7efc7e126640 1 -- 172.21.15.165:0/1677978569 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dbd8120 2024-04-27T03:30:31.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.442+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d00dcdda70 0x55d00dcdfec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.442+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dce0620 con 0x55d00dcdda70 2024-04-27T03:30:31.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.442+0000 7efc75124640 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d00dcdda70 0x55d00dcdfec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.442+0000 7efc75124640 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d00dcdda70 0x55d00dcdfec0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.443+0000 7efc75124640 1 -- 172.21.15.165:0/1677978569 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7efc680106a0 con 0x55d00dcdda70 2024-04-27T03:30:31.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.443+0000 7efc7fe1f800 1 Processor -- start 2024-04-27T03:30:31.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.443+0000 7efc7fe1f800 1 -- start start 2024-04-27T03:30:31.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.443+0000 7efc7fe1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dceea80 0x55d00dceeea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.443+0000 7efc7fe1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d00dac86b0 con 0x55d00dceea80 2024-04-27T03:30:31.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.443+0000 7efc7d925640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dceea80 0x55d00dceeea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.443+0000 7efc7d925640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dceea80 0x55d00dceeea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51924/0 (socket says 172.21.15.165:51924) 2024-04-27T03:30:31.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.443+0000 7efc7d925640 1 -- 172.21.15.165:0/4196069773 learned_addr learned my addr 172.21.15.165:0/4196069773 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.444+0000 7efc7d925640 1 -- 172.21.15.165:0/4196069773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00dcef3e0 con 0x55d00dceea80 2024-04-27T03:30:31.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.444+0000 7efc7d925640 1 --2- 172.21.15.165:0/4196069773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dceea80 0x55d00dceeea0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc70006b30 tx=0x7efc7005dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.444+0000 7efb637fe640 1 -- 172.21.15.165:0/4196069773 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc70092070 con 0x55d00dceea80 2024-04-27T03:30:31.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.444+0000 7efb637fe640 1 -- 172.21.15.165:0/4196069773 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc70008010 con 0x55d00dceea80 2024-04-27T03:30:31.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.444+0000 7efc7fe1f800 1 -- 172.21.15.165:0/4196069773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d00dcefe50 con 0x55d00dceea80 2024-04-27T03:30:31.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.444+0000 7efb637fe640 1 -- 172.21.15.165:0/4196069773 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc7008d340 con 0x55d00dceea80 2024-04-27T03:30:31.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.444+0000 7efc7fe1f800 1 -- 172.21.15.165:0/4196069773 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d00dcf0350 con 0x55d00dceea80 2024-04-27T03:30:31.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.445+0000 7efb637fe640 1 -- 172.21.15.165:0/4196069773 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7efc70095070 con 0x55d00dceea80 2024-04-27T03:30:31.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.445+0000 7efb637fe640 1 --2- 172.21.15.165:0/4196069773 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb58036950 0x7efb58038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.445+0000 7efb637fe640 1 -- 172.21.15.165:0/4196069773 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7efc70089030 con 0x55d00dceea80 2024-04-27T03:30:31.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.445+0000 7efc75124640 1 --2- 172.21.15.165:0/4196069773 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb58036950 0x7efb58038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.445+0000 7efc7fe1f800 1 Processor -- start 2024-04-27T03:30:31.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.445+0000 7efc7fe1f800 1 -- start start 2024-04-27T03:30:31.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.445+0000 7efc75124640 1 --2- 172.21.15.165:0/4196069773 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb58036950 0x7efb58038e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55d00dcd4720 tx=0x7efc68006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.445+0000 7efc7fe1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 0x55d00dcfc090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.445+0000 7efc7fe1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d00dcc8e70 con 0x55d00dcfbc70 2024-04-27T03:30:31.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.446+0000 7efc7d925640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 0x55d00dcfc090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.446+0000 7efc7d925640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 0x55d00dcfc090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51932/0 (socket says 172.21.15.165:51932) 2024-04-27T03:30:31.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.446+0000 7efc7d925640 1 -- 172.21.15.165:0/1579822894 learned_addr learned my addr 172.21.15.165:0/1579822894 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.446+0000 7efc7d925640 1 -- 172.21.15.165:0/1579822894 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00dcfc660 con 0x55d00dcfbc70 2024-04-27T03:30:31.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.446+0000 7efc7d925640 1 --2- 172.21.15.165:0/1579822894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 0x55d00dcfc090 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efc7005afd0 tx=0x7efc7008d8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88d5a9a6e395d72d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.446+0000 7efb617fa640 1 -- 172.21.15.165:0/1579822894 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc70081c70 con 0x55d00dcfbc70 2024-04-27T03:30:31.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.446+0000 7efb617fa640 1 -- 172.21.15.165:0/1579822894 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc700a9070 con 0x55d00dcfbc70 2024-04-27T03:30:31.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.446+0000 7efb617fa640 1 -- 172.21.15.165:0/1579822894 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc70078c20 con 0x55d00dcfbc70 2024-04-27T03:30:31.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1579822894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 msgr2=0x55d00dcfc090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1579822894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 0x55d00dcfc090 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7efc7005afd0 tx=0x7efc7008d8b0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1579822894 shutdown_connections 2024-04-27T03:30:31.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1579822894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 0x55d00dcfc090 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1579822894 >> 172.21.15.165:0/1579822894 conn(0x55d00dcf7f40 msgr2=0x55d00dcf5fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1579822894 shutdown_connections 2024-04-27T03:30:31.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1579822894 wait complete. 2024-04-27T03:30:31.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7fe1f800 1 Processor -- start 2024-04-27T03:30:31.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7fe1f800 1 -- start start 2024-04-27T03:30:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7fe1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 0x55d00dd03020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7fe1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d00dcfccc0 con 0x55d00dcfbc70 2024-04-27T03:30:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7d925640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 0x55d00dd03020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7d925640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 0x55d00dd03020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51936/0 (socket says 172.21.15.165:51936) 2024-04-27T03:30:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.447+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 learned_addr learned my addr 172.21.15.165:0/1117338494 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.448+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d00dd03560 con 0x55d00dcfbc70 2024-04-27T03:30:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.448+0000 7efc7d925640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 0x55d00dd03020 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc700789a0 tx=0x7efc70086ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.448+0000 7efb4f7fe640 1 -- 172.21.15.165:0/1117338494 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc700766f0 con 0x55d00dcfbc70 2024-04-27T03:30:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.448+0000 7efb4f7fe640 1 -- 172.21.15.165:0/1117338494 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efc70077730 con 0x55d00dcfbc70 2024-04-27T03:30:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.448+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d00dd03560 con 0x55d00dcfbc70 2024-04-27T03:30:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.448+0000 7efb4f7fe640 1 -- 172.21.15.165:0/1117338494 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efc7008a2d0 con 0x55d00dcfbc70 2024-04-27T03:30:31.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.448+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d00dd03560 con 0x55d00dcfbc70 2024-04-27T03:30:31.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.449+0000 7efb4f7fe640 1 -- 172.21.15.165:0/1117338494 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7efc700a3070 con 0x55d00dcfbc70 2024-04-27T03:30:31.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.449+0000 7efb4f7fe640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb48036950 0x7efb48038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.449+0000 7efb4f7fe640 1 -- 172.21.15.165:0/1117338494 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7efc700b9ae0 con 0x55d00dcfbc70 2024-04-27T03:30:31.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.449+0000 7efc75124640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb48036950 0x7efb48038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.450+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.450+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d00dd0e730 0x55d00dd10bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.450+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd11340 con 0x55d00dd0e730 2024-04-27T03:30:31.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.450+0000 7efc75124640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb48036950 0x7efb48038e10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55d00dce0a20 tx=0x7efc68011000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.450+0000 7efc7e126640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d00dd0e730 0x55d00dd10bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.450+0000 7efc7e126640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d00dd0e730 0x55d00dd10bf0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.450+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 20 realm 2024-04-27T03:30:31.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd11750 con 0x55d00dd0e730 2024-04-27T03:30:31.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:31.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd12020 con 0x55d00dd0e730 2024-04-27T03:30:31.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd0e730 2024-04-27T03:30:31.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:31.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d00dd13120 0x55d00dd15510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd15b20 con 0x55d00dd13120 2024-04-27T03:30:31.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.451+0000 7efc7d925640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d00dd13120 0x55d00dd15510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.452+0000 7efc7d925640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d00dd13120 0x55d00dd15510 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.452+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.452+0000 7efc7fe1f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:31.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.452+0000 7efc7fe1f800 20 searching for the correct realm 2024-04-27T03:30:31.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.452+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd17450 con 0x55d00dd13120 2024-04-27T03:30:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.452+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.452+0000 7efc7d925640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7efc70097060 0x7efc70097780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.452+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd17450 con 0x7efc70097060 2024-04-27T03:30:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.453+0000 7efc75124640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7efc70097060 0x7efc70097780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.453+0000 7efc75124640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7efc70097060 0x7efc70097780 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.453+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc68036040 con 0x7efc70097060 2024-04-27T03:30:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.453+0000 7efc75124640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7efc680393c0 0x7efc68039810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.453+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd17450 con 0x7efc680393c0 2024-04-27T03:30:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.453+0000 7efc7e126640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7efc680393c0 0x7efc68039810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.454+0000 7efc7e126640 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7efc680393c0 0x7efc68039810 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.454+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.454+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd0e730 2024-04-27T03:30:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.454+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.454+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd0e730 2024-04-27T03:30:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.454+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.454+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x7efc680393c0 2024-04-27T03:30:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.455+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.455+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x7efc680393c0 2024-04-27T03:30:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.455+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.455+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x7efc70097060 2024-04-27T03:30:31.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.455+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc68036040 con 0x7efc70097060 2024-04-27T03:30:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.455+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd13120 2024-04-27T03:30:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.455+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.455+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd0e730 2024-04-27T03:30:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.456+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.456+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x7efc680393c0 2024-04-27T03:30:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.456+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.456+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd0e730 2024-04-27T03:30:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.456+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.456+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x7efc70097060 2024-04-27T03:30:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.456+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7efc68036040 con 0x7efc70097060 2024-04-27T03:30:31.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.456+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd13120 2024-04-27T03:30:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.457+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.457+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd13120 2024-04-27T03:30:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.457+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.457+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x7efc680393c0 2024-04-27T03:30:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.457+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.457+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd13120 2024-04-27T03:30:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.457+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.457+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd13120 2024-04-27T03:30:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.457+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.457+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd0e730 2024-04-27T03:30:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.458+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.458+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd0e730 2024-04-27T03:30:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.458+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.458+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x7efc680393c0 2024-04-27T03:30:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.458+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.458+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd13120 2024-04-27T03:30:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.459+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.459+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x7efc70097060 2024-04-27T03:30:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.459+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc68036040 con 0x7efc70097060 2024-04-27T03:30:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.459+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd0e730 2024-04-27T03:30:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.459+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.459+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x7efc680393c0 2024-04-27T03:30:31.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.459+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.459+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd13120 2024-04-27T03:30:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.460+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.460+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x7efc680393c0 2024-04-27T03:30:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.460+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.460+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x7efc70097060 2024-04-27T03:30:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.460+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc68036040 con 0x7efc70097060 2024-04-27T03:30:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.460+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd13120 2024-04-27T03:30:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.460+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.460+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x7efc680393c0 2024-04-27T03:30:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd0e730 2024-04-27T03:30:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7efc78038040 con 0x7efc70097060 2024-04-27T03:30:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7efc68036040 con 0x7efc70097060 2024-04-27T03:30:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7fe1f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7fe1f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7fe1f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7fe1f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7fe1f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7fe1f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd123c0 con 0x55d00dd0e730 2024-04-27T03:30:31.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.461+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd163e0 con 0x55d00dd0e730 2024-04-27T03:30:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd163e0 con 0x7efc680393c0 2024-04-27T03:30:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 20 zone default found 2024-04-27T03:30:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 4 Realm: () 2024-04-27T03:30:31.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 20 zonegroup default 2024-04-27T03:30:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd16d40 con 0x7efc680393c0 2024-04-27T03:30:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.462+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd16d40 con 0x55d00dd13120 2024-04-27T03:30:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.463+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.463+0000 7efc7fe1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.463+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.463+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd192b0 con 0x55d00dd13120 2024-04-27T03:30:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.463+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.463+0000 7efc7fe1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.463+0000 7efc7fe1f800 20 started sync module instance, tier type = 2024-04-27T03:30:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.463+0000 7efc7fe1f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.463+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd18750 con 0x7efc70097060 2024-04-27T03:30:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.464+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc68036040 con 0x7efc70097060 2024-04-27T03:30:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.464+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94352073392800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd19650 con 0x7efc70097060 2024-04-27T03:30:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.464+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1a010 con 0x55d00dd0e730 2024-04-27T03:30:31.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.465+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94352073392800] v25'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc68036040 con 0x7efc70097060 2024-04-27T03:30:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.465+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.465+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94352073399136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1a430 con 0x55d00dd0e730 2024-04-27T03:30:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.465+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1ae30 con 0x55d00dd13120 2024-04-27T03:30:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.466+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.466+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94352073402688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1b6d0 con 0x55d00dd13120 2024-04-27T03:30:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.466+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1c1e0 con 0x7efc70097060 2024-04-27T03:30:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.466+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94352073399136] v25'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.467+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc68036040 con 0x7efc70097060 2024-04-27T03:30:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.467+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94352073402688] v25'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.467+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94352073400192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1ca80 con 0x7efc70097060 2024-04-27T03:30:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.467+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1b220 con 0x55d00dd0e730 2024-04-27T03:30:31.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.468+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.468+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94352073412496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1b220 con 0x55d00dd0e730 2024-04-27T03:30:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.468+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1de80 con 0x7efc680393c0 2024-04-27T03:30:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.468+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94352073400192] v25'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc68036040 con 0x7efc70097060 2024-04-27T03:30:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.469+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94352073412496] v25'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.469+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.469+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94352073413552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1d9d0 con 0x7efc680393c0 2024-04-27T03:30:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.469+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1e390 con 0x7efc680393c0 2024-04-27T03:30:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.470+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94352073413552] v25'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.471+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.471+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94352073410000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1e390 con 0x7efc680393c0 2024-04-27T03:30:31.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.471+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1ed50 con 0x7efc680393c0 2024-04-27T03:30:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.471+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'45 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.471+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94352073410000] v25'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.471+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94352073418912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1f170 con 0x7efc680393c0 2024-04-27T03:30:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.471+0000 7efc7fe1f800 20 add_watcher() i=0 2024-04-27T03:30:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.471+0000 7efc7fe1f800 20 add_watcher() i=1 2024-04-27T03:30:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.471+0000 7efc7fe1f800 20 add_watcher() i=2 2024-04-27T03:30:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.471+0000 7efc7fe1f800 20 add_watcher() i=3 2024-04-27T03:30:31.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.471+0000 7efc7fe1f800 20 add_watcher() i=4 2024-04-27T03:30:31.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.471+0000 7efc7fe1f800 20 add_watcher() i=5 2024-04-27T03:30:31.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.472+0000 7efc7fe1f800 20 add_watcher() i=6 2024-04-27T03:30:31.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.472+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94352073418912] v25'46 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.472+0000 7efc7fe1f800 20 add_watcher() i=7 2024-04-27T03:30:31.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.472+0000 7efc7fe1f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:31.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.473+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd221a0 con 0x55d00dd0e730 2024-04-27T03:30:31.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.473+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.473+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94352073419968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1f590 con 0x55d00dd0e730 2024-04-27T03:30:31.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.474+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94352073419968] v25'70 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.474+0000 7efb2f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.474+0000 7efc7fe1f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:31.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.474+0000 7efc7fe1f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:31.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.474+0000 7efc7fe1f800 5 note: GC not initialized 2024-04-27T03:30:31.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.474+0000 7efb2e7fc640 20 reqs_thread_entry: start 2024-04-27T03:30:31.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.474+0000 7efc7fe1f800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:31.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.474+0000 7efc7fe1f800 20 Filter name: none 2024-04-27T03:30:31.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.474+0000 7efa9ffff640 20 reqs_thread_entry: start 2024-04-27T03:30:31.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:31.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd55490 con 0x55d00dd0e730 2024-04-27T03:30:31.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:31.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:31.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:31.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:31.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-27T03:30:31.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd55f00 con 0x7efc680393c0 2024-04-27T03:30:31.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-27T03:30:31.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T03:30:31.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T03:30:31.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.475+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~502 in=502b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd5b8b0 con 0x55d00dd0e730 2024-04-27T03:30:31.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~502] v25'4 uv4 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efc7fe1f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:31.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efc7fe1f800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:31.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efc7fe1f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:31.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94352073667360 in=690b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd56fa0 con 0x55d00dd0e730 2024-04-27T03:30:31.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify (1) cookie 94352073412496 notify 107374182403 ret 0) ==== 720+0+0 (crc 0 0 0) 0x7efc7801e040 con 0x55d00dd0e730 2024-04-27T03:30:31.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efb617fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182403 cookie 94352073412496 notifier 4329 bl.length()=678 2024-04-27T03:30:31.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efb617fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:31.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efb617fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:31.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(65 notify.4 [notify cookie 94352073667360 out=8b] v0'0 uv45 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efb617fa640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7efb40003ea0 con 0x55d00dd0e730 2024-04-27T03:30:31.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== watch-notify(notify_complete (2) cookie 94352073667360 notify 107374182403 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7efc78027750 con 0x55d00dd0e730 2024-04-27T03:30:31.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.477+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd5b320 con 0x7efc680393c0 2024-04-27T03:30:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v25'1 uv1 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efc7fe1f800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T03:30:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efc7fe1f800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T03:30:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efc7fe1f800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T03:30:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94352073667360 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd56af0 con 0x55d00dd13120 2024-04-27T03:30:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== watch-notify(notify (1) cookie 94352073402688 notify 107374182400 ret 0) ==== 219+0+0 (crc 0 0 0) 0x55d00dcefe50 con 0x55d00dd13120 2024-04-27T03:30:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efb617fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182400 cookie 94352073402688 notifier 4329 bl.length()=177 2024-04-27T03:30:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efb617fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T03:30:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efb617fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T03:30:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(68 notify.2 [notify cookie 94352073667360 out=8b] v0'0 uv45 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efb617fa640 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7efb40003ea0 con 0x55d00dd13120 2024-04-27T03:30:31.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.479+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== watch-notify(notify_complete (2) cookie 94352073667360 notify 107374182400 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d00dcef3e0 con 0x55d00dd13120 2024-04-27T03:30:31.510 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"},{"user":"foo","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-04-27T03:30:31.476915Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:31.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.480+0000 7efb2f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.480+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94352073419968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd573d0 con 0x55d00dd0e730 2024-04-27T03:30:31.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.481+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94352073419968] v25'71 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.481+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94352073392800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1ecc0 con 0x7efc70097060 2024-04-27T03:30:31.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.482+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94352073392800] v25'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc68036040 con 0x7efc70097060 2024-04-27T03:30:31.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.482+0000 7efc7fe1f800 20 remove_watcher() i=0 2024-04-27T03:30:31.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.482+0000 7efc7fe1f800 2 removed watcher, disabling cache 2024-04-27T03:30:31.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.482+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94352073399136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1ecc0 con 0x55d00dd0e730 2024-04-27T03:30:31.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.483+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94352073399136] v25'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.483+0000 7efc7fe1f800 20 remove_watcher() i=1 2024-04-27T03:30:31.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.483+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94352073402688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1ecc0 con 0x55d00dd13120 2024-04-27T03:30:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.485+0000 7efc7d925640 1 -- 172.21.15.165:0/1117338494 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94352073402688] v25'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc700be020 con 0x55d00dd13120 2024-04-27T03:30:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.485+0000 7efc7fe1f800 20 remove_watcher() i=2 2024-04-27T03:30:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.485+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94352073400192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1ecc0 con 0x7efc70097060 2024-04-27T03:30:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.486+0000 7efc75124640 1 -- 172.21.15.165:0/1117338494 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94352073400192] v25'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc68036040 con 0x7efc70097060 2024-04-27T03:30:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.486+0000 7efc7fe1f800 20 remove_watcher() i=3 2024-04-27T03:30:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.486+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94352073412496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1ecc0 con 0x55d00dd0e730 2024-04-27T03:30:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.487+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94352073412496] v25'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x55d00dd0e730 2024-04-27T03:30:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.487+0000 7efc7fe1f800 20 remove_watcher() i=4 2024-04-27T03:30:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.487+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94352073413552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1ecc0 con 0x7efc680393c0 2024-04-27T03:30:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.488+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94352073413552] v25'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.488+0000 7efc7fe1f800 20 remove_watcher() i=5 2024-04-27T03:30:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.488+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94352073410000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1ecc0 con 0x7efc680393c0 2024-04-27T03:30:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.489+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94352073410000] v25'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.489+0000 7efc7fe1f800 20 remove_watcher() i=6 2024-04-27T03:30:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.489+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94352073418912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d00dd1ecc0 con 0x7efc680393c0 2024-04-27T03:30:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94352073418912] v25'47 uv45 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7efc7800a4a0 con 0x7efc680393c0 2024-04-27T03:30:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 20 remove_watcher() i=7 2024-04-27T03:30:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7efc680393c0 msgr2=0x7efc68039810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7efc680393c0 0x7efc68039810 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d00dd0e730 msgr2=0x55d00dd10bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d00dd0e730 0x55d00dd10bf0 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7efc70097060 msgr2=0x7efc70097780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7efc70097060 0x7efc70097780 crc :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d00dd13120 msgr2=0x55d00dd15510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d00dd13120 0x55d00dd15510 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb48036950 msgr2=0x7efb48038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb48036950 0x7efb48038e10 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x55d00dce0a20 tx=0x7efc68011000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 msgr2=0x55d00dd03020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1117338494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dcfbc70 0x55d00dd03020 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7efc700789a0 tx=0x7efc70086ea0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 reap_dead start 2024-04-27T03:30:31.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7e126640 1 -- 172.21.15.165:0/1117338494 reap_dead start 2024-04-27T03:30:31.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 shutdown_connections 2024-04-27T03:30:31.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 >> 172.21.15.165:0/1117338494 conn(0x55d00dcf7f40 msgr2=0x55d00dcf55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 shutdown_connections 2024-04-27T03:30:31.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1117338494 wait complete. 2024-04-27T03:30:31.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/4196069773 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb58036950 msgr2=0x7efb58038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/4196069773 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb58036950 0x7efb58038e10 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x55d00dcd4720 tx=0x7efc68006000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/4196069773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dceea80 msgr2=0x55d00dceeea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/4196069773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dceea80 0x55d00dceeea0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7efc70006b30 tx=0x7efc7005dde0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/4196069773 shutdown_connections 2024-04-27T03:30:31.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/4196069773 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb58036950 0x7efb58038e10 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/4196069773 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dceea80 0x55d00dceeea0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.490+0000 7efc7fe1f800 1 -- 172.21.15.165:0/4196069773 >> 172.21.15.165:0/4196069773 conn(0x55d00dce2b20 msgr2=0x55d00dce2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 -- 172.21.15.165:0/4196069773 shutdown_connections 2024-04-27T03:30:31.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 -- 172.21.15.165:0/4196069773 wait complete. 2024-04-27T03:30:31.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d00dcdda70 msgr2=0x55d00dcdfec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d00dcdda70 0x55d00dcdfec0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d00dbd8120 msgr2=0x55d00dcd7230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d00dbd8120 0x55d00dcd7230 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d00dcd9b30 msgr2=0x55d00dcdbf20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d00dcd9b30 0x55d00dcdbf20 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb64036900 msgr2=0x7efb64038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efb64036900 0x7efb64038dc0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7efc68001e10 tx=0x7efc68001390 comp rx=0 tx=0).stop 2024-04-27T03:30:31.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 msgr2=0x55d00dccfff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 --2- 172.21.15.165:0/1677978569 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d00dbd6fb0 0x55d00dccfff0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7efc7001cfe0 tx=0x7efc70003020 comp rx=0 tx=0).stop 2024-04-27T03:30:31.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7e126640 1 -- 172.21.15.165:0/1677978569 reap_dead start 2024-04-27T03:30:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 shutdown_connections 2024-04-27T03:30:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 >> 172.21.15.165:0/1677978569 conn(0x55d00dbd2550 msgr2=0x55d00dcc2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 shutdown_connections 2024-04-27T03:30:31.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.491+0000 7efc7fe1f800 1 -- 172.21.15.165:0/1677978569 wait complete. 2024-04-27T03:30:31.523 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-04-27T03:30:31.476915Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:31.523 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T03:30:31.523 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T03:30:31.524 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T03:30:31.566 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:31.567 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:31.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.561+0000 7fe08c52b800 1 Processor -- start 2024-04-27T03:30:31.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.562+0000 7fe08c52b800 1 -- start start 2024-04-27T03:30:31.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.562+0000 7fe08c52b800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x564e2dbd21b0 0x564e2dbd45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.562+0000 7fe08c52b800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x564e2da8dd40 con 0x564e2dbcdd20 2024-04-27T03:30:31.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.562+0000 7fe08c52b800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x564e2da604d0 con 0x564e2dbd21b0 2024-04-27T03:30:31.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.562+0000 7fe07ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x564e2dbd21b0 0x564e2dbd45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.562+0000 7fe0882ef640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x564e2dbcdd20 0x564e2dbce140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53676/0 (socket says 172.21.15.165:53676) 2024-04-27T03:30:31.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.562+0000 7fe07ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x564e2dbd21b0 0x564e2dbd45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51952/0 (socket says 172.21.15.165:51952) 2024-04-27T03:30:31.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.562+0000 7fe0882ef640 1 -- 172.21.15.165:0/175894022 learned_addr learned my addr 172.21.15.165:0/175894022 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:31.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.562+0000 7fe07ffff640 1 -- 172.21.15.165:0/175894022 >> v1:172.21.15.89:6789/0 conn(0x564e2dbcdd20 legacy=0x564e2dbce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.562+0000 7fe07ffff640 1 -- 172.21.15.165:0/175894022 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2da8c590 con 0x564e2dbd21b0 2024-04-27T03:30:31.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.564+0000 7fe07ffff640 1 --2- 172.21.15.165:0/175894022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbd21b0 0x564e2dbd45a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe07000a470 tx=0x7fe07001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566aa1f90bb69c37 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.564+0000 7fe087aee640 1 -- 172.21.15.165:0/175894022 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe070057070 con 0x564e2dbd21b0 2024-04-27T03:30:31.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.564+0000 7fe087aee640 1 -- 172.21.15.165:0/175894022 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe070052570 con 0x564e2dbd21b0 2024-04-27T03:30:31.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.564+0000 7fe087aee640 1 -- 172.21.15.165:0/175894022 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe070052870 con 0x564e2dbd21b0 2024-04-27T03:30:31.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.565+0000 7fe08c52b800 1 -- 172.21.15.165:0/175894022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbd21b0 msgr2=0x564e2dbd45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.565+0000 7fe08c52b800 1 --2- 172.21.15.165:0/175894022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbd21b0 0x564e2dbd45a0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7fe07000a470 tx=0x7fe07001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:31.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.565+0000 7fe08c52b800 1 -- 172.21.15.165:0/175894022 shutdown_connections 2024-04-27T03:30:31.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.565+0000 7fe08c52b800 1 --2- 172.21.15.165:0/175894022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbd21b0 0x564e2dbd45a0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.565+0000 7fe08c52b800 1 -- 172.21.15.165:0/175894022 >> 172.21.15.165:0/175894022 conn(0x564e2dbc9380 msgr2=0x564e2dbcb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.565+0000 7fe08c52b800 1 -- 172.21.15.165:0/175894022 shutdown_connections 2024-04-27T03:30:31.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.565+0000 7fe08c52b800 1 -- 172.21.15.165:0/175894022 wait complete. 2024-04-27T03:30:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.577+0000 7fe08c52b800 1 Processor -- start 2024-04-27T03:30:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.577+0000 7fe08c52b800 1 -- start start 2024-04-27T03:30:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.577+0000 7fe08c52b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 0x564e2dcbefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.577+0000 7fe08c52b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564e2db1a3c0 con 0x564e2dbcdd20 2024-04-27T03:30:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.577+0000 7fe0882ef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 0x564e2dcbefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.577+0000 7fe0882ef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 0x564e2dcbefd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51962/0 (socket says 172.21.15.165:51962) 2024-04-27T03:30:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.577+0000 7fe0882ef640 1 -- 172.21.15.165:0/2979555510 learned_addr learned my addr 172.21.15.165:0/2979555510 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.577+0000 7fe0882ef640 1 -- 172.21.15.165:0/2979555510 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2dcbf5a0 con 0x564e2dbcdd20 2024-04-27T03:30:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.578+0000 7fe0882ef640 1 --2- 172.21.15.165:0/2979555510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 0x564e2dcbefd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe078001e10 tx=0x7fe07800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a9cd504d25833ee server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.578+0000 7fdf7660c640 1 -- 172.21.15.165:0/2979555510 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe07800b200 con 0x564e2dbcdd20 2024-04-27T03:30:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.578+0000 7fdf7660c640 1 -- 172.21.15.165:0/2979555510 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe078009460 con 0x564e2dbcdd20 2024-04-27T03:30:31.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.578+0000 7fdf7660c640 1 -- 172.21.15.165:0/2979555510 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe078009760 con 0x564e2dbcdd20 2024-04-27T03:30:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.578+0000 7fe08c52b800 1 -- 172.21.15.165:0/2979555510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 msgr2=0x564e2dcbefd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.578+0000 7fe08c52b800 1 --2- 172.21.15.165:0/2979555510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 0x564e2dcbefd0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fe078001e10 tx=0x7fe07800b910 comp rx=0 tx=0).stop 2024-04-27T03:30:31.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.578+0000 7fe08c52b800 1 -- 172.21.15.165:0/2979555510 shutdown_connections 2024-04-27T03:30:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.578+0000 7fe08c52b800 1 --2- 172.21.15.165:0/2979555510 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 0x564e2dcbefd0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.578+0000 7fe08c52b800 1 -- 172.21.15.165:0/2979555510 >> 172.21.15.165:0/2979555510 conn(0x564e2dbc9380 msgr2=0x564e2dcba8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.578+0000 7fe08c52b800 1 -- 172.21.15.165:0/2979555510 shutdown_connections 2024-04-27T03:30:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.579+0000 7fe08c52b800 1 -- 172.21.15.165:0/2979555510 wait complete. 2024-04-27T03:30:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.579+0000 7fe08c52b800 1 Processor -- start 2024-04-27T03:30:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.579+0000 7fe08c52b800 1 -- start start 2024-04-27T03:30:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.579+0000 7fe08c52b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 0x564e2dcc5a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.579+0000 7fe08c52b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564e2dbd4e70 con 0x564e2dbcdd20 2024-04-27T03:30:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.579+0000 7fe0882ef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 0x564e2dcc5a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.579+0000 7fe0882ef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 0x564e2dcc5a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51972/0 (socket says 172.21.15.165:51972) 2024-04-27T03:30:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.579+0000 7fe0882ef640 1 -- 172.21.15.165:0/1808823964 learned_addr learned my addr 172.21.15.165:0/1808823964 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.579+0000 7fe0882ef640 1 -- 172.21.15.165:0/1808823964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2dcc5fd0 con 0x564e2dbcdd20 2024-04-27T03:30:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.580+0000 7fe0882ef640 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 0x564e2dcc5a90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe07800cfd0 tx=0x7fe078000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.580+0000 7fdf74e09640 1 -- 172.21.15.165:0/1808823964 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe07801b040 con 0x564e2dbcdd20 2024-04-27T03:30:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.580+0000 7fdf74e09640 1 -- 172.21.15.165:0/1808823964 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe078025070 con 0x564e2dbcdd20 2024-04-27T03:30:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.580+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564e2dcc6260 con 0x564e2dbcdd20 2024-04-27T03:30:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.580+0000 7fdf74e09640 1 -- 172.21.15.165:0/1808823964 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe078016430 con 0x564e2dbcdd20 2024-04-27T03:30:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.580+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e2dcc8f10 con 0x564e2dbcdd20 2024-04-27T03:30:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.581+0000 7fdf74e09640 1 -- 172.21.15.165:0/1808823964 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fe0780165d0 con 0x564e2dbcdd20 2024-04-27T03:30:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.581+0000 7fdf74e09640 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf70036950 0x7fdf70038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.581+0000 7fdf74e09640 1 -- 172.21.15.165:0/1808823964 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fe078011070 con 0x564e2dbcdd20 2024-04-27T03:30:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.581+0000 7fe07ffff640 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf70036950 0x7fdf70038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.581+0000 7fe08c52b800 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564e2dbd21b0 0x564e2dccccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.581+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dccd250 con 0x564e2dbd21b0 2024-04-27T03:30:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.581+0000 7fe088af0640 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564e2dbd21b0 0x564e2dccccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.581+0000 7fe07ffff640 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf70036950 0x7fdf70038e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe070054000 tx=0x7fe07004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.581+0000 7fe088af0640 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564e2dbd21b0 0x564e2dccccd0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.582+0000 7fe088af0640 1 -- 172.21.15.165:0/1808823964 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dbd21b0 2024-04-27T03:30:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.582+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dcca2a0 con 0x564e2dbd21b0 2024-04-27T03:30:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.582+0000 7fe088af0640 1 -- 172.21.15.165:0/1808823964 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dbd21b0 2024-04-27T03:30:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.582+0000 7fe08c52b800 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564e2dccf580 0x564e2dcd1970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.582+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dcd2060 con 0x564e2dccf580 2024-04-27T03:30:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.582+0000 7fe0882ef640 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564e2dccf580 0x564e2dcd1970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.582+0000 7fe0882ef640 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564e2dccf580 0x564e2dcd1970 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.583+0000 7fe0882ef640 1 -- 172.21.15.165:0/1808823964 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe07801e040 con 0x564e2dccf580 2024-04-27T03:30:31.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.583+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dcce530 con 0x564e2dbd21b0 2024-04-27T03:30:31.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.583+0000 7fe088af0640 1 -- 172.21.15.165:0/1808823964 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dbd21b0 2024-04-27T03:30:31.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.583+0000 7fe08c52b800 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x564e2dcd3510 0x564e2dcd5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.583+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dcd60c0 con 0x564e2dcd3510 2024-04-27T03:30:31.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.583+0000 7fe07ffff640 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x564e2dcd3510 0x564e2dcd5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.584+0000 7fe07ffff640 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x564e2dcd3510 0x564e2dcd5960 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.584+0000 7fe07ffff640 1 -- 172.21.15.165:0/1808823964 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe070052940 con 0x564e2dcd3510 2024-04-27T03:30:31.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.584+0000 7fe08c52b800 1 Processor -- start 2024-04-27T03:30:31.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.584+0000 7fe08c52b800 1 -- start start 2024-04-27T03:30:31.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.584+0000 7fe08c52b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dce44b0 0x564e2dce48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.584+0000 7fe08c52b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564e2dac06b0 con 0x564e2dce44b0 2024-04-27T03:30:31.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.584+0000 7fe0882ef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dce44b0 0x564e2dce48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.585+0000 7fe0882ef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dce44b0 0x564e2dce48d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51978/0 (socket says 172.21.15.165:51978) 2024-04-27T03:30:31.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.585+0000 7fe0882ef640 1 -- 172.21.15.165:0/3342051422 learned_addr learned my addr 172.21.15.165:0/3342051422 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.585+0000 7fe0882ef640 1 -- 172.21.15.165:0/3342051422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2dce4e10 con 0x564e2dce44b0 2024-04-27T03:30:31.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.585+0000 7fe0882ef640 1 --2- 172.21.15.165:0/3342051422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dce44b0 0x564e2dce48d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe07804c430 tx=0x7fe078020e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.585+0000 7fdf6dffb640 1 -- 172.21.15.165:0/3342051422 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe078041940 con 0x564e2dce44b0 2024-04-27T03:30:31.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.585+0000 7fdf6dffb640 1 -- 172.21.15.165:0/3342051422 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe078041ae0 con 0x564e2dce44b0 2024-04-27T03:30:31.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.585+0000 7fe08c52b800 1 -- 172.21.15.165:0/3342051422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564e2dce5880 con 0x564e2dce44b0 2024-04-27T03:30:31.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.585+0000 7fdf6dffb640 1 -- 172.21.15.165:0/3342051422 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe078041c90 con 0x564e2dce44b0 2024-04-27T03:30:31.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.585+0000 7fe08c52b800 1 -- 172.21.15.165:0/3342051422 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e2dce5d80 con 0x564e2dce44b0 2024-04-27T03:30:31.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.586+0000 7fdf6dffb640 1 -- 172.21.15.165:0/3342051422 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fe078045ac0 con 0x564e2dce44b0 2024-04-27T03:30:31.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.586+0000 7fdf6dffb640 1 --2- 172.21.15.165:0/3342051422 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf60036900 0x7fdf60038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.586+0000 7fdf6dffb640 1 -- 172.21.15.165:0/3342051422 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fe078053020 con 0x564e2dce44b0 2024-04-27T03:30:31.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.586+0000 7fe07ffff640 1 --2- 172.21.15.165:0/3342051422 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf60036900 0x7fdf60038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fe08c52b800 1 Processor -- start 2024-04-27T03:30:31.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fe08c52b800 1 -- start start 2024-04-27T03:30:31.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fe07ffff640 1 --2- 172.21.15.165:0/3342051422 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf60036900 0x7fdf60038dc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x564e2dccdb30 tx=0x7fe070005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fe08c52b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 0x564e2dcf1990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fe08c52b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564e2dcbf920 con 0x564e2dcf1570 2024-04-27T03:30:31.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fe0882ef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 0x564e2dcf1990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fe0882ef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 0x564e2dcf1990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51980/0 (socket says 172.21.15.165:51980) 2024-04-27T03:30:31.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fe0882ef640 1 -- 172.21.15.165:0/2531022355 learned_addr learned my addr 172.21.15.165:0/2531022355 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fe0882ef640 1 -- 172.21.15.165:0/2531022355 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2dcf1f60 con 0x564e2dcf1570 2024-04-27T03:30:31.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fe0882ef640 1 --2- 172.21.15.165:0/2531022355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 0x564e2dcf1990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe0780146f0 tx=0x7fe07803bf50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c402fa5eaefe145d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fdf677fe640 1 -- 172.21.15.165:0/2531022355 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe078015a50 con 0x564e2dcf1570 2024-04-27T03:30:31.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fdf677fe640 1 -- 172.21.15.165:0/2531022355 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe078029580 con 0x564e2dcf1570 2024-04-27T03:30:31.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.587+0000 7fdf677fe640 1 -- 172.21.15.165:0/2531022355 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe07804c840 con 0x564e2dcf1570 2024-04-27T03:30:31.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.588+0000 7fe08c52b800 1 -- 172.21.15.165:0/2531022355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 msgr2=0x564e2dcf1990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.588+0000 7fe08c52b800 1 --2- 172.21.15.165:0/2531022355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 0x564e2dcf1990 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fe0780146f0 tx=0x7fe07803bf50 comp rx=0 tx=0).stop 2024-04-27T03:30:31.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.588+0000 7fe08c52b800 1 -- 172.21.15.165:0/2531022355 shutdown_connections 2024-04-27T03:30:31.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.588+0000 7fe08c52b800 1 --2- 172.21.15.165:0/2531022355 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 0x564e2dcf1990 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.588+0000 7fe08c52b800 1 -- 172.21.15.165:0/2531022355 >> 172.21.15.165:0/2531022355 conn(0x564e2dced8d0 msgr2=0x564e2dceb930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.588+0000 7fe08c52b800 1 -- 172.21.15.165:0/2531022355 shutdown_connections 2024-04-27T03:30:31.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.588+0000 7fe08c52b800 1 -- 172.21.15.165:0/2531022355 wait complete. 2024-04-27T03:30:31.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.588+0000 7fe08c52b800 1 Processor -- start 2024-04-27T03:30:31.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.588+0000 7fe08c52b800 1 -- start start 2024-04-27T03:30:31.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.588+0000 7fe08c52b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 0x564e2dcf8990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.588+0000 7fe08c52b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564e2dcf2610 con 0x564e2dcf1570 2024-04-27T03:30:31.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.589+0000 7fe0882ef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 0x564e2dcf8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.589+0000 7fe0882ef640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 0x564e2dcf8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51988/0 (socket says 172.21.15.165:51988) 2024-04-27T03:30:31.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.589+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 learned_addr learned my addr 172.21.15.165:0/3743857535 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.589+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e2dcf8ed0 con 0x564e2dcf1570 2024-04-27T03:30:31.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.589+0000 7fe0882ef640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 0x564e2dcf8990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe078029790 tx=0x7fe07803af60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.589+0000 7fdf65ffb640 1 -- 172.21.15.165:0/3743857535 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe078070020 con 0x564e2dcf1570 2024-04-27T03:30:31.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.589+0000 7fdf65ffb640 1 -- 172.21.15.165:0/3743857535 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe07802d100 con 0x564e2dcf1570 2024-04-27T03:30:31.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.589+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564e2dcf8ed0 con 0x564e2dcf1570 2024-04-27T03:30:31.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.589+0000 7fdf65ffb640 1 -- 172.21.15.165:0/3743857535 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe07804c840 con 0x564e2dcf1570 2024-04-27T03:30:31.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.589+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e2dcf8ed0 con 0x564e2dcf1570 2024-04-27T03:30:31.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.590+0000 7fdf65ffb640 1 -- 172.21.15.165:0/3743857535 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fe078073020 con 0x564e2dcf1570 2024-04-27T03:30:31.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.590+0000 7fdf65ffb640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf58036950 0x7fdf58038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.590+0000 7fdf65ffb640 1 -- 172.21.15.165:0/3743857535 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fe078067070 con 0x564e2dcf1570 2024-04-27T03:30:31.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.590+0000 7fe07ffff640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf58036950 0x7fdf58038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe08c52b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe08c52b800 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564e2dd040b0 0x564e2dd06570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd06cc0 con 0x564e2dd040b0 2024-04-27T03:30:31.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe088af0640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564e2dd040b0 0x564e2dd06570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe07ffff640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf58036950 0x7fdf58038e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe070052fd0 tx=0x7fe07004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe088af0640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564e2dd040b0 0x564e2dd06570 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe08c52b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe08c52b800 20 realm 2024-04-27T03:30:31.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe08c52b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd070d0 con 0x564e2dd040b0 2024-04-27T03:30:31.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe08c52b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe08c52b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:31.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe08c52b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.591+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd079a0 con 0x564e2dd040b0 2024-04-27T03:30:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe08c52b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe08c52b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd07d40 con 0x564e2dd040b0 2024-04-27T03:30:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe08c52b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe08c52b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe08c52b800 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564e2dd08830 0x564e2dd0ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0b380 con 0x564e2dd08830 2024-04-27T03:30:31.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe0882ef640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564e2dd08830 0x564e2dd0ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe0882ef640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564e2dd08830 0x564e2dd0ac20 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe08c52b800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe08c52b800 20 searching for the correct realm 2024-04-27T03:30:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.592+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0cd60 con 0x564e2dd08830 2024-04-27T03:30:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.593+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.593+0000 7fe0882ef640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe078075020 0x7fe078028780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.593+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0cd60 con 0x7fe078075020 2024-04-27T03:30:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.593+0000 7fe07ffff640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe078075020 0x7fe078028780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.593+0000 7fe07ffff640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe078075020 0x7fe078028780 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.594+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe070081070 con 0x7fe078075020 2024-04-27T03:30:31.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.594+0000 7fe07ffff640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe070076040 0x7fe070083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.594+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0cd60 con 0x7fe070076040 2024-04-27T03:30:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.594+0000 7fe088af0640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe070076040 0x7fe070083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.594+0000 7fe088af0640 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe070076040 0x7fe070083410 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.595+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.595+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd07d40 con 0x564e2dd040b0 2024-04-27T03:30:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.595+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.595+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd07d40 con 0x564e2dd040b0 2024-04-27T03:30:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.595+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.595+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd07d40 con 0x7fe070076040 2024-04-27T03:30:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.595+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.595+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd07d40 con 0x7fe070076040 2024-04-27T03:30:31.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.596+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.596+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd07d40 con 0x7fe078075020 2024-04-27T03:30:31.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.596+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe070081070 con 0x7fe078075020 2024-04-27T03:30:31.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.596+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd07d40 con 0x564e2dd08830 2024-04-27T03:30:31.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.596+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.596+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd07d40 con 0x564e2dd040b0 2024-04-27T03:30:31.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.596+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.596+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd07d40 con 0x7fe070076040 2024-04-27T03:30:31.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.596+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.596+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd07d40 con 0x564e2dd040b0 2024-04-27T03:30:31.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.597+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.597+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd07d40 con 0x7fe078075020 2024-04-27T03:30:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.597+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe070081070 con 0x7fe078075020 2024-04-27T03:30:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.597+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x564e2dd08830 2024-04-27T03:30:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.597+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.597+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x564e2dd08830 2024-04-27T03:30:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.597+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.597+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x7fe070076040 2024-04-27T03:30:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.598+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.598+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x564e2dd08830 2024-04-27T03:30:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.598+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.598+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x564e2dd08830 2024-04-27T03:30:31.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.598+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.598+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x564e2dd040b0 2024-04-27T03:30:31.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.598+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.598+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x564e2dd040b0 2024-04-27T03:30:31.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.599+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.599+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x7fe070076040 2024-04-27T03:30:31.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.599+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.599+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x564e2dd08830 2024-04-27T03:30:31.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.599+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.599+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x7fe078075020 2024-04-27T03:30:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.600+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe070081070 con 0x7fe078075020 2024-04-27T03:30:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.600+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x564e2dd040b0 2024-04-27T03:30:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.600+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.600+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x7fe070076040 2024-04-27T03:30:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.600+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.600+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x564e2dd08830 2024-04-27T03:30:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.600+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.600+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x7fe070076040 2024-04-27T03:30:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.601+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.601+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x7fe078075020 2024-04-27T03:30:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.601+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe070081070 con 0x7fe078075020 2024-04-27T03:30:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.601+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x564e2dd08830 2024-04-27T03:30:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.601+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.601+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x7fe070076040 2024-04-27T03:30:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.601+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.601+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe070090060 con 0x564e2dd040b0 2024-04-27T03:30:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fe080038040 con 0x7fe078075020 2024-04-27T03:30:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe070081070 con 0x7fe078075020 2024-04-27T03:30:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:31.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0bd30 con 0x564e2dd040b0 2024-04-27T03:30:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0c120 con 0x564e2dd040b0 2024-04-27T03:30:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.602+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0c120 con 0x7fe070076040 2024-04-27T03:30:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 20 zone default found 2024-04-27T03:30:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 4 Realm: () 2024-04-27T03:30:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 20 zonegroup default 2024-04-27T03:30:31.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0c540 con 0x7fe070076040 2024-04-27T03:30:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0c540 con 0x564e2dd08830 2024-04-27T03:30:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0ec10 con 0x564e2dd08830 2024-04-27T03:30:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 20 started sync module instance, tier type = 2024-04-27T03:30:31.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.603+0000 7fe08c52b800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:31.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.604+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0e0b0 con 0x7fe078075020 2024-04-27T03:30:31.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.604+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe070081070 con 0x7fe078075020 2024-04-27T03:30:31.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.604+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94893776102608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0f460 con 0x7fe078075020 2024-04-27T03:30:31.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.604+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0fcb0 con 0x564e2dd040b0 2024-04-27T03:30:31.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.606+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94893776102608] v25'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe070081070 con 0x7fe078075020 2024-04-27T03:30:31.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.606+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.606+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94893776105392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0fcf0 con 0x564e2dd040b0 2024-04-27T03:30:31.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.606+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd10800 con 0x564e2dd08830 2024-04-27T03:30:31.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.607+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94893776105392] v25'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.607+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.607+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94893776110256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd10bf0 con 0x564e2dd08830 2024-04-27T03:30:31.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.607+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd11610 con 0x7fe078075020 2024-04-27T03:30:31.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.608+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94893776110256] v25'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.608+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe070081070 con 0x7fe078075020 2024-04-27T03:30:31.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.608+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94893776113920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd11a30 con 0x7fe078075020 2024-04-27T03:30:31.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.608+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd12450 con 0x564e2dd040b0 2024-04-27T03:30:31.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.609+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94893776113920] v25'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe070081070 con 0x7fe078075020 2024-04-27T03:30:31.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.609+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.609+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94893776117568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd12840 con 0x564e2dd040b0 2024-04-27T03:30:31.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.609+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd13240 con 0x7fe070076040 2024-04-27T03:30:31.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.610+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.610+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94893776117568] v25'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.610+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94893776121168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd13170 con 0x7fe070076040 2024-04-27T03:30:31.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.610+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd13c80 con 0x7fe070076040 2024-04-27T03:30:31.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.611+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94893776121168] v25'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.611+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.611+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94893776107520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd13c80 con 0x7fe070076040 2024-04-27T03:30:31.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.611+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd14680 con 0x7fe070076040 2024-04-27T03:30:31.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.614+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94893776107520] v25'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.614+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'48 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.614+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94893776126352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd14860 con 0x7fe070076040 2024-04-27T03:30:31.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.614+0000 7fe08c52b800 20 add_watcher() i=0 2024-04-27T03:30:31.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.614+0000 7fe08c52b800 20 add_watcher() i=1 2024-04-27T03:30:31.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.614+0000 7fe08c52b800 20 add_watcher() i=2 2024-04-27T03:30:31.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.614+0000 7fe08c52b800 20 add_watcher() i=3 2024-04-27T03:30:31.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.614+0000 7fe08c52b800 20 add_watcher() i=4 2024-04-27T03:30:31.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.614+0000 7fe08c52b800 20 add_watcher() i=5 2024-04-27T03:30:31.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.614+0000 7fe08c52b800 20 add_watcher() i=6 2024-04-27T03:30:31.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.615+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94893776126352] v25'49 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.615+0000 7fe08c52b800 20 add_watcher() i=7 2024-04-27T03:30:31.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.615+0000 7fe08c52b800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:31.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.615+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd17c30 con 0x564e2dd040b0 2024-04-27T03:30:31.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.615+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.615+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94893776127824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd15020 con 0x564e2dd040b0 2024-04-27T03:30:31.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.616+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94893776127824] v25'72 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.616+0000 7fdf3e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.616+0000 7fe08c52b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:31.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.616+0000 7fe08c52b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:31.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.616+0000 7fe08c52b800 5 note: GC not initialized 2024-04-27T03:30:31.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.617+0000 7fdf3d7e2640 20 reqs_thread_entry: start 2024-04-27T03:30:31.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.617+0000 7fe08c52b800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:31.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.617+0000 7fe08c52b800 20 Filter name: none 2024-04-27T03:30:31.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.617+0000 7fdeaeffd640 20 reqs_thread_entry: start 2024-04-27T03:30:31.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.617+0000 7fe08c52b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.617+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd4a950 con 0x564e2dd040b0 2024-04-27T03:30:31.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.617+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~502 out=502b,getxattrs out=75b] v0'0 uv4 ondisk = 0) ==== 231+0+625 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.617+0000 7fe08c52b800 20 rados_obj.operate() r=0 bl.length=502 2024-04-27T03:30:31.640 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"},{"user":"foo","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-04-27T03:30:31.476915Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:31.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.618+0000 7fdf3e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.618+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94893776127824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd49520 con 0x564e2dd040b0 2024-04-27T03:30:31.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.619+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94893776127824] v25'73 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.619+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94893776102608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd14750 con 0x7fe078075020 2024-04-27T03:30:31.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.620+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94893776102608] v25'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe070081070 con 0x7fe078075020 2024-04-27T03:30:31.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.620+0000 7fe08c52b800 20 remove_watcher() i=0 2024-04-27T03:30:31.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.620+0000 7fe08c52b800 2 removed watcher, disabling cache 2024-04-27T03:30:31.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.620+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94893776105392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd14750 con 0x564e2dd040b0 2024-04-27T03:30:31.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.622+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94893776105392] v25'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.622+0000 7fe08c52b800 20 remove_watcher() i=1 2024-04-27T03:30:31.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.622+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94893776110256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0efb0 con 0x564e2dd08830 2024-04-27T03:30:31.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.623+0000 7fe0882ef640 1 -- 172.21.15.165:0/3743857535 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94893776110256] v25'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe07803b070 con 0x564e2dd08830 2024-04-27T03:30:31.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.623+0000 7fe08c52b800 20 remove_watcher() i=2 2024-04-27T03:30:31.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.623+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94893776113920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd14750 con 0x7fe078075020 2024-04-27T03:30:31.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.623+0000 7fe07ffff640 1 -- 172.21.15.165:0/3743857535 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94893776113920] v25'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe070081070 con 0x7fe078075020 2024-04-27T03:30:31.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.624+0000 7fe08c52b800 20 remove_watcher() i=3 2024-04-27T03:30:31.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.624+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94893776117568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd102b0 con 0x564e2dd040b0 2024-04-27T03:30:31.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.624+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94893776117568] v25'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x564e2dd040b0 2024-04-27T03:30:31.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.624+0000 7fe08c52b800 20 remove_watcher() i=4 2024-04-27T03:30:31.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.624+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94893776121168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd0efb0 con 0x7fe070076040 2024-04-27T03:30:31.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.625+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94893776121168] v25'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.625+0000 7fe08c52b800 20 remove_watcher() i=5 2024-04-27T03:30:31.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.625+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94893776107520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd14750 con 0x7fe070076040 2024-04-27T03:30:31.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.626+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94893776107520] v25'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.626+0000 7fe08c52b800 20 remove_watcher() i=6 2024-04-27T03:30:31.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.626+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94893776126352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564e2dd14750 con 0x7fe070076040 2024-04-27T03:30:31.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.627+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94893776126352] v25'50 uv48 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe08000a4a0 con 0x7fe070076040 2024-04-27T03:30:31.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.627+0000 7fe08c52b800 20 remove_watcher() i=7 2024-04-27T03:30:31.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.627+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe070076040 msgr2=0x7fe070083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.627+0000 7fe08c52b800 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe070076040 0x7fe070083410 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.627+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564e2dd040b0 msgr2=0x564e2dd06570 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.627+0000 7fe08c52b800 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564e2dd040b0 0x564e2dd06570 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.627+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe078075020 msgr2=0x7fe078028780 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.627+0000 7fe08c52b800 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe078075020 0x7fe078028780 crc :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.627+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564e2dd08830 msgr2=0x564e2dd0ac20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.627+0000 7fe08c52b800 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564e2dd08830 0x564e2dd0ac20 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.627+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf58036950 msgr2=0x7fdf58038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf58036950 0x7fdf58038e10 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fe070052fd0 tx=0x7fe07004f000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 msgr2=0x564e2dcf8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 --2- 172.21.15.165:0/3743857535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dcf1570 0x564e2dcf8990 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fe078029790 tx=0x7fe07803af60 comp rx=0 tx=0).stop 2024-04-27T03:30:31.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 reap_dead start 2024-04-27T03:30:31.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe088af0640 1 -- 172.21.15.165:0/3743857535 reap_dead start 2024-04-27T03:30:31.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 shutdown_connections 2024-04-27T03:30:31.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 >> 172.21.15.165:0/3743857535 conn(0x564e2dced8d0 msgr2=0x564e2dceaf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 shutdown_connections 2024-04-27T03:30:31.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/3743857535 wait complete. 2024-04-27T03:30:31.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/3342051422 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf60036900 msgr2=0x7fdf60038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 --2- 172.21.15.165:0/3342051422 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf60036900 0x7fdf60038dc0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x564e2dccdb30 tx=0x7fe070005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:31.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/3342051422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dce44b0 msgr2=0x564e2dce48d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 --2- 172.21.15.165:0/3342051422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dce44b0 0x564e2dce48d0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fe07804c430 tx=0x7fe078020e90 comp rx=0 tx=0).stop 2024-04-27T03:30:31.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/3342051422 shutdown_connections 2024-04-27T03:30:31.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 --2- 172.21.15.165:0/3342051422 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf60036900 0x7fdf60038dc0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 --2- 172.21.15.165:0/3342051422 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dce44b0 0x564e2dce48d0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/3342051422 >> 172.21.15.165:0/3342051422 conn(0x564e2dcd8550 msgr2=0x564e2dcd8950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/3342051422 shutdown_connections 2024-04-27T03:30:31.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/3342051422 wait complete. 2024-04-27T03:30:31.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x564e2dcd3510 msgr2=0x564e2dcd5960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x564e2dcd3510 0x564e2dcd5960 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564e2dbd21b0 msgr2=0x564e2dccccd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564e2dbd21b0 0x564e2dccccd0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564e2dccf580 msgr2=0x564e2dcd1970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564e2dccf580 0x564e2dcd1970 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf70036950 msgr2=0x7fdf70038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdf70036950 0x7fdf70038e10 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fe070054000 tx=0x7fe07004e000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 msgr2=0x564e2dcc5a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 --2- 172.21.15.165:0/1808823964 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564e2dbcdd20 0x564e2dcc5a90 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fe07800cfd0 tx=0x7fe078000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe088af0640 1 -- 172.21.15.165:0/1808823964 reap_dead start 2024-04-27T03:30:31.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 shutdown_connections 2024-04-27T03:30:31.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.628+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 >> 172.21.15.165:0/1808823964 conn(0x564e2dbc9380 msgr2=0x564e2dcb91b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.629+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 shutdown_connections 2024-04-27T03:30:31.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.629+0000 7fe08c52b800 1 -- 172.21.15.165:0/1808823964 wait complete. 2024-04-27T03:30:31.654 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-04-27T03:30:31.476915Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:31.654 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-04-27T03:30:31.654 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--uid', 'foo', '--access-key', 'p5YnriCv1nAtykxBrupQ'] 2024-04-27T03:30:31.654 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --uid foo --access-key p5YnriCv1nAtykxBrupQ 2024-04-27T03:30:31.732 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:31.733 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.728+0000 7facf8333800 1 Processor -- start 2024-04-27T03:30:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.728+0000 7facf8333800 1 -- start start 2024-04-27T03:30:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.728+0000 7facf8333800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x559ce8c69230 0x559ce8c6b620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.728+0000 7facf8333800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559ce8b24d40 con 0x559ce8c65280 2024-04-27T03:30:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.728+0000 7facf8333800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x559ce8af74d0 con 0x559ce8c69230 2024-04-27T03:30:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.728+0000 7facf37d0640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x559ce8c69230 0x559ce8c6b620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.728+0000 7facf3fd1640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x559ce8c65280 0x559ce8c656a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53682/0 (socket says 172.21.15.165:53682) 2024-04-27T03:30:31.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.728+0000 7facf3fd1640 1 -- 172.21.15.165:0/4139552349 learned_addr learned my addr 172.21.15.165:0/4139552349 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:31.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.729+0000 7facf37d0640 1 -- 172.21.15.165:0/4139552349 >> v1:172.21.15.89:6789/0 conn(0x559ce8c65280 legacy=0x559ce8c656a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.729+0000 7facf2fcf640 1 -- 172.21.15.165:0/4139552349 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1113217926 0 0) 0x559ce8b24d40 con 0x559ce8c65280 2024-04-27T03:30:31.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.729+0000 7facf37d0640 1 -- 172.21.15.165:0/4139552349 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce8b23590 con 0x559ce8c69230 2024-04-27T03:30:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.730+0000 7facf37d0640 1 --2- 172.21.15.165:0/4139552349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c69230 0x559ce8c6b620 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7face800a470 tx=0x7face801cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=829c1811a1723b13 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.731+0000 7facf2fcf640 1 -- 172.21.15.165:0/4139552349 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face8057070 con 0x559ce8c69230 2024-04-27T03:30:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.731+0000 7facf2fcf640 1 -- 172.21.15.165:0/4139552349 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face8052570 con 0x559ce8c69230 2024-04-27T03:30:31.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.731+0000 7facf2fcf640 1 -- 172.21.15.165:0/4139552349 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face8052870 con 0x559ce8c69230 2024-04-27T03:30:31.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.731+0000 7facf8333800 1 -- 172.21.15.165:0/4139552349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c69230 msgr2=0x559ce8c6b620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.731+0000 7facf8333800 1 --2- 172.21.15.165:0/4139552349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c69230 0x559ce8c6b620 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7face800a470 tx=0x7face801cf20 comp rx=0 tx=0).stop 2024-04-27T03:30:31.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.731+0000 7facf8333800 1 -- 172.21.15.165:0/4139552349 shutdown_connections 2024-04-27T03:30:31.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.731+0000 7facf8333800 1 --2- 172.21.15.165:0/4139552349 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c69230 0x559ce8c6b620 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.731+0000 7facf8333800 1 -- 172.21.15.165:0/4139552349 >> 172.21.15.165:0/4139552349 conn(0x559ce8c5fb20 msgr2=0x559ce8c61f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.731+0000 7facf8333800 1 -- 172.21.15.165:0/4139552349 shutdown_connections 2024-04-27T03:30:31.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.731+0000 7facf8333800 1 -- 172.21.15.165:0/4139552349 wait complete. 2024-04-27T03:30:31.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.743+0000 7facf8333800 1 Processor -- start 2024-04-27T03:30:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.743+0000 7facf8333800 1 -- start start 2024-04-27T03:30:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.743+0000 7facf8333800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 0x559ce8c64980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.743+0000 7facf8333800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559ce8bb13b0 con 0x559ce8c65280 2024-04-27T03:30:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.743+0000 7facf3fd1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 0x559ce8c64980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.744+0000 7facf3fd1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 0x559ce8c64980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51998/0 (socket says 172.21.15.165:51998) 2024-04-27T03:30:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.744+0000 7facf3fd1640 1 -- 172.21.15.165:0/306093024 learned_addr learned my addr 172.21.15.165:0/306093024 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.744+0000 7facf3fd1640 1 -- 172.21.15.165:0/306093024 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce8d59bf0 con 0x559ce8c65280 2024-04-27T03:30:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.744+0000 7facf3fd1640 1 --2- 172.21.15.165:0/306093024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 0x559ce8c64980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7face400b240 tx=0x7face400b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f330d1aa47909a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.744+0000 7fabe1e0b640 1 -- 172.21.15.165:0/306093024 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face4008070 con 0x559ce8c65280 2024-04-27T03:30:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.744+0000 7fabe1e0b640 1 -- 172.21.15.165:0/306093024 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face400d500 con 0x559ce8c65280 2024-04-27T03:30:31.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.744+0000 7fabe1e0b640 1 -- 172.21.15.165:0/306093024 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face400d860 con 0x559ce8c65280 2024-04-27T03:30:31.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.745+0000 7facf8333800 1 -- 172.21.15.165:0/306093024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 msgr2=0x559ce8c64980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.745+0000 7facf8333800 1 --2- 172.21.15.165:0/306093024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 0x559ce8c64980 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7face400b240 tx=0x7face400b700 comp rx=0 tx=0).stop 2024-04-27T03:30:31.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.745+0000 7facf8333800 1 -- 172.21.15.165:0/306093024 shutdown_connections 2024-04-27T03:30:31.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.745+0000 7facf8333800 1 --2- 172.21.15.165:0/306093024 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 0x559ce8c64980 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.745+0000 7facf8333800 1 -- 172.21.15.165:0/306093024 >> 172.21.15.165:0/306093024 conn(0x559ce8c5fb20 msgr2=0x559ce8d516e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.745+0000 7facf8333800 1 -- 172.21.15.165:0/306093024 shutdown_connections 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.745+0000 7facf8333800 1 -- 172.21.15.165:0/306093024 wait complete. 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.745+0000 7facf8333800 1 Processor -- start 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.745+0000 7facf8333800 1 -- start start 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.745+0000 7facf8333800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 0x559ce8d5eb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.745+0000 7facf8333800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559ce8c6be60 con 0x559ce8c65280 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.746+0000 7facf3fd1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 0x559ce8d5eb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.746+0000 7facf3fd1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 0x559ce8d5eb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52010/0 (socket says 172.21.15.165:52010) 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.746+0000 7facf3fd1640 1 -- 172.21.15.165:0/950685711 learned_addr learned my addr 172.21.15.165:0/950685711 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.746+0000 7facf3fd1640 1 -- 172.21.15.165:0/950685711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce8d5f070 con 0x559ce8c65280 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.746+0000 7facf3fd1640 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 0x559ce8d5eb30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7face4000f60 tx=0x7face4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.746+0000 7fabdbfff640 1 -- 172.21.15.165:0/950685711 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face4019870 con 0x559ce8c65280 2024-04-27T03:30:31.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.746+0000 7fabdbfff640 1 -- 172.21.15.165:0/950685711 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face4019a10 con 0x559ce8c65280 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.746+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559ce8d5f300 con 0x559ce8c65280 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.746+0000 7fabdbfff640 1 -- 172.21.15.165:0/950685711 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face4019d10 con 0x559ce8c65280 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.746+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce8d61fb0 con 0x559ce8c65280 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.747+0000 7fabdbfff640 1 -- 172.21.15.165:0/950685711 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7face4017040 con 0x559ce8c65280 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.747+0000 7fabdbfff640 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabdc036900 0x7fabdc038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.747+0000 7fabdbfff640 1 -- 172.21.15.165:0/950685711 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7face4015020 con 0x559ce8c65280 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.747+0000 7facf37d0640 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabdc036900 0x7fabdc038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.748+0000 7facf8333800 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559ce8c69230 0x559ce8d65d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.748+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8d662b0 con 0x559ce8c69230 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.748+0000 7facf47d2640 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559ce8c69230 0x559ce8d65d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.748+0000 7facf37d0640 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabdc036900 0x7fabdc038dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7face8054000 tx=0x7face804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.748+0000 7facf47d2640 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559ce8c69230 0x559ce8d65d70 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.748+0000 7facf47d2640 1 -- 172.21.15.165:0/950685711 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8c69230 2024-04-27T03:30:31.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.748+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8d63340 con 0x559ce8c69230 2024-04-27T03:30:31.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.749+0000 7facf47d2640 1 -- 172.21.15.165:0/950685711 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8c69230 2024-04-27T03:30:31.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.749+0000 7facf8333800 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559ce8d685a0 0x559ce8d6a990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.749+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8d6b080 con 0x559ce8d685a0 2024-04-27T03:30:31.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.749+0000 7facf3fd1640 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559ce8d685a0 0x559ce8d6a990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.749+0000 7facf3fd1640 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559ce8d685a0 0x559ce8d6a990 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.749+0000 7facf3fd1640 1 -- 172.21.15.165:0/950685711 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7face402d070 con 0x559ce8d685a0 2024-04-27T03:30:31.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.749+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8d67550 con 0x559ce8c69230 2024-04-27T03:30:31.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.750+0000 7facf47d2640 1 -- 172.21.15.165:0/950685711 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8c69230 2024-04-27T03:30:31.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.750+0000 7facf8333800 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559ce8d6c530 0x559ce8d6e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.750+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8d6f0e0 con 0x559ce8d6c530 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.750+0000 7facf37d0640 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559ce8d6c530 0x559ce8d6e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.750+0000 7facf37d0640 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559ce8d6c530 0x559ce8d6e980 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.750+0000 7facf37d0640 1 -- 172.21.15.165:0/950685711 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7face80528e0 con 0x559ce8d6c530 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.751+0000 7facf8333800 1 Processor -- start 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.751+0000 7facf8333800 1 -- start start 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.751+0000 7facf8333800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d7d4b0 0x559ce8d7d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.751+0000 7facf8333800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559ce8b576b0 con 0x559ce8d7d4b0 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.751+0000 7facf3fd1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d7d4b0 0x559ce8d7d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.751+0000 7facf3fd1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d7d4b0 0x559ce8d7d8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52020/0 (socket says 172.21.15.165:52020) 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.751+0000 7facf3fd1640 1 -- 172.21.15.165:0/3028630598 learned_addr learned my addr 172.21.15.165:0/3028630598 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.751+0000 7facf3fd1640 1 -- 172.21.15.165:0/3028630598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce8d7de10 con 0x559ce8d7d4b0 2024-04-27T03:30:31.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.751+0000 7facf3fd1640 1 --2- 172.21.15.165:0/3028630598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d7d4b0 0x559ce8d7d8d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7face400eb40 tx=0x7face401aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.752+0000 7fabd97fa640 1 -- 172.21.15.165:0/3028630598 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face4040950 con 0x559ce8d7d4b0 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.752+0000 7fabd97fa640 1 -- 172.21.15.165:0/3028630598 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face4040af0 con 0x559ce8d7d4b0 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.752+0000 7fabd97fa640 1 -- 172.21.15.165:0/3028630598 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face4040dd0 con 0x559ce8d7d4b0 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.752+0000 7facf8333800 1 -- 172.21.15.165:0/3028630598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559ce8d7e880 con 0x559ce8d7d4b0 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.752+0000 7facf8333800 1 -- 172.21.15.165:0/3028630598 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce8d7ed80 con 0x559ce8d7d4b0 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.752+0000 7fabd97fa640 1 -- 172.21.15.165:0/3028630598 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7face4049040 con 0x559ce8d7d4b0 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.752+0000 7fabd97fa640 1 --2- 172.21.15.165:0/3028630598 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabcc036900 0x7fabcc038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.752+0000 7fabd97fa640 1 -- 172.21.15.165:0/3028630598 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7face4051060 con 0x559ce8d7d4b0 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.753+0000 7facf37d0640 1 --2- 172.21.15.165:0/3028630598 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabcc036900 0x7fabcc038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.753+0000 7facf8333800 1 Processor -- start 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.753+0000 7facf8333800 1 -- start start 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.753+0000 7facf37d0640 1 --2- 172.21.15.165:0/3028630598 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabcc036900 0x7fabcc038dc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x559ce8d66b50 tx=0x7face8005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.753+0000 7facf8333800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 0x559ce8d559d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.753+0000 7facf8333800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559ce8d59f40 con 0x559ce8d572e0 2024-04-27T03:30:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.753+0000 7facf3fd1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 0x559ce8d559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.753+0000 7facf3fd1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 0x559ce8d559d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52024/0 (socket says 172.21.15.165:52024) 2024-04-27T03:30:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.753+0000 7facf3fd1640 1 -- 172.21.15.165:0/2663146724 learned_addr learned my addr 172.21.15.165:0/2663146724 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.753+0000 7facf3fd1640 1 -- 172.21.15.165:0/2663146724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce8d57790 con 0x559ce8d572e0 2024-04-27T03:30:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.754+0000 7facf3fd1640 1 --2- 172.21.15.165:0/2663146724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 0x559ce8d559d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7face401ac40 tx=0x7face400f530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d7d58622a7653e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.754+0000 7fabd2ffd640 1 -- 172.21.15.165:0/2663146724 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face4024340 con 0x559ce8d572e0 2024-04-27T03:30:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.754+0000 7fabd2ffd640 1 -- 172.21.15.165:0/2663146724 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face4050020 con 0x559ce8d572e0 2024-04-27T03:30:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.754+0000 7fabd2ffd640 1 -- 172.21.15.165:0/2663146724 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face401c5e0 con 0x559ce8d572e0 2024-04-27T03:30:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.754+0000 7facf8333800 1 -- 172.21.15.165:0/2663146724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 msgr2=0x559ce8d559d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.754+0000 7facf8333800 1 --2- 172.21.15.165:0/2663146724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 0x559ce8d559d0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7face401ac40 tx=0x7face400f530 comp rx=0 tx=0).stop 2024-04-27T03:30:31.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.754+0000 7facf8333800 1 -- 172.21.15.165:0/2663146724 shutdown_connections 2024-04-27T03:30:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.754+0000 7facf8333800 1 --2- 172.21.15.165:0/2663146724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 0x559ce8d559d0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.754+0000 7facf8333800 1 -- 172.21.15.165:0/2663146724 >> 172.21.15.165:0/2663146724 conn(0x559ce8d86b10 msgr2=0x559ce8d84b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.754+0000 7facf8333800 1 -- 172.21.15.165:0/2663146724 shutdown_connections 2024-04-27T03:30:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.754+0000 7facf8333800 1 -- 172.21.15.165:0/2663146724 wait complete. 2024-04-27T03:30:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.755+0000 7facf8333800 1 Processor -- start 2024-04-27T03:30:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.755+0000 7facf8333800 1 -- start start 2024-04-27T03:30:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.755+0000 7facf8333800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 0x559ce8d93da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.755+0000 7facf8333800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559ce8d55f10 con 0x559ce8d572e0 2024-04-27T03:30:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.755+0000 7facf3fd1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 0x559ce8d93da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.755+0000 7facf3fd1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 0x559ce8d93da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52026/0 (socket says 172.21.15.165:52026) 2024-04-27T03:30:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.755+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 learned_addr learned my addr 172.21.15.165:0/801877615 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.755+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559ce8d942e0 con 0x559ce8d572e0 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.755+0000 7facf3fd1640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 0x559ce8d93da0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7face401ac40 tx=0x7face4005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.755+0000 7fabd17fa640 1 -- 172.21.15.165:0/801877615 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face4030d40 con 0x559ce8d572e0 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.755+0000 7fabd17fa640 1 -- 172.21.15.165:0/801877615 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7face4044e50 con 0x559ce8d572e0 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.755+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559ce8d942e0 con 0x559ce8d572e0 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.756+0000 7fabd17fa640 1 -- 172.21.15.165:0/801877615 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7face4030450 con 0x559ce8d572e0 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.756+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559ce8d942e0 con 0x559ce8d572e0 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.756+0000 7fabd17fa640 1 -- 172.21.15.165:0/801877615 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7face406c070 con 0x559ce8d572e0 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.756+0000 7fabd17fa640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabbc036950 0x7fabbc038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.756+0000 7fabd17fa640 1 -- 172.21.15.165:0/801877615 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7face4070ad0 con 0x559ce8d572e0 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.756+0000 7facf37d0640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabbc036950 0x7fabbc038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.757+0000 7facf37d0640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabbc036950 0x7fabbc038e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7face8052fd0 tx=0x7face804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.757+0000 7facf8333800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.757+0000 7facf8333800 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559ce8d9f400 0x559ce8da18c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.757+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da2010 con 0x559ce8d9f400 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.757+0000 7facf47d2640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559ce8d9f400 0x559ce8da18c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.757+0000 7facf47d2640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559ce8d9f400 0x559ce8da18c0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8d9f400 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 20 realm 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da2420 con 0x559ce8d9f400 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8d9f400 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da2cf0 con 0x559ce8d9f400 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8d9f400 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da3090 con 0x559ce8d9f400 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8d9f400 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559ce8da3df0 0x559ce8da61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da67f0 con 0x559ce8da3df0 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.758+0000 7facf3fd1640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559ce8da3df0 0x559ce8da61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.759+0000 7facf3fd1640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559ce8da3df0 0x559ce8da61e0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.759+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.759+0000 7facf8333800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.759+0000 7facf8333800 20 searching for the correct realm 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.759+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da8120 con 0x559ce8da3df0 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.759+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.759+0000 7facf3fd1640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7face4026840 0x7face4013130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.760+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da8120 con 0x7face4026840 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.760+0000 7facf37d0640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7face4026840 0x7face4013130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.760+0000 7facf37d0640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7face4026840 0x7face4013130 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.760+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face8081070 con 0x7face4026840 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.760+0000 7facf37d0640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7face8076040 0x7face805e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.760+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da8120 con 0x7face8076040 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.760+0000 7facf47d2640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7face8076040 0x7face805e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.761+0000 7facf47d2640 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7face8076040 0x7face805e480 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.761+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7facec00a4a0 con 0x7face8076040 2024-04-27T03:30:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.761+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da3090 con 0x559ce8d9f400 2024-04-27T03:30:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.761+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8d9f400 2024-04-27T03:30:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.761+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da3090 con 0x559ce8d9f400 2024-04-27T03:30:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.762+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8d9f400 2024-04-27T03:30:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.762+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da3090 con 0x7face8076040 2024-04-27T03:30:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.762+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec00a4a0 con 0x7face8076040 2024-04-27T03:30:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.762+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da3090 con 0x7face8076040 2024-04-27T03:30:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.762+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec00a4a0 con 0x7face8076040 2024-04-27T03:30:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.762+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da3090 con 0x7face4026840 2024-04-27T03:30:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.762+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face8081070 con 0x7face4026840 2024-04-27T03:30:31.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.762+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da3090 con 0x559ce8da3df0 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.762+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.763+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da3090 con 0x559ce8d9f400 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.763+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8d9f400 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.763+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da3090 con 0x7face8076040 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.763+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec00a4a0 con 0x7face8076040 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.763+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da3090 con 0x559ce8d9f400 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.763+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8d9f400 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.763+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da3090 con 0x7face4026840 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.763+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7face8081070 con 0x7face4026840 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.763+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x559ce8da3df0 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.764+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.764+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x559ce8da3df0 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.764+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.764+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x7face8076040 2024-04-27T03:30:31.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.764+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec00a4a0 con 0x7face8076040 2024-04-27T03:30:31.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.764+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x559ce8da3df0 2024-04-27T03:30:31.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.764+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.764+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x559ce8da3df0 2024-04-27T03:30:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.765+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.765+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x559ce8d9f400 2024-04-27T03:30:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.765+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8d9f400 2024-04-27T03:30:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.765+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x559ce8d9f400 2024-04-27T03:30:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.765+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7facec00a4a0 con 0x559ce8d9f400 2024-04-27T03:30:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.765+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x7face8076040 2024-04-27T03:30:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.765+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec033040 con 0x7face8076040 2024-04-27T03:30:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.765+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x559ce8da3df0 2024-04-27T03:30:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.766+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.766+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x7face4026840 2024-04-27T03:30:31.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.766+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face8081070 con 0x7face4026840 2024-04-27T03:30:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.766+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x559ce8d9f400 2024-04-27T03:30:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.766+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.766+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x7face8076040 2024-04-27T03:30:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.767+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.767+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x559ce8da3df0 2024-04-27T03:30:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.767+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.767+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x7face8076040 2024-04-27T03:30:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.767+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.767+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x7face4026840 2024-04-27T03:30:31.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.767+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face8081070 con 0x7face4026840 2024-04-27T03:30:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.767+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x559ce8da3df0 2024-04-27T03:30:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x7face8076040 2024-04-27T03:30:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x559ce8d9f400 2024-04-27T03:30:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7face8090060 con 0x7face4026840 2024-04-27T03:30:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7face8081070 con 0x7face4026840 2024-04-27T03:30:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf8333800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf8333800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf8333800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:31.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf8333800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf8333800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf8333800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf8333800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.768+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da70b0 con 0x559ce8d9f400 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da74a0 con 0x559ce8d9f400 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da74a0 con 0x7face8076040 2024-04-27T03:30:31.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:31.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 20 zone default found 2024-04-27T03:30:31.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 4 Realm: () 2024-04-27T03:30:31.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:31.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:31.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:31.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 20 zonegroup default 2024-04-27T03:30:31.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da6d10 con 0x7face8076040 2024-04-27T03:30:31.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.769+0000 7facf8333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.770+0000 7facf8333800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.770+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da6d10 con 0x559ce8da3df0 2024-04-27T03:30:31.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.770+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.770+0000 7facf8333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.770+0000 7facf8333800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.770+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8daa000 con 0x559ce8da3df0 2024-04-27T03:30:31.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.770+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.770+0000 7facf8333800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.770+0000 7facf8333800 20 started sync module instance, tier type = 2024-04-27T03:30:31.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.770+0000 7facf8333800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:31.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.770+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da9420 con 0x7face4026840 2024-04-27T03:30:31.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.771+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7face8081070 con 0x7face4026840 2024-04-27T03:30:31.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.771+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94132409899024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da9080 con 0x7face4026840 2024-04-27T03:30:31.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.771+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8daaf30 con 0x559ce8d9f400 2024-04-27T03:30:31.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.772+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94132409899024] v25'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7face8081070 con 0x7face4026840 2024-04-27T03:30:31.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.772+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.772+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94132409903744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8dab350 con 0x559ce8d9f400 2024-04-27T03:30:31.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.772+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8dabd10 con 0x559ce8da3df0 2024-04-27T03:30:31.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.773+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.773+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94132409907296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8dac5b0 con 0x559ce8da3df0 2024-04-27T03:30:31.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.773+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8dad0b0 con 0x7face4026840 2024-04-27T03:30:31.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.774+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94132409903744] v25'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.774+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94132409907296] v25'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.775+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7face8081070 con 0x7face4026840 2024-04-27T03:30:31.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.775+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94132409904800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8dac5b0 con 0x7face4026840 2024-04-27T03:30:31.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.775+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8dad950 con 0x559ce8d9f400 2024-04-27T03:30:31.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.776+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.776+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94132409915040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8dae1f0 con 0x559ce8d9f400 2024-04-27T03:30:31.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.776+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8daece0 con 0x7face8076040 2024-04-27T03:30:31.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.776+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94132409904800] v25'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7face8081070 con 0x7face4026840 2024-04-27T03:30:31.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.777+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.777+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94132409919408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8daeca0 con 0x7face8076040 2024-04-27T03:30:31.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.777+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8daf6f0 con 0x7face8076040 2024-04-27T03:30:31.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.777+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94132409915040] v25'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.777+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94132409919408] v25'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.778+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.778+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94132409909504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8daf240 con 0x7face8076040 2024-04-27T03:30:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.778+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8dafc00 con 0x7face8076040 2024-04-27T03:30:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.779+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94132409909504] v25'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.779+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'51 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.779+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94132409916736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8dafc00 con 0x7face8076040 2024-04-27T03:30:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.779+0000 7facf8333800 20 add_watcher() i=0 2024-04-27T03:30:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.779+0000 7facf8333800 20 add_watcher() i=1 2024-04-27T03:30:31.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.779+0000 7facf8333800 20 add_watcher() i=2 2024-04-27T03:30:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.779+0000 7facf8333800 20 add_watcher() i=3 2024-04-27T03:30:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.779+0000 7facf8333800 20 add_watcher() i=4 2024-04-27T03:30:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.779+0000 7facf8333800 20 add_watcher() i=5 2024-04-27T03:30:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.779+0000 7facf8333800 20 add_watcher() i=6 2024-04-27T03:30:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.780+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94132409916736] v25'52 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.780+0000 7facf8333800 20 add_watcher() i=7 2024-04-27T03:30:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.780+0000 7facf8333800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.780+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8db2f40 con 0x559ce8d9f400 2024-04-27T03:30:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.780+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.780+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94132409923776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8db0190 con 0x559ce8d9f400 2024-04-27T03:30:31.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.781+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94132409923776] v25'74 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.781+0000 7faba9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.781+0000 7facf8333800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.781+0000 7facf8333800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.781+0000 7facf8333800 5 note: GC not initialized 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7faba8fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7facf8333800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7facf8333800 20 Filter name: none 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7fab227fc640 20 reqs_thread_entry: start 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7facf8333800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7facf8333800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8de62f0 con 0x559ce8d9f400 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~502 out=502b,getxattrs out=75b] v0'0 uv4 ondisk = 0) ==== 231+0+625 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7facf8333800 20 rados_obj.operate() r=0 bl.length=502 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7facf8333800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7facf8333800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7facf8333800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.782+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8dec990 con 0x559ce8d9f400 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~391] v25'5 uv5 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7facf8333800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7facf8333800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7facf8333800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94132410172416 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8de8060 con 0x559ce8d9f400 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify (1) cookie 94132409915040 notify 107374182404 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7facec01e040 con 0x559ce8d9f400 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7fabd2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182404 cookie 94132409915040 notifier 4343 bl.length()=567 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7fabd2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:31.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7fabd2ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(64 notify.4 [notify cookie 94132410172416 out=8b] v0'0 uv51 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7fabd2ffd640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fabb8003ea0 con 0x559ce8d9f400 2024-04-27T03:30:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== watch-notify(notify_complete (2) cookie 94132410172416 notify 107374182404 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7facec018e10 con 0x559ce8d9f400 2024-04-27T03:30:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.784+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:66 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8dec400 con 0x7face8076040 2024-04-27T03:30:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.785+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(66 p5YnriCv1nAtykxBrupQ [delete] v25'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.785+0000 7facf8333800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T03:30:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.785+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [notify cookie 94132410172416 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559ce8de7bb0 con 0x559ce8da3df0 2024-04-27T03:30:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.786+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== watch-notify(notify (1) cookie 94132409907296 notify 107374182401 ret 0) ==== 212+0+0 (crc 0 0 0) 0x559ce8d7ed80 con 0x559ce8da3df0 2024-04-27T03:30:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.786+0000 7fabd27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182401 cookie 94132409907296 notifier 4343 bl.length()=170 2024-04-27T03:30:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.786+0000 7fabd27fc640 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fab280032c0 con 0x559ce8da3df0 2024-04-27T03:30:31.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.786+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 notify.2 [notify cookie 94132410172416 out=8b] v0'0 uv51 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.786+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(68 notify.2 [notify-ack] v0'0 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.786+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== watch-notify(notify_complete (2) cookie 94132410172416 notify 107374182401 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7face4098040 con 0x559ce8da3df0 2024-04-27T03:30:31.806 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:31.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.787+0000 7faba9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.787+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94132409923776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8de8490 con 0x559ce8d9f400 2024-04-27T03:30:31.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.788+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94132409923776] v25'75 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.788+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94132409899024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8de8490 con 0x7face4026840 2024-04-27T03:30:31.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.789+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94132409899024] v25'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7face8081070 con 0x7face4026840 2024-04-27T03:30:31.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.789+0000 7facf8333800 20 remove_watcher() i=0 2024-04-27T03:30:31.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.789+0000 7facf8333800 2 removed watcher, disabling cache 2024-04-27T03:30:31.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.789+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94132409903744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da9810 con 0x559ce8d9f400 2024-04-27T03:30:31.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.790+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94132409903744] v25'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.790+0000 7facf8333800 20 remove_watcher() i=1 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.790+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94132409907296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da9810 con 0x559ce8da3df0 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.791+0000 7facf3fd1640 1 -- 172.21.15.165:0/801877615 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94132409907296] v25'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7face4075020 con 0x559ce8da3df0 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.791+0000 7facf8333800 20 remove_watcher() i=2 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.791+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94132409904800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da9810 con 0x7face4026840 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.792+0000 7facf37d0640 1 -- 172.21.15.165:0/801877615 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94132409904800] v25'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7face8081070 con 0x7face4026840 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.792+0000 7facf8333800 20 remove_watcher() i=3 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.792+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94132409915040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da9810 con 0x559ce8d9f400 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.793+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94132409915040] v25'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x559ce8d9f400 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.793+0000 7facf8333800 20 remove_watcher() i=4 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.793+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94132409919408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da9810 con 0x7face8076040 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.794+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94132409919408] v25'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.794+0000 7facf8333800 20 remove_watcher() i=5 2024-04-27T03:30:31.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.794+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94132409909504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da9810 con 0x7face8076040 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.795+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94132409909504] v25'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.795+0000 7facf8333800 20 remove_watcher() i=6 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.795+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94132409916736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559ce8da9810 con 0x7face8076040 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94132409916736] v25'53 uv51 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7facec031040 con 0x7face8076040 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 20 remove_watcher() i=7 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7face8076040 msgr2=0x7face805e480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7face8076040 0x7face805e480 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559ce8d9f400 msgr2=0x559ce8da18c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559ce8d9f400 0x559ce8da18c0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7face4026840 msgr2=0x7face4013130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7face4026840 0x7face4013130 crc :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559ce8da3df0 msgr2=0x559ce8da61e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559ce8da3df0 0x559ce8da61e0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabbc036950 msgr2=0x7fabbc038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabbc036950 0x7fabbc038e10 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7face8052fd0 tx=0x7face804f000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 msgr2=0x559ce8d93da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 --2- 172.21.15.165:0/801877615 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d572e0 0x559ce8d93da0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7face401ac40 tx=0x7face4005ef0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 reap_dead start 2024-04-27T03:30:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf47d2640 1 -- 172.21.15.165:0/801877615 reap_dead start 2024-04-27T03:30:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 shutdown_connections 2024-04-27T03:30:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 >> 172.21.15.165:0/801877615 conn(0x559ce8d86b10 msgr2=0x559ce8d84170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 shutdown_connections 2024-04-27T03:30:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/801877615 wait complete. 2024-04-27T03:30:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/3028630598 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabcc036900 msgr2=0x7fabcc038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 --2- 172.21.15.165:0/3028630598 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabcc036900 0x7fabcc038dc0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x559ce8d66b50 tx=0x7face8005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/3028630598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d7d4b0 msgr2=0x559ce8d7d8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 --2- 172.21.15.165:0/3028630598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d7d4b0 0x559ce8d7d8d0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7face400eb40 tx=0x7face401aaf0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/3028630598 shutdown_connections 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 --2- 172.21.15.165:0/3028630598 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabcc036900 0x7fabcc038dc0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 --2- 172.21.15.165:0/3028630598 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8d7d4b0 0x559ce8d7d8d0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/3028630598 >> 172.21.15.165:0/3028630598 conn(0x559ce8d71550 msgr2=0x559ce8d71950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/3028630598 shutdown_connections 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.796+0000 7facf8333800 1 -- 172.21.15.165:0/3028630598 wait complete. 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559ce8d6c530 msgr2=0x559ce8d6e980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559ce8d6c530 0x559ce8d6e980 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559ce8c69230 msgr2=0x559ce8d65d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559ce8c69230 0x559ce8d65d70 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559ce8d685a0 msgr2=0x559ce8d6a990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559ce8d685a0 0x559ce8d6a990 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabdc036900 msgr2=0x7fabdc038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fabdc036900 0x7fabdc038dc0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7face8054000 tx=0x7face804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 msgr2=0x559ce8d5eb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 --2- 172.21.15.165:0/950685711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559ce8c65280 0x559ce8d5eb30 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7face4000f60 tx=0x7face4000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf47d2640 1 -- 172.21.15.165:0/950685711 reap_dead start 2024-04-27T03:30:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 shutdown_connections 2024-04-27T03:30:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 >> 172.21.15.165:0/950685711 conn(0x559ce8c5fb20 msgr2=0x559ce8d4ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 shutdown_connections 2024-04-27T03:30:31.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.797+0000 7facf8333800 1 -- 172.21.15.165:0/950685711 wait complete. 2024-04-27T03:30:31.817 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:31.817 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-04-27T03:30:31.818 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo1', '--access', 'full'] 2024-04-27T03:30:31.818 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo1 --access full 2024-04-27T03:30:31.862 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:31.862 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:31.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.857+0000 7f1b6372d800 1 Processor -- start 2024-04-27T03:30:31.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.857+0000 7f1b6372d800 1 -- start start 2024-04-27T03:30:31.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.858+0000 7f1b6372d800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d8cf4032a0 0x55d8cf40bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.858+0000 7f1b6372d800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d8cf2c1d40 con 0x55d8cf401760 2024-04-27T03:30:31.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.858+0000 7f1b6372d800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55d8cf2944d0 con 0x55d8cf4032a0 2024-04-27T03:30:31.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.858+0000 7f1b57fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d8cf4032a0 0x55d8cf40bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.858+0000 7f1b5f378640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55d8cf401760 0x55d8cf401b80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53684/0 (socket says 172.21.15.165:53684) 2024-04-27T03:30:31.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.858+0000 7f1b5f378640 1 -- 172.21.15.165:0/3389515262 learned_addr learned my addr 172.21.15.165:0/3389515262 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:31.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.858+0000 7f1b57fff640 1 -- 172.21.15.165:0/3389515262 >> v1:172.21.15.89:6789/0 conn(0x55d8cf401760 legacy=0x55d8cf401b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.858+0000 7f1b57fff640 1 -- 172.21.15.165:0/3389515262 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cf2c0590 con 0x55d8cf4032a0 2024-04-27T03:30:31.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.860+0000 7f1b57fff640 1 --2- 172.21.15.165:0/3389515262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf4032a0 0x55d8cf40bda0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800a470 tx=0x7f1b4801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef949f044e590cb0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.860+0000 7f1b5eb77640 1 -- 172.21.15.165:0/3389515262 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b48057070 con 0x55d8cf4032a0 2024-04-27T03:30:31.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.860+0000 7f1b5eb77640 1 -- 172.21.15.165:0/3389515262 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b4804adc0 con 0x55d8cf4032a0 2024-04-27T03:30:31.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.860+0000 7f1b5eb77640 1 -- 172.21.15.165:0/3389515262 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b48052590 con 0x55d8cf4032a0 2024-04-27T03:30:31.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.860+0000 7f1b6372d800 1 -- 172.21.15.165:0/3389515262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf4032a0 msgr2=0x55d8cf40bda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.861+0000 7f1b6372d800 1 --2- 172.21.15.165:0/3389515262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf4032a0 0x55d8cf40bda0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f1b4800a470 tx=0x7f1b4801ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:31.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.861+0000 7f1b6372d800 1 -- 172.21.15.165:0/3389515262 shutdown_connections 2024-04-27T03:30:31.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.861+0000 7f1b6372d800 1 --2- 172.21.15.165:0/3389515262 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf4032a0 0x55d8cf40bda0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.861+0000 7f1b6372d800 1 -- 172.21.15.165:0/3389515262 >> 172.21.15.165:0/3389515262 conn(0x55d8cf3fd3a0 msgr2=0x55d8cf3ff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.861+0000 7f1b6372d800 1 -- 172.21.15.165:0/3389515262 shutdown_connections 2024-04-27T03:30:31.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.861+0000 7f1b6372d800 1 -- 172.21.15.165:0/3389515262 wait complete. 2024-04-27T03:30:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.873+0000 7f1b6372d800 1 Processor -- start 2024-04-27T03:30:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.873+0000 7f1b6372d800 1 -- start start 2024-04-27T03:30:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.873+0000 7f1b6372d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 0x55d8cf4f6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.873+0000 7f1b6372d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d8cf34e3b0 con 0x55d8cf401760 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.873+0000 7f1b5f378640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 0x55d8cf4f6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.873+0000 7f1b5f378640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 0x55d8cf4f6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52046/0 (socket says 172.21.15.165:52046) 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.873+0000 7f1b5f378640 1 -- 172.21.15.165:0/680395675 learned_addr learned my addr 172.21.15.165:0/680395675 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.873+0000 7f1b5f378640 1 -- 172.21.15.165:0/680395675 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cf4f6bd0 con 0x55d8cf401760 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.874+0000 7f1b5f378640 1 --2- 172.21.15.165:0/680395675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 0x55d8cf4f6600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1b50001de0 tx=0x7f1b5000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=810426436ac9a83f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.874+0000 7f1a4d60a640 1 -- 172.21.15.165:0/680395675 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b50012020 con 0x55d8cf401760 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.874+0000 7f1a4d60a640 1 -- 172.21.15.165:0/680395675 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5000e5a0 con 0x55d8cf401760 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.874+0000 7f1a4d60a640 1 -- 172.21.15.165:0/680395675 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b5000e8c0 con 0x55d8cf401760 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.874+0000 7f1b6372d800 1 -- 172.21.15.165:0/680395675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 msgr2=0x55d8cf4f6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.874+0000 7f1b6372d800 1 --2- 172.21.15.165:0/680395675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 0x55d8cf4f6600 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f1b50001de0 tx=0x7f1b5000b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.874+0000 7f1b6372d800 1 -- 172.21.15.165:0/680395675 shutdown_connections 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.874+0000 7f1b6372d800 1 --2- 172.21.15.165:0/680395675 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 0x55d8cf4f6600 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.874+0000 7f1b6372d800 1 -- 172.21.15.165:0/680395675 >> 172.21.15.165:0/680395675 conn(0x55d8cf3fd3a0 msgr2=0x55d8cf4f1f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.874+0000 7f1b6372d800 1 -- 172.21.15.165:0/680395675 shutdown_connections 2024-04-27T03:30:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.875+0000 7f1b6372d800 1 -- 172.21.15.165:0/680395675 wait complete. 2024-04-27T03:30:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.875+0000 7f1b6372d800 1 Processor -- start 2024-04-27T03:30:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.875+0000 7f1b6372d800 1 -- start start 2024-04-27T03:30:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.875+0000 7f1b6372d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 0x55d8cf4fe120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.875+0000 7f1b6372d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d8cf40e5c0 con 0x55d8cf401760 2024-04-27T03:30:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.875+0000 7f1b5f378640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 0x55d8cf4fe120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.875+0000 7f1b5f378640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 0x55d8cf4fe120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52048/0 (socket says 172.21.15.165:52048) 2024-04-27T03:30:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.875+0000 7f1b5f378640 1 -- 172.21.15.165:0/410092163 learned_addr learned my addr 172.21.15.165:0/410092163 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.875+0000 7f1b5f378640 1 -- 172.21.15.165:0/410092163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cf4fe660 con 0x55d8cf401760 2024-04-27T03:30:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.876+0000 7f1b5f378640 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 0x55d8cf4fe120 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1b50001de0 tx=0x7f1b50000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.876+0000 7f1a4be07640 1 -- 172.21.15.165:0/410092163 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b50012030 con 0x55d8cf401760 2024-04-27T03:30:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.876+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d8cf4fe8f0 con 0x55d8cf401760 2024-04-27T03:30:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.876+0000 7f1a4be07640 1 -- 172.21.15.165:0/410092163 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5001a7a0 con 0x55d8cf401760 2024-04-27T03:30:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.876+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8cf5015a0 con 0x55d8cf401760 2024-04-27T03:30:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.876+0000 7f1a4be07640 1 -- 172.21.15.165:0/410092163 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b5001aa80 con 0x55d8cf401760 2024-04-27T03:30:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.877+0000 7f1a4be07640 1 -- 172.21.15.165:0/410092163 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f1b50015020 con 0x55d8cf401760 2024-04-27T03:30:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.877+0000 7f1a4be07640 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a44036900 0x7f1a44038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.877+0000 7f1a4be07640 1 -- 172.21.15.165:0/410092163 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f1b50020070 con 0x55d8cf401760 2024-04-27T03:30:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.877+0000 7f1b57fff640 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a44036900 0x7f1a44038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.877+0000 7f1b6372d800 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d8cf4032a0 0x55d8cf505360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.877+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5058a0 con 0x55d8cf4032a0 2024-04-27T03:30:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.877+0000 7f1b5fb79640 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d8cf4032a0 0x55d8cf505360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.877+0000 7f1b57fff640 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a44036900 0x7f1a44038dc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1b48054000 tx=0x7f1b4804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.877+0000 7f1b5fb79640 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d8cf4032a0 0x55d8cf505360 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.878+0000 7f1b5fb79640 1 -- 172.21.15.165:0/410092163 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf4032a0 2024-04-27T03:30:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.878+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf502930 con 0x55d8cf4032a0 2024-04-27T03:30:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.878+0000 7f1b5fb79640 1 -- 172.21.15.165:0/410092163 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf4032a0 2024-04-27T03:30:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.878+0000 7f1b6372d800 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d8cf507c00 0x55d8cf509ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.878+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf50a6e0 con 0x55d8cf507c00 2024-04-27T03:30:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.878+0000 7f1b5f378640 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d8cf507c00 0x55d8cf509ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.878+0000 7f1b5f378640 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d8cf507c00 0x55d8cf509ff0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.879+0000 7f1b5f378640 1 -- 172.21.15.165:0/410092163 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f1b5002dcf0 con 0x55d8cf507c00 2024-04-27T03:30:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.879+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf506bb0 con 0x55d8cf4032a0 2024-04-27T03:30:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.879+0000 7f1b5fb79640 1 -- 172.21.15.165:0/410092163 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf4032a0 2024-04-27T03:30:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.879+0000 7f1b6372d800 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d8cf50bb90 0x55d8cf50dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.879+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf50e740 con 0x55d8cf50bb90 2024-04-27T03:30:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.879+0000 7f1b57fff640 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d8cf50bb90 0x55d8cf50dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.880+0000 7f1b57fff640 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d8cf50bb90 0x55d8cf50dfe0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.880+0000 7f1b57fff640 1 -- 172.21.15.165:0/410092163 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f1b4805a040 con 0x55d8cf50bb90 2024-04-27T03:30:31.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.881+0000 7f1b6372d800 1 Processor -- start 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.881+0000 7f1b6372d800 1 -- start start 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.881+0000 7f1b6372d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf51cb50 0x55d8cf51cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.881+0000 7f1b6372d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d8cf2f46b0 con 0x55d8cf51cb50 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.881+0000 7f1b5f378640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf51cb50 0x55d8cf51cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.881+0000 7f1b5f378640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf51cb50 0x55d8cf51cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52050/0 (socket says 172.21.15.165:52050) 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.881+0000 7f1b5f378640 1 -- 172.21.15.165:0/2209909507 learned_addr learned my addr 172.21.15.165:0/2209909507 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.881+0000 7f1b5f378640 1 -- 172.21.15.165:0/2209909507 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cf51d4b0 con 0x55d8cf51cb50 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.881+0000 7f1b5f378640 1 --2- 172.21.15.165:0/2209909507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf51cb50 0x55d8cf51cf70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b5004b090 tx=0x7f1b5000d600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.882+0000 7f1a49602640 1 -- 172.21.15.165:0/2209909507 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b5000d880 con 0x55d8cf51cb50 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.882+0000 7f1a49602640 1 -- 172.21.15.165:0/2209909507 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b5000da20 con 0x55d8cf51cb50 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.882+0000 7f1b6372d800 1 -- 172.21.15.165:0/2209909507 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d8cf51df20 con 0x55d8cf51cb50 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.882+0000 7f1a49602640 1 -- 172.21.15.165:0/2209909507 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b50017560 con 0x55d8cf51cb50 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.882+0000 7f1b6372d800 1 -- 172.21.15.165:0/2209909507 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8cf51e420 con 0x55d8cf51cb50 2024-04-27T03:30:31.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.882+0000 7f1a49602640 1 -- 172.21.15.165:0/2209909507 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f1b50017700 con 0x55d8cf51cb50 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.882+0000 7f1a49602640 1 --2- 172.21.15.165:0/2209909507 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a380369c0 0x7f1a38038e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.882+0000 7f1a49602640 1 -- 172.21.15.165:0/2209909507 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f1b5001b040 con 0x55d8cf51cb50 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.882+0000 7f1b57fff640 1 --2- 172.21.15.165:0/2209909507 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a380369c0 0x7f1a38038e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.883+0000 7f1b6372d800 1 Processor -- start 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.883+0000 7f1b6372d800 1 -- start start 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.883+0000 7f1b57fff640 1 --2- 172.21.15.165:0/2209909507 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a380369c0 0x7f1a38038e80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55d8cf5061b0 tx=0x7f1b48005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.883+0000 7f1b6372d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 0x55d8cf52a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.883+0000 7f1b6372d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d8cf4f6f50 con 0x55d8cf529c80 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.883+0000 7f1b5f378640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 0x55d8cf52a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.883+0000 7f1b5f378640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 0x55d8cf52a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52060/0 (socket says 172.21.15.165:52060) 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.883+0000 7f1b5f378640 1 -- 172.21.15.165:0/1031066021 learned_addr learned my addr 172.21.15.165:0/1031066021 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.883+0000 7f1b5f378640 1 -- 172.21.15.165:0/1031066021 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cf52a670 con 0x55d8cf529c80 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.884+0000 7f1b5f378640 1 --2- 172.21.15.165:0/1031066021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 0x55d8cf52a0a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000dbf0 tx=0x7f1b5004ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98297f99384e607 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.884+0000 7f1a42ffd640 1 -- 172.21.15.165:0/1031066021 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b50051070 con 0x55d8cf529c80 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.884+0000 7f1a42ffd640 1 -- 172.21.15.165:0/1031066021 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b50044580 con 0x55d8cf529c80 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.884+0000 7f1a42ffd640 1 -- 172.21.15.165:0/1031066021 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b5004c570 con 0x55d8cf529c80 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.884+0000 7f1b6372d800 1 -- 172.21.15.165:0/1031066021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 msgr2=0x55d8cf52a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.884+0000 7f1b6372d800 1 --2- 172.21.15.165:0/1031066021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 0x55d8cf52a0a0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1b5000dbf0 tx=0x7f1b5004ac50 comp rx=0 tx=0).stop 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.884+0000 7f1b6372d800 1 -- 172.21.15.165:0/1031066021 shutdown_connections 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.884+0000 7f1b6372d800 1 --2- 172.21.15.165:0/1031066021 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 0x55d8cf52a0a0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.884+0000 7f1b6372d800 1 -- 172.21.15.165:0/1031066021 >> 172.21.15.165:0/1031066021 conn(0x55d8cf525fa0 msgr2=0x55d8cf524000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.884+0000 7f1b6372d800 1 -- 172.21.15.165:0/1031066021 shutdown_connections 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.884+0000 7f1b6372d800 1 -- 172.21.15.165:0/1031066021 wait complete. 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.885+0000 7f1b6372d800 1 Processor -- start 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.885+0000 7f1b6372d800 1 -- start start 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.885+0000 7f1b6372d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 0x55d8cf5310b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.885+0000 7f1b6372d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d8cf52ad90 con 0x55d8cf529c80 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.885+0000 7f1b5f378640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 0x55d8cf5310b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.885+0000 7f1b5f378640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 0x55d8cf5310b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52074/0 (socket says 172.21.15.165:52074) 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.885+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 learned_addr learned my addr 172.21.15.165:0/3105267111 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.885+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8cf5315f0 con 0x55d8cf529c80 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.885+0000 7f1b5f378640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 0x55d8cf5310b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1b50032010 tx=0x7f1b50020f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.885+0000 7f1a417fa640 1 -- 172.21.15.165:0/3105267111 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b500097b0 con 0x55d8cf529c80 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.885+0000 7f1a417fa640 1 -- 172.21.15.165:0/3105267111 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b50070070 con 0x55d8cf529c80 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.885+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d8cf5315f0 con 0x55d8cf529c80 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.886+0000 7f1a417fa640 1 -- 172.21.15.165:0/3105267111 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b5004c1b0 con 0x55d8cf529c80 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.886+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8cf5315f0 con 0x55d8cf529c80 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.886+0000 7f1a417fa640 1 -- 172.21.15.165:0/3105267111 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f1b50068030 con 0x55d8cf529c80 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.886+0000 7f1a417fa640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a28036950 0x7f1a28038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.886+0000 7f1a417fa640 1 -- 172.21.15.165:0/3105267111 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f1b50032260 con 0x55d8cf529c80 2024-04-27T03:30:31.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.886+0000 7f1b57fff640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a28036950 0x7f1a28038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.887+0000 7f1b57fff640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a28036950 0x7f1a28038e10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1b48000c00 tx=0x7f1b48060040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.887+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.887+0000 7f1b6372d800 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d8cf53c7d0 0x55d8cf53ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.887+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf53f3e0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.887+0000 7f1b5fb79640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d8cf53c7d0 0x55d8cf53ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.887+0000 7f1b5fb79640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d8cf53c7d0 0x55d8cf53ec90 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 20 realm 2024-04-27T03:30:31.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf53f7f0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5400c0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf53c7d0 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:31.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d8cf5411c0 0x55d8cf5435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.888+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf543bc0 con 0x55d8cf5411c0 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.889+0000 7f1b5f378640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d8cf5411c0 0x55d8cf5435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.889+0000 7f1b5f378640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d8cf5411c0 0x55d8cf5435b0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.889+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.889+0000 7f1b6372d800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.889+0000 7f1b6372d800 20 searching for the correct realm 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.889+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5454f0 con 0x55d8cf5411c0 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.890+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.890+0000 7f1b5f378640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1b50017050 0x7f1b5006b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.890+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5454f0 con 0x7f1b50017050 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.890+0000 7f1b57fff640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1b50017050 0x7f1b5006b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.890+0000 7f1b57fff640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1b50017050 0x7f1b5006b840 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.890+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.890+0000 7f1b57fff640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1b48052b30 0x7f1b48080930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.890+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5454f0 con 0x7f1b48052b30 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.890+0000 7f1b5fb79640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1b48052b30 0x7f1b48080930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.891+0000 7f1b5fb79640 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1b48052b30 0x7f1b48080930 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.891+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f1b5800a4a0 con 0x7f1b48052b30 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.891+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf53c7d0 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.891+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.891+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf53c7d0 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.891+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.891+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b48052b30 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.892+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5800a4a0 con 0x7f1b48052b30 2024-04-27T03:30:31.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.892+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b48052b30 2024-04-27T03:30:31.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.892+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5800a4a0 con 0x7f1b48052b30 2024-04-27T03:30:31.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.892+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b50017050 2024-04-27T03:30:31.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.892+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.892+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf5411c0 2024-04-27T03:30:31.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.892+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.892+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf53c7d0 2024-04-27T03:30:31.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.893+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.893+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b48052b30 2024-04-27T03:30:31.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.893+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5800a4a0 con 0x7f1b48052b30 2024-04-27T03:30:31.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.893+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf53c7d0 2024-04-27T03:30:31.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.893+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.893+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b50017050 2024-04-27T03:30:31.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.894+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.894+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf5411c0 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.894+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.894+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf5411c0 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.894+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.894+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b48052b30 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.894+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5800a4a0 con 0x7f1b48052b30 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.894+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf5411c0 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.895+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.895+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf5411c0 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.895+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.895+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf53c7d0 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.895+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.895+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf53c7d0 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.895+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f1b5800a4a0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.895+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b48052b30 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.896+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b58033040 con 0x7f1b48052b30 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.896+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf5411c0 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.896+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.896+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b50017050 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.896+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.896+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf53c7d0 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.896+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.896+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b48052b30 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.897+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.897+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf5411c0 2024-04-27T03:30:31.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.897+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.897+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b48052b30 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.897+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.897+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b50017050 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.897+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.897+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf5411c0 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.898+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.898+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b48052b30 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.898+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.898+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x55d8cf53c7d0 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf540460 con 0x7f1b50017050 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:31.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf544480 con 0x55d8cf53c7d0 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf544870 con 0x55d8cf53c7d0 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.899+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf544870 con 0x7f1b48052b30 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 20 zone default found 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 4 Realm: () 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 20 zonegroup default 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5440e0 con 0x7f1b48052b30 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5440e0 con 0x55d8cf5411c0 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5473f0 con 0x55d8cf5411c0 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 20 started sync module instance, tier type = 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.900+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5467f0 con 0x7f1b50017050 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.901+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.902+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94389679713248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf547d00 con 0x7f1b50017050 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.902+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf548550 con 0x55d8cf53c7d0 2024-04-27T03:30:31.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.903+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94389679713248] v25'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.903+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.903+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94389679716432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5485b0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.903+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5490c0 con 0x55d8cf5411c0 2024-04-27T03:30:31.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.904+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94389679716432] v25'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.904+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.904+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94389679721328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5494b0 con 0x55d8cf5411c0 2024-04-27T03:30:31.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.904+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf549ef0 con 0x7f1b50017050 2024-04-27T03:30:31.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.905+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94389679721328] v25'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.905+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.905+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94389679724992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54a2e0 con 0x7f1b50017050 2024-04-27T03:30:31.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.905+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54ad20 con 0x55d8cf53c7d0 2024-04-27T03:30:31.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.906+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.906+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94389679728624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54b5c0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.906+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94389679724992] v25'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.906+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54c080 con 0x7f1b48052b30 2024-04-27T03:30:31.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.907+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.907+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94389679718640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54c080 con 0x7f1b48052b30 2024-04-27T03:30:31.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.907+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54cad0 con 0x7f1b48052b30 2024-04-27T03:30:31.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.907+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94389679728624] v25'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.908+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94389679718640] v25'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.909+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.909+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94389679733680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54ca90 con 0x7f1b48052b30 2024-04-27T03:30:31.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.909+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54cf50 con 0x7f1b48052b30 2024-04-27T03:30:31.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.909+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94389679733680] v25'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.910+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'54 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.910+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94389679730960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54cf10 con 0x7f1b48052b30 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.910+0000 7f1b6372d800 20 add_watcher() i=0 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.910+0000 7f1b6372d800 20 add_watcher() i=1 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.910+0000 7f1b6372d800 20 add_watcher() i=2 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.910+0000 7f1b6372d800 20 add_watcher() i=3 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.910+0000 7f1b6372d800 20 add_watcher() i=4 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.910+0000 7f1b6372d800 20 add_watcher() i=5 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.910+0000 7f1b6372d800 20 add_watcher() i=6 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.910+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94389679730960] v25'55 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.910+0000 7f1b6372d800 20 add_watcher() i=7 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.911+0000 7f1b6372d800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.911+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf550370 con 0x55d8cf53c7d0 2024-04-27T03:30:31.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.911+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.911+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94389679736256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54d870 con 0x55d8cf53c7d0 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.912+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94389679736256] v25'76 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.912+0000 7f1a15fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.912+0000 7f1b6372d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.912+0000 7f1b6372d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.913+0000 7f1b6372d800 5 note: GC not initialized 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.913+0000 7f1a14fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.913+0000 7f1b6372d800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.913+0000 7f1b6372d800 20 Filter name: none 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.913+0000 7f198e7fc640 20 reqs_thread_entry: start 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.913+0000 7f1b6372d800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.913+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.913+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf5836a0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.913+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b6372d800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b6372d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b6372d800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b6372d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b6372d800 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : miss 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b6372d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.8 8:17d68d04:users.swift::foo%3afoo1:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf584110 con 0x55d8cf53c7d0 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo:foo1 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 194+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b6372d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b6372d800 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x0 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b6372d800 10 adding default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b6372d800 10 cache get: name=default.rgw.meta+users.swift+foo:foo1 : hit (negative entry) 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.914+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~556 in=556b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf589b00 con 0x55d8cf53c7d0 2024-04-27T03:30:31.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.915+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~556] v25'6 uv6 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.915+0000 7f1b6372d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.915+0000 7f1b6372d800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.915+0000 7f1b6372d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.915+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94389679986032 in=744b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf585120 con 0x55d8cf53c7d0 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.915+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify (1) cookie 94389679728624 notify 107374182405 ret 0) ==== 774+0+0 (crc 0 0 0) 0x7f1b5801e040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.915+0000 7f1a42ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182405 cookie 94389679728624 notifier 4350 bl.length()=732 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.915+0000 7f1a42ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.915+0000 7f1a42ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.915+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(65 notify.4 [notify cookie 94389679986032 out=8b] v0'0 uv54 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.915+0000 7f1a42ffd640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f1a24003ea0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.915+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.916+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== watch-notify(notify_complete (2) cookie 94389679986032 notify 107374182405 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f1b58027420 con 0x55d8cf53c7d0 2024-04-27T03:30:31.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.916+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf589570 con 0x55d8cf53c7d0 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(67 foo:foo1 [delete,create,writefull 0~7] v25'1 uv1 ondisk = 0) ==== 236+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1b6372d800 10 cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1b6372d800 10 moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1b6372d800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94389679986032 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf584c70 con 0x7f1b50017050 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== watch-notify(notify (1) cookie 94389679713248 notify 107374182402 ret 0) ==== 208+0+0 (crc 0 0 0) 0x7f1b48089060 con 0x7f1b50017050 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1a427fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182402 cookie 94389679713248 notifier 4350 bl.length()=166 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1a427fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo1 info.flags=0x7 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1a427fc640 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo1 to cache LRU end 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1a427fc640 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f1994003400 con 0x7f1b50017050 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(68 notify.0 [notify cookie 94389679986032 out=8b] v0'0 uv54 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.917+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== watch-notify(notify_complete (2) cookie 94389679986032 notify 107374182402 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d8cf2c0590 con 0x7f1b50017050 2024-04-27T03:30:31.940 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"c5voHhtxYBX0oDePFiYgLvk5Mqz50MtS6MABF7b9","active":true,"create_date":"2024-04-27T03:30:31.915251Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.918+0000 7f1a15fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.918+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94389679736256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf588a00 con 0x55d8cf53c7d0 2024-04-27T03:30:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.919+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94389679736256] v25'77 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.919+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94389679713248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54c5c0 con 0x7f1b50017050 2024-04-27T03:30:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.920+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94389679713248] v25'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.920+0000 7f1b6372d800 20 remove_watcher() i=0 2024-04-27T03:30:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.920+0000 7f1b6372d800 2 removed watcher, disabling cache 2024-04-27T03:30:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.920+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94389679716432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54c5c0 con 0x55d8cf53c7d0 2024-04-27T03:30:31.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.921+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94389679716432] v25'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.921+0000 7f1b6372d800 20 remove_watcher() i=1 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.921+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94389679721328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf547850 con 0x55d8cf5411c0 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.922+0000 7f1b5f378640 1 -- 172.21.15.165:0/3105267111 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94389679721328] v25'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b50053070 con 0x55d8cf5411c0 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.922+0000 7f1b6372d800 20 remove_watcher() i=2 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.922+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94389679724992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54c5c0 con 0x7f1b50017050 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.923+0000 7f1b57fff640 1 -- 172.21.15.165:0/3105267111 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94389679724992] v25'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b48070070 con 0x7f1b50017050 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.923+0000 7f1b6372d800 20 remove_watcher() i=3 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.923+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94389679728624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf547850 con 0x55d8cf53c7d0 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.924+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94389679728624] v25'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x55d8cf53c7d0 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.924+0000 7f1b6372d800 20 remove_watcher() i=4 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.924+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94389679718640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54c5c0 con 0x7f1b48052b30 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.925+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94389679718640] v25'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.925+0000 7f1b6372d800 20 remove_watcher() i=5 2024-04-27T03:30:31.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.925+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94389679733680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54c5c0 con 0x7f1b48052b30 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.926+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94389679733680] v25'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.926+0000 7f1b6372d800 20 remove_watcher() i=6 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.926+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94389679730960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d8cf54bbb0 con 0x7f1b48052b30 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94389679730960] v25'56 uv54 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b58031040 con 0x7f1b48052b30 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 20 remove_watcher() i=7 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1b48052b30 msgr2=0x7f1b48080930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1b48052b30 0x7f1b48080930 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d8cf53c7d0 msgr2=0x55d8cf53ec90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d8cf53c7d0 0x55d8cf53ec90 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1b50017050 msgr2=0x7f1b5006b840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1b50017050 0x7f1b5006b840 crc :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d8cf5411c0 msgr2=0x55d8cf5435b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d8cf5411c0 0x55d8cf5435b0 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a28036950 msgr2=0x7f1a28038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a28036950 0x7f1a28038e10 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1b48000c00 tx=0x7f1b48060040 comp rx=0 tx=0).stop 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b5fb79640 1 -- 172.21.15.165:0/3105267111 reap_dead start 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 msgr2=0x55d8cf5310b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 0x55d8cf5310b0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1b50032010 tx=0x7f1b50020f00 comp rx=0 tx=0).stop 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 shutdown_connections 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 --2- 172.21.15.165:0/3105267111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf529c80 0x55d8cf5310b0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 >> 172.21.15.165:0/3105267111 conn(0x55d8cf525fa0 msgr2=0x55d8cf523600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 shutdown_connections 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.927+0000 7f1b6372d800 1 -- 172.21.15.165:0/3105267111 wait complete. 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/2209909507 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a380369c0 msgr2=0x7f1a38038e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 --2- 172.21.15.165:0/2209909507 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a380369c0 0x7f1a38038e80 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x55d8cf5061b0 tx=0x7f1b48005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/2209909507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf51cb50 msgr2=0x55d8cf51cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 --2- 172.21.15.165:0/2209909507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf51cb50 0x55d8cf51cf70 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f1b5004b090 tx=0x7f1b5000d600 comp rx=0 tx=0).stop 2024-04-27T03:30:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/2209909507 shutdown_connections 2024-04-27T03:30:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 --2- 172.21.15.165:0/2209909507 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a380369c0 0x7f1a38038e80 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 --2- 172.21.15.165:0/2209909507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf51cb50 0x55d8cf51cf70 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/2209909507 >> 172.21.15.165:0/2209909507 conn(0x55d8cf510bf0 msgr2=0x55d8cf510ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/2209909507 shutdown_connections 2024-04-27T03:30:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/2209909507 wait complete. 2024-04-27T03:30:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d8cf50bb90 msgr2=0x55d8cf50dfe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d8cf50bb90 0x55d8cf50dfe0 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d8cf4032a0 msgr2=0x55d8cf505360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d8cf4032a0 0x55d8cf505360 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d8cf507c00 msgr2=0x55d8cf509ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d8cf507c00 0x55d8cf509ff0 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a44036900 msgr2=0x7f1a44038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a44036900 0x7f1a44038dc0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f1b48054000 tx=0x7f1b4804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 msgr2=0x55d8cf4fe120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 --2- 172.21.15.165:0/410092163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d8cf401760 0x55d8cf4fe120 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1b50001de0 tx=0x7f1b50000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b5fb79640 1 -- 172.21.15.165:0/410092163 reap_dead start 2024-04-27T03:30:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 shutdown_connections 2024-04-27T03:30:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 >> 172.21.15.165:0/410092163 conn(0x55d8cf3fd3a0 msgr2=0x55d8cf4f07e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 shutdown_connections 2024-04-27T03:30:31.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.928+0000 7f1b6372d800 1 -- 172.21.15.165:0/410092163 wait complete. 2024-04-27T03:30:31.951 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'c5voHhtxYBX0oDePFiYgLvk5Mqz50MtS6MABF7b9', 'active': True, 'create_date': '2024-04-27T03:30:31.915251Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:31.952 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-04-27T03:30:31.952 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'modify', '--subuser', 'foo:foo1', '--secret', 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', '--key-type', 'swift'] 2024-04-27T03:30:31.952 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser modify --subuser foo:foo1 --secret gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL --key-type swift 2024-04-27T03:30:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:31.998 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.994+0000 7f2dcaa1f800 1 Processor -- start 2024-04-27T03:30:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.994+0000 7f2dcaa1f800 1 -- start start 2024-04-27T03:30:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.994+0000 7f2dcaa1f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x555916d25d40 0x555916d26160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:31.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.994+0000 7f2dcaa1f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x555916be5d40 con 0x555916d26fa0 2024-04-27T03:30:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.994+0000 7f2dcaa1f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x555916bb84d0 con 0x555916d25d40 2024-04-27T03:30:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.994+0000 7f2dc8525640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x555916d25d40 0x555916d26160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.994+0000 7f2dc8525640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x555916d25d40 0x555916d26160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52082/0 (socket says 172.21.15.165:52082) 2024-04-27T03:30:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.994+0000 7f2dc8525640 1 -- 172.21.15.165:0/4190815175 learned_addr learned my addr 172.21.15.165:0/4190815175 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.994+0000 7f2dc8525640 1 -- 172.21.15.165:0/4190815175 >> v1:172.21.15.89:6789/0 conn(0x555916d26fa0 legacy=0x555916d27420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.995+0000 7f2dc8525640 1 -- 172.21.15.165:0/4190815175 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x555916be4590 con 0x555916d25d40 2024-04-27T03:30:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.996+0000 7f2dc8525640 1 --2- 172.21.15.165:0/4190815175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916d26160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000a520 tx=0x7f2dc001d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b362b2f95c26971 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.996+0000 7f2dc7523640 1 -- 172.21.15.165:0/4190815175 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc0057070 con 0x555916d25d40 2024-04-27T03:30:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.996+0000 7f2dc7523640 1 -- 172.21.15.165:0/4190815175 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc0052570 con 0x555916d25d40 2024-04-27T03:30:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.997+0000 7f2dc7523640 1 -- 172.21.15.165:0/4190815175 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc004ae60 con 0x555916d25d40 2024-04-27T03:30:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.997+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4190815175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 msgr2=0x555916d26160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.997+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/4190815175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916d26160 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2dc000a520 tx=0x7f2dc001d010 comp rx=0 tx=0).stop 2024-04-27T03:30:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.997+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4190815175 shutdown_connections 2024-04-27T03:30:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.997+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/4190815175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916d26160 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.997+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4190815175 >> 172.21.15.165:0/4190815175 conn(0x555916d213a0 msgr2=0x555916d237c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.997+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4190815175 shutdown_connections 2024-04-27T03:30:32.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:31.997+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4190815175 wait complete. 2024-04-27T03:30:32.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.009+0000 7f2dcaa1f800 1 Processor -- start 2024-04-27T03:30:32.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.009+0000 7f2dcaa1f800 1 -- start start 2024-04-27T03:30:32.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.009+0000 7f2dcaa1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916c9a010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.009+0000 7f2dcaa1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555916c723e0 con 0x555916d25d40 2024-04-27T03:30:32.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.009+0000 7f2dc8525640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916c9a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.009+0000 7f2dc8525640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916c9a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52086/0 (socket says 172.21.15.165:52086) 2024-04-27T03:30:32.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.009+0000 7f2dc8525640 1 -- 172.21.15.165:0/641869109 learned_addr learned my addr 172.21.15.165:0/641869109 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.010+0000 7f2dc8525640 1 -- 172.21.15.165:0/641869109 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555916e1b290 con 0x555916d25d40 2024-04-27T03:30:32.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.010+0000 7f2dc8525640 1 --2- 172.21.15.165:0/641869109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916c9a010 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2dc001d050 tx=0x7f2dc004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f8fa0855ee0dfb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.010+0000 7f2cb660c640 1 -- 172.21.15.165:0/641869109 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc0057040 con 0x555916d25d40 2024-04-27T03:30:32.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.010+0000 7f2cb660c640 1 -- 172.21.15.165:0/641869109 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc0052720 con 0x555916d25d40 2024-04-27T03:30:32.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.010+0000 7f2cb660c640 1 -- 172.21.15.165:0/641869109 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc0052a40 con 0x555916d25d40 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/641869109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 msgr2=0x555916c9a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/641869109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916c9a010 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f2dc001d050 tx=0x7f2dc004c000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/641869109 shutdown_connections 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/641869109 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916c9a010 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/641869109 >> 172.21.15.165:0/641869109 conn(0x555916d213a0 msgr2=0x555916e12d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/641869109 shutdown_connections 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/641869109 wait complete. 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dcaa1f800 1 Processor -- start 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dcaa1f800 1 -- start start 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dcaa1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916e18bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dcaa1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555916d2f410 con 0x555916d25d40 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dc8525640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916e18bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dc8525640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916e18bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52094/0 (socket says 172.21.15.165:52094) 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.011+0000 7f2dc8525640 1 -- 172.21.15.165:0/2389152877 learned_addr learned my addr 172.21.15.165:0/2389152877 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.012+0000 7f2dc8525640 1 -- 172.21.15.165:0/2389152877 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555916e172b0 con 0x555916d25d40 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.012+0000 7f2dc8525640 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916e18bc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2dc001cfe0 tx=0x7f2dc0003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.012+0000 7f2cb4e09640 1 -- 172.21.15.165:0/2389152877 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc0073070 con 0x555916d25d40 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.012+0000 7f2cb4e09640 1 -- 172.21.15.165:0/2389152877 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc0052da0 con 0x555916d25d40 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.012+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555916e17540 con 0x555916d25d40 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.012+0000 7f2cb4e09640 1 -- 172.21.15.165:0/2389152877 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc0057070 con 0x555916d25d40 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.012+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555916e17a60 con 0x555916d25d40 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.013+0000 7f2cb4e09640 1 -- 172.21.15.165:0/2389152877 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2dc0076070 con 0x555916d25d40 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.013+0000 7f2cb4e09640 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2cb0036900 0x7f2cb0038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.013+0000 7f2cb4e09640 1 -- 172.21.15.165:0/2389152877 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2dc008d9a0 con 0x555916d25d40 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.013+0000 7f2dc7d24640 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2cb0036900 0x7f2cb0038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.013+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555916d26fa0 0x555916e29390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.013+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e299b0 con 0x555916d26fa0 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.013+0000 7f2dc8d26640 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555916d26fa0 0x555916e29390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.013+0000 7f2dc7d24640 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2cb0036900 0x7f2cb0038dc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2db0001e10 tx=0x7f2db0001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.013+0000 7f2dc8d26640 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555916d26fa0 0x555916e29390 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.014+0000 7f2dc8d26640 1 -- 172.21.15.165:0/2389152877 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916d26fa0 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.014+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e2a250 con 0x555916d26fa0 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.014+0000 7f2dc8d26640 1 -- 172.21.15.165:0/2389152877 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916d26fa0 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.014+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555916e2bcd0 0x555916e2e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.014+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e2e8f0 con 0x555916e2bcd0 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.014+0000 7f2dc8525640 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555916e2bcd0 0x555916e2e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.014+0000 7f2dc8525640 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555916e2bcd0 0x555916e2e160 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.015+0000 7f2dc8525640 1 -- 172.21.15.165:0/2389152877 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2dc0075030 con 0x555916e2bcd0 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.015+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e2aad0 con 0x555916d26fa0 2024-04-27T03:30:32.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.015+0000 7f2dc8d26640 1 -- 172.21.15.165:0/2389152877 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916d26fa0 2024-04-27T03:30:32.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.015+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x555916e2fda0 0x555916e32210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.015+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e32970 con 0x555916e2fda0 2024-04-27T03:30:32.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.015+0000 7f2dc7d24640 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x555916e2fda0 0x555916e32210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.016+0000 7f2dc7d24640 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x555916e2fda0 0x555916e32210 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.016+0000 7f2dc7d24640 1 -- 172.21.15.165:0/2389152877 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2db00106a0 con 0x555916e2fda0 2024-04-27T03:30:32.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.016+0000 7f2dcaa1f800 1 Processor -- start 2024-04-27T03:30:32.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.016+0000 7f2dcaa1f800 1 -- start start 2024-04-27T03:30:32.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.016+0000 7f2dcaa1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e40dd0 0x555916e411f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.016+0000 7f2dcaa1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555916e19100 con 0x555916e40dd0 2024-04-27T03:30:32.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.017+0000 7f2dc8525640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e40dd0 0x555916e411f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.017+0000 7f2dc8525640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e40dd0 0x555916e411f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52106/0 (socket says 172.21.15.165:52106) 2024-04-27T03:30:32.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.017+0000 7f2dc8525640 1 -- 172.21.15.165:0/4005365518 learned_addr learned my addr 172.21.15.165:0/4005365518 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.017+0000 7f2dc8525640 1 -- 172.21.15.165:0/4005365518 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555916e41730 con 0x555916e40dd0 2024-04-27T03:30:32.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.017+0000 7f2dc8525640 1 --2- 172.21.15.165:0/4005365518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e40dd0 0x555916e411f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0006b30 tx=0x7f2dc005dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.017+0000 7f2cadffb640 1 -- 172.21.15.165:0/4005365518 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc0092070 con 0x555916e40dd0 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.017+0000 7f2cadffb640 1 -- 172.21.15.165:0/4005365518 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc0008010 con 0x555916e40dd0 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.017+0000 7f2cadffb640 1 -- 172.21.15.165:0/4005365518 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc008d300 con 0x555916e40dd0 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.017+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4005365518 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555916e421a0 con 0x555916e40dd0 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.018+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4005365518 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555916e426a0 con 0x555916e40dd0 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.018+0000 7f2cadffb640 1 -- 172.21.15.165:0/4005365518 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2dc0095070 con 0x555916e40dd0 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.018+0000 7f2cadffb640 1 --2- 172.21.15.165:0/4005365518 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ca0036950 0x7f2ca0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.018+0000 7f2cadffb640 1 -- 172.21.15.165:0/4005365518 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2dc0089030 con 0x555916e40dd0 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.019+0000 7f2dc7d24640 1 --2- 172.21.15.165:0/4005365518 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ca0036950 0x7f2ca0038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.019+0000 7f2dcaa1f800 1 Processor -- start 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.019+0000 7f2dcaa1f800 1 -- start start 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.019+0000 7f2dc7d24640 1 --2- 172.21.15.165:0/4005365518 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ca0036950 0x7f2ca0038e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x555916e17e80 tx=0x7f2db0006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.019+0000 7f2dcaa1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 0x555916e4e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.019+0000 7f2dcaa1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555916c186b0 con 0x555916e4dfc0 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.019+0000 7f2dc8525640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 0x555916e4e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.019+0000 7f2dc8525640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 0x555916e4e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52112/0 (socket says 172.21.15.165:52112) 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.019+0000 7f2dc8525640 1 -- 172.21.15.165:0/701252955 learned_addr learned my addr 172.21.15.165:0/701252955 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.019+0000 7f2dc8525640 1 -- 172.21.15.165:0/701252955 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555916e4e9b0 con 0x555916e4dfc0 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.020+0000 7f2dc8525640 1 --2- 172.21.15.165:0/701252955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 0x555916e4e3e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0002f40 tx=0x7f2dc008d870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7c55251183f022 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.020+0000 7f2c9f7fe640 1 -- 172.21.15.165:0/701252955 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc0078a10 con 0x555916e4dfc0 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.020+0000 7f2c9f7fe640 1 -- 172.21.15.165:0/701252955 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc0081c70 con 0x555916e4dfc0 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.020+0000 7f2c9f7fe640 1 -- 172.21.15.165:0/701252955 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc0077730 con 0x555916e4dfc0 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.020+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/701252955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 msgr2=0x555916e4e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.020+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/701252955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 0x555916e4e3e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0002f40 tx=0x7f2dc008d870 comp rx=0 tx=0).stop 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.020+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/701252955 shutdown_connections 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.020+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/701252955 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 0x555916e4e3e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.020+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/701252955 >> 172.21.15.165:0/701252955 conn(0x555916e4a290 msgr2=0x555916e482f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.020+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/701252955 shutdown_connections 2024-04-27T03:30:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.020+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/701252955 wait complete. 2024-04-27T03:30:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.020+0000 7f2dcaa1f800 1 Processor -- start 2024-04-27T03:30:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.021+0000 7f2dcaa1f800 1 -- start start 2024-04-27T03:30:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.021+0000 7f2dcaa1f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 0x555916e55170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.021+0000 7f2dcaa1f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x555916e4ee90 con 0x555916e4dfc0 2024-04-27T03:30:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.021+0000 7f2dc8525640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 0x555916e55170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.021+0000 7f2dc8525640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 0x555916e55170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52120/0 (socket says 172.21.15.165:52120) 2024-04-27T03:30:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.021+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 learned_addr learned my addr 172.21.15.165:0/3155431318 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.021+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555916e556b0 con 0x555916e4dfc0 2024-04-27T03:30:32.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.021+0000 7f2dc8525640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 0x555916e55170 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2dc005cf70 tx=0x7f2dc004ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.021+0000 7f2c9dffb640 1 -- 172.21.15.165:0/3155431318 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc0078a10 con 0x555916e4dfc0 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.021+0000 7f2c9dffb640 1 -- 172.21.15.165:0/3155431318 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2dc00766f0 con 0x555916e4dfc0 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.021+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x555916e556b0 con 0x555916e4dfc0 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.022+0000 7f2c9dffb640 1 -- 172.21.15.165:0/3155431318 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2dc00077c0 con 0x555916e4dfc0 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.022+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555916e556b0 con 0x555916e4dfc0 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.022+0000 7f2c9dffb640 1 -- 172.21.15.165:0/3155431318 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2dc0007960 con 0x555916e4dfc0 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.022+0000 7f2c9dffb640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c90036950 0x7f2c90038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.022+0000 7f2c9dffb640 1 -- 172.21.15.165:0/3155431318 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2dc00c2aa0 con 0x555916e4dfc0 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.022+0000 7f2dc7d24640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c90036950 0x7f2c90038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.023+0000 7f2dc7d24640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c90036950 0x7f2c90038e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x555916e32d70 tx=0x7f2db0011000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.023+0000 7f2dcaa1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.023+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555916e60910 0x555916e62dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.023+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e63520 con 0x555916e60910 2024-04-27T03:30:32.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.023+0000 7f2dc8d26640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555916e60910 0x555916e62dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.023+0000 7f2dc8d26640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555916e60910 0x555916e62dd0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 20 realm 2024-04-27T03:30:32.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e63930 con 0x555916e60910 2024-04-27T03:30:32.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:32.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e64200 con 0x555916e60910 2024-04-27T03:30:32.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e60910 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555916e65300 0x555916e676f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e67d00 con 0x555916e65300 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.024+0000 7f2dc8525640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555916e65300 0x555916e676f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.025+0000 7f2dc8525640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555916e65300 0x555916e676f0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.025+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.025+0000 7f2dcaa1f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:32.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.025+0000 7f2dcaa1f800 20 searching for the correct realm 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.025+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e69630 con 0x555916e65300 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.025+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.025+0000 7f2dc8525640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2dc00af040 0x7f2dc006d840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.026+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e69630 con 0x7f2dc00af040 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.026+0000 7f2dc7d24640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2dc00af040 0x7f2dc006d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.026+0000 7f2dc7d24640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2dc00af040 0x7f2dc006d840 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.026+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db0036040 con 0x7f2dc00af040 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.026+0000 7f2dc7d24640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2db00393c0 0x7f2db0039810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.026+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e69630 con 0x7f2db00393c0 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.026+0000 7f2dc8d26640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2db00393c0 0x7f2db0039810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.027+0000 7f2dc8d26640 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2db00393c0 0x7f2db0039810 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.027+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.027+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e60910 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.027+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.027+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e60910 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.027+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.027+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x7f2db00393c0 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.028+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.028+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x7f2db00393c0 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.028+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.028+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x7f2dc00af040 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.028+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db0036040 con 0x7f2dc00af040 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.028+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e65300 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.028+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.028+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e60910 2024-04-27T03:30:32.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.029+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.029+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x7f2db00393c0 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.029+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.029+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e60910 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.029+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.029+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x7f2dc00af040 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.030+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2db0036040 con 0x7f2dc00af040 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.030+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e65300 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.030+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.030+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e65300 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.030+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.030+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x7f2db00393c0 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.031+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.031+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e65300 2024-04-27T03:30:32.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.031+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.031+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e65300 2024-04-27T03:30:32.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.031+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.031+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e60910 2024-04-27T03:30:32.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.031+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.031+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e60910 2024-04-27T03:30:32.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.032+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.032+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x7f2db00393c0 2024-04-27T03:30:32.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.032+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.032+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e65300 2024-04-27T03:30:32.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.032+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.032+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x7f2dc00af040 2024-04-27T03:30:32.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.033+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db0036040 con 0x7f2dc00af040 2024-04-27T03:30:32.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.033+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e60910 2024-04-27T03:30:32.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.033+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.033+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x7f2db00393c0 2024-04-27T03:30:32.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.033+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.033+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e65300 2024-04-27T03:30:32.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.033+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.033+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x7f2db00393c0 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.034+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.034+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x7f2dc00af040 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.034+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db0036040 con 0x7f2dc00af040 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.034+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e65300 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.034+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.034+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x7f2db00393c0 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.034+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.034+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e60910 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.034+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2db8038040 con 0x7f2dc00af040 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2db0036040 con 0x7f2dc00af040 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:32.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e645a0 con 0x555916e60910 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e685c0 con 0x555916e60910 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.035+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e685c0 con 0x7f2db00393c0 2024-04-27T03:30:32.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:32.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 20 zone default found 2024-04-27T03:30:32.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 4 Realm: () 2024-04-27T03:30:32.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:32.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:32.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:32.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 20 zonegroup default 2024-04-27T03:30:32.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e685b0 con 0x7f2db00393c0 2024-04-27T03:30:32.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e68fa0 con 0x555916e65300 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e6b490 con 0x555916e65300 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 20 started sync module instance, tier type = 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.036+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6a930 con 0x7f2dc00af040 2024-04-27T03:30:32.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.037+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db0036040 con 0x7f2dc00af040 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.037+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93841124664448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6b830 con 0x7f2dc00af040 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.037+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6c1f0 con 0x555916e60910 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.038+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93841124664448] v25'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db0036040 con 0x7f2dc00af040 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.039+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.039+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93841124670784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6c610 con 0x555916e60910 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.039+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6d010 con 0x555916e65300 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.040+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93841124670784] v25'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.040+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.040+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93841124671840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6c580 con 0x555916e65300 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.040+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6ded0 con 0x7f2dc00af040 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.041+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93841124671840] v25'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.041+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db0036040 con 0x7f2dc00af040 2024-04-27T03:30:32.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.041+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93841124676608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6e770 con 0x7f2dc00af040 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.041+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6f020 con 0x555916e60910 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.042+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.042+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93841124681584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6ebd0 con 0x555916e60910 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.042+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e70010 con 0x7f2db00393c0 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.042+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93841124676608] v25'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db0036040 con 0x7f2dc00af040 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.043+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93841124681584] v25'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.043+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.043+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93841124686528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6ffd0 con 0x7f2db00393c0 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.043+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6f530 con 0x7f2db00393c0 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.044+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93841124686528] v25'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.044+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.044+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93841124680384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6f530 con 0x7f2db00393c0 2024-04-27T03:30:32.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.044+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e70ed0 con 0x7f2db00393c0 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.045+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93841124680384] v25'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.045+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'57 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.045+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93841124688640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e70ed0 con 0x7f2db00393c0 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.045+0000 7f2dcaa1f800 20 add_watcher() i=0 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.045+0000 7f2dcaa1f800 20 add_watcher() i=1 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.045+0000 7f2dcaa1f800 20 add_watcher() i=2 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.045+0000 7f2dcaa1f800 20 add_watcher() i=3 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.045+0000 7f2dcaa1f800 20 add_watcher() i=4 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.045+0000 7f2dcaa1f800 20 add_watcher() i=5 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.045+0000 7f2dcaa1f800 20 add_watcher() i=6 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.046+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93841124688640] v25'58 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.046+0000 7f2dcaa1f800 20 add_watcher() i=7 2024-04-27T03:30:32.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.046+0000 7f2dcaa1f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.046+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916e741f0 con 0x555916e60910 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.046+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.046+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93841124690832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e71460 con 0x555916e60910 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.047+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93841124690832] v25'78 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.047+0000 7f2c7e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.047+0000 7f2dcaa1f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.047+0000 7f2dcaa1f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.047+0000 7f2dcaa1f800 5 note: GC not initialized 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.048+0000 7f2c7d7e2640 20 reqs_thread_entry: start 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.048+0000 7f2dcaa1f800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.048+0000 7f2dcaa1f800 20 Filter name: none 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.048+0000 7f2beeffd640 20 reqs_thread_entry: start 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.048+0000 7f2dcaa1f800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:32.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.048+0000 7f2dcaa1f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.048+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916ea75d0 con 0x555916e60910 2024-04-27T03:30:32.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.049+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv6 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.049+0000 7f2dcaa1f800 20 rados_obj.operate() r=0 bl.length=556 2024-04-27T03:30:32.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.049+0000 7f2dcaa1f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:32.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.049+0000 7f2dcaa1f800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:32.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.049+0000 7f2dcaa1f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:32.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.049+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~556 in=556b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916eadd70 con 0x555916e60910 2024-04-27T03:30:32.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~556] v25'7 uv7 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2dcaa1f800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:32.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2dcaa1f800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:32.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2dcaa1f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:32.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 93841124939744 in=744b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x555916ea94a0 con 0x555916e60910 2024-04-27T03:30:32.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify (1) cookie 93841124681584 notify 107374182406 ret 0) ==== 774+0+0 (crc 0 0 0) 0x7f2db801e040 con 0x555916e60910 2024-04-27T03:30:32.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2c9f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182406 cookie 93841124681584 notifier 4357 bl.length()=732 2024-04-27T03:30:32.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2c9f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:32.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2c9f7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:32.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2c9f7fe640 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f2c8c003ea0 con 0x555916e60910 2024-04-27T03:30:32.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(64 notify.4 [notify cookie 93841124939744 out=8b] v0'0 uv57 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.050+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== watch-notify(notify_complete (2) cookie 93841124939744 notify 107374182406 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f2db8018db0 con 0x555916e60910 2024-04-27T03:30:32.069 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:32.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.051+0000 7f2c7e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.051+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93841124690832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916ea8ff0 con 0x555916e60910 2024-04-27T03:30:32.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.052+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 93841124690832] v25'79 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.052+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93841124664448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916ea94a0 con 0x7f2dc00af040 2024-04-27T03:30:32.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.053+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 93841124664448] v25'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db0036040 con 0x7f2dc00af040 2024-04-27T03:30:32.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.053+0000 7f2dcaa1f800 20 remove_watcher() i=0 2024-04-27T03:30:32.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.053+0000 7f2dcaa1f800 2 removed watcher, disabling cache 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.053+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93841124670784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916ea94a0 con 0x555916e60910 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.054+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 93841124670784] v25'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.054+0000 7f2dcaa1f800 20 remove_watcher() i=1 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.054+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93841124671840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916ea94a0 con 0x555916e65300 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.055+0000 7f2dc8525640 1 -- 172.21.15.165:0/3155431318 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 93841124671840] v25'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2dc00be020 con 0x555916e65300 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.055+0000 7f2dcaa1f800 20 remove_watcher() i=2 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.055+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93841124676608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e70b90 con 0x7f2dc00af040 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.056+0000 7f2dc7d24640 1 -- 172.21.15.165:0/3155431318 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 93841124676608] v25'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db0036040 con 0x7f2dc00af040 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.056+0000 7f2dcaa1f800 20 remove_watcher() i=3 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.056+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93841124681584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e70b90 con 0x555916e60910 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.057+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 93841124681584] v25'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x555916e60910 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.057+0000 7f2dcaa1f800 20 remove_watcher() i=4 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.057+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93841124686528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e70b90 con 0x7f2db00393c0 2024-04-27T03:30:32.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.058+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 93841124686528] v25'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.058+0000 7f2dcaa1f800 20 remove_watcher() i=5 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.058+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93841124680384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e6fac0 con 0x7f2db00393c0 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.059+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 93841124680384] v25'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.059+0000 7f2dcaa1f800 20 remove_watcher() i=6 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.059+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93841124688640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x555916e70b90 con 0x7f2db00393c0 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 93841124688640] v25'59 uv57 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2db800a4a0 con 0x7f2db00393c0 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 20 remove_watcher() i=7 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2db00393c0 msgr2=0x7f2db0039810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2db00393c0 0x7f2db0039810 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555916e60910 msgr2=0x555916e62dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555916e60910 0x555916e62dd0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2dc00af040 msgr2=0x7f2dc006d840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2dc00af040 0x7f2dc006d840 crc :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555916e65300 msgr2=0x555916e676f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555916e65300 0x555916e676f0 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c90036950 msgr2=0x7f2c90038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c90036950 0x7f2c90038e10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x555916e32d70 tx=0x7f2db0011000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 msgr2=0x555916e55170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 0x555916e55170 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f2dc005cf70 tx=0x7f2dc004ee90 comp rx=0 tx=0).stop 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dc8d26640 1 -- 172.21.15.165:0/3155431318 reap_dead start 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 shutdown_connections 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/3155431318 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e4dfc0 0x555916e55170 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 >> 172.21.15.165:0/3155431318 conn(0x555916e4a290 msgr2=0x555916e478f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 shutdown_connections 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.060+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/3155431318 wait complete. 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4005365518 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ca0036950 msgr2=0x7f2ca0038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/4005365518 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ca0036950 0x7f2ca0038e10 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x555916e17e80 tx=0x7f2db0006000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4005365518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e40dd0 msgr2=0x555916e411f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/4005365518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e40dd0 0x555916e411f0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f2dc0006b30 tx=0x7f2dc005dde0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4005365518 shutdown_connections 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/4005365518 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2ca0036950 0x7f2ca0038e10 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/4005365518 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916e40dd0 0x555916e411f0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4005365518 >> 172.21.15.165:0/4005365518 conn(0x555916e34e70 msgr2=0x555916e35270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4005365518 shutdown_connections 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/4005365518 wait complete. 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x555916e2fda0 msgr2=0x555916e32210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x555916e2fda0 0x555916e32210 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555916d26fa0 msgr2=0x555916e29390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x555916d26fa0 0x555916e29390 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555916e2bcd0 msgr2=0x555916e2e160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x555916e2bcd0 0x555916e2e160 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2cb0036900 msgr2=0x7f2cb0038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2cb0036900 0x7f2cb0038dc0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f2db0001e10 tx=0x7f2db0001390 comp rx=0 tx=0).stop 2024-04-27T03:30:32.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 msgr2=0x555916e18bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 --2- 172.21.15.165:0/2389152877 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x555916d25d40 0x555916e18bc0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f2dc001cfe0 tx=0x7f2dc0003020 comp rx=0 tx=0).stop 2024-04-27T03:30:32.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dc8d26640 1 -- 172.21.15.165:0/2389152877 reap_dead start 2024-04-27T03:30:32.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 shutdown_connections 2024-04-27T03:30:32.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 >> 172.21.15.165:0/2389152877 conn(0x555916d213a0 msgr2=0x555916e11620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 shutdown_connections 2024-04-27T03:30:32.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.061+0000 7f2dcaa1f800 1 -- 172.21.15.165:0/2389152877 wait complete. 2024-04-27T03:30:32.080 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:32.080 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T03:30:32.080 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T03:30:32.081 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T03:30:32.123 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:32.123 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:32.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.118+0000 7f0d55013800 1 Processor -- start 2024-04-27T03:30:32.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.118+0000 7f0d55013800 1 -- start start 2024-04-27T03:30:32.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.118+0000 7f0d55013800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562cf3ca03c0 0x562cf3ca27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.118+0000 7f0d55013800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562cf3b5ad40 con 0x562cf3c9c8b0 2024-04-27T03:30:32.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.118+0000 7f0d55013800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x562cf3b2d4d0 con 0x562cf3ca03c0 2024-04-27T03:30:32.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.119+0000 7f0d47fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562cf3ca03c0 0x562cf3ca27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.119+0000 7f0d47fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562cf3ca03c0 0x562cf3ca27b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52128/0 (socket says 172.21.15.165:52128) 2024-04-27T03:30:32.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.119+0000 7f0d47fff640 1 -- 172.21.15.165:0/3374617469 learned_addr learned my addr 172.21.15.165:0/3374617469 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.119+0000 7f0d5066e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x562cf3c9c8b0 0x562cf3c9eca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53692/0 (socket says 172.21.15.165:53692) 2024-04-27T03:30:32.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.119+0000 7f0d4fe6d640 1 -- 172.21.15.165:0/3374617469 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1314056306 0 0) 0x562cf3b5ad40 con 0x562cf3c9c8b0 2024-04-27T03:30:32.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.119+0000 7f0d4fe6d640 1 -- 172.21.15.165:0/3374617469 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d3c001e10 con 0x562cf3c9c8b0 2024-04-27T03:30:32.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.119+0000 7f0d47fff640 1 -- 172.21.15.165:0/3374617469 >> v1:172.21.15.89:6789/0 conn(0x562cf3c9c8b0 legacy=0x562cf3c9eca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.119+0000 7f0d47fff640 1 -- 172.21.15.165:0/3374617469 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf3b59590 con 0x562cf3ca03c0 2024-04-27T03:30:32.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.121+0000 7f0d47fff640 1 --2- 172.21.15.165:0/3374617469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3ca03c0 0x562cf3ca27b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800a520 tx=0x7f0d3801cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a2ad0c2f9347ddc server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.121+0000 7f0d4fe6d640 1 -- 172.21.15.165:0/3374617469 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d38057070 con 0x562cf3ca03c0 2024-04-27T03:30:32.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.121+0000 7f0d4fe6d640 1 -- 172.21.15.165:0/3374617469 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d38052570 con 0x562cf3ca03c0 2024-04-27T03:30:32.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.121+0000 7f0d4fe6d640 1 -- 172.21.15.165:0/3374617469 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d38052870 con 0x562cf3ca03c0 2024-04-27T03:30:32.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.122+0000 7f0d55013800 1 -- 172.21.15.165:0/3374617469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3ca03c0 msgr2=0x562cf3ca27b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.122+0000 7f0d55013800 1 --2- 172.21.15.165:0/3374617469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3ca03c0 0x562cf3ca27b0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0d3800a520 tx=0x7f0d3801cf30 comp rx=0 tx=0).stop 2024-04-27T03:30:32.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.122+0000 7f0d55013800 1 -- 172.21.15.165:0/3374617469 shutdown_connections 2024-04-27T03:30:32.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.122+0000 7f0d55013800 1 --2- 172.21.15.165:0/3374617469 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3ca03c0 0x562cf3ca27b0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.122+0000 7f0d55013800 1 -- 172.21.15.165:0/3374617469 >> 172.21.15.165:0/3374617469 conn(0x562cf3c96380 msgr2=0x562cf3c987a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.122+0000 7f0d55013800 1 -- 172.21.15.165:0/3374617469 shutdown_connections 2024-04-27T03:30:32.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.122+0000 7f0d55013800 1 -- 172.21.15.165:0/3374617469 wait complete. 2024-04-27T03:30:32.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.134+0000 7f0d55013800 1 Processor -- start 2024-04-27T03:30:32.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.134+0000 7f0d55013800 1 -- start start 2024-04-27T03:30:32.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.134+0000 7f0d55013800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 0x562cf3d8ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.134+0000 7f0d55013800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562cf3be73c0 con 0x562cf3c9c8b0 2024-04-27T03:30:32.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.134+0000 7f0d5066e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 0x562cf3d8ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.134+0000 7f0d5066e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 0x562cf3d8ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52138/0 (socket says 172.21.15.165:52138) 2024-04-27T03:30:32.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.134+0000 7f0d5066e640 1 -- 172.21.15.165:0/1844790255 learned_addr learned my addr 172.21.15.165:0/1844790255 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.135+0000 7f0d5066e640 1 -- 172.21.15.165:0/1844790255 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf3d8d4a0 con 0x562cf3c9c8b0 2024-04-27T03:30:32.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.135+0000 7f0d5066e640 1 --2- 172.21.15.165:0/1844790255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 0x562cf3d8ced0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b230 tx=0x7f0d4000b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31734f83cd063e2e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.135+0000 7f0c3e60c640 1 -- 172.21.15.165:0/1844790255 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d4000e5c0 con 0x562cf3c9c8b0 2024-04-27T03:30:32.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.135+0000 7f0c3e60c640 1 -- 172.21.15.165:0/1844790255 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d40006070 con 0x562cf3c9c8b0 2024-04-27T03:30:32.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.135+0000 7f0c3e60c640 1 -- 172.21.15.165:0/1844790255 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d4000e8d0 con 0x562cf3c9c8b0 2024-04-27T03:30:32.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d55013800 1 -- 172.21.15.165:0/1844790255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 msgr2=0x562cf3d8ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d55013800 1 --2- 172.21.15.165:0/1844790255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 0x562cf3d8ced0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000b230 tx=0x7f0d4000b6f0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d55013800 1 -- 172.21.15.165:0/1844790255 shutdown_connections 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d55013800 1 --2- 172.21.15.165:0/1844790255 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 0x562cf3d8ced0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d55013800 1 -- 172.21.15.165:0/1844790255 >> 172.21.15.165:0/1844790255 conn(0x562cf3c96380 msgr2=0x562cf3d887f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d55013800 1 -- 172.21.15.165:0/1844790255 shutdown_connections 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d55013800 1 -- 172.21.15.165:0/1844790255 wait complete. 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d55013800 1 Processor -- start 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d55013800 1 -- start start 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d55013800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 0x562cf3d94a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d55013800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562cf3ca2ff0 con 0x562cf3c9c8b0 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d5066e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 0x562cf3d94a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d5066e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 0x562cf3d94a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52152/0 (socket says 172.21.15.165:52152) 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.136+0000 7f0d5066e640 1 -- 172.21.15.165:0/3602302201 learned_addr learned my addr 172.21.15.165:0/3602302201 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.137+0000 7f0d5066e640 1 -- 172.21.15.165:0/3602302201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf3d94f90 con 0x562cf3c9c8b0 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.137+0000 7f0d5066e640 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 0x562cf3d94a50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0d400011e0 tx=0x7f0d40000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.137+0000 7f0c3ce09640 1 -- 172.21.15.165:0/3602302201 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d4000f8d0 con 0x562cf3c9c8b0 2024-04-27T03:30:32.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.137+0000 7f0c3ce09640 1 -- 172.21.15.165:0/3602302201 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4000fab0 con 0x562cf3c9c8b0 2024-04-27T03:30:32.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.137+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562cf3d95220 con 0x562cf3c9c8b0 2024-04-27T03:30:32.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.137+0000 7f0c3ce09640 1 -- 172.21.15.165:0/3602302201 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d4000fdd0 con 0x562cf3c9c8b0 2024-04-27T03:30:32.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.137+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf3d97ed0 con 0x562cf3c9c8b0 2024-04-27T03:30:32.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.138+0000 7f0c3ce09640 1 -- 172.21.15.165:0/3602302201 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0d40017070 con 0x562cf3c9c8b0 2024-04-27T03:30:32.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.138+0000 7f0c3ce09640 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c38036950 0x7f0c38038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.138+0000 7f0c3ce09640 1 -- 172.21.15.165:0/3602302201 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f0d40015020 con 0x562cf3c9c8b0 2024-04-27T03:30:32.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.138+0000 7f0d47fff640 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c38036950 0x7f0c38038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.138+0000 7f0d55013800 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562cf3ca03c0 0x562cf3d9bc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.138+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3d9c210 con 0x562cf3ca03c0 2024-04-27T03:30:32.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.138+0000 7f0d50e6f640 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562cf3ca03c0 0x562cf3d9bc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.139+0000 7f0d47fff640 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c38036950 0x7f0c38038e10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d38054000 tx=0x7f0d3804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.139+0000 7f0d50e6f640 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562cf3ca03c0 0x562cf3d9bc90 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.139+0000 7f0d50e6f640 1 -- 172.21.15.165:0/3602302201 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3ca03c0 2024-04-27T03:30:32.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.139+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3d99260 con 0x562cf3ca03c0 2024-04-27T03:30:32.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.139+0000 7f0d50e6f640 1 -- 172.21.15.165:0/3602302201 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3ca03c0 2024-04-27T03:30:32.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.139+0000 7f0d55013800 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562cf3d9e540 0x562cf3da0930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.139+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3da1020 con 0x562cf3d9e540 2024-04-27T03:30:32.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.139+0000 7f0d5066e640 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562cf3d9e540 0x562cf3da0930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.140+0000 7f0d5066e640 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562cf3d9e540 0x562cf3da0930 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.140+0000 7f0d5066e640 1 -- 172.21.15.165:0/3602302201 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0d4002e070 con 0x562cf3d9e540 2024-04-27T03:30:32.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.140+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3d9d4f0 con 0x562cf3ca03c0 2024-04-27T03:30:32.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.140+0000 7f0d50e6f640 1 -- 172.21.15.165:0/3602302201 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3ca03c0 2024-04-27T03:30:32.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.140+0000 7f0d55013800 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562cf3da24d0 0x562cf3da4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.140+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3da5080 con 0x562cf3da24d0 2024-04-27T03:30:32.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.140+0000 7f0d47fff640 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562cf3da24d0 0x562cf3da4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.141+0000 7f0d47fff640 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562cf3da24d0 0x562cf3da4920 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.141+0000 7f0d47fff640 1 -- 172.21.15.165:0/3602302201 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0d3805a040 con 0x562cf3da24d0 2024-04-27T03:30:32.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.141+0000 7f0d55013800 1 Processor -- start 2024-04-27T03:30:32.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.141+0000 7f0d55013800 1 -- start start 2024-04-27T03:30:32.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.142+0000 7f0d55013800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3db34e0 0x562cf3db3900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.142+0000 7f0d55013800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562cf3b8d6b0 con 0x562cf3db34e0 2024-04-27T03:30:32.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.142+0000 7f0d5066e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3db34e0 0x562cf3db3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.142+0000 7f0d5066e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3db34e0 0x562cf3db3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52160/0 (socket says 172.21.15.165:52160) 2024-04-27T03:30:32.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.142+0000 7f0d5066e640 1 -- 172.21.15.165:0/1925009740 learned_addr learned my addr 172.21.15.165:0/1925009740 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.142+0000 7f0d5066e640 1 -- 172.21.15.165:0/1925009740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf3db3e40 con 0x562cf3db34e0 2024-04-27T03:30:32.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.142+0000 7f0d5066e640 1 --2- 172.21.15.165:0/1925009740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3db34e0 0x562cf3db3900 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0d4004b4c0 tx=0x7f0d40017d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.142+0000 7f0c35ffb640 1 -- 172.21.15.165:0/1925009740 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d4001a5b0 con 0x562cf3db34e0 2024-04-27T03:30:32.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.142+0000 7f0c35ffb640 1 -- 172.21.15.165:0/1925009740 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4001a750 con 0x562cf3db34e0 2024-04-27T03:30:32.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.142+0000 7f0c35ffb640 1 -- 172.21.15.165:0/1925009740 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d40044b20 con 0x562cf3db34e0 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.142+0000 7f0d55013800 1 -- 172.21.15.165:0/1925009740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562cf3db48b0 con 0x562cf3db34e0 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.142+0000 7f0d55013800 1 -- 172.21.15.165:0/1925009740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf3db4db0 con 0x562cf3db34e0 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.143+0000 7f0c35ffb640 1 -- 172.21.15.165:0/1925009740 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0d40044cc0 con 0x562cf3db34e0 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.143+0000 7f0c35ffb640 1 --2- 172.21.15.165:0/1925009740 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c28036950 0x7f0c28038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.143+0000 7f0c35ffb640 1 -- 172.21.15.165:0/1925009740 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f0d40038070 con 0x562cf3db34e0 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.143+0000 7f0d47fff640 1 --2- 172.21.15.165:0/1925009740 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c28036950 0x7f0c28038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.144+0000 7f0d55013800 1 Processor -- start 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.144+0000 7f0d55013800 1 -- start start 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.144+0000 7f0d47fff640 1 --2- 172.21.15.165:0/1925009740 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c28036950 0x7f0c28038e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x562cf3d9caf0 tx=0x7f0d38005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.144+0000 7f0d55013800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 0x562cf3dc09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.144+0000 7f0d55013800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562cf3d8d820 con 0x562cf3dc05a0 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.144+0000 7f0d5066e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 0x562cf3dc09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.144+0000 7f0d5066e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 0x562cf3dc09c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52164/0 (socket says 172.21.15.165:52164) 2024-04-27T03:30:32.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.144+0000 7f0d5066e640 1 -- 172.21.15.165:0/1230543420 learned_addr learned my addr 172.21.15.165:0/1230543420 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.144+0000 7f0d5066e640 1 -- 172.21.15.165:0/1230543420 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf3dc0f90 con 0x562cf3dc05a0 2024-04-27T03:30:32.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.144+0000 7f0d5066e640 1 --2- 172.21.15.165:0/1230543420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 0x562cf3dc09c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000ff90 tx=0x7f0d40036e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29731d7de3862c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0c277fe640 1 -- 172.21.15.165:0/1230543420 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d40026bb0 con 0x562cf3dc05a0 2024-04-27T03:30:32.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0c277fe640 1 -- 172.21.15.165:0/1230543420 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d40032df0 con 0x562cf3dc05a0 2024-04-27T03:30:32.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0c277fe640 1 -- 172.21.15.165:0/1230543420 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d4004b790 con 0x562cf3dc05a0 2024-04-27T03:30:32.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0d55013800 1 -- 172.21.15.165:0/1230543420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 msgr2=0x562cf3dc09c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0d55013800 1 --2- 172.21.15.165:0/1230543420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 0x562cf3dc09c0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000ff90 tx=0x7f0d40036e70 comp rx=0 tx=0).stop 2024-04-27T03:30:32.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0d55013800 1 -- 172.21.15.165:0/1230543420 shutdown_connections 2024-04-27T03:30:32.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0d55013800 1 --2- 172.21.15.165:0/1230543420 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 0x562cf3dc09c0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0d55013800 1 -- 172.21.15.165:0/1230543420 >> 172.21.15.165:0/1230543420 conn(0x562cf3dbc900 msgr2=0x562cf3dba960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0d55013800 1 -- 172.21.15.165:0/1230543420 shutdown_connections 2024-04-27T03:30:32.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0d55013800 1 -- 172.21.15.165:0/1230543420 wait complete. 2024-04-27T03:30:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0d55013800 1 Processor -- start 2024-04-27T03:30:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0d55013800 1 -- start start 2024-04-27T03:30:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.145+0000 7f0d55013800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 0x562cf3dc79c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.146+0000 7f0d55013800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562cf3dc1640 con 0x562cf3dc05a0 2024-04-27T03:30:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.146+0000 7f0d5066e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 0x562cf3dc79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.146+0000 7f0d5066e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 0x562cf3dc79c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52170/0 (socket says 172.21.15.165:52170) 2024-04-27T03:30:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.146+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 learned_addr learned my addr 172.21.15.165:0/165919165 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.146+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562cf3dc7f00 con 0x562cf3dc05a0 2024-04-27T03:30:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.146+0000 7f0d5066e640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 0x562cf3dc79c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d40001600 tx=0x7f0d4002ef00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.146+0000 7f0c25ffb640 1 -- 172.21.15.165:0/165919165 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d40044fb0 con 0x562cf3dc05a0 2024-04-27T03:30:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.146+0000 7f0c25ffb640 1 -- 172.21.15.165:0/165919165 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d4003f080 con 0x562cf3dc05a0 2024-04-27T03:30:32.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.146+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562cf3dc7f00 con 0x562cf3dc05a0 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.146+0000 7f0c25ffb640 1 -- 172.21.15.165:0/165919165 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d40019b60 con 0x562cf3dc05a0 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.146+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562cf3dc7f00 con 0x562cf3dc05a0 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.147+0000 7f0c25ffb640 1 -- 172.21.15.165:0/165919165 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0d4006d070 con 0x562cf3dc05a0 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.147+0000 7f0c25ffb640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c18036950 0x7f0c18038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.147+0000 7f0c25ffb640 1 -- 172.21.15.165:0/165919165 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f0d40071ad0 con 0x562cf3dc05a0 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.147+0000 7f0d55013800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.147+0000 7f0d47fff640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c18036950 0x7f0c18038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d55013800 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562cf3dd3140 0x562cf3dd55e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd5d30 con 0x562cf3dd3140 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d50e6f640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562cf3dd3140 0x562cf3dd55e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d47fff640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c18036950 0x7f0c18038e10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000c00 tx=0x7f0d3804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d50e6f640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562cf3dd3140 0x562cf3dd55e0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3dd3140 2024-04-27T03:30:32.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d55013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d55013800 20 realm 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d55013800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd6140 con 0x562cf3dd3140 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3dd3140 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d55013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d55013800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d55013800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.148+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd69e0 con 0x562cf3dd3140 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3dd3140 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d55013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d55013800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd6d80 con 0x562cf3dd3140 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3dd3140 2024-04-27T03:30:32.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d55013800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d55013800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d55013800 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562cf3dd7ae0 0x562cf3dd9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dda450 con 0x562cf3dd7ae0 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d5066e640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562cf3dd7ae0 0x562cf3dd9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d5066e640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562cf3dd7ae0 0x562cf3dd9ed0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d55013800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d55013800 20 searching for the correct realm 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.149+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddbdd0 con 0x562cf3dd7ae0 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.150+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.150+0000 7f0d5066e640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d4000f070 0x7f0d400713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.150+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddbdd0 con 0x7f0d4000f070 2024-04-27T03:30:32.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.150+0000 7f0d47fff640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d4000f070 0x7f0d400713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.150+0000 7f0d47fff640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d4000f070 0x7f0d400713f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.151+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3807f070 con 0x7f0d4000f070 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.151+0000 7f0d47fff640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0d38074040 0x7f0d38081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.151+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddbdd0 con 0x7f0d38074040 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.151+0000 7f0d50e6f640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0d38074040 0x7f0d38081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.151+0000 7f0d50e6f640 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0d38074040 0x7f0d38081410 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.152+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f0d4800a4a0 con 0x7f0d38074040 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.152+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd6d80 con 0x562cf3dd3140 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.152+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3dd3140 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.152+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd6d80 con 0x562cf3dd3140 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.152+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3dd3140 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.152+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd6d80 con 0x7f0d38074040 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.152+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4800a4a0 con 0x7f0d38074040 2024-04-27T03:30:32.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.152+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd6d80 con 0x7f0d38074040 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.153+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4800a4a0 con 0x7f0d38074040 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.153+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd6d80 con 0x7f0d4000f070 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.153+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3807f070 con 0x7f0d4000f070 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.153+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd6d80 con 0x562cf3dd7ae0 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.153+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.153+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd6d80 con 0x562cf3dd3140 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.153+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3dd3140 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.153+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd6d80 con 0x7f0d38074040 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.154+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4800a4a0 con 0x7f0d38074040 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.154+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd6d80 con 0x562cf3dd3140 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.154+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3dd3140 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.154+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dd6d80 con 0x7f0d4000f070 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.154+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0d3807f070 con 0x7f0d4000f070 2024-04-27T03:30:32.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.154+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x562cf3dd7ae0 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.155+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.155+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x562cf3dd7ae0 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.155+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.155+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x7f0d38074040 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.155+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4800a4a0 con 0x7f0d38074040 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.155+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x562cf3dd7ae0 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.155+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.155+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x562cf3dd7ae0 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.155+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.156+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x562cf3dd3140 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.156+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3dd3140 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.156+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x562cf3dd3140 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.156+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0d4800a4a0 con 0x562cf3dd3140 2024-04-27T03:30:32.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.156+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x7f0d38074040 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.156+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d48033040 con 0x7f0d38074040 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.156+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x562cf3dd7ae0 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.156+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.156+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x7f0d4000f070 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.157+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3807f070 con 0x7f0d4000f070 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.157+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x562cf3dd3140 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.157+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.157+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x7f0d38074040 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.157+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.157+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x562cf3dd7ae0 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.158+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.158+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x7f0d38074040 2024-04-27T03:30:32.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.158+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.158+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x7f0d4000f070 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.158+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3807f070 con 0x7f0d4000f070 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.158+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x562cf3dd7ae0 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.158+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.158+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x7f0d38074040 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.158+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x562cf3dd3140 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0d3808e060 con 0x7f0d4000f070 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3807f070 con 0x7f0d4000f070 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:32.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddada0 con 0x562cf3dd3140 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddb190 con 0x562cf3dd3140 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.159+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddb190 con 0x7f0d38074040 2024-04-27T03:30:32.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 20 zone default found 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 4 Realm: () 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 20 zonegroup default 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddb5b0 con 0x7f0d38074040 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddb9d0 con 0x562cf3dd7ae0 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.160+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dddc80 con 0x562cf3dd7ae0 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.161+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.161+0000 7f0d55013800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.161+0000 7f0d55013800 20 started sync module instance, tier type = 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.161+0000 7f0d55013800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.161+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddd120 con 0x7f0d4000f070 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.162+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d3807f070 con 0x7f0d4000f070 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.162+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94751069951296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dde4d0 con 0x7f0d4000f070 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.162+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dded20 con 0x562cf3dd3140 2024-04-27T03:30:32.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.163+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94751069951296] v25'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d3807f070 con 0x7f0d4000f070 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.163+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.163+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94751069954080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dded20 con 0x562cf3dd3140 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.163+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddf810 con 0x562cf3dd7ae0 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.164+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94751069954080] v25'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.164+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.164+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94751069956208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddec90 con 0x562cf3dd7ae0 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.164+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de06d0 con 0x7f0d4000f070 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.165+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d3807f070 con 0x7f0d4000f070 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.165+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94751069964992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de1390 con 0x7f0d4000f070 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.165+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94751069956208] v25'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.165+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de1c10 con 0x562cf3dd3140 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.166+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.166+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94751069968224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de20f0 con 0x562cf3dd3140 2024-04-27T03:30:32.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.166+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de2c10 con 0x7f0d38074040 2024-04-27T03:30:32.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.166+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94751069964992] v25'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d3807f070 con 0x7f0d4000f070 2024-04-27T03:30:32.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.167+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.167+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94751069968224] v25'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.167+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94751069972192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de2bd0 con 0x7f0d38074040 2024-04-27T03:30:32.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.167+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de1b80 con 0x7f0d38074040 2024-04-27T03:30:32.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.168+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94751069972192] v25'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.168+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.168+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94751069966048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de3170 con 0x7f0d38074040 2024-04-27T03:30:32.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.168+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de3b30 con 0x7f0d38074040 2024-04-27T03:30:32.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.169+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94751069966048] v25'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.170+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'60 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.170+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94751069969280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de3b30 con 0x7f0d38074040 2024-04-27T03:30:32.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.170+0000 7f0d55013800 20 add_watcher() i=0 2024-04-27T03:30:32.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.170+0000 7f0d55013800 20 add_watcher() i=1 2024-04-27T03:30:32.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.170+0000 7f0d55013800 20 add_watcher() i=2 2024-04-27T03:30:32.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.170+0000 7f0d55013800 20 add_watcher() i=3 2024-04-27T03:30:32.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.170+0000 7f0d55013800 20 add_watcher() i=4 2024-04-27T03:30:32.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.171+0000 7f0d55013800 20 add_watcher() i=5 2024-04-27T03:30:32.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.171+0000 7f0d55013800 20 add_watcher() i=6 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.172+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94751069969280] v25'61 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.172+0000 7f0d55013800 20 add_watcher() i=7 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.172+0000 7f0d55013800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.172+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de6e70 con 0x562cf3dd3140 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.172+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.172+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94751069976560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de40c0 con 0x562cf3dd3140 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.173+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94751069976560] v25'80 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.173+0000 7f0c067e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.173+0000 7f0d55013800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.173+0000 7f0d55013800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.173+0000 7f0d55013800 5 note: GC not initialized 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.173+0000 7f0c057e2640 20 reqs_thread_entry: start 2024-04-27T03:30:32.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.173+0000 7f0d55013800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:32.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.174+0000 7f0d55013800 20 Filter name: none 2024-04-27T03:30:32.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.174+0000 7f0b76ffd640 20 reqs_thread_entry: start 2024-04-27T03:30:32.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.174+0000 7f0d55013800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.174+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562cf3e19bf0 con 0x562cf3dd3140 2024-04-27T03:30:32.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.174+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.174+0000 7f0d55013800 20 rados_obj.operate() r=0 bl.length=556 2024-04-27T03:30:32.193 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:32.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.175+0000 7f0c067e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.175+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94751069976560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3e187e0 con 0x562cf3dd3140 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.176+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94751069976560] v25'81 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.176+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94751069951296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3e18c90 con 0x7f0d4000f070 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.177+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94751069951296] v25'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d3807f070 con 0x7f0d4000f070 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.177+0000 7f0d55013800 20 remove_watcher() i=0 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.177+0000 7f0d55013800 2 removed watcher, disabling cache 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.177+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94751069954080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddd540 con 0x562cf3dd3140 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.178+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94751069954080] v25'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.178+0000 7f0d55013800 20 remove_watcher() i=1 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.178+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94751069956208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddd540 con 0x562cf3dd7ae0 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.179+0000 7f0d5066e640 1 -- 172.21.15.165:0/165919165 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94751069956208] v25'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d40076020 con 0x562cf3dd7ae0 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.179+0000 7f0d55013800 20 remove_watcher() i=2 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.179+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94751069964992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dde870 con 0x7f0d4000f070 2024-04-27T03:30:32.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.180+0000 7f0d47fff640 1 -- 172.21.15.165:0/165919165 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94751069964992] v25'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d3807f070 con 0x7f0d4000f070 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.180+0000 7f0d55013800 20 remove_watcher() i=3 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.180+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94751069968224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3ddd540 con 0x562cf3dd3140 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.182+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94751069968224] v25'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x562cf3dd3140 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.182+0000 7f0d55013800 20 remove_watcher() i=4 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.182+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94751069972192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3de0ac0 con 0x7f0d38074040 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.182+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94751069972192] v25'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.182+0000 7f0d55013800 20 remove_watcher() i=5 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.183+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94751069966048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dde870 con 0x7f0d38074040 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.184+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94751069966048] v25'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.184+0000 7f0d55013800 20 remove_watcher() i=6 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.184+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94751069969280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562cf3dde870 con 0x7f0d38074040 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94751069969280] v25'62 uv60 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d48031040 con 0x7f0d38074040 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 20 remove_watcher() i=7 2024-04-27T03:30:32.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0d38074040 msgr2=0x7f0d38081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0d38074040 0x7f0d38081410 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562cf3dd3140 msgr2=0x562cf3dd55e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562cf3dd3140 0x562cf3dd55e0 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d4000f070 msgr2=0x7f0d400713f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d4000f070 0x7f0d400713f0 crc :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562cf3dd7ae0 msgr2=0x562cf3dd9ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562cf3dd7ae0 0x562cf3dd9ed0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c18036950 msgr2=0x7f0c18038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c18036950 0x7f0c18038e10 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000c00 tx=0x7f0d3804f000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d50e6f640 1 -- 172.21.15.165:0/165919165 reap_dead start 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 msgr2=0x562cf3dc79c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 0x562cf3dc79c0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f0d40001600 tx=0x7f0d4002ef00 comp rx=0 tx=0).stop 2024-04-27T03:30:32.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 shutdown_connections 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 --2- 172.21.15.165:0/165919165 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3dc05a0 0x562cf3dc79c0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 >> 172.21.15.165:0/165919165 conn(0x562cf3dbc900 msgr2=0x562cf3db9f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 shutdown_connections 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 -- 172.21.15.165:0/165919165 wait complete. 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 -- 172.21.15.165:0/1925009740 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c28036950 msgr2=0x7f0c28038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.185+0000 7f0d55013800 1 --2- 172.21.15.165:0/1925009740 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c28036950 0x7f0c28038e10 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x562cf3d9caf0 tx=0x7f0d38005ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/1925009740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3db34e0 msgr2=0x562cf3db3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 --2- 172.21.15.165:0/1925009740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3db34e0 0x562cf3db3900 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f0d4004b4c0 tx=0x7f0d40017d50 comp rx=0 tx=0).stop 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/1925009740 shutdown_connections 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 --2- 172.21.15.165:0/1925009740 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c28036950 0x7f0c28038e10 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 --2- 172.21.15.165:0/1925009740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3db34e0 0x562cf3db3900 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/1925009740 >> 172.21.15.165:0/1925009740 conn(0x562cf3da7580 msgr2=0x562cf3da7980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/1925009740 shutdown_connections 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/1925009740 wait complete. 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562cf3da24d0 msgr2=0x562cf3da4920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562cf3da24d0 0x562cf3da4920 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562cf3ca03c0 msgr2=0x562cf3d9bc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562cf3ca03c0 0x562cf3d9bc90 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562cf3d9e540 msgr2=0x562cf3da0930 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562cf3d9e540 0x562cf3da0930 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c38036950 msgr2=0x7f0c38038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c38036950 0x7f0c38038e10 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0d38054000 tx=0x7f0d3804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 msgr2=0x562cf3d94a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 --2- 172.21.15.165:0/3602302201 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562cf3c9c8b0 0x562cf3d94a50 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f0d400011e0 tx=0x7f0d40000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d50e6f640 1 -- 172.21.15.165:0/3602302201 reap_dead start 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 shutdown_connections 2024-04-27T03:30:32.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 >> 172.21.15.165:0/3602302201 conn(0x562cf3c96380 msgr2=0x562cf3d87070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 shutdown_connections 2024-04-27T03:30:32.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.186+0000 7f0d55013800 1 -- 172.21.15.165:0/3602302201 wait complete. 2024-04-27T03:30:32.205 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:32.205 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T03:30:32.205 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T03:30:32.205 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T03:30:32.288 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:32.288 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:32.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.283+0000 7f2b2d72b800 1 Processor -- start 2024-04-27T03:30:32.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.283+0000 7f2b2d72b800 1 -- start start 2024-04-27T03:30:32.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.283+0000 7f2b2d72b800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560f0acbf8b0 0x560f0acc1ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.283+0000 7f2b2d72b800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x560f0ab7dd40 con 0x560f0acc21e0 2024-04-27T03:30:32.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.283+0000 7f2b2d72b800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x560f0ab504d0 con 0x560f0acbf8b0 2024-04-27T03:30:32.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.284+0000 7f2b29352640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560f0acbf8b0 0x560f0acc1ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.284+0000 7f2b29352640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560f0acbf8b0 0x560f0acc1ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52176/0 (socket says 172.21.15.165:52176) 2024-04-27T03:30:32.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.284+0000 7f2b20b51640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x560f0acc21e0 0x560f0acc45d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53706/0 (socket says 172.21.15.165:53706) 2024-04-27T03:30:32.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.284+0000 7f2b29352640 1 -- 172.21.15.165:0/212273023 learned_addr learned my addr 172.21.15.165:0/212273023 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.284+0000 7f2b29352640 1 -- 172.21.15.165:0/212273023 >> v1:172.21.15.89:6789/0 conn(0x560f0acc21e0 legacy=0x560f0acc45d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.290 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.284+0000 7f2b29352640 1 -- 172.21.15.165:0/212273023 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0ab7c590 con 0x560f0acbf8b0 2024-04-27T03:30:32.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.286+0000 7f2b29352640 1 --2- 172.21.15.165:0/212273023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0acc1ca0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00a520 tx=0x7f2b1c01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a11fd1addb9afda server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.286+0000 7f2b28b51640 1 -- 172.21.15.165:0/212273023 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b1c057070 con 0x560f0acbf8b0 2024-04-27T03:30:32.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.286+0000 7f2b28b51640 1 -- 172.21.15.165:0/212273023 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c052570 con 0x560f0acbf8b0 2024-04-27T03:30:32.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.286+0000 7f2b28b51640 1 -- 172.21.15.165:0/212273023 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b1c04ae60 con 0x560f0acbf8b0 2024-04-27T03:30:32.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.286+0000 7f2b2d72b800 1 -- 172.21.15.165:0/212273023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 msgr2=0x560f0acc1ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.286+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/212273023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0acc1ca0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c00a520 tx=0x7f2b1c01d010 comp rx=0 tx=0).stop 2024-04-27T03:30:32.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.286+0000 7f2b2d72b800 1 -- 172.21.15.165:0/212273023 shutdown_connections 2024-04-27T03:30:32.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.286+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/212273023 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0acc1ca0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.286+0000 7f2b2d72b800 1 -- 172.21.15.165:0/212273023 >> 172.21.15.165:0/212273023 conn(0x560f0acb9380 msgr2=0x560f0acbb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.286+0000 7f2b2d72b800 1 -- 172.21.15.165:0/212273023 shutdown_connections 2024-04-27T03:30:32.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.286+0000 7f2b2d72b800 1 -- 172.21.15.165:0/212273023 wait complete. 2024-04-27T03:30:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.298+0000 7f2b2d72b800 1 Processor -- start 2024-04-27T03:30:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.298+0000 7f2b2d72b800 1 -- start start 2024-04-27T03:30:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.298+0000 7f2b2d72b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0adb00d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.298+0000 7f2b2d72b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560f0ac0a3c0 con 0x560f0acbf8b0 2024-04-27T03:30:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.299+0000 7f2b29352640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0adb00d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.299+0000 7f2b29352640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0adb00d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52182/0 (socket says 172.21.15.165:52182) 2024-04-27T03:30:32.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.299+0000 7f2b29352640 1 -- 172.21.15.165:0/341883890 learned_addr learned my addr 172.21.15.165:0/341883890 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.299+0000 7f2b29352640 1 -- 172.21.15.165:0/341883890 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0adb06a0 con 0x560f0acbf8b0 2024-04-27T03:30:32.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.299+0000 7f2b29352640 1 --2- 172.21.15.165:0/341883890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0adb00d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c055040 tx=0x7f2b1c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc248f4617e54c0f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.299+0000 7f2a17606640 1 -- 172.21.15.165:0/341883890 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b1c057040 con 0x560f0acbf8b0 2024-04-27T03:30:32.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.299+0000 7f2a17606640 1 -- 172.21.15.165:0/341883890 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c052720 con 0x560f0acbf8b0 2024-04-27T03:30:32.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.299+0000 7f2a17606640 1 -- 172.21.15.165:0/341883890 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b1c052a40 con 0x560f0acbf8b0 2024-04-27T03:30:32.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.300+0000 7f2b2d72b800 1 -- 172.21.15.165:0/341883890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 msgr2=0x560f0adb00d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.300+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/341883890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0adb00d0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c055040 tx=0x7f2b1c04c000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.300+0000 7f2b2d72b800 1 -- 172.21.15.165:0/341883890 shutdown_connections 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.300+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/341883890 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0adb00d0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.300+0000 7f2b2d72b800 1 -- 172.21.15.165:0/341883890 >> 172.21.15.165:0/341883890 conn(0x560f0acb9380 msgr2=0x560f0adab9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.300+0000 7f2b2d72b800 1 -- 172.21.15.165:0/341883890 shutdown_connections 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.300+0000 7f2b2d72b800 1 -- 172.21.15.165:0/341883890 wait complete. 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.300+0000 7f2b2d72b800 1 Processor -- start 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.300+0000 7f2b2d72b800 1 -- start start 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.301+0000 7f2b2d72b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0adb7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.301+0000 7f2b2d72b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560f0acc5f90 con 0x560f0acbf8b0 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.301+0000 7f2b29352640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0adb7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.301+0000 7f2b29352640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0adb7b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52196/0 (socket says 172.21.15.165:52196) 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.301+0000 7f2b29352640 1 -- 172.21.15.165:0/2713031148 learned_addr learned my addr 172.21.15.165:0/2713031148 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.301+0000 7f2b29352640 1 -- 172.21.15.165:0/2713031148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0adb8080 con 0x560f0acbf8b0 2024-04-27T03:30:32.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.301+0000 7f2b29352640 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0adb7b40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c0524f0 tx=0x7f2b1c003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.301+0000 7f2a15e03640 1 -- 172.21.15.165:0/2713031148 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b1c073070 con 0x560f0acbf8b0 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.301+0000 7f2a15e03640 1 -- 172.21.15.165:0/2713031148 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c05d440 con 0x560f0acbf8b0 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.301+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f0adb8310 con 0x560f0acbf8b0 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.301+0000 7f2a15e03640 1 -- 172.21.15.165:0/2713031148 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b1c057070 con 0x560f0acbf8b0 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.301+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f0adbafc0 con 0x560f0acbf8b0 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.302+0000 7f2a15e03640 1 -- 172.21.15.165:0/2713031148 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2b1c076070 con 0x560f0acbf8b0 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.302+0000 7f2a15e03640 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2a10036900 0x7f2a10038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.302+0000 7f2a15e03640 1 -- 172.21.15.165:0/2713031148 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2b1c08db90 con 0x560f0acbf8b0 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.302+0000 7f2b20b51640 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2a10036900 0x7f2a10038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.302+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560f0acc21e0 0x560f0adbed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.302+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adbf2c0 con 0x560f0acc21e0 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.303+0000 7f2b29b53640 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560f0acc21e0 0x560f0adbed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.303+0000 7f2b20b51640 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2a10036900 0x7f2a10038dc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b14000f60 tx=0x7f2b14001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.303+0000 7f2b29b53640 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560f0acc21e0 0x560f0adbed80 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.303+0000 7f2b29b53640 1 -- 172.21.15.165:0/2713031148 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0acc21e0 2024-04-27T03:30:32.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.303+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adbc350 con 0x560f0acc21e0 2024-04-27T03:30:32.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.303+0000 7f2b29b53640 1 -- 172.21.15.165:0/2713031148 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0acc21e0 2024-04-27T03:30:32.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.303+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560f0adc1610 0x560f0adc3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.303+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adc40f0 con 0x560f0adc1610 2024-04-27T03:30:32.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.304+0000 7f2b29352640 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560f0adc1610 0x560f0adc3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.304+0000 7f2b29352640 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560f0adc1610 0x560f0adc3a00 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.304+0000 7f2b29352640 1 -- 172.21.15.165:0/2713031148 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adc1610 2024-04-27T03:30:32.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.304+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adc05c0 con 0x560f0acc21e0 2024-04-27T03:30:32.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.304+0000 7f2b29b53640 1 -- 172.21.15.165:0/2713031148 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0acc21e0 2024-04-27T03:30:32.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.304+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560f0adc55a0 0x560f0adc79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.304+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adc8150 con 0x560f0adc55a0 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.305+0000 7f2b20b51640 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560f0adc55a0 0x560f0adc79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.305+0000 7f2b20b51640 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560f0adc55a0 0x560f0adc79f0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.305+0000 7f2b20b51640 1 -- 172.21.15.165:0/2713031148 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2b1400f690 con 0x560f0adc55a0 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.306+0000 7f2b2d72b800 1 Processor -- start 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.306+0000 7f2b2d72b800 1 -- start start 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.306+0000 7f2b2d72b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0add6540 0x560f0add6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.306+0000 7f2b2d72b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560f0abb06b0 con 0x560f0add6540 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.306+0000 7f2b29352640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0add6540 0x560f0add6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.306+0000 7f2b29352640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0add6540 0x560f0add6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52212/0 (socket says 172.21.15.165:52212) 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.306+0000 7f2b29352640 1 -- 172.21.15.165:0/3002273061 learned_addr learned my addr 172.21.15.165:0/3002273061 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.306+0000 7f2b29352640 1 -- 172.21.15.165:0/3002273061 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0add6ea0 con 0x560f0add6540 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.306+0000 7f2b29352640 1 --2- 172.21.15.165:0/3002273061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0add6540 0x560f0add6960 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c08c0d0 tx=0x7f2b1c076650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.307+0000 7f2a0effd640 1 -- 172.21.15.165:0/3002273061 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b1c076820 con 0x560f0add6540 2024-04-27T03:30:32.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.307+0000 7f2a0effd640 1 -- 172.21.15.165:0/3002273061 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c0769c0 con 0x560f0add6540 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.307+0000 7f2a0effd640 1 -- 172.21.15.165:0/3002273061 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b1c007f70 con 0x560f0add6540 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.307+0000 7f2b2d72b800 1 -- 172.21.15.165:0/3002273061 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f0add7910 con 0x560f0add6540 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.307+0000 7f2b2d72b800 1 -- 172.21.15.165:0/3002273061 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f0add7e10 con 0x560f0add6540 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.307+0000 7f2a0effd640 1 -- 172.21.15.165:0/3002273061 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2b1c077a50 con 0x560f0add6540 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.308+0000 7f2a0effd640 1 --2- 172.21.15.165:0/3002273061 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2a04036950 0x7f2a04038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.308+0000 7f2a0effd640 1 -- 172.21.15.165:0/3002273061 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2b1c089070 con 0x560f0add6540 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.308+0000 7f2b20b51640 1 --2- 172.21.15.165:0/3002273061 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2a04036950 0x7f2a04038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.308+0000 7f2b2d72b800 1 Processor -- start 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.308+0000 7f2b2d72b800 1 -- start start 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.308+0000 7f2b20b51640 1 --2- 172.21.15.165:0/3002273061 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2a04036950 0x7f2a04038e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x560f0adbfbc0 tx=0x7f2b1400f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.308+0000 7f2b2d72b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 0x560f0ade3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.308+0000 7f2b2d72b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560f0adb0960 con 0x560f0ade3690 2024-04-27T03:30:32.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.308+0000 7f2b29352640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 0x560f0ade3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.308+0000 7f2b29352640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 0x560f0ade3ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52222/0 (socket says 172.21.15.165:52222) 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.308+0000 7f2b29352640 1 -- 172.21.15.165:0/516027040 learned_addr learned my addr 172.21.15.165:0/516027040 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.309+0000 7f2b29352640 1 -- 172.21.15.165:0/516027040 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0ade4080 con 0x560f0ade3690 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.309+0000 7f2b29352640 1 --2- 172.21.15.165:0/516027040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 0x560f0ade3ab0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c08ae50 tx=0x7f2b1c07cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8887fb850d05d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.309+0000 7f2a0cff9640 1 -- 172.21.15.165:0/516027040 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b1c096400 con 0x560f0ade3690 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.309+0000 7f2a0cff9640 1 -- 172.21.15.165:0/516027040 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c06e400 con 0x560f0ade3690 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.309+0000 7f2a0cff9640 1 -- 172.21.15.165:0/516027040 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b1c08aba0 con 0x560f0ade3690 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.309+0000 7f2b2d72b800 1 -- 172.21.15.165:0/516027040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 msgr2=0x560f0ade3ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.309+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/516027040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 0x560f0ade3ab0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c08ae50 tx=0x7f2b1c07cf70 comp rx=0 tx=0).stop 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.309+0000 7f2b2d72b800 1 -- 172.21.15.165:0/516027040 shutdown_connections 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.309+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/516027040 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 0x560f0ade3ab0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.309+0000 7f2b2d72b800 1 -- 172.21.15.165:0/516027040 >> 172.21.15.165:0/516027040 conn(0x560f0addf9a0 msgr2=0x560f0addda00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.310+0000 7f2b2d72b800 1 -- 172.21.15.165:0/516027040 shutdown_connections 2024-04-27T03:30:32.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.310+0000 7f2b2d72b800 1 -- 172.21.15.165:0/516027040 wait complete. 2024-04-27T03:30:32.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.310+0000 7f2b2d72b800 1 Processor -- start 2024-04-27T03:30:32.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.310+0000 7f2b2d72b800 1 -- start start 2024-04-27T03:30:32.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.310+0000 7f2b2d72b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 0x560f0adeaa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.310+0000 7f2b2d72b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560f0ade4710 con 0x560f0ade3690 2024-04-27T03:30:32.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.310+0000 7f2b29352640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 0x560f0adeaa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.310+0000 7f2b29352640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 0x560f0adeaa50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52224/0 (socket says 172.21.15.165:52224) 2024-04-27T03:30:32.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.310+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 learned_addr learned my addr 172.21.15.165:0/616150353 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.311+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560f0adeaf90 con 0x560f0ade3690 2024-04-27T03:30:32.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.311+0000 7f2b29352640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 0x560f0adeaa50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c08ae50 tx=0x7f2b1c0a4070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.311+0000 7f29faffd640 1 -- 172.21.15.165:0/616150353 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b1c0b9020 con 0x560f0ade3690 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.311+0000 7f29faffd640 1 -- 172.21.15.165:0/616150353 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2b1c007b70 con 0x560f0ade3690 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.311+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560f0adeaf90 con 0x560f0ade3690 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.311+0000 7f29faffd640 1 -- 172.21.15.165:0/616150353 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2b1c078c80 con 0x560f0ade3690 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.311+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560f0adeaf90 con 0x560f0ade3690 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.312+0000 7f29faffd640 1 -- 172.21.15.165:0/616150353 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2b1c0bc020 con 0x560f0ade3690 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.312+0000 7f29faffd640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f29f4036900 0x7f29f4038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.312+0000 7f29faffd640 1 -- 172.21.15.165:0/616150353 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2b1c0b0050 con 0x560f0ade3690 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.312+0000 7f2b20b51640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f29f4036900 0x7f29f4038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.312+0000 7f2b2d72b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.312+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560f0adf60d0 0x560f0adf8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.312+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf8ce0 con 0x560f0adf60d0 2024-04-27T03:30:32.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.312+0000 7f2b20b51640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f29f4036900 0x7f29f4038dc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x560f0adc8550 tx=0x7f2b14010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.312+0000 7f2b29b53640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560f0adf60d0 0x560f0adf8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b29b53640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560f0adf60d0 0x560f0adf8590 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b2d72b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b2d72b800 20 realm 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b2d72b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf90f0 con 0x560f0adf60d0 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b2d72b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b2d72b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b2d72b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf99c0 con 0x560f0adf60d0 2024-04-27T03:30:32.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b2d72b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b2d72b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.313+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adf60d0 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.314+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.314+0000 7f2b2d72b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.314+0000 7f2b2d72b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.314+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560f0adfa850 0x560f0adfcc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.314+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adfd3a0 con 0x560f0adfa850 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.314+0000 7f2b29352640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560f0adfa850 0x560f0adfcc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.314+0000 7f2b29352640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560f0adfa850 0x560f0adfcc40 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.314+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.314+0000 7f2b2d72b800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.314+0000 7f2b2d72b800 20 searching for the correct realm 2024-04-27T03:30:32.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.314+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adfed80 con 0x560f0adfa850 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.315+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.315+0000 7f2b29352640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2b1c0be020 0x7f2b1c0a28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.315+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adfed80 con 0x7f2b1c0be020 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.315+0000 7f2b20b51640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2b1c0be020 0x7f2b1c0a28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.315+0000 7f2b20b51640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2b1c0be020 0x7f2b1c0a28d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.315+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b14037040 con 0x7f2b1c0be020 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.316+0000 7f2b20b51640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2b1400f070 0x7f2b1402e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.316+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adfed80 con 0x7f2b1400f070 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.316+0000 7f2b29b53640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2b1400f070 0x7f2b1402e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.316+0000 7f2b29b53640 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2b1400f070 0x7f2b1402e910 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.316+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.316+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adf60d0 2024-04-27T03:30:32.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.317+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.317+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adf60d0 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.317+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.317+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x7f2b1400f070 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.317+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.317+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x7f2b1400f070 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.317+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.317+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x7f2b1c0be020 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.318+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b14037040 con 0x7f2b1c0be020 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.318+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adfa850 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.318+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.318+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adf60d0 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.318+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.318+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x7f2b1400f070 2024-04-27T03:30:32.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.319+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.319+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adf60d0 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.319+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.319+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x7f2b1c0be020 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.319+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2b14037040 con 0x7f2b1c0be020 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.319+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adfa850 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.319+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.319+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adfa850 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.320+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.320+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x7f2b1400f070 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.320+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.320+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adfa850 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.320+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.320+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adfa850 2024-04-27T03:30:32.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.320+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.320+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adf60d0 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.321+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.321+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adf60d0 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.321+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.321+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x7f2b1400f070 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.321+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.321+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adfa850 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.321+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.321+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x7f2b1c0be020 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.322+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b14037040 con 0x7f2b1c0be020 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.322+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adf60d0 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.322+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.322+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x7f2b1400f070 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.322+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.322+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adfa850 2024-04-27T03:30:32.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.323+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.323+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x7f2b1400f070 2024-04-27T03:30:32.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.323+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.323+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x7f2b1c0be020 2024-04-27T03:30:32.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.323+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b14037040 con 0x7f2b1c0be020 2024-04-27T03:30:32.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.323+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adfa850 2024-04-27T03:30:32.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.323+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.323+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x7f2b1400f070 2024-04-27T03:30:32.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adf60d0 2024-04-27T03:30:32.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2b24038040 con 0x7f2b1c0be020 2024-04-27T03:30:32.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2b14037040 con 0x7f2b1c0be020 2024-04-27T03:30:32.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:32.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:32.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:32.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:32.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:32.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:32.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adf9d60 con 0x560f0adf60d0 2024-04-27T03:30:32.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adfdd50 con 0x560f0adf60d0 2024-04-27T03:30:32.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.324+0000 7f2b2d72b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adfdd50 con 0x7f2b1400f070 2024-04-27T03:30:32.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:32.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 20 zone default found 2024-04-27T03:30:32.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 4 Realm: () 2024-04-27T03:30:32.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:32.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:32.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:32.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 20 zonegroup default 2024-04-27T03:30:32.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adfe170 con 0x7f2b1400f070 2024-04-27T03:30:32.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.325+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0adfe590 con 0x560f0adfa850 2024-04-27T03:30:32.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.326+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.326+0000 7f2b2d72b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.326+0000 7f2b2d72b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.326+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae00c30 con 0x560f0adfa850 2024-04-27T03:30:32.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.326+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.326+0000 7f2b2d72b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.326+0000 7f2b2d72b800 20 started sync module instance, tier type = 2024-04-27T03:30:32.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.326+0000 7f2b2d72b800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:32.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.326+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae000d0 con 0x7f2b1c0be020 2024-04-27T03:30:32.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.327+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b14037040 con 0x7f2b1c0be020 2024-04-27T03:30:32.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.327+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94622606949408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae00fd0 con 0x7f2b1c0be020 2024-04-27T03:30:32.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.327+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae01820 con 0x560f0adf60d0 2024-04-27T03:30:32.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.328+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94622606949408] v25'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b14037040 con 0x7f2b1c0be020 2024-04-27T03:30:32.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.328+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.328+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94622606955376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae01c40 con 0x560f0adf60d0 2024-04-27T03:30:32.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.328+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae02600 con 0x560f0adfa850 2024-04-27T03:30:32.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.330+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94622606955376] v25'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.330+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.330+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94622606956432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae01bb0 con 0x560f0adfa850 2024-04-27T03:30:32.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.330+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae03360 con 0x7f2b1c0be020 2024-04-27T03:30:32.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.331+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b14037040 con 0x7f2b1c0be020 2024-04-27T03:30:32.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.331+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94622606964608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae04050 con 0x7f2b1c0be020 2024-04-27T03:30:32.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.331+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae04ac0 con 0x560f0adf60d0 2024-04-27T03:30:32.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.331+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94622606956432] v25'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.332+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94622606964608] v25'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b14037040 con 0x7f2b1c0be020 2024-04-27T03:30:32.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.332+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.332+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94622606965664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae029a0 con 0x560f0adf60d0 2024-04-27T03:30:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.332+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae053a0 con 0x7f2b1400f070 2024-04-27T03:30:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.333+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.333+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94622606965664] v25'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.333+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94622606970464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae057d0 con 0x7f2b1400f070 2024-04-27T03:30:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.333+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae04590 con 0x7f2b1400f070 2024-04-27T03:30:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.334+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94622606970464] v25'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.334+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.334+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94622606968208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae05dc0 con 0x7f2b1400f070 2024-04-27T03:30:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.334+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae06780 con 0x7f2b1400f070 2024-04-27T03:30:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.335+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94622606968208] v25'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.335+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'63 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.335+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94622606971520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae06780 con 0x7f2b1400f070 2024-04-27T03:30:32.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.335+0000 7f2b2d72b800 20 add_watcher() i=0 2024-04-27T03:30:32.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.335+0000 7f2b2d72b800 20 add_watcher() i=1 2024-04-27T03:30:32.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.335+0000 7f2b2d72b800 20 add_watcher() i=2 2024-04-27T03:30:32.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.335+0000 7f2b2d72b800 20 add_watcher() i=3 2024-04-27T03:30:32.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.335+0000 7f2b2d72b800 20 add_watcher() i=4 2024-04-27T03:30:32.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.335+0000 7f2b2d72b800 20 add_watcher() i=5 2024-04-27T03:30:32.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.335+0000 7f2b2d72b800 20 add_watcher() i=6 2024-04-27T03:30:32.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.336+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94622606971520] v25'64 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.336+0000 7f2b2d72b800 20 add_watcher() i=7 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.336+0000 7f2b2d72b800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.336+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae09b80 con 0x560f0adf60d0 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.336+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.336+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94622606976160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae070a0 con 0x560f0adf60d0 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.337+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94622606976160] v25'82 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.337+0000 7f29df7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.337+0000 7f2b2d72b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.337+0000 7f2b2d72b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.337+0000 7f2b2d72b800 5 note: GC not initialized 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.338+0000 7f29de7e4640 20 reqs_thread_entry: start 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.338+0000 7f2b2d72b800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:32.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.338+0000 7f2b2d72b800 20 Filter name: none 2024-04-27T03:30:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.338+0000 7f294ffff640 20 reqs_thread_entry: start 2024-04-27T03:30:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.338+0000 7f2b2d72b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.338+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae3c9f0 con 0x560f0adf60d0 2024-04-27T03:30:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.338+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.338+0000 7f2b2d72b800 20 rados_obj.operate() r=0 bl.length=556 2024-04-27T03:30:32.368 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:32.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.339+0000 7f29df7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.339+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94622606976160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae3b600 con 0x560f0adf60d0 2024-04-27T03:30:32.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.340+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94622606976160] v25'83 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.340+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94622606949408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae06470 con 0x7f2b1c0be020 2024-04-27T03:30:32.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.341+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94622606949408] v25'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b14037040 con 0x7f2b1c0be020 2024-04-27T03:30:32.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.341+0000 7f2b2d72b800 20 remove_watcher() i=0 2024-04-27T03:30:32.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.341+0000 7f2b2d72b800 2 removed watcher, disabling cache 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.341+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94622606955376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0adffc20 con 0x560f0adf60d0 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.342+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94622606955376] v25'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.342+0000 7f2b2d72b800 20 remove_watcher() i=1 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.342+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94622606956432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0adffc20 con 0x560f0adfa850 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.343+0000 7f2b29352640 1 -- 172.21.15.165:0/616150353 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94622606956432] v25'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b1c075030 con 0x560f0adfa850 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.343+0000 7f2b2d72b800 20 remove_watcher() i=2 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.343+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94622606964608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae06470 con 0x7f2b1c0be020 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.344+0000 7f2b20b51640 1 -- 172.21.15.165:0/616150353 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94622606964608] v25'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b14037040 con 0x7f2b1c0be020 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.344+0000 7f2b2d72b800 20 remove_watcher() i=3 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.344+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94622606965664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae03780 con 0x560f0adf60d0 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.345+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94622606965664] v25'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x560f0adf60d0 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.345+0000 7f2b2d72b800 20 remove_watcher() i=4 2024-04-27T03:30:32.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.345+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94622606970464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae06470 con 0x7f2b1400f070 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.346+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94622606970464] v25'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.346+0000 7f2b2d72b800 20 remove_watcher() i=5 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.346+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94622606968208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae04e60 con 0x7f2b1400f070 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.347+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94622606968208] v25'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.347+0000 7f2b2d72b800 20 remove_watcher() i=6 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.347+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94622606971520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560f0ae04e60 con 0x7f2b1400f070 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94622606971520] v25'65 uv63 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2b2400a4a0 con 0x7f2b1400f070 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 20 remove_watcher() i=7 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2b1400f070 msgr2=0x7f2b1402e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2b1400f070 0x7f2b1402e910 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560f0adf60d0 msgr2=0x560f0adf8590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560f0adf60d0 0x560f0adf8590 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2b1c0be020 msgr2=0x7f2b1c0a28d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2b1c0be020 0x7f2b1c0a28d0 crc :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560f0adfa850 msgr2=0x560f0adfcc40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560f0adfa850 0x560f0adfcc40 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f29f4036900 msgr2=0x7f29f4038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f29f4036900 0x7f29f4038dc0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x560f0adc8550 tx=0x7f2b14010000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 msgr2=0x560f0adeaa50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 0x560f0adeaa50 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c08ae50 tx=0x7f2b1c0a4070 comp rx=0 tx=0).stop 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b29b53640 1 -- 172.21.15.165:0/616150353 reap_dead start 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 shutdown_connections 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/616150353 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0ade3690 0x560f0adeaa50 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 >> 172.21.15.165:0/616150353 conn(0x560f0addf9a0 msgr2=0x560f0addd000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 shutdown_connections 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/616150353 wait complete. 2024-04-27T03:30:32.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/3002273061 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2a04036950 msgr2=0x7f2a04038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/3002273061 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2a04036950 0x7f2a04038e10 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x560f0adbfbc0 tx=0x7f2b1400f000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/3002273061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0add6540 msgr2=0x560f0add6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/3002273061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0add6540 0x560f0add6960 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c08c0d0 tx=0x7f2b1c076650 comp rx=0 tx=0).stop 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/3002273061 shutdown_connections 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/3002273061 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2a04036950 0x7f2a04038e10 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/3002273061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0add6540 0x560f0add6960 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.348+0000 7f2b2d72b800 1 -- 172.21.15.165:0/3002273061 >> 172.21.15.165:0/3002273061 conn(0x560f0adca5e0 msgr2=0x560f0adca9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 -- 172.21.15.165:0/3002273061 shutdown_connections 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 -- 172.21.15.165:0/3002273061 wait complete. 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560f0adc55a0 msgr2=0x560f0adc79f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560f0adc55a0 0x560f0adc79f0 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560f0acc21e0 msgr2=0x560f0adbed80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560f0acc21e0 0x560f0adbed80 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560f0adc1610 msgr2=0x560f0adc3a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560f0adc1610 0x560f0adc3a00 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2a10036900 msgr2=0x7f2a10038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2a10036900 0x7f2a10038dc0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f2b14000f60 tx=0x7f2b14001370 comp rx=0 tx=0).stop 2024-04-27T03:30:32.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 msgr2=0x560f0adb7b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 --2- 172.21.15.165:0/2713031148 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560f0acbf8b0 0x560f0adb7b40 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f2b1c0524f0 tx=0x7f2b1c003020 comp rx=0 tx=0).stop 2024-04-27T03:30:32.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b29b53640 1 -- 172.21.15.165:0/2713031148 reap_dead start 2024-04-27T03:30:32.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 shutdown_connections 2024-04-27T03:30:32.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 >> 172.21.15.165:0/2713031148 conn(0x560f0acb9380 msgr2=0x560f0adaa210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 shutdown_connections 2024-04-27T03:30:32.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.349+0000 7f2b2d72b800 1 -- 172.21.15.165:0/2713031148 wait complete. 2024-04-27T03:30:32.379 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:32.380 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-04-27T03:30:32.380 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'create', '--subuser', 'foo:foo2', '--secret', 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', '--key-type', 'swift'] 2024-04-27T03:30:32.380 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser create --subuser foo:foo2 --secret ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy --key-type swift 2024-04-27T03:30:32.461 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:32.461 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:32.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.456+0000 7f97f5430800 1 Processor -- start 2024-04-27T03:30:32.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.456+0000 7f97f5430800 1 -- start start 2024-04-27T03:30:32.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.456+0000 7f97f5430800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d445ac01d0 0x55d445ac25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.456+0000 7f97f5430800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d44597bd40 con 0x55d445abbd40 2024-04-27T03:30:32.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.456+0000 7f97f5430800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55d44594e4d0 con 0x55d445ac01d0 2024-04-27T03:30:32.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.457+0000 7f97e7fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d445ac01d0 0x55d445ac25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.457+0000 7f97e7fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d445ac01d0 0x55d445ac25c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52230/0 (socket says 172.21.15.165:52230) 2024-04-27T03:30:32.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.457+0000 7f97e7fff640 1 -- 172.21.15.165:0/1542565590 learned_addr learned my addr 172.21.15.165:0/1542565590 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.457+0000 7f97e7fff640 1 -- 172.21.15.165:0/1542565590 >> v1:172.21.15.89:6789/0 conn(0x55d445abbd40 legacy=0x55d445abc160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.457+0000 7f97e7fff640 1 -- 172.21.15.165:0/1542565590 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d44597a590 con 0x55d445ac01d0 2024-04-27T03:30:32.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.459+0000 7f97e7fff640 1 --2- 172.21.15.165:0/1542565590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445ac01d0 0x55d445ac25c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f97d800a520 tx=0x7f97d801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad4d6f88b89a748 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.459+0000 7f97efe8a640 1 -- 172.21.15.165:0/1542565590 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97d8057070 con 0x55d445ac01d0 2024-04-27T03:30:32.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.459+0000 7f97efe8a640 1 -- 172.21.15.165:0/1542565590 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97d8052570 con 0x55d445ac01d0 2024-04-27T03:30:32.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.459+0000 7f97efe8a640 1 -- 172.21.15.165:0/1542565590 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97d804ae60 con 0x55d445ac01d0 2024-04-27T03:30:32.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.459+0000 7f97f5430800 1 -- 172.21.15.165:0/1542565590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445ac01d0 msgr2=0x55d445ac25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.459+0000 7f97f5430800 1 --2- 172.21.15.165:0/1542565590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445ac01d0 0x55d445ac25c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f97d800a520 tx=0x7f97d801d010 comp rx=0 tx=0).stop 2024-04-27T03:30:32.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.459+0000 7f97f5430800 1 -- 172.21.15.165:0/1542565590 shutdown_connections 2024-04-27T03:30:32.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.459+0000 7f97f5430800 1 --2- 172.21.15.165:0/1542565590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445ac01d0 0x55d445ac25c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.459+0000 7f97f5430800 1 -- 172.21.15.165:0/1542565590 >> 172.21.15.165:0/1542565590 conn(0x55d445ab73a0 msgr2=0x55d445ab97c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.460+0000 7f97f5430800 1 -- 172.21.15.165:0/1542565590 shutdown_connections 2024-04-27T03:30:32.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.460+0000 7f97f5430800 1 -- 172.21.15.165:0/1542565590 wait complete. 2024-04-27T03:30:32.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.471+0000 7f97f5430800 1 Processor -- start 2024-04-27T03:30:32.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.471+0000 7f97f5430800 1 -- start start 2024-04-27T03:30:32.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.472+0000 7f97f5430800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 0x55d445a30010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.472+0000 7f97f5430800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d445a083e0 con 0x55d445abbd40 2024-04-27T03:30:32.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.472+0000 7f97f068b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 0x55d445a30010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.472+0000 7f97f068b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 0x55d445a30010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52242/0 (socket says 172.21.15.165:52242) 2024-04-27T03:30:32.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.472+0000 7f97f068b640 1 -- 172.21.15.165:0/750941260 learned_addr learned my addr 172.21.15.165:0/750941260 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.472+0000 7f97f068b640 1 -- 172.21.15.165:0/750941260 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d445bb0eb0 con 0x55d445abbd40 2024-04-27T03:30:32.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.472+0000 7f97f068b640 1 --2- 172.21.15.165:0/750941260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 0x55d445a30010 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f97e0001d50 tx=0x7f97e000b5b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=539dcde6ebfc1624 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.472+0000 7f96de60c640 1 -- 172.21.15.165:0/750941260 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e000f400 con 0x55d445abbd40 2024-04-27T03:30:32.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.472+0000 7f96de60c640 1 -- 172.21.15.165:0/750941260 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e000f5a0 con 0x55d445abbd40 2024-04-27T03:30:32.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.472+0000 7f96de60c640 1 -- 172.21.15.165:0/750941260 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e000f7d0 con 0x55d445abbd40 2024-04-27T03:30:32.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.473+0000 7f97f5430800 1 -- 172.21.15.165:0/750941260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 msgr2=0x55d445a30010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.473+0000 7f97f5430800 1 --2- 172.21.15.165:0/750941260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 0x55d445a30010 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7f97e0001d50 tx=0x7f97e000b5b0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.473+0000 7f97f5430800 1 -- 172.21.15.165:0/750941260 shutdown_connections 2024-04-27T03:30:32.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.473+0000 7f97f5430800 1 --2- 172.21.15.165:0/750941260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 0x55d445a30010 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.473+0000 7f97f5430800 1 -- 172.21.15.165:0/750941260 >> 172.21.15.165:0/750941260 conn(0x55d445ab73a0 msgr2=0x55d445ba89a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.473+0000 7f97f5430800 1 -- 172.21.15.165:0/750941260 shutdown_connections 2024-04-27T03:30:32.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.473+0000 7f97f5430800 1 -- 172.21.15.165:0/750941260 wait complete. 2024-04-27T03:30:32.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.473+0000 7f97f5430800 1 Processor -- start 2024-04-27T03:30:32.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.473+0000 7f97f5430800 1 -- start start 2024-04-27T03:30:32.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.474+0000 7f97f5430800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 0x55d445bb5d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.474+0000 7f97f5430800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d445ac2e90 con 0x55d445abbd40 2024-04-27T03:30:32.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.474+0000 7f97f068b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 0x55d445bb5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.474+0000 7f97f068b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 0x55d445bb5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52244/0 (socket says 172.21.15.165:52244) 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.474+0000 7f97f068b640 1 -- 172.21.15.165:0/1619435367 learned_addr learned my addr 172.21.15.165:0/1619435367 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.474+0000 7f97f068b640 1 -- 172.21.15.165:0/1619435367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d445bb62d0 con 0x55d445abbd40 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.474+0000 7f97f068b640 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 0x55d445bb5d90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f97e000b5f0 tx=0x7f97e0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.474+0000 7f96dce09640 1 -- 172.21.15.165:0/1619435367 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e001a7c0 con 0x55d445abbd40 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.474+0000 7f96dce09640 1 -- 172.21.15.165:0/1619435367 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e001a960 con 0x55d445abbd40 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.474+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d445bb6560 con 0x55d445abbd40 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.474+0000 7f96dce09640 1 -- 172.21.15.165:0/1619435367 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e001ac40 con 0x55d445abbd40 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.474+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d445bb9210 con 0x55d445abbd40 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.475+0000 7f96dce09640 1 -- 172.21.15.165:0/1619435367 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f97e0017070 con 0x55d445abbd40 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.475+0000 7f96dce09640 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96d8036900 0x7f96d8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.475+0000 7f96dce09640 1 -- 172.21.15.165:0/1619435367 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f97e0015020 con 0x55d445abbd40 2024-04-27T03:30:32.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.475+0000 7f97e7fff640 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96d8036900 0x7f96d8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.476+0000 7f97f5430800 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d445ac01d0 0x55d445bbcfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.476+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bbd550 con 0x55d445ac01d0 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.476+0000 7f97f0e8c640 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d445ac01d0 0x55d445bbcfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.476+0000 7f97e7fff640 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96d8036900 0x7f96d8038dc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f97d8054000 tx=0x7f97d804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.476+0000 7f97f0e8c640 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d445ac01d0 0x55d445bbcfd0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.476+0000 7f97f0e8c640 1 -- 172.21.15.165:0/1619435367 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445ac01d0 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.476+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bba5a0 con 0x55d445ac01d0 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.477+0000 7f97f0e8c640 1 -- 172.21.15.165:0/1619435367 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445ac01d0 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.477+0000 7f97f5430800 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d445bbf880 0x55d445bc1c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.477+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bc2360 con 0x55d445bbf880 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.477+0000 7f97f068b640 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d445bbf880 0x55d445bc1c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.477+0000 7f97f068b640 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d445bbf880 0x55d445bc1c70 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.477+0000 7f97f068b640 1 -- 172.21.15.165:0/1619435367 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f97e002e070 con 0x55d445bbf880 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.477+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bbe800 con 0x55d445ac01d0 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.478+0000 7f97f0e8c640 1 -- 172.21.15.165:0/1619435367 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445ac01d0 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.478+0000 7f97f5430800 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d445bc37c0 0x55d445bc5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.478+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bc6390 con 0x55d445bc37c0 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.478+0000 7f97e7fff640 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d445bc37c0 0x55d445bc5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.478+0000 7f97e7fff640 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d445bc37c0 0x55d445bc5c30 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.478+0000 7f97e7fff640 1 -- 172.21.15.165:0/1619435367 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f97d805a040 con 0x55d445bc37c0 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.479+0000 7f97f5430800 1 Processor -- start 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.479+0000 7f97f5430800 1 -- start start 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.479+0000 7f97f5430800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bd4780 0x55d445bd4ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.479+0000 7f97f5430800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d4459ae6b0 con 0x55d445bd4780 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.479+0000 7f97f068b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bd4780 0x55d445bd4ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.479+0000 7f97f068b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bd4780 0x55d445bd4ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52256/0 (socket says 172.21.15.165:52256) 2024-04-27T03:30:32.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.479+0000 7f97f068b640 1 -- 172.21.15.165:0/1599566596 learned_addr learned my addr 172.21.15.165:0/1599566596 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.479+0000 7f97f068b640 1 -- 172.21.15.165:0/1599566596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d445bd50e0 con 0x55d445bd4780 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.480+0000 7f97f068b640 1 --2- 172.21.15.165:0/1599566596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bd4780 0x55d445bd4ba0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f97e001cfd0 tx=0x7f97e00178f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.480+0000 7f96d5ffb640 1 -- 172.21.15.165:0/1599566596 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e000ee10 con 0x55d445bd4780 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.480+0000 7f96d5ffb640 1 -- 172.21.15.165:0/1599566596 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e001c660 con 0x55d445bd4780 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.480+0000 7f97f5430800 1 -- 172.21.15.165:0/1599566596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d445bd5b50 con 0x55d445bd4780 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.480+0000 7f96d5ffb640 1 -- 172.21.15.165:0/1599566596 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e001c940 con 0x55d445bd4780 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.480+0000 7f97f5430800 1 -- 172.21.15.165:0/1599566596 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d445bd5b50 con 0x55d445bd4780 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.480+0000 7f96d5ffb640 1 -- 172.21.15.165:0/1599566596 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f97e000e070 con 0x55d445bd4780 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.481+0000 7f96d5ffb640 1 --2- 172.21.15.165:0/1599566596 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96c8036900 0x7f96c8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.481+0000 7f96d5ffb640 1 -- 172.21.15.165:0/1599566596 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f97e003cae0 con 0x55d445bd4780 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.481+0000 7f97e7fff640 1 --2- 172.21.15.165:0/1599566596 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96c8036900 0x7f96c8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.481+0000 7f97f5430800 1 Processor -- start 2024-04-27T03:30:32.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.481+0000 7f97f5430800 1 -- start start 2024-04-27T03:30:32.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.481+0000 7f97e7fff640 1 --2- 172.21.15.165:0/1599566596 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96c8036900 0x7f96c8038dc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55d445bbde30 tx=0x7f97d8005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.481+0000 7f97f5430800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 0x55d445bae8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.481+0000 7f97f5430800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d445bb1200 con 0x55d445bae490 2024-04-27T03:30:32.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.481+0000 7f97f068b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 0x55d445bae8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.481+0000 7f97f068b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 0x55d445bae8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52264/0 (socket says 172.21.15.165:52264) 2024-04-27T03:30:32.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.481+0000 7f97f068b640 1 -- 172.21.15.165:0/1175248567 learned_addr learned my addr 172.21.15.165:0/1175248567 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.482+0000 7f97f068b640 1 -- 172.21.15.165:0/1175248567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d445bacc90 con 0x55d445bae490 2024-04-27T03:30:32.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.482+0000 7f97f068b640 1 --2- 172.21.15.165:0/1175248567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 0x55d445bae8b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f97e001ae40 tx=0x7f97e00274f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95a625e5c76e0b88 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.482+0000 7f96cf7fe640 1 -- 172.21.15.165:0/1175248567 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e0044930 con 0x55d445bae490 2024-04-27T03:30:32.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.482+0000 7f96cf7fe640 1 -- 172.21.15.165:0/1175248567 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e005c030 con 0x55d445bae490 2024-04-27T03:30:32.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.482+0000 7f96cf7fe640 1 -- 172.21.15.165:0/1175248567 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e0040430 con 0x55d445bae490 2024-04-27T03:30:32.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.482+0000 7f97f5430800 1 -- 172.21.15.165:0/1175248567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 msgr2=0x55d445bae8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.482+0000 7f97f5430800 1 --2- 172.21.15.165:0/1175248567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 0x55d445bae8b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f97e001ae40 tx=0x7f97e00274f0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.482+0000 7f97f5430800 1 -- 172.21.15.165:0/1175248567 shutdown_connections 2024-04-27T03:30:32.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.482+0000 7f97f5430800 1 --2- 172.21.15.165:0/1175248567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 0x55d445bae8b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.482+0000 7f97f5430800 1 -- 172.21.15.165:0/1175248567 >> 172.21.15.165:0/1175248567 conn(0x55d445bdd9d0 msgr2=0x55d445bdba30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.482+0000 7f97f5430800 1 -- 172.21.15.165:0/1175248567 shutdown_connections 2024-04-27T03:30:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.482+0000 7f97f5430800 1 -- 172.21.15.165:0/1175248567 wait complete. 2024-04-27T03:30:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.483+0000 7f97f5430800 1 Processor -- start 2024-04-27T03:30:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.483+0000 7f97f5430800 1 -- start start 2024-04-27T03:30:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.483+0000 7f97f5430800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 0x55d445beab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.483+0000 7f97f5430800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d445bad0a0 con 0x55d445bae490 2024-04-27T03:30:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.483+0000 7f97f068b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 0x55d445beab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.483+0000 7f97f068b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 0x55d445beab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52268/0 (socket says 172.21.15.165:52268) 2024-04-27T03:30:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.483+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 learned_addr learned my addr 172.21.15.165:0/3947179833 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.483+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d445beb0d0 con 0x55d445bae490 2024-04-27T03:30:32.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.484+0000 7f97f068b640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 0x55d445beab90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f97e0044720 tx=0x7f97e002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.484+0000 7f96cdffb640 1 -- 172.21.15.165:0/3947179833 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e0071070 con 0x55d445bae490 2024-04-27T03:30:32.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.484+0000 7f96cdffb640 1 -- 172.21.15.165:0/3947179833 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f97e004c6e0 con 0x55d445bae490 2024-04-27T03:30:32.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.484+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d445beb0d0 con 0x55d445bae490 2024-04-27T03:30:32.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.484+0000 7f96cdffb640 1 -- 172.21.15.165:0/3947179833 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f97e0040900 con 0x55d445bae490 2024-04-27T03:30:32.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.484+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d445beb0d0 con 0x55d445bae490 2024-04-27T03:30:32.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.484+0000 7f96cdffb640 1 -- 172.21.15.165:0/3947179833 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f97e0074070 con 0x55d445bae490 2024-04-27T03:30:32.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.485+0000 7f96cdffb640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96c0036950 0x7f96c0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.485+0000 7f96cdffb640 1 -- 172.21.15.165:0/3947179833 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f97e0069070 con 0x55d445bae490 2024-04-27T03:30:32.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.485+0000 7f97e7fff640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96c0036950 0x7f96c0038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.485+0000 7f97e7fff640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96c0036950 0x7f96c0038e10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f97d8000c00 tx=0x7f97d804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.485+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.485+0000 7f97f5430800 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d445bf6350 0x55d445bf8810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.485+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf8f60 con 0x55d445bf6350 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.485+0000 7f97f0e8c640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d445bf6350 0x55d445bf8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f0e8c640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d445bf6350 0x55d445bf8810 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445bf6350 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f5430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f5430800 20 realm 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9370 con 0x55d445bf6350 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445bf6350 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f5430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f5430800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:32.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9c40 con 0x55d445bf6350 2024-04-27T03:30:32.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445bf6350 2024-04-27T03:30:32.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f5430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.486+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9fe0 con 0x55d445bf6350 2024-04-27T03:30:32.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.487+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445bf6350 2024-04-27T03:30:32.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.487+0000 7f97f5430800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.487+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.487+0000 7f97f5430800 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d445bfad40 0x55d445bfd130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.487+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bfd740 con 0x55d445bfad40 2024-04-27T03:30:32.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.487+0000 7f97f068b640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d445bfad40 0x55d445bfd130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.487+0000 7f97f068b640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d445bfad40 0x55d445bfd130 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.487+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.487+0000 7f97f5430800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.487+0000 7f97f5430800 20 searching for the correct realm 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.487+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bff070 con 0x55d445bfad40 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.488+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.488+0000 7f97f068b640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f97e0076060 0x7f97e00805e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.488+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bff070 con 0x7f97e0076060 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.488+0000 7f97e7fff640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f97e0076060 0x7f97e00805e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.488+0000 7f97e7fff640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f97e0076060 0x7f97e00805e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.488+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.488+0000 7f97e7fff640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f97d8074040 0x7f97d8081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.488+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bff070 con 0x7f97d8074040 2024-04-27T03:30:32.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.489+0000 7f97f0e8c640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f97d8074040 0x7f97d8081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.489+0000 7f97f0e8c640 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f97d8074040 0x7f97d8081410 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.489+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f97e800a4a0 con 0x7f97d8074040 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.489+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9fe0 con 0x55d445bf6350 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.490+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445bf6350 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.490+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9fe0 con 0x55d445bf6350 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.490+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445bf6350 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.490+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9fe0 con 0x7f97d8074040 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.490+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e800a4a0 con 0x7f97d8074040 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.490+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9fe0 con 0x7f97d8074040 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.490+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e800a4a0 con 0x7f97d8074040 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.490+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9fe0 con 0x7f97e0076060 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.491+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.491+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9fe0 con 0x55d445bfad40 2024-04-27T03:30:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.491+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.491+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9fe0 con 0x55d445bf6350 2024-04-27T03:30:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.491+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445bf6350 2024-04-27T03:30:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.491+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9fe0 con 0x7f97d8074040 2024-04-27T03:30:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.491+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e800a4a0 con 0x7f97d8074040 2024-04-27T03:30:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.491+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9fe0 con 0x55d445bf6350 2024-04-27T03:30:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.492+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445bf6350 2024-04-27T03:30:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.492+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55d445bf9fe0 con 0x7f97e0076060 2024-04-27T03:30:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.492+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.492+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x55d445bfad40 2024-04-27T03:30:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.492+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.492+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x55d445bfad40 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.492+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.492+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x7f97d8074040 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.493+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e800a4a0 con 0x7f97d8074040 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.493+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x55d445bfad40 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.493+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.493+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x55d445bfad40 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.493+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.493+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x55d445bf6350 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.493+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445bf6350 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.493+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x55d445bf6350 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.494+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f97e800a4a0 con 0x55d445bf6350 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.494+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x7f97d8074040 2024-04-27T03:30:32.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.494+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e8033040 con 0x7f97d8074040 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.494+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x55d445bfad40 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.494+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.494+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x7f97e0076060 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.494+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.494+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x55d445bf6350 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.494+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.494+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x7f97d8074040 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.495+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.495+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x55d445bfad40 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.495+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.495+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x7f97d8074040 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.495+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.495+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x7f97e0076060 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.495+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.495+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x55d445bfad40 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x7f97d8074040 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x55d445bf6350 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f97d808e060 con 0x7f97e0076060 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f5430800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f5430800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f5430800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:32.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f5430800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f5430800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f5430800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.496+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bfe000 con 0x55d445bf6350 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bfe3f0 con 0x55d445bf6350 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bfe3f0 con 0x7f97d8074040 2024-04-27T03:30:32.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 20 zone default found 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 4 Realm: () 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 20 zonegroup default 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.497+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445bff4f0 con 0x7f97d8074040 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f5430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445c00ed0 con 0x55d445bfad40 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f5430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445c01270 con 0x55d445bfad40 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f5430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f5430800 20 started sync module instance, tier type = 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f5430800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.498+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c00370 con 0x7f97e0076060 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.499+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.499+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94370191638208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c01610 con 0x7f97e0076060 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.499+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c01ea0 con 0x55d445bf6350 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.500+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94370191638208] v25'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.500+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.500+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94370191640416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c022c0 con 0x55d445bf6350 2024-04-27T03:30:32.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.500+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c02d00 con 0x55d445bfad40 2024-04-27T03:30:32.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.501+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.501+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94370191648720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c035a0 con 0x55d445bfad40 2024-04-27T03:30:32.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.501+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c040a0 con 0x7f97e0076060 2024-04-27T03:30:32.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.502+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94370191640416] v25'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.503+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94370191648720] v25'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.503+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.503+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94370191653776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c04490 con 0x7f97e0076060 2024-04-27T03:30:32.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.503+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c04ce0 con 0x55d445bf6350 2024-04-27T03:30:32.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.504+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.504+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94370191657488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c055b0 con 0x55d445bf6350 2024-04-27T03:30:32.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.504+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c06030 con 0x7f97d8074040 2024-04-27T03:30:32.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.504+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94370191653776] v25'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.505+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.505+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94370191657488] v25'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.505+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94370191661856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c05ff0 con 0x7f97d8074040 2024-04-27T03:30:32.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.505+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c05100 con 0x7f97d8074040 2024-04-27T03:30:32.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.506+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94370191661856] v25'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.506+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.506+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94370191659264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c065c0 con 0x7f97d8074040 2024-04-27T03:30:32.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.506+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c06f80 con 0x7f97d8074040 2024-04-27T03:30:32.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.507+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94370191659264] v25'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.507+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'66 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.507+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94370191645104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c06f80 con 0x7f97d8074040 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.507+0000 7f97f5430800 20 add_watcher() i=0 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.507+0000 7f97f5430800 20 add_watcher() i=1 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.507+0000 7f97f5430800 20 add_watcher() i=2 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.507+0000 7f97f5430800 20 add_watcher() i=3 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.507+0000 7f97f5430800 20 add_watcher() i=4 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.507+0000 7f97f5430800 20 add_watcher() i=5 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.507+0000 7f97f5430800 20 add_watcher() i=6 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.508+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94370191645104] v25'67 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.508+0000 7f97f5430800 20 add_watcher() i=7 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.508+0000 7f97f5430800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.508+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445c0a2c0 con 0x55d445bf6350 2024-04-27T03:30:32.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.508+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.508+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94370191666240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c07510 con 0x55d445bf6350 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.509+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94370191666240] v25'84 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.509+0000 7f96a67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.509+0000 7f97f5430800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.509+0000 7f97f5430800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.509+0000 7f97f5430800 5 note: GC not initialized 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.509+0000 7f96a57e2640 20 reqs_thread_entry: start 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.509+0000 7f97f5430800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 20 Filter name: none 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f9616ffd640 20 reqs_thread_entry: start 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445c3d630 con 0x55d445bf6350 2024-04-27T03:30:32.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 20 rados_obj.operate() r=0 bl.length=556 2024-04-27T03:30:32.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:32.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:32.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:32.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : miss 2024-04-27T03:30:32.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.2 8:45671577:users.swift::foo%3afoo2:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445c3e350 con 0x7f97e0076060 2024-04-27T03:30:32.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(63 foo:foo2 [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 194+0+0 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x0 2024-04-27T03:30:32.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 10 adding default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-04-27T03:30:32.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 10 cache get: name=default.rgw.meta+users.swift+foo:foo2 : hit (negative entry) 2024-04-27T03:30:32.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.510+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~673 in=673b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c440e0 con 0x55d445bf6350 2024-04-27T03:30:32.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 foo [delete,create,call,call,writefull 0~673] v25'8 uv8 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f97f5430800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:32.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f97f5430800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:32.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f97f5430800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:32.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94370191915904 in=861b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445c40c40 con 0x55d445bf6350 2024-04-27T03:30:32.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify (1) cookie 94370191657488 notify 107374182407 ret 0) ==== 891+0+0 (crc 0 0 0) 0x7f97e801e040 con 0x55d445bf6350 2024-04-27T03:30:32.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f96cf7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182407 cookie 94370191657488 notifier 4378 bl.length()=849 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f96cf7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f96cf7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(65 notify.4 [notify cookie 94370191915904 out=8b] v0'0 uv66 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f96cf7fe640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f96b4003ea0 con 0x55d445bf6350 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== watch-notify(notify_complete (2) cookie 94370191915904 notify 107374182407 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f97e8018db0 con 0x55d445bf6350 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.512+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 8.2 8:45671577:users.swift::foo%3afoo2:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c43b80 con 0x7f97e0076060 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(67 foo:foo2 [delete,create,writefull 0~7] v25'1 uv1 ondisk = 0) ==== 236+0+0 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f97f5430800 10 cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f97f5430800 10 moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f97f5430800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94370191915904 in=178b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55d445c40c40 con 0x7f97e0076060 2024-04-27T03:30:32.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== watch-notify(notify (1) cookie 94370191638208 notify 107374182403 ret 0) ==== 208+0+0 (crc 0 0 0) 0x7f97d8088040 con 0x7f97e0076060 2024-04-27T03:30:32.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f96ceffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182403 cookie 94370191638208 notifier 4378 bl.length()=166 2024-04-27T03:30:32.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f96ceffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.swift+foo:foo2 info.flags=0x7 2024-04-27T03:30:32.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f96ceffd640 10 rgw watcher librados: moving default.rgw.meta+users.swift+foo:foo2 to cache LRU end 2024-04-27T03:30:32.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(68 notify.0 [notify cookie 94370191915904 out=8b] v0'0 uv66 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f96ceffd640 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f9624003400 con 0x7f97e0076060 2024-04-27T03:30:32.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(69 notify.0 [notify-ack] v0'0 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.514+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== watch-notify(notify_complete (2) cookie 94370191915904 notify 107374182403 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d44597a590 con 0x7f97e0076060 2024-04-27T03:30:32.537 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"},{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-04-27T03:30:32.511777Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:32.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.515+0000 7f96a67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.515+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94370191666240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c3e6f0 con 0x55d445bf6350 2024-04-27T03:30:32.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.516+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94370191666240] v25'85 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.516+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94370191638208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c3e6f0 con 0x7f97e0076060 2024-04-27T03:30:32.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.517+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94370191638208] v25'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.517+0000 7f97f5430800 20 remove_watcher() i=0 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.517+0000 7f97f5430800 2 removed watcher, disabling cache 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.517+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94370191640416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445bffec0 con 0x55d445bf6350 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.518+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94370191640416] v25'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.518+0000 7f97f5430800 20 remove_watcher() i=1 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.518+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94370191648720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445bffec0 con 0x55d445bfad40 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.519+0000 7f97f068b640 1 -- 172.21.15.165:0/3947179833 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94370191648720] v25'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e0073030 con 0x55d445bfad40 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.519+0000 7f97f5430800 20 remove_watcher() i=2 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.519+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94370191653776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445bffec0 con 0x7f97e0076060 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.520+0000 7f97e7fff640 1 -- 172.21.15.165:0/3947179833 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94370191653776] v25'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97d807f070 con 0x7f97e0076060 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.520+0000 7f97f5430800 20 remove_watcher() i=3 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.520+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94370191657488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c3e6f0 con 0x55d445bf6350 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.521+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94370191657488] v25'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x55d445bf6350 2024-04-27T03:30:32.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.521+0000 7f97f5430800 20 remove_watcher() i=4 2024-04-27T03:30:32.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.521+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94370191661856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c3e6f0 con 0x7f97d8074040 2024-04-27T03:30:32.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.522+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94370191661856] v25'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.522+0000 7f97f5430800 20 remove_watcher() i=5 2024-04-27T03:30:32.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.522+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94370191659264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c05b20 con 0x7f97d8074040 2024-04-27T03:30:32.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.523+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94370191659264] v25'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.523+0000 7f97f5430800 20 remove_watcher() i=6 2024-04-27T03:30:32.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.523+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94370191645104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55d445c05b20 con 0x7f97d8074040 2024-04-27T03:30:32.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94370191645104] v25'68 uv66 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f97e8031040 con 0x7f97d8074040 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 20 remove_watcher() i=7 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f97d8074040 msgr2=0x7f97d8081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f97d8074040 0x7f97d8081410 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d445bf6350 msgr2=0x55d445bf8810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d445bf6350 0x55d445bf8810 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f97e0076060 msgr2=0x7f97e00805e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f97e0076060 0x7f97e00805e0 crc :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d445bfad40 msgr2=0x55d445bfd130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d445bfad40 0x55d445bfd130 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96c0036950 msgr2=0x7f96c0038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96c0036950 0x7f96c0038e10 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f97d8000c00 tx=0x7f97d804f000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f0e8c640 1 -- 172.21.15.165:0/3947179833 reap_dead start 2024-04-27T03:30:32.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 msgr2=0x55d445beab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 0x55d445beab90 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f97e0044720 tx=0x7f97e002c5d0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 shutdown_connections 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 --2- 172.21.15.165:0/3947179833 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bae490 0x55d445beab90 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 >> 172.21.15.165:0/3947179833 conn(0x55d445bdd9d0 msgr2=0x55d445bdb030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 shutdown_connections 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.524+0000 7f97f5430800 1 -- 172.21.15.165:0/3947179833 wait complete. 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1599566596 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96c8036900 msgr2=0x7f96c8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 --2- 172.21.15.165:0/1599566596 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96c8036900 0x7f96c8038dc0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x55d445bbde30 tx=0x7f97d8005ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1599566596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bd4780 msgr2=0x55d445bd4ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 --2- 172.21.15.165:0/1599566596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bd4780 0x55d445bd4ba0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f97e001cfd0 tx=0x7f97e00178f0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1599566596 shutdown_connections 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 --2- 172.21.15.165:0/1599566596 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96c8036900 0x7f96c8038dc0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 --2- 172.21.15.165:0/1599566596 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445bd4780 0x55d445bd4ba0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1599566596 >> 172.21.15.165:0/1599566596 conn(0x55d445bc8820 msgr2=0x55d445bc8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1599566596 shutdown_connections 2024-04-27T03:30:32.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1599566596 wait complete. 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d445bc37c0 msgr2=0x55d445bc5c30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55d445bc37c0 0x55d445bc5c30 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d445ac01d0 msgr2=0x55d445bbcfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d445ac01d0 0x55d445bbcfd0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d445bbf880 msgr2=0x55d445bc1c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d445bbf880 0x55d445bc1c70 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96d8036900 msgr2=0x7f96d8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f96d8036900 0x7f96d8038dc0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f97d8054000 tx=0x7f97d804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 msgr2=0x55d445bb5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 --2- 172.21.15.165:0/1619435367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d445abbd40 0x55d445bb5d90 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f97e000b5f0 tx=0x7f97e0000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f0e8c640 1 -- 172.21.15.165:0/1619435367 reap_dead start 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 shutdown_connections 2024-04-27T03:30:32.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 >> 172.21.15.165:0/1619435367 conn(0x55d445ab73a0 msgr2=0x55d445ba7240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 shutdown_connections 2024-04-27T03:30:32.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.525+0000 7f97f5430800 1 -- 172.21.15.165:0/1619435367 wait complete. 2024-04-27T03:30:32.551 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-04-27T03:30:32.511777Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:32.551 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T03:30:32.551 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T03:30:32.551 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T03:30:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:32.596 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:32.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.591+0000 7f880752d800 1 Processor -- start 2024-04-27T03:30:32.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.591+0000 7f880752d800 1 -- start start 2024-04-27T03:30:32.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.591+0000 7f880752d800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562363ea5330 0x562363ea7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.591+0000 7f880752d800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562363d5fd40 con 0x562363ea0f90 2024-04-27T03:30:32.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.591+0000 7f880752d800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x562363d324d0 con 0x562363ea5330 2024-04-27T03:30:32.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.591+0000 7f8802977640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562363ea5330 0x562363ea7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.591+0000 7f8803178640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x562363ea0f90 0x562363ea13b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53720/0 (socket says 172.21.15.165:53720) 2024-04-27T03:30:32.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.592+0000 7f8803178640 1 -- 172.21.15.165:0/783438020 learned_addr learned my addr 172.21.15.165:0/783438020 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:32.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.592+0000 7f8802977640 1 -- 172.21.15.165:0/783438020 >> v1:172.21.15.89:6789/0 conn(0x562363ea0f90 legacy=0x562363ea13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.592+0000 7f8802977640 1 -- 172.21.15.165:0/783438020 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562363d5e590 con 0x562363ea5330 2024-04-27T03:30:32.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.594+0000 7f8802977640 1 --2- 172.21.15.165:0/783438020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea5330 0x562363ea7720 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f87f800a470 tx=0x7f87f801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd0dcd39b604cfe1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.594+0000 7f8802176640 1 -- 172.21.15.165:0/783438020 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f8057070 con 0x562363ea5330 2024-04-27T03:30:32.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.594+0000 7f8802176640 1 -- 172.21.15.165:0/783438020 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f8052570 con 0x562363ea5330 2024-04-27T03:30:32.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.594+0000 7f8802176640 1 -- 172.21.15.165:0/783438020 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f8052870 con 0x562363ea5330 2024-04-27T03:30:32.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.594+0000 7f880752d800 1 -- 172.21.15.165:0/783438020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea5330 msgr2=0x562363ea7720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.594+0000 7f880752d800 1 --2- 172.21.15.165:0/783438020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea5330 0x562363ea7720 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f87f800a470 tx=0x7f87f801ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:32.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.594+0000 7f880752d800 1 -- 172.21.15.165:0/783438020 shutdown_connections 2024-04-27T03:30:32.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.594+0000 7f880752d800 1 --2- 172.21.15.165:0/783438020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea5330 0x562363ea7720 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.594+0000 7f880752d800 1 -- 172.21.15.165:0/783438020 >> 172.21.15.165:0/783438020 conn(0x562363e9c530 msgr2=0x562363e9e950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.594+0000 7f880752d800 1 -- 172.21.15.165:0/783438020 shutdown_connections 2024-04-27T03:30:32.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.594+0000 7f880752d800 1 -- 172.21.15.165:0/783438020 wait complete. 2024-04-27T03:30:32.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.606+0000 7f880752d800 1 Processor -- start 2024-04-27T03:30:32.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.606+0000 7f880752d800 1 -- start start 2024-04-27T03:30:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.606+0000 7f880752d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 0x562363f920f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.606+0000 7f880752d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562363dec3c0 con 0x562363ea0f90 2024-04-27T03:30:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.606+0000 7f8803178640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 0x562363f920f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.606+0000 7f8803178640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 0x562363f920f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52288/0 (socket says 172.21.15.165:52288) 2024-04-27T03:30:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.606+0000 7f8803178640 1 -- 172.21.15.165:0/1501240745 learned_addr learned my addr 172.21.15.165:0/1501240745 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.607+0000 7f8803178640 1 -- 172.21.15.165:0/1501240745 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562363f926c0 con 0x562363ea0f90 2024-04-27T03:30:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.607+0000 7f8803178640 1 --2- 172.21.15.165:0/1501240745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 0x562363f920f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f87f4001e10 tx=0x7f87f400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f92ec694f74c239 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.607+0000 7f86f160a640 1 -- 172.21.15.165:0/1501240745 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f4012020 con 0x562363ea0f90 2024-04-27T03:30:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.607+0000 7f86f160a640 1 -- 172.21.15.165:0/1501240745 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f400e5a0 con 0x562363ea0f90 2024-04-27T03:30:32.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.607+0000 7f86f160a640 1 -- 172.21.15.165:0/1501240745 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f400e8e0 con 0x562363ea0f90 2024-04-27T03:30:32.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f880752d800 1 -- 172.21.15.165:0/1501240745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 msgr2=0x562363f920f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f880752d800 1 --2- 172.21.15.165:0/1501240745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 0x562363f920f0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f87f4001e10 tx=0x7f87f400b780 comp rx=0 tx=0).stop 2024-04-27T03:30:32.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f880752d800 1 -- 172.21.15.165:0/1501240745 shutdown_connections 2024-04-27T03:30:32.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f880752d800 1 --2- 172.21.15.165:0/1501240745 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 0x562363f920f0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f880752d800 1 -- 172.21.15.165:0/1501240745 >> 172.21.15.165:0/1501240745 conn(0x562363e9c530 msgr2=0x562363f8da10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f880752d800 1 -- 172.21.15.165:0/1501240745 shutdown_connections 2024-04-27T03:30:32.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f880752d800 1 -- 172.21.15.165:0/1501240745 wait complete. 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f880752d800 1 Processor -- start 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f880752d800 1 -- start start 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f880752d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 0x562363f99ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f880752d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562363ea7ff0 con 0x562363ea0f90 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f8803178640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 0x562363f99ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f8803178640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 0x562363f99ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52300/0 (socket says 172.21.15.165:52300) 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.608+0000 7f8803178640 1 -- 172.21.15.165:0/3878216064 learned_addr learned my addr 172.21.15.165:0/3878216064 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.609+0000 7f8803178640 1 -- 172.21.15.165:0/3878216064 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562363f9a1e0 con 0x562363ea0f90 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.609+0000 7f8803178640 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 0x562363f99ca0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f87f400b7c0 tx=0x7f87f4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.609+0000 7f86efe07640 1 -- 172.21.15.165:0/3878216064 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f4012030 con 0x562363ea0f90 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.609+0000 7f86efe07640 1 -- 172.21.15.165:0/3878216064 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f4001470 con 0x562363ea0f90 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.609+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562363f9a470 con 0x562363ea0f90 2024-04-27T03:30:32.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.609+0000 7f86efe07640 1 -- 172.21.15.165:0/3878216064 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f401a8d0 con 0x562363ea0f90 2024-04-27T03:30:32.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.609+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562363f9d120 con 0x562363ea0f90 2024-04-27T03:30:32.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.610+0000 7f86efe07640 1 -- 172.21.15.165:0/3878216064 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f87f4015020 con 0x562363ea0f90 2024-04-27T03:30:32.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.610+0000 7f86efe07640 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86e8036900 0x7f86e8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.610+0000 7f86efe07640 1 -- 172.21.15.165:0/3878216064 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f87f4020030 con 0x562363ea0f90 2024-04-27T03:30:32.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.610+0000 7f8802977640 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86e8036900 0x7f86e8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.610+0000 7f880752d800 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562363ea5330 0x562363fa0ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.610+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fa14b0 con 0x562363ea5330 2024-04-27T03:30:32.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.610+0000 7f8803979640 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562363ea5330 0x562363fa0ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.610+0000 7f8802977640 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86e8036900 0x7f86e8038dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f87f8054000 tx=0x7f87f804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.611+0000 7f8803979640 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562363ea5330 0x562363fa0ee0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.611+0000 7f8803979640 1 -- 172.21.15.165:0/3878216064 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363ea5330 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.611+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363f9e4b0 con 0x562363ea5330 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.611+0000 7f8803979640 1 -- 172.21.15.165:0/3878216064 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363ea5330 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.611+0000 7f880752d800 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562363fa37c0 0x562363fa5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.611+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fa6250 con 0x562363fa37c0 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.611+0000 7f8803178640 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562363fa37c0 0x562363fa5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.612+0000 7f8803178640 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562363fa37c0 0x562363fa5bb0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.612+0000 7f8803178640 1 -- 172.21.15.165:0/3878216064 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fa37c0 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.612+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fa2770 con 0x562363ea5330 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.612+0000 7f8803979640 1 -- 172.21.15.165:0/3878216064 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363ea5330 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.612+0000 7f880752d800 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562363fa7700 0x562363fa9b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.612+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363faa2b0 con 0x562363fa7700 2024-04-27T03:30:32.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.612+0000 7f8802977640 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562363fa7700 0x562363fa9b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.613+0000 7f8802977640 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562363fa7700 0x562363fa9b50 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.613+0000 7f8802977640 1 -- 172.21.15.165:0/3878216064 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f87f8052a40 con 0x562363fa7700 2024-04-27T03:30:32.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.613+0000 7f880752d800 1 Processor -- start 2024-04-27T03:30:32.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.613+0000 7f880752d800 1 -- start start 2024-04-27T03:30:32.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.614+0000 7f880752d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fb86d0 0x562363fb8af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.614+0000 7f880752d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562363d926b0 con 0x562363fb86d0 2024-04-27T03:30:32.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.614+0000 7f8803178640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fb86d0 0x562363fb8af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.614+0000 7f8803178640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fb86d0 0x562363fb8af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52316/0 (socket says 172.21.15.165:52316) 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.614+0000 7f8803178640 1 -- 172.21.15.165:0/577705444 learned_addr learned my addr 172.21.15.165:0/577705444 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.614+0000 7f8803178640 1 -- 172.21.15.165:0/577705444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562363fb9030 con 0x562363fb86d0 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.614+0000 7f8803178640 1 --2- 172.21.15.165:0/577705444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fb86d0 0x562363fb8af0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f87f404ad60 tx=0x7f87f400db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.614+0000 7f86ed602640 1 -- 172.21.15.165:0/577705444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f40185b0 con 0x562363fb86d0 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.614+0000 7f86ed602640 1 -- 172.21.15.165:0/577705444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f4017c50 con 0x562363fb86d0 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.614+0000 7f880752d800 1 -- 172.21.15.165:0/577705444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562363fb9aa0 con 0x562363fb86d0 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.614+0000 7f86ed602640 1 -- 172.21.15.165:0/577705444 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f401ac50 con 0x562363fb86d0 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.614+0000 7f880752d800 1 -- 172.21.15.165:0/577705444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562363fb9fa0 con 0x562363fb86d0 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.615+0000 7f86ed602640 1 -- 172.21.15.165:0/577705444 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f87f4044490 con 0x562363fb86d0 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.615+0000 7f86ed602640 1 --2- 172.21.15.165:0/577705444 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86dc036900 0x7f86dc038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.615+0000 7f86ed602640 1 -- 172.21.15.165:0/577705444 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f87f4036050 con 0x562363fb86d0 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.615+0000 7f8802977640 1 --2- 172.21.15.165:0/577705444 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86dc036900 0x7f86dc038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.615+0000 7f880752d800 1 Processor -- start 2024-04-27T03:30:32.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.616+0000 7f880752d800 1 -- start start 2024-04-27T03:30:32.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.616+0000 7f8802977640 1 --2- 172.21.15.165:0/577705444 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86dc036900 0x7f86dc038dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x562363f9e3d0 tx=0x7f87f8005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.616+0000 7f880752d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 0x562363fc5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.616+0000 7f880752d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562363f92a40 con 0x562363fc58c0 2024-04-27T03:30:32.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.616+0000 7f8803178640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 0x562363fc5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.616+0000 7f8803178640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 0x562363fc5ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52322/0 (socket says 172.21.15.165:52322) 2024-04-27T03:30:32.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.616+0000 7f8803178640 1 -- 172.21.15.165:0/793319374 learned_addr learned my addr 172.21.15.165:0/793319374 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.616+0000 7f8803178640 1 -- 172.21.15.165:0/793319374 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562363fc62b0 con 0x562363fc58c0 2024-04-27T03:30:32.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.617+0000 7f8803178640 1 --2- 172.21.15.165:0/793319374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 0x562363fc5ce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f87f4044420 tx=0x7f87f4023d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dce0af9ed2d49404 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.617+0000 7f86e6ffd640 1 -- 172.21.15.165:0/793319374 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f402fe50 con 0x562363fc58c0 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.617+0000 7f86e6ffd640 1 -- 172.21.15.165:0/793319374 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f4024420 con 0x562363fc58c0 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.617+0000 7f86e6ffd640 1 -- 172.21.15.165:0/793319374 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f4044780 con 0x562363fc58c0 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.617+0000 7f880752d800 1 -- 172.21.15.165:0/793319374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 msgr2=0x562363fc5ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.617+0000 7f880752d800 1 --2- 172.21.15.165:0/793319374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 0x562363fc5ce0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f87f4044420 tx=0x7f87f4023d80 comp rx=0 tx=0).stop 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.617+0000 7f880752d800 1 -- 172.21.15.165:0/793319374 shutdown_connections 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.617+0000 7f880752d800 1 --2- 172.21.15.165:0/793319374 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 0x562363fc5ce0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.617+0000 7f880752d800 1 -- 172.21.15.165:0/793319374 >> 172.21.15.165:0/793319374 conn(0x562363fc1b90 msgr2=0x562363fbfbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.617+0000 7f880752d800 1 -- 172.21.15.165:0/793319374 shutdown_connections 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.617+0000 7f880752d800 1 -- 172.21.15.165:0/793319374 wait complete. 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.617+0000 7f880752d800 1 Processor -- start 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.618+0000 7f880752d800 1 -- start start 2024-04-27T03:30:32.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.618+0000 7f880752d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 0x562363e16900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.618+0000 7f880752d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562363fc6980 con 0x562363fc58c0 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.618+0000 7f8803178640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 0x562363e16900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.618+0000 7f8803178640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 0x562363e16900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52332/0 (socket says 172.21.15.165:52332) 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.618+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 learned_addr learned my addr 172.21.15.165:0/3995934998 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.618+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562363e16e40 con 0x562363fc58c0 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.618+0000 7f8803178640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 0x562363e16900 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f87f401bfd0 tx=0x7f87f4022f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.619+0000 7f86e57fa640 1 -- 172.21.15.165:0/3995934998 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f406f020 con 0x562363fc58c0 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.619+0000 7f86e57fa640 1 -- 172.21.15.165:0/3995934998 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f87f404cb70 con 0x562363fc58c0 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.619+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562363e16e40 con 0x562363fc58c0 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.619+0000 7f86e57fa640 1 -- 172.21.15.165:0/3995934998 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f87f40490c0 con 0x562363fc58c0 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.619+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562363e16e40 con 0x562363fc58c0 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.619+0000 7f86e57fa640 1 -- 172.21.15.165:0/3995934998 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f87f4072020 con 0x562363fc58c0 2024-04-27T03:30:32.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.619+0000 7f86e57fa640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86cc036950 0x7f86cc038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f86e57fa640 1 -- 172.21.15.165:0/3995934998 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f87f4065040 con 0x562363fc58c0 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f8802977640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86cc036950 0x7f86cc038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f880752d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f880752d800 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562363fda5d0 0x562363fdca90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fdd1e0 con 0x562363fda5d0 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f8803979640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562363fda5d0 0x562363fdca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f8803979640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562363fda5d0 0x562363fdca90 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f8802977640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86cc036950 0x7f86cc038e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f87f8052fd0 tx=0x7f87f804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f880752d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f880752d800 20 realm 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f880752d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.620+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fdd5f0 con 0x562363fda5d0 2024-04-27T03:30:32.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f880752d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f880752d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f880752d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fdde90 con 0x562363fda5d0 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f880752d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f880752d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fde230 con 0x562363fda5d0 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f880752d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f880752d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f880752d800 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562363fdef90 0x562363fe1380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fe18c0 con 0x562363fdef90 2024-04-27T03:30:32.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f8803178640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562363fdef90 0x562363fe1380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.621+0000 7f8803178640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562363fdef90 0x562363fe1380 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.622+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.622+0000 7f880752d800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:32.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.622+0000 7f880752d800 20 searching for the correct realm 2024-04-27T03:30:32.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.622+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fe3260 con 0x562363fdef90 2024-04-27T03:30:32.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.622+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.622+0000 7f8803178640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f87f4074020 0x7f87f40608a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.622+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fe3260 con 0x7f87f4074020 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.622+0000 7f8802977640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f87f4074020 0x7f87f40608a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.623+0000 7f8802977640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f87f4074020 0x7f87f40608a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.623+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f8081070 con 0x7f87f4074020 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.623+0000 7f8802977640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f87f8076040 0x7f87f8083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.623+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fe3260 con 0x7f87f8076040 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.623+0000 7f8803979640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f87f8076040 0x7f87f8083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.624+0000 7f8803979640 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f87f8076040 0x7f87f8083410 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.624+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.624+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fde230 con 0x562363fda5d0 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.624+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.624+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fde230 con 0x562363fda5d0 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.624+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.624+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fde230 con 0x7f87f8076040 2024-04-27T03:30:32.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.625+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.625+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fde230 con 0x7f87f8076040 2024-04-27T03:30:32.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.625+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.625+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fde230 con 0x7f87f4074020 2024-04-27T03:30:32.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.625+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f8081070 con 0x7f87f4074020 2024-04-27T03:30:32.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.625+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fde230 con 0x562363fdef90 2024-04-27T03:30:32.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.625+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.625+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fde230 con 0x562363fda5d0 2024-04-27T03:30:32.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.626+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.626+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fde230 con 0x7f87f8076040 2024-04-27T03:30:32.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.626+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.626+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fde230 con 0x562363fda5d0 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.626+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.626+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562363fde230 con 0x7f87f4074020 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.627+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f87f8081070 con 0x7f87f4074020 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.627+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x562363fdef90 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.627+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.627+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x562363fdef90 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.627+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.627+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x7f87f8076040 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.627+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.628+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x562363fdef90 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.628+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.628+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x562363fdef90 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.628+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.628+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x562363fda5d0 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.628+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.628+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x562363fda5d0 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.628+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.628+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x7f87f8076040 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.629+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.629+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x562363fdef90 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.629+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.629+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x7f87f4074020 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.629+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f8081070 con 0x7f87f4074020 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.629+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x562363fda5d0 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.629+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.629+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x7f87f8076040 2024-04-27T03:30:32.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.630+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.630+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x562363fdef90 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.630+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.630+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x7f87f8076040 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.630+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.630+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x7f87f4074020 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.630+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f8081070 con 0x7f87f4074020 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.630+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x562363fdef90 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x7f87f8076040 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87f8090060 con 0x562363fda5d0 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f87fc038040 con 0x7f87f4074020 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f87f8081070 con 0x7f87f4074020 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f880752d800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f880752d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f880752d800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f880752d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f880752d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f880752d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f880752d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.631+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fe2230 con 0x562363fda5d0 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fe2620 con 0x562363fda5d0 2024-04-27T03:30:32.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fe2620 con 0x7f87f8076040 2024-04-27T03:30:32.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:32.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 20 zone default found 2024-04-27T03:30:32.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 4 Realm: () 2024-04-27T03:30:32.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:32.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:32.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:32.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 20 zonegroup default 2024-04-27T03:30:32.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fe2a40 con 0x7f87f8076040 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.632+0000 7f880752d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.633+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fe2a40 con 0x562363fdef90 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.633+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.633+0000 7f880752d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.633+0000 7f880752d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.633+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fe5110 con 0x562363fdef90 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.633+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.633+0000 7f880752d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.633+0000 7f880752d800 20 started sync module instance, tier type = 2024-04-27T03:30:32.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.633+0000 7f880752d800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:32.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.633+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe45b0 con 0x7f87f4074020 2024-04-27T03:30:32.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.634+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87f8081070 con 0x7f87f4074020 2024-04-27T03:30:32.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.634+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94710001453472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe4210 con 0x7f87f4074020 2024-04-27T03:30:32.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.634+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe5e70 con 0x562363fda5d0 2024-04-27T03:30:32.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.635+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94710001453472] v25'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87f8081070 con 0x7f87f4074020 2024-04-27T03:30:32.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.635+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.635+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94710001456304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe6260 con 0x562363fda5d0 2024-04-27T03:30:32.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.635+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe6cc0 con 0x562363fdef90 2024-04-27T03:30:32.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.636+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94710001456304] v25'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.636+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.636+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94710001461104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe70b0 con 0x562363fdef90 2024-04-27T03:30:32.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.636+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe7b10 con 0x7f87f4074020 2024-04-27T03:30:32.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.637+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94710001461104] v25'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.637+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87f8081070 con 0x7f87f4074020 2024-04-27T03:30:32.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.637+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94710001457360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe83b0 con 0x7f87f4074020 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.637+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe8d70 con 0x562363fda5d0 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.638+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94710001457360] v25'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87f8081070 con 0x7f87f4074020 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.638+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.639+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94710001469632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe9190 con 0x562363fda5d0 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.639+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe9b50 con 0x7f87f8076040 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.639+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.640+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94710001469632] v25'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.640+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94710001473184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe9f70 con 0x7f87f8076040 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.640+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe8d00 con 0x7f87f8076040 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.641+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94710001473184] v25'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.641+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.641+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94710001474240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe9ee0 con 0x7f87f8076040 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.641+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363feaec0 con 0x7f87f8076040 2024-04-27T03:30:32.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.642+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94710001474240] v25'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.642+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'69 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.642+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94710001476960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363feae30 con 0x7f87f8076040 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.642+0000 7f880752d800 20 add_watcher() i=0 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.642+0000 7f880752d800 20 add_watcher() i=1 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.642+0000 7f880752d800 20 add_watcher() i=2 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.642+0000 7f880752d800 20 add_watcher() i=3 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.642+0000 7f880752d800 20 add_watcher() i=4 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.642+0000 7f880752d800 20 add_watcher() i=5 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.642+0000 7f880752d800 20 add_watcher() i=6 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.643+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94710001476960] v25'70 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.643+0000 7f880752d800 20 add_watcher() i=7 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.643+0000 7f880752d800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.643+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562363fee340 con 0x562363fda5d0 2024-04-27T03:30:32.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.643+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.643+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94710001478480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363feb860 con 0x562363fda5d0 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.644+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94710001478480] v25'86 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.645+0000 7f86b9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.645+0000 7f880752d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.645+0000 7f880752d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.645+0000 7f880752d800 5 note: GC not initialized 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.645+0000 7f86b8fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.645+0000 7f880752d800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.645+0000 7f880752d800 20 Filter name: none 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.645+0000 7f86327fc640 20 reqs_thread_entry: start 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.645+0000 7f880752d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.645+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562364021170 con 0x562363fda5d0 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.646+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~673 out=673b,getxattrs out=75b] v0'0 uv8 ondisk = 0) ==== 231+0+796 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.646+0000 7f880752d800 20 rados_obj.operate() r=0 bl.length=673 2024-04-27T03:30:32.665 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[{"user":"foo:foo1","secret_key":"gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL","active":true,"create_date":"0.000000"},{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-04-27T03:30:32.511777Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:32.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.646+0000 7f86b9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.646+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94710001478480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5623640216b0 con 0x562363fda5d0 2024-04-27T03:30:32.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.647+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94710001478480] v25'87 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.647+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94710001453472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5623640216b0 con 0x7f87f4074020 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.648+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94710001453472] v25'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87f8081070 con 0x7f87f4074020 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.648+0000 7f880752d800 20 remove_watcher() i=0 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.648+0000 7f880752d800 2 removed watcher, disabling cache 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.648+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94710001456304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe49a0 con 0x562363fda5d0 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.649+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94710001456304] v25'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.649+0000 7f880752d800 20 remove_watcher() i=1 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.649+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94710001461104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe54b0 con 0x562363fdef90 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.650+0000 7f8803178640 1 -- 172.21.15.165:0/3995934998 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94710001461104] v25'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87f402dcf0 con 0x562363fdef90 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.651+0000 7f880752d800 20 remove_watcher() i=2 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.651+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94710001457360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe54b0 con 0x7f87f4074020 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.651+0000 7f8802977640 1 -- 172.21.15.165:0/3995934998 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94710001457360] v25'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87f8081070 con 0x7f87f4074020 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.651+0000 7f880752d800 20 remove_watcher() i=3 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.651+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94710001469632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe6770 con 0x562363fda5d0 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.653+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94710001469632] v25'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x562363fda5d0 2024-04-27T03:30:32.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.653+0000 7f880752d800 20 remove_watcher() i=4 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.653+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94710001473184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe6770 con 0x7f87f8076040 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.653+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94710001473184] v25'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.653+0000 7f880752d800 20 remove_watcher() i=5 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.653+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94710001474240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe96a0 con 0x7f87f8076040 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.654+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94710001474240] v25'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.654+0000 7f880752d800 20 remove_watcher() i=6 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.654+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94710001476960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562363fe6770 con 0x7f87f8076040 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.655+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94710001476960] v25'71 uv69 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f87fc00a4a0 con 0x7f87f8076040 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.655+0000 7f880752d800 20 remove_watcher() i=7 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.655+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f87f8076040 msgr2=0x7f87f8083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.655+0000 7f880752d800 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f87f8076040 0x7f87f8083410 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.655+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562363fda5d0 msgr2=0x562363fdca90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.655+0000 7f880752d800 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562363fda5d0 0x562363fdca90 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.655+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f87f4074020 msgr2=0x7f87f40608a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.655+0000 7f880752d800 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f87f4074020 0x7f87f40608a0 crc :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.655+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562363fdef90 msgr2=0x562363fe1380 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.655+0000 7f880752d800 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562363fdef90 0x562363fe1380 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.655+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86cc036950 msgr2=0x7f86cc038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.655+0000 7f880752d800 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86cc036950 0x7f86cc038e10 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f87f8052fd0 tx=0x7f87f804f000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 msgr2=0x562363e16900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 0x562363e16900 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f87f401bfd0 tx=0x7f87f4022f20 comp rx=0 tx=0).stop 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f8803979640 1 -- 172.21.15.165:0/3995934998 reap_dead start 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 shutdown_connections 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 --2- 172.21.15.165:0/3995934998 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fc58c0 0x562363e16900 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 >> 172.21.15.165:0/3995934998 conn(0x562363fc1b90 msgr2=0x562363fbf1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 shutdown_connections 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/3995934998 wait complete. 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/577705444 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86dc036900 msgr2=0x7f86dc038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 --2- 172.21.15.165:0/577705444 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86dc036900 0x7f86dc038dc0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x562363f9e3d0 tx=0x7f87f8005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:32.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/577705444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fb86d0 msgr2=0x562363fb8af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 --2- 172.21.15.165:0/577705444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fb86d0 0x562363fb8af0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f87f404ad60 tx=0x7f87f400db20 comp rx=0 tx=0).stop 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/577705444 shutdown_connections 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 --2- 172.21.15.165:0/577705444 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86dc036900 0x7f86dc038dc0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 --2- 172.21.15.165:0/577705444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363fb86d0 0x562363fb8af0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/577705444 >> 172.21.15.165:0/577705444 conn(0x562363fac770 msgr2=0x562363facb70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/577705444 shutdown_connections 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/577705444 wait complete. 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562363fa7700 msgr2=0x562363fa9b50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562363fa7700 0x562363fa9b50 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562363ea5330 msgr2=0x562363fa0ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562363ea5330 0x562363fa0ee0 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562363fa37c0 msgr2=0x562363fa5bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.656+0000 7f880752d800 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562363fa37c0 0x562363fa5bb0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.657+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86e8036900 msgr2=0x7f86e8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.657+0000 7f880752d800 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f86e8036900 0x7f86e8038dc0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f87f8054000 tx=0x7f87f804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.657+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 msgr2=0x562363f99ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.657+0000 7f880752d800 1 --2- 172.21.15.165:0/3878216064 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562363ea0f90 0x562363f99ca0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f87f400b7c0 tx=0x7f87f4000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.657+0000 7f8803979640 1 -- 172.21.15.165:0/3878216064 reap_dead start 2024-04-27T03:30:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.657+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 shutdown_connections 2024-04-27T03:30:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.657+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 >> 172.21.15.165:0/3878216064 conn(0x562363e9c530 msgr2=0x562363f8c210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.657+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 shutdown_connections 2024-04-27T03:30:32.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.657+0000 7f880752d800 1 -- 172.21.15.165:0/3878216064 wait complete. 2024-04-27T03:30:32.675 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True, 'create_date': '0.000000'}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-04-27T03:30:32.511777Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:32.675 INFO:tasks.util.rgw:rgwadmin: client.0 : ['key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-04-27T03:30:32.676 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'key', 'rm', '--subuser', 'foo:foo1', '--key-type', 'swift'] 2024-04-27T03:30:32.676 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph key rm --subuser foo:foo1 --key-type swift 2024-04-27T03:30:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:32.720 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:32.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.715+0000 7f6e06346800 1 Processor -- start 2024-04-27T03:30:32.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.715+0000 7f6e06346800 1 -- start start 2024-04-27T03:30:32.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.715+0000 7f6e06346800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x556f76d21990 0x556f76d23d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.715+0000 7f6e06346800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x556f76bdfd40 con 0x556f76d242c0 2024-04-27T03:30:32.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.715+0000 7f6e06346800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x556f76bb24d0 con 0x556f76d21990 2024-04-27T03:30:32.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.716+0000 7f6e03e4c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x556f76d21990 0x556f76d23d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.716+0000 7f6dfb64b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x556f76d242c0 0x556f76d266b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53736/0 (socket says 172.21.15.165:53736) 2024-04-27T03:30:32.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.716+0000 7f6dfb64b640 1 -- 172.21.15.165:0/2808071564 learned_addr learned my addr 172.21.15.165:0/2808071564 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:32.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.716+0000 7f6e03e4c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x556f76d21990 0x556f76d23d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52338/0 (socket says 172.21.15.165:52338) 2024-04-27T03:30:32.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.716+0000 7f6e03e4c640 1 -- 172.21.15.165:0/2808071564 >> v1:172.21.15.89:6789/0 conn(0x556f76d242c0 legacy=0x556f76d266b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.716+0000 7f6e03e4c640 1 -- 172.21.15.165:0/2808071564 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f76bde590 con 0x556f76d21990 2024-04-27T03:30:32.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.718+0000 7f6e03e4c640 1 --2- 172.21.15.165:0/2808071564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76d23d80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6df400a470 tx=0x7f6df401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf8c995f512305c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.718+0000 7f6e0364b640 1 -- 172.21.15.165:0/2808071564 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df4057070 con 0x556f76d21990 2024-04-27T03:30:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.718+0000 7f6e0364b640 1 -- 172.21.15.165:0/2808071564 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df4052570 con 0x556f76d21990 2024-04-27T03:30:32.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.718+0000 7f6e0364b640 1 -- 172.21.15.165:0/2808071564 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df4052870 con 0x556f76d21990 2024-04-27T03:30:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.718+0000 7f6e06346800 1 -- 172.21.15.165:0/2808071564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 msgr2=0x556f76d23d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.719+0000 7f6e06346800 1 --2- 172.21.15.165:0/2808071564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76d23d80 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f6df400a470 tx=0x7f6df401ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.719+0000 7f6e06346800 1 -- 172.21.15.165:0/2808071564 shutdown_connections 2024-04-27T03:30:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.719+0000 7f6e06346800 1 --2- 172.21.15.165:0/2808071564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76d23d80 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.719+0000 7f6e06346800 1 -- 172.21.15.165:0/2808071564 >> 172.21.15.165:0/2808071564 conn(0x556f76d1b340 msgr2=0x556f76d1d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.719+0000 7f6e06346800 1 -- 172.21.15.165:0/2808071564 shutdown_connections 2024-04-27T03:30:32.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.719+0000 7f6e06346800 1 -- 172.21.15.165:0/2808071564 wait complete. 2024-04-27T03:30:32.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.730+0000 7f6e06346800 1 Processor -- start 2024-04-27T03:30:32.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.730+0000 7f6e06346800 1 -- start start 2024-04-27T03:30:32.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.731+0000 7f6e06346800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76e12130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.731+0000 7f6e06346800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556f76c6c3b0 con 0x556f76d21990 2024-04-27T03:30:32.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.731+0000 7f6e03e4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76e12130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.731+0000 7f6e03e4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76e12130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52346/0 (socket says 172.21.15.165:52346) 2024-04-27T03:30:32.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.731+0000 7f6e03e4c640 1 -- 172.21.15.165:0/707164079 learned_addr learned my addr 172.21.15.165:0/707164079 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.731+0000 7f6e03e4c640 1 -- 172.21.15.165:0/707164079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f76e12700 con 0x556f76d21990 2024-04-27T03:30:32.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.731+0000 7f6e03e4c640 1 --2- 172.21.15.165:0/707164079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76e12130 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6df4055040 tx=0x7f6df404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf0d4c8f0842141 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.731+0000 7f6cf1e0b640 1 -- 172.21.15.165:0/707164079 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df4057070 con 0x556f76d21990 2024-04-27T03:30:32.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.732+0000 7f6cf1e0b640 1 -- 172.21.15.165:0/707164079 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df4063070 con 0x556f76d21990 2024-04-27T03:30:32.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.732+0000 7f6cf1e0b640 1 -- 172.21.15.165:0/707164079 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df4052720 con 0x556f76d21990 2024-04-27T03:30:32.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.732+0000 7f6e06346800 1 -- 172.21.15.165:0/707164079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 msgr2=0x556f76e12130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.732+0000 7f6e06346800 1 --2- 172.21.15.165:0/707164079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76e12130 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f6df4055040 tx=0x7f6df404c000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.732+0000 7f6e06346800 1 -- 172.21.15.165:0/707164079 shutdown_connections 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.732+0000 7f6e06346800 1 --2- 172.21.15.165:0/707164079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76e12130 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.732+0000 7f6e06346800 1 -- 172.21.15.165:0/707164079 >> 172.21.15.165:0/707164079 conn(0x556f76d1b340 msgr2=0x556f76e0da50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.732+0000 7f6e06346800 1 -- 172.21.15.165:0/707164079 shutdown_connections 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.732+0000 7f6e06346800 1 -- 172.21.15.165:0/707164079 wait complete. 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.733+0000 7f6e06346800 1 Processor -- start 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.733+0000 7f6e06346800 1 -- start start 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.733+0000 7f6e06346800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76e19b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.733+0000 7f6e06346800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556f76d2a0e0 con 0x556f76d21990 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.733+0000 7f6e03e4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76e19b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.733+0000 7f6e03e4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76e19b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52362/0 (socket says 172.21.15.165:52362) 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.733+0000 7f6e03e4c640 1 -- 172.21.15.165:0/1053841030 learned_addr learned my addr 172.21.15.165:0/1053841030 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.733+0000 7f6e03e4c640 1 -- 172.21.15.165:0/1053841030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f76e1a0c0 con 0x556f76d21990 2024-04-27T03:30:32.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.733+0000 7f6e03e4c640 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76e19b80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6df4060040 tx=0x7f6df4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.734+0000 7f6cebfff640 1 -- 172.21.15.165:0/1053841030 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df400a2d0 con 0x556f76d21990 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.734+0000 7f6cebfff640 1 -- 172.21.15.165:0/1053841030 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df406c070 con 0x556f76d21990 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.734+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556f76e1a350 con 0x556f76d21990 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.734+0000 7f6cebfff640 1 -- 172.21.15.165:0/1053841030 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df405e5d0 con 0x556f76d21990 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.734+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f76e1cf60 con 0x556f76d21990 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.734+0000 7f6cebfff640 1 -- 172.21.15.165:0/1053841030 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6df405e770 con 0x556f76d21990 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.734+0000 7f6cebfff640 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6cec036950 0x7f6cec038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.735+0000 7f6cebfff640 1 -- 172.21.15.165:0/1053841030 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f6df408e960 con 0x556f76d21990 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.735+0000 7f6dfb64b640 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6cec036950 0x7f6cec038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.735+0000 7f6e06346800 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556f76d242c0 0x556f76e20d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.735+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e212f0 con 0x556f76d242c0 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.735+0000 7f6e0464d640 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556f76d242c0 0x556f76e20d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.735+0000 7f6dfb64b640 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6cec036950 0x7f6cec038e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6dec000f60 tx=0x7f6dec0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.735+0000 7f6e0464d640 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556f76d242c0 0x556f76e20d20 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.735+0000 7f6e0464d640 1 -- 172.21.15.165:0/1053841030 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76d242c0 2024-04-27T03:30:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.735+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e1e2f0 con 0x556f76d242c0 2024-04-27T03:30:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.736+0000 7f6e0464d640 1 -- 172.21.15.165:0/1053841030 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76d242c0 2024-04-27T03:30:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.736+0000 7f6e06346800 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556f76e23670 0x556f76e25a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.736+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e26100 con 0x556f76e23670 2024-04-27T03:30:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.736+0000 7f6e03e4c640 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556f76e23670 0x556f76e25a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.736+0000 7f6e03e4c640 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556f76e23670 0x556f76e25a60 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.736+0000 7f6e03e4c640 1 -- 172.21.15.165:0/1053841030 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e23670 2024-04-27T03:30:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.736+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e22620 con 0x556f76d242c0 2024-04-27T03:30:32.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.737+0000 7f6e0464d640 1 -- 172.21.15.165:0/1053841030 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76d242c0 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.737+0000 7f6e06346800 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x556f76e275b0 0x556f76e29a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.737+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e2a160 con 0x556f76e275b0 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.737+0000 7f6dfb64b640 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x556f76e275b0 0x556f76e29a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.737+0000 7f6dfb64b640 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x556f76e275b0 0x556f76e29a00 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.738+0000 7f6dfb64b640 1 -- 172.21.15.165:0/1053841030 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f6dec009050 con 0x556f76e275b0 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.738+0000 7f6e06346800 1 Processor -- start 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.738+0000 7f6e06346800 1 -- start start 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.738+0000 7f6e06346800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21530 0x556f76d1f9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.738+0000 7f6e06346800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556f76c126b0 con 0x556f76d21530 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.738+0000 7f6e03e4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21530 0x556f76d1f9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.738+0000 7f6e03e4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21530 0x556f76d1f9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52364/0 (socket says 172.21.15.165:52364) 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.738+0000 7f6e03e4c640 1 -- 172.21.15.165:0/1463329389 learned_addr learned my addr 172.21.15.165:0/1463329389 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.738+0000 7f6e03e4c640 1 -- 172.21.15.165:0/1463329389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f76d1fee0 con 0x556f76d21530 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.739+0000 7f6e03e4c640 1 --2- 172.21.15.165:0/1463329389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21530 0x556f76d1f9a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6df4005200 tx=0x7f6df4071680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.739+0000 7f6ce97fa640 1 -- 172.21.15.165:0/1463329389 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df4068dc0 con 0x556f76d21530 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.739+0000 7f6ce97fa640 1 -- 172.21.15.165:0/1463329389 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df4093070 con 0x556f76d21530 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.739+0000 7f6e06346800 1 -- 172.21.15.165:0/1463329389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556f76d20170 con 0x556f76d21530 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.739+0000 7f6ce97fa640 1 -- 172.21.15.165:0/1463329389 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df4009790 con 0x556f76d21530 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.739+0000 7f6e06346800 1 -- 172.21.15.165:0/1463329389 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f76d20670 con 0x556f76d21530 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.739+0000 7f6ce97fa640 1 -- 172.21.15.165:0/1463329389 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6df4009930 con 0x556f76d21530 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.740+0000 7f6ce97fa640 1 --2- 172.21.15.165:0/1463329389 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6cdc0369a0 0x7f6cdc038e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.740+0000 7f6ce97fa640 1 -- 172.21.15.165:0/1463329389 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f6df408b050 con 0x556f76d21530 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.740+0000 7f6dfb64b640 1 --2- 172.21.15.165:0/1463329389 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6cdc0369a0 0x7f6cdc038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.740+0000 7f6e06346800 1 Processor -- start 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.740+0000 7f6e06346800 1 -- start start 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.740+0000 7f6e06346800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 0x556f76e47cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.740+0000 7f6dfb64b640 1 --2- 172.21.15.165:0/1463329389 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6cdc0369a0 0x7f6cdc038e60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00bf50 tx=0x7f6dec00f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.740+0000 7f6e06346800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556f76e129c0 con 0x556f76e478f0 2024-04-27T03:30:32.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.740+0000 7f6e03e4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 0x556f76e47cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.740+0000 7f6e03e4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 0x556f76e47cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52378/0 (socket says 172.21.15.165:52378) 2024-04-27T03:30:32.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.740+0000 7f6e03e4c640 1 -- 172.21.15.165:0/2498745105 learned_addr learned my addr 172.21.15.165:0/2498745105 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.741+0000 7f6e03e4c640 1 -- 172.21.15.165:0/2498745105 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f76e482c0 con 0x556f76e478f0 2024-04-27T03:30:32.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.741+0000 7f6e03e4c640 1 --2- 172.21.15.165:0/2498745105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 0x556f76e47cf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6df4052540 tx=0x7f6df4067e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=146a3ccec3d3d76e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.741+0000 7f6ce2ffd640 1 -- 172.21.15.165:0/2498745105 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df4066a60 con 0x556f76e478f0 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.741+0000 7f6ce2ffd640 1 -- 172.21.15.165:0/2498745105 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df4058040 con 0x556f76e478f0 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.741+0000 7f6ce2ffd640 1 -- 172.21.15.165:0/2498745105 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df40836e0 con 0x556f76e478f0 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.741+0000 7f6e06346800 1 -- 172.21.15.165:0/2498745105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 msgr2=0x556f76e47cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.741+0000 7f6e06346800 1 --2- 172.21.15.165:0/2498745105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 0x556f76e47cf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6df4052540 tx=0x7f6df4067e90 comp rx=0 tx=0).stop 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.741+0000 7f6e06346800 1 -- 172.21.15.165:0/2498745105 shutdown_connections 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.741+0000 7f6e06346800 1 --2- 172.21.15.165:0/2498745105 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 0x556f76e47cf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.741+0000 7f6e06346800 1 -- 172.21.15.165:0/2498745105 >> 172.21.15.165:0/2498745105 conn(0x556f76e43bc0 msgr2=0x556f76e41c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.741+0000 7f6e06346800 1 -- 172.21.15.165:0/2498745105 shutdown_connections 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.741+0000 7f6e06346800 1 -- 172.21.15.165:0/2498745105 wait complete. 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.742+0000 7f6e06346800 1 Processor -- start 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.742+0000 7f6e06346800 1 -- start start 2024-04-27T03:30:32.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.742+0000 7f6e06346800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 0x556f76e4ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.742+0000 7f6e06346800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556f76e489b0 con 0x556f76e478f0 2024-04-27T03:30:32.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.742+0000 7f6e03e4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 0x556f76e4ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.742+0000 7f6e03e4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 0x556f76e4ec30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52394/0 (socket says 172.21.15.165:52394) 2024-04-27T03:30:32.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.742+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 learned_addr learned my addr 172.21.15.165:0/3892759289 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.743+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f76e4f170 con 0x556f76e478f0 2024-04-27T03:30:32.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.743+0000 7f6e03e4c640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 0x556f76e4ec30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6df4062000 tx=0x7f6df407ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.743+0000 7f6ce17fa640 1 -- 172.21.15.165:0/3892759289 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df4083a40 con 0x556f76e478f0 2024-04-27T03:30:32.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.743+0000 7f6ce17fa640 1 -- 172.21.15.165:0/3892759289 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6df4083be0 con 0x556f76e478f0 2024-04-27T03:30:32.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.743+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556f76e4f170 con 0x556f76e478f0 2024-04-27T03:30:32.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.743+0000 7f6ce17fa640 1 -- 172.21.15.165:0/3892759289 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6df4004280 con 0x556f76e478f0 2024-04-27T03:30:32.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.743+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f76e4f170 con 0x556f76e478f0 2024-04-27T03:30:32.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.744+0000 7f6ce17fa640 1 -- 172.21.15.165:0/3892759289 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6df40be060 con 0x556f76e478f0 2024-04-27T03:30:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.744+0000 7f6ce17fa640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6ccc036950 0x7f6ccc038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.744+0000 7f6ce17fa640 1 -- 172.21.15.165:0/3892759289 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f6df40b8b20 con 0x556f76e478f0 2024-04-27T03:30:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.744+0000 7f6dfb64b640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6ccc036950 0x7f6ccc038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.744+0000 7f6dfb64b640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6ccc036950 0x7f6ccc038e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6dec001b60 tx=0x7f6dec010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.744+0000 7f6e06346800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.744+0000 7f6e06346800 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556f76e5a3f0 0x556f76e5c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.744+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5d000 con 0x556f76e5a3f0 2024-04-27T03:30:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.744+0000 7f6e0464d640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556f76e5a3f0 0x556f76e5c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e0464d640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556f76e5a3f0 0x556f76e5c8b0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76e5a3f0 2024-04-27T03:30:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e06346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e06346800 20 realm 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e06346800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5d410 con 0x556f76e5a3f0 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76e5a3f0 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e06346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e06346800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e06346800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5dce0 con 0x556f76e5a3f0 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76e5a3f0 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e06346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e06346800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.745+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5a3f0 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.746+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76e5a3f0 2024-04-27T03:30:32.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.746+0000 7f6e06346800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.746+0000 7f6e06346800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.746+0000 7f6e06346800 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556f76e5ede0 0x556f76e611d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.746+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e617e0 con 0x556f76e5ede0 2024-04-27T03:30:32.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.746+0000 7f6e03e4c640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556f76e5ede0 0x556f76e611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.746+0000 7f6e03e4c640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556f76e5ede0 0x556f76e611d0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.746+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.746+0000 7f6e06346800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:32.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.746+0000 7f6e06346800 20 searching for the correct realm 2024-04-27T03:30:32.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.746+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e63110 con 0x556f76e5ede0 2024-04-27T03:30:32.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.747+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.747+0000 7f6e03e4c640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6df40c0070 0x7f6df4065770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.747+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e63110 con 0x7f6df40c0070 2024-04-27T03:30:32.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.747+0000 7f6dfb64b640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6df40c0070 0x7f6df4065770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.747+0000 7f6dfb64b640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6df40c0070 0x7f6df4065770 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.747+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.747+0000 7f6dfb64b640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6dec037040 0x7f6dec036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.748+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e63110 con 0x7f6dec037040 2024-04-27T03:30:32.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.748+0000 7f6e0464d640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6dec037040 0x7f6dec036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.748+0000 7f6e0464d640 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6dec037040 0x7f6dec036410 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.748+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x7f6dec037040 2024-04-27T03:30:32.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.748+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5a3f0 2024-04-27T03:30:32.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.749+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76e5a3f0 2024-04-27T03:30:32.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.749+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5a3f0 2024-04-27T03:30:32.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.749+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76e5a3f0 2024-04-27T03:30:32.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.749+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6dec037040 2024-04-27T03:30:32.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.749+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x7f6dec037040 2024-04-27T03:30:32.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.749+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6dec037040 2024-04-27T03:30:32.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.749+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x7f6dec037040 2024-04-27T03:30:32.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.749+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6df40c0070 2024-04-27T03:30:32.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.750+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.750+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5ede0 2024-04-27T03:30:32.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.750+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.750+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5a3f0 2024-04-27T03:30:32.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.750+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76e5a3f0 2024-04-27T03:30:32.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.750+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6dec037040 2024-04-27T03:30:32.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.750+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x7f6dec037040 2024-04-27T03:30:32.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.750+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5a3f0 2024-04-27T03:30:32.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.751+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76e5a3f0 2024-04-27T03:30:32.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.751+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6df40c0070 2024-04-27T03:30:32.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.751+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.751+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5ede0 2024-04-27T03:30:32.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.751+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.751+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5ede0 2024-04-27T03:30:32.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.751+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.751+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6dec037040 2024-04-27T03:30:32.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.752+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x7f6dec037040 2024-04-27T03:30:32.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.752+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5ede0 2024-04-27T03:30:32.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.752+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.752+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5ede0 2024-04-27T03:30:32.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.752+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.752+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5a3f0 2024-04-27T03:30:32.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.752+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76e5a3f0 2024-04-27T03:30:32.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.752+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5a3f0 2024-04-27T03:30:32.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.753+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f6dfc00a4a0 con 0x556f76e5a3f0 2024-04-27T03:30:32.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.753+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6dec037040 2024-04-27T03:30:32.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.753+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc033040 con 0x7f6dec037040 2024-04-27T03:30:32.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.753+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5ede0 2024-04-27T03:30:32.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.753+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.753+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6df40c0070 2024-04-27T03:30:32.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.753+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.753+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5a3f0 2024-04-27T03:30:32.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.754+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.754+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6dec037040 2024-04-27T03:30:32.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.754+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.754+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5ede0 2024-04-27T03:30:32.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.754+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.754+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6dec037040 2024-04-27T03:30:32.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.755+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.755+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6df40c0070 2024-04-27T03:30:32.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.755+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.755+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5ede0 2024-04-27T03:30:32.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.755+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.755+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6dec037040 2024-04-27T03:30:32.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.755+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.755+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x556f76e5a3f0 2024-04-27T03:30:32.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556f76e5e080 con 0x7f6df40c0070 2024-04-27T03:30:32.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:32.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:32.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:32.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:32.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:32.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:32.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e620a0 con 0x556f76e5a3f0 2024-04-27T03:30:32.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e62490 con 0x556f76e5a3f0 2024-04-27T03:30:32.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.756+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e62490 con 0x7f6dec037040 2024-04-27T03:30:32.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:32.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 20 zone default found 2024-04-27T03:30:32.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 4 Realm: () 2024-04-27T03:30:32.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:32.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:32.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:32.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 20 zonegroup default 2024-04-27T03:30:32.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e63590 con 0x7f6dec037040 2024-04-27T03:30:32.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e64f70 con 0x556f76e5ede0 2024-04-27T03:30:32.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e65310 con 0x556f76e5ede0 2024-04-27T03:30:32.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.757+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.758+0000 7f6e06346800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.758+0000 7f6e06346800 20 started sync module instance, tier type = 2024-04-27T03:30:32.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.758+0000 7f6e06346800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:32.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.758+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e64410 con 0x7f6df40c0070 2024-04-27T03:30:32.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.759+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.759+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93937224531808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e656b0 con 0x7f6df40c0070 2024-04-27T03:30:32.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.759+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e66070 con 0x556f76e5a3f0 2024-04-27T03:30:32.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.760+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93937224531808] v25'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.760+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.760+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93937224539072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e66490 con 0x556f76e5a3f0 2024-04-27T03:30:32.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.760+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e66e50 con 0x556f76e5ede0 2024-04-27T03:30:32.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.761+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93937224539072] v25'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.761+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.761+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93937224540128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e66400 con 0x556f76e5ede0 2024-04-27T03:30:32.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.761+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e67cc0 con 0x7f6df40c0070 2024-04-27T03:30:32.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.762+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93937224540128] v25'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.762+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.762+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93937224546320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e68060 con 0x7f6df40c0070 2024-04-27T03:30:32.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.762+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e68a80 con 0x556f76e5a3f0 2024-04-27T03:30:32.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.763+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.763+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93937224549744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e69350 con 0x556f76e5a3f0 2024-04-27T03:30:32.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.763+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e69e20 con 0x7f6dec037040 2024-04-27T03:30:32.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.763+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93937224546320] v25'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.764+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.764+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93937224554768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e69de0 con 0x7f6dec037040 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.764+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6a830 con 0x7f6dec037040 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.764+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93937224549744] v25'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.765+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.765+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93937224552096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6a830 con 0x7f6dec037040 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.765+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6b230 con 0x7f6dec037040 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.765+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 93937224554768] v25'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.766+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93937224552096] v25'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.766+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'72 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.766+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93937224544752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6a380 con 0x7f6dec037040 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.766+0000 7f6e06346800 20 add_watcher() i=0 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.766+0000 7f6e06346800 20 add_watcher() i=1 2024-04-27T03:30:32.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.766+0000 7f6e06346800 20 add_watcher() i=2 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.766+0000 7f6e06346800 20 add_watcher() i=3 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.766+0000 7f6e06346800 20 add_watcher() i=4 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.766+0000 7f6e06346800 20 add_watcher() i=5 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.766+0000 7f6e06346800 20 add_watcher() i=6 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.767+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93937224544752] v25'73 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.767+0000 7f6e06346800 20 add_watcher() i=7 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.767+0000 7f6e06346800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.767+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6e440 con 0x556f76e5a3f0 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.767+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.767+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93937224559936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6b8c0 con 0x556f76e5a3f0 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.768+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93937224559936] v25'88 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.768+0000 7f6cb9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.768+0000 7f6e06346800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.768+0000 7f6e06346800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:32.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.768+0000 7f6e06346800 5 note: GC not initialized 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6cb8fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6e06346800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6e06346800 20 Filter name: none 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6c327fc640 20 reqs_thread_entry: start 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6e06346800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6e06346800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76ea13c0 con 0x556f76e5a3f0 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~673 out=673b,getxattrs out=75b] v0'0 uv8 ondisk = 0) ==== 231+0+796 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6e06346800 20 rados_obj.operate() r=0 bl.length=673 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6e06346800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6e06346800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6e06346800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.769+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~582 in=582b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76ea7fb0 con 0x556f76e5a3f0 2024-04-27T03:30:32.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~582] v25'9 uv9 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6e06346800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6e06346800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6e06346800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 93937224808992 in=770b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76ea3700 con 0x556f76e5a3f0 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify (1) cookie 93937224549744 notify 107374182408 ret 0) ==== 800+0+0 (crc 0 0 0) 0x7f6dfc01e040 con 0x556f76e5a3f0 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6ce2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182408 cookie 93937224549744 notifier 4392 bl.length()=758 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6ce2ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6ce2ffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(64 notify.4 [notify cookie 93937224808992 out=8b] v0'0 uv72 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6ce2ffd640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f6cc8003ea0 con 0x556f76e5a3f0 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== watch-notify(notify_complete (2) cookie 93937224808992 notify 107374182408 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f6dfc018db0 con 0x556f76e5a3f0 2024-04-27T03:30:32.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.771+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 8.8 8:17d68d04:users.swift::foo%3afoo1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76ea7a20 con 0x556f76e5a3f0 2024-04-27T03:30:32.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.772+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(66 foo:foo1 [delete] v25'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.772+0000 7f6e06346800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo1, ofs0, ns] 2024-04-27T03:30:32.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.772+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 93937224808992 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556f76ea3250 con 0x7f6df40c0070 2024-04-27T03:30:32.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.772+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== watch-notify(notify (1) cookie 93937224531808 notify 107374182404 ret 0) ==== 201+0+0 (crc 0 0 0) 0x7f6dec041060 con 0x7f6df40c0070 2024-04-27T03:30:32.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.772+0000 7f6ce27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182404 cookie 93937224531808 notifier 4392 bl.length()=159 2024-04-27T03:30:32.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.772+0000 7f6ce27fc640 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f6c380032c0 con 0x7f6df40c0070 2024-04-27T03:30:32.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.772+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(67 notify.0 [notify cookie 93937224808992 out=8b] v0'0 uv72 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.773+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.773+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== watch-notify(notify_complete (2) cookie 93937224808992 notify 107374182404 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556f76bdfd40 con 0x7f6df40c0070 2024-04-27T03:30:32.795 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo1","permissions":"full-control"},{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-04-27T03:30:32.511777Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:32.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.774+0000 7f6cb9fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.774+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93937224559936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e9faa0 con 0x556f76e5a3f0 2024-04-27T03:30:32.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.775+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 93937224559936] v25'89 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.775+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93937224531808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6ad40 con 0x7f6df40c0070 2024-04-27T03:30:32.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.776+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 93937224531808] v25'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.776+0000 7f6e06346800 20 remove_watcher() i=0 2024-04-27T03:30:32.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.776+0000 7f6e06346800 2 removed watcher, disabling cache 2024-04-27T03:30:32.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.776+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93937224539072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6ad40 con 0x556f76e5a3f0 2024-04-27T03:30:32.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.777+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 93937224539072] v25'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.777+0000 7f6e06346800 20 remove_watcher() i=1 2024-04-27T03:30:32.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.777+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93937224540128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6ad40 con 0x556f76e5ede0 2024-04-27T03:30:32.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.778+0000 7f6e03e4c640 1 -- 172.21.15.165:0/3892759289 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 93937224540128] v25'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6df405ec40 con 0x556f76e5ede0 2024-04-27T03:30:32.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.778+0000 7f6e06346800 20 remove_watcher() i=2 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.778+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93937224546320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6ad40 con 0x7f6df40c0070 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.779+0000 7f6dfb64b640 1 -- 172.21.15.165:0/3892759289 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 93937224546320] v25'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dec009050 con 0x7f6df40c0070 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.779+0000 7f6e06346800 20 remove_watcher() i=3 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.779+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93937224549744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6ad40 con 0x556f76e5a3f0 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.780+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 93937224549744] v25'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x556f76e5a3f0 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.780+0000 7f6e06346800 20 remove_watcher() i=4 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.780+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93937224554768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6ad40 con 0x7f6dec037040 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.781+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 93937224554768] v25'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.781+0000 7f6e06346800 20 remove_watcher() i=5 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.781+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93937224552096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e69910 con 0x7f6dec037040 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.782+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 93937224552096] v25'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.782+0000 7f6e06346800 20 remove_watcher() i=6 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.782+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93937224544752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556f76e6ad40 con 0x7f6dec037040 2024-04-27T03:30:32.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 93937224544752] v25'74 uv72 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6dfc031040 con 0x7f6dec037040 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 20 remove_watcher() i=7 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6dec037040 msgr2=0x7f6dec036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6dec037040 0x7f6dec036410 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556f76e5a3f0 msgr2=0x556f76e5c8b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556f76e5a3f0 0x556f76e5c8b0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6df40c0070 msgr2=0x7f6df4065770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6df40c0070 0x7f6df4065770 crc :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556f76e5ede0 msgr2=0x556f76e611d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556f76e5ede0 0x556f76e611d0 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6ccc036950 msgr2=0x7f6ccc038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6ccc036950 0x7f6ccc038e10 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f6dec001b60 tx=0x7f6dec010000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 msgr2=0x556f76e4ec30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e0464d640 1 -- 172.21.15.165:0/3892759289 reap_dead start 2024-04-27T03:30:32.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 0x556f76e4ec30 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f6df4062000 tx=0x7f6df407ab70 comp rx=0 tx=0).stop 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 shutdown_connections 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 --2- 172.21.15.165:0/3892759289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76e478f0 0x556f76e4ec30 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 >> 172.21.15.165:0/3892759289 conn(0x556f76e43bc0 msgr2=0x556f76e41220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 shutdown_connections 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/3892759289 wait complete. 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/1463329389 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6cdc0369a0 msgr2=0x7f6cdc038e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 --2- 172.21.15.165:0/1463329389 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6cdc0369a0 0x7f6cdc038e60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f6dec00bf50 tx=0x7f6dec00f000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/1463329389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21530 msgr2=0x556f76d1f9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 --2- 172.21.15.165:0/1463329389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21530 0x556f76d1f9a0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f6df4005200 tx=0x7f6df4071680 comp rx=0 tx=0).stop 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/1463329389 shutdown_connections 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 --2- 172.21.15.165:0/1463329389 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6cdc0369a0 0x7f6cdc038e60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 --2- 172.21.15.165:0/1463329389 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21530 0x556f76d1f9a0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/1463329389 >> 172.21.15.165:0/1463329389 conn(0x556f76e2c660 msgr2=0x556f76e2ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.783+0000 7f6e06346800 1 -- 172.21.15.165:0/1463329389 shutdown_connections 2024-04-27T03:30:32.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 -- 172.21.15.165:0/1463329389 wait complete. 2024-04-27T03:30:32.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x556f76e275b0 msgr2=0x556f76e29a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x556f76e275b0 0x556f76e29a00 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556f76d242c0 msgr2=0x556f76e20d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556f76d242c0 0x556f76e20d20 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556f76e23670 msgr2=0x556f76e25a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556f76e23670 0x556f76e25a60 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6cec036950 msgr2=0x7f6cec038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6cec036950 0x7f6cec038e10 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6dec000f60 tx=0x7f6dec0012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 msgr2=0x556f76e19b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 --2- 172.21.15.165:0/1053841030 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556f76d21990 0x556f76e19b80 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f6df4060040 tx=0x7f6df4002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e0464d640 1 -- 172.21.15.165:0/1053841030 reap_dead start 2024-04-27T03:30:32.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 shutdown_connections 2024-04-27T03:30:32.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 >> 172.21.15.165:0/1053841030 conn(0x556f76d1b340 msgr2=0x556f76e0c250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 shutdown_connections 2024-04-27T03:30:32.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.784+0000 7f6e06346800 1 -- 172.21.15.165:0/1053841030 wait complete. 2024-04-27T03:30:32.807 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': 'full-control'}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-04-27T03:30:32.511777Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:32.807 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo1'] 2024-04-27T03:30:32.807 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo1'] 2024-04-27T03:30:32.807 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo1 2024-04-27T03:30:32.850 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:32.850 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:32.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.845+0000 7fa412a62800 1 Processor -- start 2024-04-27T03:30:32.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.845+0000 7fa412a62800 1 -- start start 2024-04-27T03:30:32.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.845+0000 7fa412a62800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a8a09062c0 0x55a8a090edc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.846+0000 7fa412a62800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a8a07c4d40 con 0x55a8a0904780 2024-04-27T03:30:32.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.846+0000 7fa412a62800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55a8a07974d0 con 0x55a8a09062c0 2024-04-27T03:30:32.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.846+0000 7fa40fd67640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a8a09062c0 0x55a8a090edc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.846+0000 7fa410568640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55a8a0904780 0x55a8a0904ba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53748/0 (socket says 172.21.15.165:53748) 2024-04-27T03:30:32.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.846+0000 7fa410568640 1 -- 172.21.15.165:0/1239857238 learned_addr learned my addr 172.21.15.165:0/1239857238 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:32.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.846+0000 7fa40f566640 1 -- 172.21.15.165:0/1239857238 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3044658353 0 0) 0x55a8a07c4d40 con 0x55a8a0904780 2024-04-27T03:30:32.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.846+0000 7fa40f566640 1 -- 172.21.15.165:0/1239857238 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fa3f8001e10 con 0x55a8a0904780 2024-04-27T03:30:32.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.846+0000 7fa40fd67640 1 -- 172.21.15.165:0/1239857238 >> v1:172.21.15.89:6789/0 conn(0x55a8a0904780 legacy=0x55a8a0904ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.846+0000 7fa40fd67640 1 -- 172.21.15.165:0/1239857238 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a07c3590 con 0x55a8a09062c0 2024-04-27T03:30:32.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.848+0000 7fa40fd67640 1 --2- 172.21.15.165:0/1239857238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a09062c0 0x55a8a090edc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa40400a470 tx=0x7fa40401cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8dade55ef0e1842 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.848+0000 7fa40f566640 1 -- 172.21.15.165:0/1239857238 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa404057070 con 0x55a8a09062c0 2024-04-27T03:30:32.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.848+0000 7fa40f566640 1 -- 172.21.15.165:0/1239857238 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40404acd0 con 0x55a8a09062c0 2024-04-27T03:30:32.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.848+0000 7fa40f566640 1 -- 172.21.15.165:0/1239857238 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa404052440 con 0x55a8a09062c0 2024-04-27T03:30:32.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.849+0000 7fa412a62800 1 -- 172.21.15.165:0/1239857238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a09062c0 msgr2=0x55a8a090edc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.849+0000 7fa412a62800 1 --2- 172.21.15.165:0/1239857238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a09062c0 0x55a8a090edc0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fa40400a470 tx=0x7fa40401cdb0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.849+0000 7fa412a62800 1 -- 172.21.15.165:0/1239857238 shutdown_connections 2024-04-27T03:30:32.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.849+0000 7fa412a62800 1 --2- 172.21.15.165:0/1239857238 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a09062c0 0x55a8a090edc0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.849+0000 7fa412a62800 1 -- 172.21.15.165:0/1239857238 >> 172.21.15.165:0/1239857238 conn(0x55a8a0900320 msgr2=0x55a8a0902740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.849+0000 7fa412a62800 1 -- 172.21.15.165:0/1239857238 shutdown_connections 2024-04-27T03:30:32.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.849+0000 7fa412a62800 1 -- 172.21.15.165:0/1239857238 wait complete. 2024-04-27T03:30:32.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.860+0000 7fa412a62800 1 Processor -- start 2024-04-27T03:30:32.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.860+0000 7fa412a62800 1 -- start start 2024-04-27T03:30:32.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.861+0000 7fa412a62800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 0x55a8a09f93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.861+0000 7fa412a62800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a8a08513c0 con 0x55a8a0904780 2024-04-27T03:30:32.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.861+0000 7fa410568640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 0x55a8a09f93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.861+0000 7fa410568640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 0x55a8a09f93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52420/0 (socket says 172.21.15.165:52420) 2024-04-27T03:30:32.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.861+0000 7fa410568640 1 -- 172.21.15.165:0/4282911531 learned_addr learned my addr 172.21.15.165:0/4282911531 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.861+0000 7fa410568640 1 -- 172.21.15.165:0/4282911531 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a09f9990 con 0x55a8a0904780 2024-04-27T03:30:32.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.861+0000 7fa410568640 1 --2- 172.21.15.165:0/4282911531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 0x55a8a09f93c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa400001d60 tx=0x7fa40000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc536d9d21381665 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.862+0000 7fa2fe60c640 1 -- 172.21.15.165:0/4282911531 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa400012020 con 0x55a8a0904780 2024-04-27T03:30:32.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.862+0000 7fa2fe60c640 1 -- 172.21.15.165:0/4282911531 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40000e470 con 0x55a8a0904780 2024-04-27T03:30:32.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.862+0000 7fa2fe60c640 1 -- 172.21.15.165:0/4282911531 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa40000e810 con 0x55a8a0904780 2024-04-27T03:30:32.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.862+0000 7fa412a62800 1 -- 172.21.15.165:0/4282911531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 msgr2=0x55a8a09f93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.862+0000 7fa412a62800 1 --2- 172.21.15.165:0/4282911531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 0x55a8a09f93c0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fa400001d60 tx=0x7fa40000b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.862+0000 7fa412a62800 1 -- 172.21.15.165:0/4282911531 shutdown_connections 2024-04-27T03:30:32.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.862+0000 7fa412a62800 1 --2- 172.21.15.165:0/4282911531 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 0x55a8a09f93c0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.862+0000 7fa412a62800 1 -- 172.21.15.165:0/4282911531 >> 172.21.15.165:0/4282911531 conn(0x55a8a0900320 msgr2=0x55a8a09f4ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.862+0000 7fa412a62800 1 -- 172.21.15.165:0/4282911531 shutdown_connections 2024-04-27T03:30:32.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.862+0000 7fa412a62800 1 -- 172.21.15.165:0/4282911531 wait complete. 2024-04-27T03:30:32.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.863+0000 7fa412a62800 1 Processor -- start 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.863+0000 7fa412a62800 1 -- start start 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.863+0000 7fa412a62800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 0x55a8a0a00f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.863+0000 7fa412a62800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a8a09115e0 con 0x55a8a0904780 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.863+0000 7fa410568640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 0x55a8a0a00f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.863+0000 7fa410568640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 0x55a8a0a00f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52432/0 (socket says 172.21.15.165:52432) 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.863+0000 7fa410568640 1 -- 172.21.15.165:0/1550963153 learned_addr learned my addr 172.21.15.165:0/1550963153 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.863+0000 7fa410568640 1 -- 172.21.15.165:0/1550963153 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a0a014b0 con 0x55a8a0904780 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.864+0000 7fa410568640 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 0x55a8a0a00f70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa40000b700 tx=0x7fa400000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.864+0000 7fa2fce09640 1 -- 172.21.15.165:0/1550963153 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa400012030 con 0x55a8a0904780 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.864+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a8a0a01740 con 0x55a8a0904780 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.864+0000 7fa2fce09640 1 -- 172.21.15.165:0/1550963153 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40001a7a0 con 0x55a8a0904780 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.864+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a0a043f0 con 0x55a8a0904780 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.864+0000 7fa2fce09640 1 -- 172.21.15.165:0/1550963153 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa40001a950 con 0x55a8a0904780 2024-04-27T03:30:32.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.865+0000 7fa2fce09640 1 -- 172.21.15.165:0/1550963153 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa400015020 con 0x55a8a0904780 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.865+0000 7fa2fce09640 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2f8036900 0x7fa2f8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.865+0000 7fa2fce09640 1 -- 172.21.15.165:0/1550963153 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fa400020030 con 0x55a8a0904780 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.865+0000 7fa40fd67640 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2f8036900 0x7fa2f8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.865+0000 7fa412a62800 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a8a09062c0 0x55a8a0a081b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.865+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a08730 con 0x55a8a09062c0 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.865+0000 7fa410d69640 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a8a09062c0 0x55a8a0a081b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.865+0000 7fa40fd67640 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2f8036900 0x7fa2f8038dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa404054000 tx=0x7fa40404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.865+0000 7fa410d69640 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a8a09062c0 0x55a8a0a081b0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.866+0000 7fa410d69640 1 -- 172.21.15.165:0/1550963153 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a09062c0 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.866+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a05780 con 0x55a8a09062c0 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.866+0000 7fa410d69640 1 -- 172.21.15.165:0/1550963153 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a09062c0 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.866+0000 7fa412a62800 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a8a0a0aa40 0x55a8a0a0ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.866+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a0d520 con 0x55a8a0a0aa40 2024-04-27T03:30:32.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.866+0000 7fa410568640 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a8a0a0aa40 0x55a8a0a0ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.866+0000 7fa410568640 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a8a0a0aa40 0x55a8a0a0ce30 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.867+0000 7fa410568640 1 -- 172.21.15.165:0/1550963153 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa40004ad70 con 0x55a8a0a0aa40 2024-04-27T03:30:32.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.867+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a099f0 con 0x55a8a09062c0 2024-04-27T03:30:32.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.867+0000 7fa410d69640 1 -- 172.21.15.165:0/1550963153 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a09062c0 2024-04-27T03:30:32.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.867+0000 7fa412a62800 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55a8a0a0e9d0 0x55a8a0a10e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.867+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a11580 con 0x55a8a0a0e9d0 2024-04-27T03:30:32.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.867+0000 7fa40fd67640 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55a8a0a0e9d0 0x55a8a0a10e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.867+0000 7fa40fd67640 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55a8a0a0e9d0 0x55a8a0a10e20 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.868+0000 7fa40fd67640 1 -- 172.21.15.165:0/1550963153 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fa40405a040 con 0x55a8a0a0e9d0 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.868+0000 7fa412a62800 1 Processor -- start 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.868+0000 7fa412a62800 1 -- start start 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.868+0000 7fa412a62800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a1f9c0 0x55a8a0a1fde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.868+0000 7fa412a62800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a8a07f76b0 con 0x55a8a0a1f9c0 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.868+0000 7fa410568640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a1f9c0 0x55a8a0a1fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.868+0000 7fa410568640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a1f9c0 0x55a8a0a1fde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52436/0 (socket says 172.21.15.165:52436) 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.868+0000 7fa410568640 1 -- 172.21.15.165:0/3062104352 learned_addr learned my addr 172.21.15.165:0/3062104352 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.869+0000 7fa410568640 1 -- 172.21.15.165:0/3062104352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a0a20320 con 0x55a8a0a1f9c0 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.869+0000 7fa410568640 1 --2- 172.21.15.165:0/3062104352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a1f9c0 0x55a8a0a1fde0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa400020000 tx=0x7fa400018bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.869+0000 7fa2f5ffb640 1 -- 172.21.15.165:0/3062104352 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa400044b50 con 0x55a8a0a1f9c0 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.869+0000 7fa2f5ffb640 1 -- 172.21.15.165:0/3062104352 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa400044cf0 con 0x55a8a0a1f9c0 2024-04-27T03:30:32.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.869+0000 7fa412a62800 1 -- 172.21.15.165:0/3062104352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a8a0a20d90 con 0x55a8a0a1f9c0 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.869+0000 7fa2f5ffb640 1 -- 172.21.15.165:0/3062104352 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4000442c0 con 0x55a8a0a1f9c0 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.869+0000 7fa412a62800 1 -- 172.21.15.165:0/3062104352 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a0a20d90 con 0x55a8a0a1f9c0 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.870+0000 7fa2f5ffb640 1 -- 172.21.15.165:0/3062104352 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa40003d020 con 0x55a8a0a1f9c0 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.870+0000 7fa2f5ffb640 1 --2- 172.21.15.165:0/3062104352 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2e8036950 0x7fa2e8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.870+0000 7fa2f5ffb640 1 -- 172.21.15.165:0/3062104352 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fa40003b020 con 0x55a8a0a1f9c0 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.870+0000 7fa40fd67640 1 --2- 172.21.15.165:0/3062104352 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2e8036950 0x7fa2e8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.870+0000 7fa412a62800 1 Processor -- start 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.870+0000 7fa412a62800 1 -- start start 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.870+0000 7fa40fd67640 1 --2- 172.21.15.165:0/3062104352 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2e8036950 0x7fa2e8038e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55a8a0a08ff0 tx=0x7fa404005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.870+0000 7fa412a62800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 0x55a8a0a2cd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.870+0000 7fa412a62800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a8a09f9d10 con 0x55a8a0a2c960 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.870+0000 7fa410568640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 0x55a8a0a2cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.871+0000 7fa410568640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 0x55a8a0a2cd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52440/0 (socket says 172.21.15.165:52440) 2024-04-27T03:30:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.871+0000 7fa410568640 1 -- 172.21.15.165:0/837353497 learned_addr learned my addr 172.21.15.165:0/837353497 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.871+0000 7fa410568640 1 -- 172.21.15.165:0/837353497 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a0a2d350 con 0x55a8a0a2c960 2024-04-27T03:30:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.871+0000 7fa410568640 1 --2- 172.21.15.165:0/837353497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 0x55a8a0a2cd80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa40000d6e0 tx=0x7fa40004ff90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed14b660a153deb7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.871+0000 7fa2ef7fe640 1 -- 172.21.15.165:0/837353497 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa40002da70 con 0x55a8a0a2c960 2024-04-27T03:30:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.871+0000 7fa2ef7fe640 1 -- 172.21.15.165:0/837353497 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa400061070 con 0x55a8a0a2c960 2024-04-27T03:30:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.871+0000 7fa2ef7fe640 1 -- 172.21.15.165:0/837353497 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4000245e0 con 0x55a8a0a2c960 2024-04-27T03:30:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.871+0000 7fa412a62800 1 -- 172.21.15.165:0/837353497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 msgr2=0x55a8a0a2cd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.871+0000 7fa412a62800 1 --2- 172.21.15.165:0/837353497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 0x55a8a0a2cd80 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7fa40000d6e0 tx=0x7fa40004ff90 comp rx=0 tx=0).stop 2024-04-27T03:30:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.872+0000 7fa412a62800 1 -- 172.21.15.165:0/837353497 shutdown_connections 2024-04-27T03:30:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.872+0000 7fa412a62800 1 --2- 172.21.15.165:0/837353497 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 0x55a8a0a2cd80 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.872+0000 7fa412a62800 1 -- 172.21.15.165:0/837353497 >> 172.21.15.165:0/837353497 conn(0x55a8a0a28c30 msgr2=0x55a8a0a26c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.872+0000 7fa412a62800 1 -- 172.21.15.165:0/837353497 shutdown_connections 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.872+0000 7fa412a62800 1 -- 172.21.15.165:0/837353497 wait complete. 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.872+0000 7fa412a62800 1 Processor -- start 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.872+0000 7fa412a62800 1 -- start start 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.872+0000 7fa412a62800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 0x55a8a0a33cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.872+0000 7fa412a62800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a8a0a2da20 con 0x55a8a0a2c960 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.872+0000 7fa410568640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 0x55a8a0a33cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.872+0000 7fa410568640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 0x55a8a0a33cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52446/0 (socket says 172.21.15.165:52446) 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.872+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 learned_addr learned my addr 172.21.15.165:0/2916692655 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.873+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8a0a34230 con 0x55a8a0a2c960 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.873+0000 7fa410568640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 0x55a8a0a33cf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa40001a6a0 tx=0x7fa400049f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.873+0000 7fa2edffb640 1 -- 172.21.15.165:0/2916692655 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa400049c20 con 0x55a8a0a2c960 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.873+0000 7fa2edffb640 1 -- 172.21.15.165:0/2916692655 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa40004c870 con 0x55a8a0a2c960 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.873+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a8a0a34230 con 0x55a8a0a2c960 2024-04-27T03:30:32.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.873+0000 7fa2edffb640 1 -- 172.21.15.165:0/2916692655 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa400022320 con 0x55a8a0a2c960 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.873+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8a0a34230 con 0x55a8a0a2c960 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.874+0000 7fa2edffb640 1 -- 172.21.15.165:0/2916692655 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa40005a040 con 0x55a8a0a2c960 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.874+0000 7fa2edffb640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2e0036950 0x7fa2e0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.874+0000 7fa2edffb640 1 -- 172.21.15.165:0/2916692655 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fa400057040 con 0x55a8a0a2c960 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.874+0000 7fa40fd67640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2e0036950 0x7fa2e0038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.874+0000 7fa412a62800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.874+0000 7fa40fd67640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2e0036950 0x7fa2e0038e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa404055040 tx=0x7fa404060040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.874+0000 7fa412a62800 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a8a0a3f460 0x55a8a0a41920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.874+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a42070 con 0x55a8a0a3f460 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa410d69640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a8a0a3f460 0x55a8a0a41920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa410d69640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a8a0a3f460 0x55a8a0a41920 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a0a3f460 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa412a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa412a62800 20 realm 2024-04-27T03:30:32.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa412a62800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a42480 con 0x55a8a0a3f460 2024-04-27T03:30:32.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a0a3f460 2024-04-27T03:30:32.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa412a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa412a62800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:32.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa412a62800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a42d50 con 0x55a8a0a3f460 2024-04-27T03:30:32.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a0a3f460 2024-04-27T03:30:32.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa412a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa412a62800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.875+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a3f460 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.876+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a0a3f460 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.876+0000 7fa412a62800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.876+0000 7fa412a62800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.876+0000 7fa412a62800 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a8a0a43e50 0x55a8a0a46240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.876+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a46850 con 0x55a8a0a43e50 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.876+0000 7fa410568640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a8a0a43e50 0x55a8a0a46240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.876+0000 7fa410568640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a8a0a43e50 0x55a8a0a46240 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.876+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.876+0000 7fa412a62800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.876+0000 7fa412a62800 20 searching for the correct realm 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.876+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a48180 con 0x55a8a0a43e50 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.877+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.877+0000 7fa410568640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa400017050 0x7fa40004a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.877+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a48180 con 0x7fa400017050 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.877+0000 7fa40fd67640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa400017050 0x7fa40004a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.877+0000 7fa40fd67640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa400017050 0x7fa40004a080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.878+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa404070070 con 0x7fa400017050 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.878+0000 7fa40fd67640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa404081040 0x7fa404082430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.878+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a48180 con 0x7fa404081040 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.878+0000 7fa410d69640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa404081040 0x7fa404082430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.878+0000 7fa410d69640 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa404081040 0x7fa404082430 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.879+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fa40800a4a0 con 0x7fa404081040 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.879+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a3f460 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.879+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a0a3f460 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.879+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a3f460 2024-04-27T03:30:32.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.879+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a0a3f460 2024-04-27T03:30:32.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.879+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa404081040 2024-04-27T03:30:32.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.879+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa40800a4a0 con 0x7fa404081040 2024-04-27T03:30:32.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.879+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa404081040 2024-04-27T03:30:32.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.880+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa40800a4a0 con 0x7fa404081040 2024-04-27T03:30:32.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.880+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa400017050 2024-04-27T03:30:32.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.880+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa404070070 con 0x7fa400017050 2024-04-27T03:30:32.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.880+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a43e50 2024-04-27T03:30:32.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.880+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.880+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a3f460 2024-04-27T03:30:32.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.880+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a0a3f460 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.880+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa404081040 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.881+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa40800a4a0 con 0x7fa404081040 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.881+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a3f460 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.881+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a0a3f460 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.881+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa400017050 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.881+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa404070070 con 0x7fa400017050 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.881+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a43e50 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.881+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.881+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a43e50 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.881+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.881+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa404081040 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.882+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa40800a4a0 con 0x7fa404081040 2024-04-27T03:30:32.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.882+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a43e50 2024-04-27T03:30:32.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.882+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.882+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a43e50 2024-04-27T03:30:32.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.882+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.882+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a3f460 2024-04-27T03:30:32.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.882+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a0a3f460 2024-04-27T03:30:32.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.883+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a3f460 2024-04-27T03:30:32.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.883+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa40800a4a0 con 0x55a8a0a3f460 2024-04-27T03:30:32.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.883+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa404081040 2024-04-27T03:30:32.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.883+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa408033040 con 0x7fa404081040 2024-04-27T03:30:32.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.883+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a43e50 2024-04-27T03:30:32.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.883+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.883+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa400017050 2024-04-27T03:30:32.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.884+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa404070070 con 0x7fa400017050 2024-04-27T03:30:32.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.884+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a3f460 2024-04-27T03:30:32.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.884+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.884+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa404081040 2024-04-27T03:30:32.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.884+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.884+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a43e50 2024-04-27T03:30:32.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.884+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.884+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa404081040 2024-04-27T03:30:32.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.885+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.885+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa400017050 2024-04-27T03:30:32.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.885+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa404070070 con 0x7fa400017050 2024-04-27T03:30:32.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.885+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a43e50 2024-04-27T03:30:32.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.885+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.885+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa404081040 2024-04-27T03:30:32.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.885+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.885+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x55a8a0a3f460 2024-04-27T03:30:32.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a430f0 con 0x7fa400017050 2024-04-27T03:30:32.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa404070070 con 0x7fa400017050 2024-04-27T03:30:32.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:32.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:32.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:32.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:32.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:32.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:32.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a47110 con 0x55a8a0a3f460 2024-04-27T03:30:32.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a47500 con 0x55a8a0a3f460 2024-04-27T03:30:32.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:32.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.886+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a47500 con 0x7fa404081040 2024-04-27T03:30:32.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa412a62800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa412a62800 20 zone default found 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa412a62800 4 Realm: () 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa412a62800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa412a62800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa412a62800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa412a62800 20 zonegroup default 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa412a62800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a46c90 con 0x7fa404081040 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa412a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa412a62800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.887+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a46c90 con 0x55a8a0a43e50 2024-04-27T03:30:32.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.888+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.888+0000 7fa412a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.888+0000 7fa412a62800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.888+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4a080 con 0x55a8a0a43e50 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.888+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.888+0000 7fa412a62800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.888+0000 7fa412a62800 20 started sync module instance, tier type = 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.888+0000 7fa412a62800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.888+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a49480 con 0x7fa400017050 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.889+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa404070070 con 0x7fa400017050 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.889+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94182738008176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a490e0 con 0x7fa400017050 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.889+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4afb0 con 0x55a8a0a3f460 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.890+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94182738008176] v25'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa404070070 con 0x7fa400017050 2024-04-27T03:30:32.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.891+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.891+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94182738012928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4b3d0 con 0x55a8a0a3f460 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.891+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4bd90 con 0x55a8a0a43e50 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.892+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94182738012928] v25'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.892+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.892+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94182738013984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4b340 con 0x55a8a0a43e50 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.892+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4cc00 con 0x7fa400017050 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.893+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94182738013984] v25'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.893+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa404070070 con 0x7fa400017050 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.893+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94182738018608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4d4a0 con 0x7fa400017050 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.893+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4de60 con 0x55a8a0a3f460 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.894+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94182738018608] v25'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa404070070 con 0x7fa400017050 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.894+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.894+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94182738024880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4e280 con 0x55a8a0a3f460 2024-04-27T03:30:32.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.894+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4ec40 con 0x7fa404081040 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.895+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.895+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94182738024880] v25'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.895+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94182738028432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4f060 con 0x7fa404081040 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.895+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4ddd0 con 0x7fa404081040 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.896+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.896+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94182738025936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4fb00 con 0x7fa404081040 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.896+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a50530 con 0x7fa404081040 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.897+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94182738028432] v25'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.897+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94182738025936] v25'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'75 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94182738029488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4f650 con 0x7fa404081040 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa412a62800 20 add_watcher() i=0 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa412a62800 20 add_watcher() i=1 2024-04-27T03:30:32.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa412a62800 20 add_watcher() i=2 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa412a62800 20 add_watcher() i=3 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa412a62800 20 add_watcher() i=4 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa412a62800 20 add_watcher() i=5 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa412a62800 20 add_watcher() i=6 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94182738029488] v25'76 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa412a62800 20 add_watcher() i=7 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa412a62800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.898+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4fae0 con 0x55a8a0a3f460 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.899+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.899+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94182738034752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4fae0 con 0x55a8a0a3f460 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.900+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94182738034752] v25'90 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.900+0000 7fa2c67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.900+0000 7fa412a62800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:32.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.900+0000 7fa412a62800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.900+0000 7fa412a62800 5 note: GC not initialized 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.900+0000 7fa2c57e2640 20 reqs_thread_entry: start 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.900+0000 7fa412a62800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.901+0000 7fa412a62800 20 Filter name: none 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.901+0000 7fa236ffd640 20 reqs_thread_entry: start 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.901+0000 7fa412a62800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.901+0000 7fa412a62800 20 rados->read ofs=0 len=0 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.901+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a86730 con 0x55a8a0a3f460 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.901+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~582 out=582b,getxattrs out=75b] v0'0 uv9 ondisk = 0) ==== 231+0+705 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.901+0000 7fa412a62800 20 rados_obj.operate() r=0 bl.length=582 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.901+0000 7fa412a62800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.901+0000 7fa412a62800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:32.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.901+0000 7fa412a62800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.901+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~556 in=556b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a8cd90 con 0x55a8a0a3f460 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.902+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~556] v25'10 uv10 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.902+0000 7fa412a62800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.902+0000 7fa412a62800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.902+0000 7fa412a62800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.902+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94182738282496 in=744b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a88730 con 0x55a8a0a3f460 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.902+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify (1) cookie 94182738024880 notify 107374182409 ret 0) ==== 774+0+0 (crc 0 0 0) 0x7fa40801e040 con 0x55a8a0a3f460 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.902+0000 7fa2ef7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182409 cookie 94182738024880 notifier 4399 bl.length()=732 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.902+0000 7fa2ef7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.902+0000 7fa2ef7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.902+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(64 notify.4 [notify cookie 94182738282496 out=8b] v0'0 uv75 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.902+0000 7fa2ef7fe640 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fa2d4003ea0 con 0x55a8a0a3f460 2024-04-27T03:30:32.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.903+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.903+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== watch-notify(notify_complete (2) cookie 94182738282496 notify 107374182409 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fa408018d90 con 0x55a8a0a3f460 2024-04-27T03:30:32.924 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[{"id":"foo:foo2","permissions":""}],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[{"user":"foo:foo2","secret_key":"ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy","active":true,"create_date":"2024-04-27T03:30:32.511777Z"}],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:32.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.904+0000 7fa2c67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:32.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.904+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94182738034752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a84e10 con 0x55a8a0a3f460 2024-04-27T03:30:32.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.905+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94182738034752] v25'91 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.905+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94182738008176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a50040 con 0x7fa400017050 2024-04-27T03:30:32.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.906+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94182738008176] v25'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa404070070 con 0x7fa400017050 2024-04-27T03:30:32.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.906+0000 7fa412a62800 20 remove_watcher() i=0 2024-04-27T03:30:32.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.906+0000 7fa412a62800 2 removed watcher, disabling cache 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.906+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94182738012928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a50040 con 0x55a8a0a3f460 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.907+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94182738012928] v25'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.907+0000 7fa412a62800 20 remove_watcher() i=1 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.907+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94182738013984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4ab00 con 0x55a8a0a43e50 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.908+0000 7fa410568640 1 -- 172.21.15.165:0/2916692655 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94182738013984] v25'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa400077020 con 0x55a8a0a43e50 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.908+0000 7fa412a62800 20 remove_watcher() i=2 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.908+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94182738018608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a50040 con 0x7fa400017050 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.909+0000 7fa40fd67640 1 -- 172.21.15.165:0/2916692655 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94182738018608] v25'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa404070070 con 0x7fa400017050 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.909+0000 7fa412a62800 20 remove_watcher() i=3 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.909+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94182738024880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4c130 con 0x55a8a0a3f460 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.910+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94182738024880] v25'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x55a8a0a3f460 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.910+0000 7fa412a62800 20 remove_watcher() i=4 2024-04-27T03:30:32.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.910+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94182738028432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a50040 con 0x7fa404081040 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.910+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94182738028432] v25'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.910+0000 7fa412a62800 20 remove_watcher() i=5 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.910+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94182738025936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4d9b0 con 0x7fa404081040 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.911+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94182738025936] v25'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.911+0000 7fa412a62800 20 remove_watcher() i=6 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.911+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94182738029488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a8a0a4e790 con 0x7fa404081040 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.912+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94182738029488] v25'77 uv75 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa408031040 con 0x7fa404081040 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.912+0000 7fa412a62800 20 remove_watcher() i=7 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa404081040 msgr2=0x7fa404082430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa404081040 0x7fa404082430 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a8a0a3f460 msgr2=0x55a8a0a41920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a8a0a3f460 0x55a8a0a41920 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa400017050 msgr2=0x7fa40004a080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa400017050 0x7fa40004a080 crc :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a8a0a43e50 msgr2=0x55a8a0a46240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a8a0a43e50 0x55a8a0a46240 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2e0036950 msgr2=0x7fa2e0038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2e0036950 0x7fa2e0038e10 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fa404055040 tx=0x7fa404060040 comp rx=0 tx=0).stop 2024-04-27T03:30:32.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa410d69640 1 -- 172.21.15.165:0/2916692655 reap_dead start 2024-04-27T03:30:32.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 msgr2=0x55a8a0a33cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 0x55a8a0a33cf0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7fa40001a6a0 tx=0x7fa400049f00 comp rx=0 tx=0).stop 2024-04-27T03:30:32.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 shutdown_connections 2024-04-27T03:30:32.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 --2- 172.21.15.165:0/2916692655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a2c960 0x55a8a0a33cf0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 >> 172.21.15.165:0/2916692655 conn(0x55a8a0a28c30 msgr2=0x55a8a0a26290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 shutdown_connections 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/2916692655 wait complete. 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/3062104352 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2e8036950 msgr2=0x7fa2e8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 --2- 172.21.15.165:0/3062104352 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2e8036950 0x7fa2e8038e10 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x55a8a0a08ff0 tx=0x7fa404005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/3062104352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a1f9c0 msgr2=0x55a8a0a1fde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 --2- 172.21.15.165:0/3062104352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a1f9c0 0x55a8a0a1fde0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fa400020000 tx=0x7fa400018bf0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/3062104352 shutdown_connections 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 --2- 172.21.15.165:0/3062104352 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2e8036950 0x7fa2e8038e10 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 --2- 172.21.15.165:0/3062104352 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0a1f9c0 0x55a8a0a1fde0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/3062104352 >> 172.21.15.165:0/3062104352 conn(0x55a8a0a13a60 msgr2=0x55a8a0a13e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/3062104352 shutdown_connections 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.913+0000 7fa412a62800 1 -- 172.21.15.165:0/3062104352 wait complete. 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55a8a0a0e9d0 msgr2=0x55a8a0a10e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55a8a0a0e9d0 0x55a8a0a10e20 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a8a09062c0 msgr2=0x55a8a0a081b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a8a09062c0 0x55a8a0a081b0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a8a0a0aa40 msgr2=0x55a8a0a0ce30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a8a0a0aa40 0x55a8a0a0ce30 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2f8036900 msgr2=0x7fa2f8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2f8036900 0x7fa2f8038dc0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fa404054000 tx=0x7fa40404e000 comp rx=0 tx=0).stop 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 msgr2=0x55a8a0a00f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 --2- 172.21.15.165:0/1550963153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a8a0904780 0x55a8a0a00f70 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fa40000b700 tx=0x7fa400000bc0 comp rx=0 tx=0).stop 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa410d69640 1 -- 172.21.15.165:0/1550963153 reap_dead start 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 shutdown_connections 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 >> 172.21.15.165:0/1550963153 conn(0x55a8a0900320 msgr2=0x55a8a09f34e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 shutdown_connections 2024-04-27T03:30:32.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:32.914+0000 7fa412a62800 1 -- 172.21.15.165:0/1550963153 wait complete. 2024-04-27T03:30:32.937 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True, 'create_date': '2024-04-27T03:30:32.511777Z'}], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:32.937 INFO:tasks.util.rgw:rgwadmin: client.0 : ['subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-04-27T03:30:32.937 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'subuser', 'rm', '--subuser', 'foo:foo2', '--key-type', 'swift', '--purge-keys'] 2024-04-27T03:30:32.937 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph subuser rm --subuser foo:foo2 --key-type swift --purge-keys 2024-04-27T03:30:33.017 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:33.017 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:33.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.012+0000 7fc2e9cdf800 1 Processor -- start 2024-04-27T03:30:33.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.012+0000 7fc2e9cdf800 1 -- start start 2024-04-27T03:30:33.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.012+0000 7fc2e9cdf800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b6f36ec4c0 0x55b6f36ee8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.012+0000 7fc2e9cdf800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55b6f35a6d40 con 0x55b6f36e89b0 2024-04-27T03:30:33.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.012+0000 7fc2e9cdf800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55b6f35794d0 con 0x55b6f36ec4c0 2024-04-27T03:30:33.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.012+0000 7fc2e6fe4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b6f36ec4c0 0x55b6f36ee8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.012+0000 7fc2e6fe4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b6f36ec4c0 0x55b6f36ee8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52460/0 (socket says 172.21.15.165:52460) 2024-04-27T03:30:33.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.012+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/366672154 learned_addr learned my addr 172.21.15.165:0/366672154 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.012+0000 7fc2e77e5640 1 --1- 172.21.15.165:0/366672154 >> v1:172.21.15.89:6789/0 conn(0x55b6f36e89b0 0x55b6f36eada0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53750/0 (socket says 172.21.15.165:53750) 2024-04-27T03:30:33.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.013+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/366672154 >> v1:172.21.15.89:6789/0 conn(0x55b6f36e89b0 legacy=0x55b6f36eada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.013+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/366672154 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f35a5590 con 0x55b6f36ec4c0 2024-04-27T03:30:33.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.015+0000 7fc2e6fe4640 1 --2- 172.21.15.165:0/366672154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36ec4c0 0x55b6f36ee8b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800a520 tx=0x7fc2d801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3543e969a6132eee server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.015+0000 7fc2e67e3640 1 -- 172.21.15.165:0/366672154 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2d8057070 con 0x55b6f36ec4c0 2024-04-27T03:30:33.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.015+0000 7fc2e67e3640 1 -- 172.21.15.165:0/366672154 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2d8052570 con 0x55b6f36ec4c0 2024-04-27T03:30:33.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.015+0000 7fc2e67e3640 1 -- 172.21.15.165:0/366672154 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2d804ae60 con 0x55b6f36ec4c0 2024-04-27T03:30:33.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.015+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/366672154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36ec4c0 msgr2=0x55b6f36ee8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.015+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/366672154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36ec4c0 0x55b6f36ee8b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc2d800a520 tx=0x7fc2d801d010 comp rx=0 tx=0).stop 2024-04-27T03:30:33.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.015+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/366672154 shutdown_connections 2024-04-27T03:30:33.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.015+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/366672154 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36ec4c0 0x55b6f36ee8b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.015+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/366672154 >> 172.21.15.165:0/366672154 conn(0x55b6f36e2340 msgr2=0x55b6f36e4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.015+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/366672154 shutdown_connections 2024-04-27T03:30:33.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.015+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/366672154 wait complete. 2024-04-27T03:30:33.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.027+0000 7fc2e9cdf800 1 Processor -- start 2024-04-27T03:30:33.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.027+0000 7fc2e9cdf800 1 -- start start 2024-04-27T03:30:33.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.027+0000 7fc2e9cdf800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 0x55b6f37d90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.027+0000 7fc2e9cdf800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b6f36333e0 con 0x55b6f36e89b0 2024-04-27T03:30:33.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.028+0000 7fc2e77e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 0x55b6f37d90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.028+0000 7fc2e77e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 0x55b6f37d90e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52470/0 (socket says 172.21.15.165:52470) 2024-04-27T03:30:33.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.028+0000 7fc2e77e5640 1 -- 172.21.15.165:0/1185209546 learned_addr learned my addr 172.21.15.165:0/1185209546 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.028+0000 7fc2e77e5640 1 -- 172.21.15.165:0/1185209546 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f37d96b0 con 0x55b6f36e89b0 2024-04-27T03:30:33.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.028+0000 7fc2e77e5640 1 --2- 172.21.15.165:0/1185209546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 0x55b6f37d90e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc001e10 tx=0x7fc2dc00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92090bbad7d172 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.029+0000 7fc1d560a640 1 -- 172.21.15.165:0/1185209546 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc00e5c0 con 0x55b6f36e89b0 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.029+0000 7fc1d560a640 1 -- 172.21.15.165:0/1185209546 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2dc006070 con 0x55b6f36e89b0 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.029+0000 7fc1d560a640 1 -- 172.21.15.165:0/1185209546 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc00e930 con 0x55b6f36e89b0 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.029+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/1185209546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 msgr2=0x55b6f37d90e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.029+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/1185209546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 0x55b6f37d90e0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc001e10 tx=0x7fc2dc00b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.029+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/1185209546 shutdown_connections 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.029+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/1185209546 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 0x55b6f37d90e0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.029+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/1185209546 >> 172.21.15.165:0/1185209546 conn(0x55b6f36e2340 msgr2=0x55b6f37d4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.029+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/1185209546 shutdown_connections 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.029+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/1185209546 wait complete. 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.030+0000 7fc2e9cdf800 1 Processor -- start 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.030+0000 7fc2e9cdf800 1 -- start start 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.030+0000 7fc2e9cdf800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 0x55b6f37e0c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.030+0000 7fc2e9cdf800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b6f36f1100 con 0x55b6f36e89b0 2024-04-27T03:30:33.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.030+0000 7fc2e77e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 0x55b6f37e0c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.030+0000 7fc2e77e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 0x55b6f37e0c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52486/0 (socket says 172.21.15.165:52486) 2024-04-27T03:30:33.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.030+0000 7fc2e77e5640 1 -- 172.21.15.165:0/2257198306 learned_addr learned my addr 172.21.15.165:0/2257198306 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.030+0000 7fc2e77e5640 1 -- 172.21.15.165:0/2257198306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f37e1190 con 0x55b6f36e89b0 2024-04-27T03:30:33.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.031+0000 7fc2e77e5640 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 0x55b6f37e0c50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00b700 tx=0x7fc2dc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.031+0000 7fc1d3e07640 1 -- 172.21.15.165:0/2257198306 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc00f8d0 con 0x55b6f36e89b0 2024-04-27T03:30:33.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.031+0000 7fc1d3e07640 1 -- 172.21.15.165:0/2257198306 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2dc00fa70 con 0x55b6f36e89b0 2024-04-27T03:30:33.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.031+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b6f37e1420 con 0x55b6f36e89b0 2024-04-27T03:30:33.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.031+0000 7fc1d3e07640 1 -- 172.21.15.165:0/2257198306 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc00fd50 con 0x55b6f36e89b0 2024-04-27T03:30:33.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.031+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f37e40d0 con 0x55b6f36e89b0 2024-04-27T03:30:33.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.032+0000 7fc1d3e07640 1 -- 172.21.15.165:0/2257198306 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc2dc017070 con 0x55b6f36e89b0 2024-04-27T03:30:33.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.032+0000 7fc1d3e07640 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1cc036900 0x7fc1cc038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.032+0000 7fc1d3e07640 1 -- 172.21.15.165:0/2257198306 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fc2dc015020 con 0x55b6f36e89b0 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.032+0000 7fc2e6fe4640 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1cc036900 0x7fc1cc038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.032+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b6f36ec4c0 0x55b6f37e7e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.032+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f37e83d0 con 0x55b6f36ec4c0 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.032+0000 7fc2e7fe6640 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b6f36ec4c0 0x55b6f37e7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.032+0000 7fc2e6fe4640 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1cc036900 0x7fc1cc038dc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8054000 tx=0x7fc2d804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.032+0000 7fc2e7fe6640 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b6f36ec4c0 0x55b6f37e7e90 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.033+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/2257198306 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f36ec4c0 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.033+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f37e5440 con 0x55b6f36ec4c0 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.033+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/2257198306 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f36ec4c0 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.033+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b6f37ea700 0x55b6f37ecaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.033+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f37ed190 con 0x55b6f37ea700 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.033+0000 7fc2e77e5640 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b6f37ea700 0x55b6f37ecaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.047 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.033+0000 7fc2e77e5640 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b6f37ea700 0x55b6f37ecaf0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.034+0000 7fc2e77e5640 1 -- 172.21.15.165:0/2257198306 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc2dc02e070 con 0x55b6f37ea700 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.034+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f37e96b0 con 0x55b6f36ec4c0 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.034+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/2257198306 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f36ec4c0 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.034+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b6f37ee640 0x55b6f37f0a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.034+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f37f11f0 con 0x55b6f37ee640 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.034+0000 7fc2e6fe4640 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b6f37ee640 0x55b6f37f0a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.035+0000 7fc2e6fe4640 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b6f37ee640 0x55b6f37f0a90 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.035+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/2257198306 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fc2d805a040 con 0x55b6f37ee640 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.035+0000 7fc2e9cdf800 1 Processor -- start 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.035+0000 7fc2e9cdf800 1 -- start start 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.036+0000 7fc2e9cdf800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e8550 0x55b6f36e69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.036+0000 7fc2e9cdf800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b6f35d96b0 con 0x55b6f36e8550 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.036+0000 7fc2e77e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e8550 0x55b6f36e69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.036+0000 7fc2e77e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e8550 0x55b6f36e69c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52498/0 (socket says 172.21.15.165:52498) 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.036+0000 7fc2e77e5640 1 -- 172.21.15.165:0/4239081076 learned_addr learned my addr 172.21.15.165:0/4239081076 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.036+0000 7fc2e77e5640 1 -- 172.21.15.165:0/4239081076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f36e6f00 con 0x55b6f36e8550 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.036+0000 7fc2e77e5640 1 --2- 172.21.15.165:0/4239081076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e8550 0x55b6f36e69c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0178e0 tx=0x7fc2dc041d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.036+0000 7fc1d1602640 1 -- 172.21.15.165:0/4239081076 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc01a690 con 0x55b6f36e8550 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.037+0000 7fc1d1602640 1 -- 172.21.15.165:0/4239081076 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2dc01a830 con 0x55b6f36e8550 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.037+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4239081076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b6f36e7190 con 0x55b6f36e8550 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.037+0000 7fc1d1602640 1 -- 172.21.15.165:0/4239081076 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc044c70 con 0x55b6f36e8550 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.037+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4239081076 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f36e7690 con 0x55b6f36e8550 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.037+0000 7fc1d1602640 1 -- 172.21.15.165:0/4239081076 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc2dc044e10 con 0x55b6f36e8550 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.037+0000 7fc1d1602640 1 --2- 172.21.15.165:0/4239081076 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1c0036900 0x7fc1c0038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.037+0000 7fc1d1602640 1 -- 172.21.15.165:0/4239081076 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fc2dc03cae0 con 0x55b6f36e8550 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.037+0000 7fc2e6fe4640 1 --2- 172.21.15.165:0/4239081076 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1c0036900 0x7fc1c0038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.038+0000 7fc2e9cdf800 1 Processor -- start 2024-04-27T03:30:33.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.038+0000 7fc2e9cdf800 1 -- start start 2024-04-27T03:30:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.038+0000 7fc2e6fe4640 1 --2- 172.21.15.165:0/4239081076 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1c0036900 0x7fc1c0038dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55b6f37e5380 tx=0x7fc2d8005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.038+0000 7fc2e9cdf800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 0x55b6f380ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.038+0000 7fc2e9cdf800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b6f37d9a30 con 0x55b6f380e950 2024-04-27T03:30:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.038+0000 7fc2e77e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 0x55b6f380ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.038+0000 7fc2e77e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 0x55b6f380ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52508/0 (socket says 172.21.15.165:52508) 2024-04-27T03:30:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.038+0000 7fc2e77e5640 1 -- 172.21.15.165:0/4145050592 learned_addr learned my addr 172.21.15.165:0/4145050592 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.038+0000 7fc2e77e5640 1 -- 172.21.15.165:0/4145050592 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f380f340 con 0x55b6f380e950 2024-04-27T03:30:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.039+0000 7fc2e77e5640 1 --2- 172.21.15.165:0/4145050592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 0x55b6f380ed70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc01ba90 tx=0x7fc2dc027610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e430ed42ffa7ea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.039+0000 7fc1caffd640 1 -- 172.21.15.165:0/4145050592 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc010df0 con 0x55b6f380e950 2024-04-27T03:30:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.039+0000 7fc1caffd640 1 -- 172.21.15.165:0/4145050592 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2dc026390 con 0x55b6f380e950 2024-04-27T03:30:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.039+0000 7fc1caffd640 1 -- 172.21.15.165:0/4145050592 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc026710 con 0x55b6f380e950 2024-04-27T03:30:33.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.039+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4145050592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 msgr2=0x55b6f380ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.039+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/4145050592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 0x55b6f380ed70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc01ba90 tx=0x7fc2dc027610 comp rx=0 tx=0).stop 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.039+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4145050592 shutdown_connections 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.039+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/4145050592 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 0x55b6f380ed70 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.039+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4145050592 >> 172.21.15.165:0/4145050592 conn(0x55b6f380ac70 msgr2=0x55b6f3808cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.039+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4145050592 shutdown_connections 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.039+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4145050592 wait complete. 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.039+0000 7fc2e9cdf800 1 Processor -- start 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.040+0000 7fc2e9cdf800 1 -- start start 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.040+0000 7fc2e9cdf800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 0x55b6f3815dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.040+0000 7fc2e9cdf800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b6f380fa60 con 0x55b6f380e950 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.040+0000 7fc2e77e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 0x55b6f3815dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.040+0000 7fc2e77e5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 0x55b6f3815dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52524/0 (socket says 172.21.15.165:52524) 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.040+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 learned_addr learned my addr 172.21.15.165:0/3803654555 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.040+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6f3816310 con 0x55b6f380e950 2024-04-27T03:30:33.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.041+0000 7fc2e77e5640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 0x55b6f3815dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc03cd50 tx=0x7fc2dc041f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.041+0000 7fc1c97fa640 1 -- 172.21.15.165:0/3803654555 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc071070 con 0x55b6f380e950 2024-04-27T03:30:33.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.041+0000 7fc1c97fa640 1 -- 172.21.15.165:0/3803654555 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc2dc031450 con 0x55b6f380e950 2024-04-27T03:30:33.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.041+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b6f3816310 con 0x55b6f380e950 2024-04-27T03:30:33.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.041+0000 7fc1c97fa640 1 -- 172.21.15.165:0/3803654555 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc2dc04ba70 con 0x55b6f380e950 2024-04-27T03:30:33.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.041+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6f3816310 con 0x55b6f380e950 2024-04-27T03:30:33.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.041+0000 7fc1c97fa640 1 -- 172.21.15.165:0/3803654555 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc2dc074070 con 0x55b6f380e950 2024-04-27T03:30:33.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.042+0000 7fc1c97fa640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1b0036900 0x7fc1b0038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.042+0000 7fc1c97fa640 1 -- 172.21.15.165:0/3803654555 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fc2dc05c070 con 0x55b6f380e950 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.042+0000 7fc2e6fe4640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1b0036900 0x7fc1b0038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.042+0000 7fc2e9cdf800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.042+0000 7fc2e6fe4640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1b0036900 0x7fc1b0038dc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8055040 tx=0x7fc2d804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.042+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b6f3821200 0x55b6f38236c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.042+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3823e10 con 0x55b6f3821200 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.042+0000 7fc2e7fe6640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b6f3821200 0x55b6f38236c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.042+0000 7fc2e7fe6640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b6f3821200 0x55b6f38236c0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f3821200 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 20 realm 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824220 con 0x55b6f3821200 2024-04-27T03:30:33.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f3821200 2024-04-27T03:30:33.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:33.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824af0 con 0x55b6f3821200 2024-04-27T03:30:33.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f3821200 2024-04-27T03:30:33.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824e90 con 0x55b6f3821200 2024-04-27T03:30:33.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f3821200 2024-04-27T03:30:33.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:33.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b6f3825bf0 0x55b6f3827fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.043+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f38285f0 con 0x55b6f3825bf0 2024-04-27T03:30:33.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.044+0000 7fc2e77e5640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b6f3825bf0 0x55b6f3827fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.044+0000 7fc2e77e5640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b6f3825bf0 0x55b6f3827fe0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.044+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.044+0000 7fc2e9cdf800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:33.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.044+0000 7fc2e9cdf800 20 searching for the correct realm 2024-04-27T03:30:33.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.044+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3829f20 con 0x55b6f3825bf0 2024-04-27T03:30:33.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.045+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.045+0000 7fc2e77e5640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc2dc076060 0x7fc2dc031670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.045+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3829f20 con 0x7fc2dc076060 2024-04-27T03:30:33.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.045+0000 7fc2e6fe4640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc2dc076060 0x7fc2dc031670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.045+0000 7fc2e6fe4640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc2dc076060 0x7fc2dc031670 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.045+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.045+0000 7fc2e6fe4640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc2d8074040 0x7fc2d8081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.045+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3829f20 con 0x7fc2d8074040 2024-04-27T03:30:33.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.046+0000 7fc2e7fe6640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc2d8074040 0x7fc2d8081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.046+0000 7fc2e7fe6640 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc2d8074040 0x7fc2d8081410 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.046+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fc2e000a4a0 con 0x7fc2d8074040 2024-04-27T03:30:33.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.046+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824e90 con 0x55b6f3821200 2024-04-27T03:30:33.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.047+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f3821200 2024-04-27T03:30:33.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.047+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824e90 con 0x55b6f3821200 2024-04-27T03:30:33.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.047+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f3821200 2024-04-27T03:30:33.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.047+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824e90 con 0x7fc2d8074040 2024-04-27T03:30:33.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.047+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e000a4a0 con 0x7fc2d8074040 2024-04-27T03:30:33.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.047+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824e90 con 0x7fc2d8074040 2024-04-27T03:30:33.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.047+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e000a4a0 con 0x7fc2d8074040 2024-04-27T03:30:33.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.047+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824e90 con 0x7fc2dc076060 2024-04-27T03:30:33.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.047+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.047+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824e90 con 0x55b6f3825bf0 2024-04-27T03:30:33.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.048+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.048+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824e90 con 0x55b6f3821200 2024-04-27T03:30:33.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.048+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f3821200 2024-04-27T03:30:33.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.048+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824e90 con 0x7fc2d8074040 2024-04-27T03:30:33.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.048+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e000a4a0 con 0x7fc2d8074040 2024-04-27T03:30:33.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.048+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824e90 con 0x55b6f3821200 2024-04-27T03:30:33.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.048+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f3821200 2024-04-27T03:30:33.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.048+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3824e90 con 0x7fc2dc076060 2024-04-27T03:30:33.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.049+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.049+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x55b6f3825bf0 2024-04-27T03:30:33.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.049+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.049+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x55b6f3825bf0 2024-04-27T03:30:33.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.049+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.049+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x7fc2d8074040 2024-04-27T03:30:33.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.050+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e000a4a0 con 0x7fc2d8074040 2024-04-27T03:30:33.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.050+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x55b6f3825bf0 2024-04-27T03:30:33.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.050+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.050+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x55b6f3825bf0 2024-04-27T03:30:33.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.050+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.050+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x55b6f3821200 2024-04-27T03:30:33.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.050+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f3821200 2024-04-27T03:30:33.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.050+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x55b6f3821200 2024-04-27T03:30:33.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.051+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc2e000a4a0 con 0x55b6f3821200 2024-04-27T03:30:33.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.051+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x7fc2d8074040 2024-04-27T03:30:33.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.051+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e0033040 con 0x7fc2d8074040 2024-04-27T03:30:33.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.051+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x55b6f3825bf0 2024-04-27T03:30:33.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.051+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.051+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x7fc2dc076060 2024-04-27T03:30:33.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.051+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.051+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x55b6f3821200 2024-04-27T03:30:33.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.052+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.052+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x7fc2d8074040 2024-04-27T03:30:33.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.052+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.052+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x55b6f3825bf0 2024-04-27T03:30:33.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.052+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.052+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x7fc2d8074040 2024-04-27T03:30:33.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.053+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.053+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x7fc2dc076060 2024-04-27T03:30:33.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.053+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.053+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x55b6f3825bf0 2024-04-27T03:30:33.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.053+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.053+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x7fc2d8074040 2024-04-27T03:30:33.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.053+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.053+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x55b6f3821200 2024-04-27T03:30:33.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc2d808e060 con 0x7fc2dc076060 2024-04-27T03:30:33.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:33.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3828eb0 con 0x55b6f3821200 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f38292a0 con 0x55b6f3821200 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:33.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.054+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f38292a0 con 0x7fc2d8074040 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 20 zone default found 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 4 Realm: () 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 20 zonegroup default 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3828b10 con 0x7fc2d8074040 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3828b10 con 0x55b6f3825bf0 2024-04-27T03:30:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.055+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382be60 con 0x55b6f3825bf0 2024-04-27T03:30:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.056+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.056+0000 7fc2e9cdf800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.056+0000 7fc2e9cdf800 20 started sync module instance, tier type = 2024-04-27T03:30:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.056+0000 7fc2e9cdf800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.056+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382b220 con 0x7fc2dc076060 2024-04-27T03:30:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.057+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.057+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94244257838608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382c770 con 0x7fc2dc076060 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.057+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382cfc0 con 0x55b6f3821200 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.058+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.058+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94244257843984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382d540 con 0x55b6f3821200 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.058+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94244257838608] v25'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.058+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382e060 con 0x55b6f3825bf0 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.059+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94244257843984] v25'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.059+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.059+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94244257841936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382cf90 con 0x55b6f3825bf0 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.059+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382ea20 con 0x7fc2dc076060 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.060+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.060+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94244257841936] v25'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.060+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94244257850736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382f270 con 0x7fc2dc076060 2024-04-27T03:30:33.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.060+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382fc30 con 0x55b6f3821200 2024-04-27T03:30:33.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.061+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94244257850736] v25'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.061+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.061+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94244257855360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3830050 con 0x55b6f3821200 2024-04-27T03:30:33.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.061+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3830a10 con 0x7fc2d8074040 2024-04-27T03:30:33.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.062+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.062+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94244257855360] v25'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.062+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94244257858912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3830e30 con 0x7fc2d8074040 2024-04-27T03:30:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.062+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382fbe0 con 0x7fc2d8074040 2024-04-27T03:30:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.063+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94244257858912] v25'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.063+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.064+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94244257859968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3830da0 con 0x7fc2d8074040 2024-04-27T03:30:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.064+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3831d80 con 0x7fc2d8074040 2024-04-27T03:30:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.066+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94244257859968] v25'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.066+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'78 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.066+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94244257862688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3831cf0 con 0x7fc2d8074040 2024-04-27T03:30:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.066+0000 7fc2e9cdf800 20 add_watcher() i=0 2024-04-27T03:30:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.066+0000 7fc2e9cdf800 20 add_watcher() i=1 2024-04-27T03:30:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.066+0000 7fc2e9cdf800 20 add_watcher() i=2 2024-04-27T03:30:33.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.066+0000 7fc2e9cdf800 20 add_watcher() i=3 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.066+0000 7fc2e9cdf800 20 add_watcher() i=4 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.066+0000 7fc2e9cdf800 20 add_watcher() i=5 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.066+0000 7fc2e9cdf800 20 add_watcher() i=6 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.067+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94244257862688] v25'79 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.067+0000 7fc2e9cdf800 20 add_watcher() i=7 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.067+0000 7fc2e9cdf800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.067+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f38351d0 con 0x55b6f3821200 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.067+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.067+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94244257864208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f38326f0 con 0x55b6f3821200 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.068+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94244257864208] v25'92 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.068+0000 7fc19dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.069+0000 7fc2e9cdf800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.069+0000 7fc2e9cdf800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:33.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.069+0000 7fc2e9cdf800 5 note: GC not initialized 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.069+0000 7fc19cfe1640 20 reqs_thread_entry: start 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.069+0000 7fc2e9cdf800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.069+0000 7fc2e9cdf800 20 Filter name: none 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.069+0000 7fc1167fc640 20 reqs_thread_entry: start 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.069+0000 7fc2e9cdf800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.069+0000 7fc2e9cdf800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.069+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f38684f0 con 0x55b6f3821200 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.070+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~556 out=556b,getxattrs out=75b] v0'0 uv10 ondisk = 0) ==== 231+0+679 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.070+0000 7fc2e9cdf800 20 rados_obj.operate() r=0 bl.length=556 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.070+0000 7fc2e9cdf800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.070+0000 7fc2e9cdf800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.070+0000 7fc2e9cdf800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.070+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f386ec50 con 0x55b6f3821200 2024-04-27T03:30:33.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.071+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~391] v25'11 uv11 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.071+0000 7fc2e9cdf800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.071+0000 7fc2e9cdf800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.071+0000 7fc2e9cdf800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.071+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94244258113216 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f386a3c0 con 0x55b6f3821200 2024-04-27T03:30:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.071+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify (1) cookie 94244257855360 notify 107374182410 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7fc2e001e040 con 0x55b6f3821200 2024-04-27T03:30:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.071+0000 7fc1caffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182410 cookie 94244257855360 notifier 4406 bl.length()=567 2024-04-27T03:30:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.071+0000 7fc1caffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.071+0000 7fc1caffd640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.071+0000 7fc1caffd640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fc1ac003ea0 con 0x55b6f3821200 2024-04-27T03:30:33.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.071+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(64 notify.4 [notify cookie 94244258113216 out=8b] v0'0 uv78 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.072+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.072+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== watch-notify(notify_complete (2) cookie 94244258113216 notify 107374182410 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fc2e0018dd0 con 0x55b6f3821200 2024-04-27T03:30:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.072+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 8.2 8:45671577:users.swift::foo%3afoo2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f386e6c0 con 0x7fc2dc076060 2024-04-27T03:30:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.073+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 foo:foo2 [delete] v25'2 uv2 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.073+0000 7fc2e9cdf800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.swift:foo:foo2, ofs0, ns] 2024-04-27T03:30:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.073+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94244258113216 in=171b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b6f3869f10 con 0x7fc2dc076060 2024-04-27T03:30:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.073+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== watch-notify(notify (1) cookie 94244257838608 notify 107374182405 ret 0) ==== 201+0+0 (crc 0 0 0) 0x7fc2d8088040 con 0x7fc2dc076060 2024-04-27T03:30:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.073+0000 7fc1ca7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182405 cookie 94244257838608 notifier 4406 bl.length()=159 2024-04-27T03:30:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.073+0000 7fc1ca7fc640 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fc11c0032c0 con 0x7fc2dc076060 2024-04-27T03:30:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.073+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(67 notify.0 [notify cookie 94244258113216 out=8b] v0'0 uv78 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.074+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== watch-notify(notify_complete (2) cookie 94244258113216 notify 107374182405 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55b6f35a5590 con 0x7fc2dc076060 2024-04-27T03:30:33.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.074+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.091 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:33.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.074+0000 7fc19dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:33.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.074+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94244257864208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f386a3c0 con 0x55b6f3821200 2024-04-27T03:30:33.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.075+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(69 data_loggenerations_metadata [watch unwatch cookie 94244257864208] v25'93 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.075+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94244257838608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f386a390 con 0x7fc2dc076060 2024-04-27T03:30:33.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.076+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(70 notify.0 [watch unwatch cookie 94244257838608] v25'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.076+0000 7fc2e9cdf800 20 remove_watcher() i=0 2024-04-27T03:30:33.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.076+0000 7fc2e9cdf800 2 removed watcher, disabling cache 2024-04-27T03:30:33.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.076+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94244257843984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f386a390 con 0x55b6f3821200 2024-04-27T03:30:33.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.077+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(71 notify.1 [watch unwatch cookie 94244257843984] v25'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.077+0000 7fc2e9cdf800 20 remove_watcher() i=1 2024-04-27T03:30:33.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.077+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:72 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94244257841936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382cb10 con 0x55b6f3825bf0 2024-04-27T03:30:33.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.078+0000 7fc2e77e5640 1 -- 172.21.15.165:0/3803654555 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(72 notify.2 [watch unwatch cookie 94244257841936] v25'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2dc073030 con 0x55b6f3825bf0 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.078+0000 7fc2e9cdf800 20 remove_watcher() i=2 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.078+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:73 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94244257850736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382b610 con 0x7fc2dc076060 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.079+0000 7fc2e6fe4640 1 -- 172.21.15.165:0/3803654555 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(73 notify.3 [watch unwatch cookie 94244257850736] v25'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2d807f070 con 0x7fc2dc076060 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.079+0000 7fc2e9cdf800 20 remove_watcher() i=3 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.079+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94244257855360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382b610 con 0x55b6f3821200 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.080+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(74 notify.4 [watch unwatch cookie 94244257855360] v25'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x55b6f3821200 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.080+0000 7fc2e9cdf800 20 remove_watcher() i=4 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.080+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94244257858912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382b610 con 0x7fc2d8074040 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.081+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(75 notify.5 [watch unwatch cookie 94244257858912] v25'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.081+0000 7fc2e9cdf800 20 remove_watcher() i=5 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.081+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94244257859968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382b610 con 0x7fc2d8074040 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.082+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(76 notify.6 [watch unwatch cookie 94244257859968] v25'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.082+0000 7fc2e9cdf800 20 remove_watcher() i=6 2024-04-27T03:30:33.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.082+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94244257862688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b6f382b610 con 0x7fc2d8074040 2024-04-27T03:30:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.083+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(77 notify.7 [watch unwatch cookie 94244257862688] v25'80 uv78 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc2e0031040 con 0x7fc2d8074040 2024-04-27T03:30:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.083+0000 7fc2e9cdf800 20 remove_watcher() i=7 2024-04-27T03:30:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc2d8074040 msgr2=0x7fc2d8081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc2d8074040 0x7fc2d8081410 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b6f3821200 msgr2=0x55b6f38236c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b6f3821200 0x55b6f38236c0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc2dc076060 msgr2=0x7fc2dc031670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc2dc076060 0x7fc2dc031670 crc :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b6f3825bf0 msgr2=0x55b6f3827fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b6f3825bf0 0x55b6f3827fe0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1b0036900 msgr2=0x7fc1b0038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1b0036900 0x7fc1b0038dc0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8055040 tx=0x7fc2d804f000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 msgr2=0x55b6f3815dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/3803654555 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f380e950 0x55b6f3815dd0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc03cd50 tx=0x7fc2dc041f70 comp rx=0 tx=0).stop 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 reap_dead start 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/3803654555 reap_dead start 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 shutdown_connections 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 >> 172.21.15.165:0/3803654555 conn(0x55b6f380ac70 msgr2=0x55b6f38082d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 shutdown_connections 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/3803654555 wait complete. 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4239081076 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1c0036900 msgr2=0x7fc1c0038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/4239081076 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1c0036900 0x7fc1c0038dc0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x55b6f37e5380 tx=0x7fc2d8005ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4239081076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e8550 msgr2=0x55b6f36e69c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/4239081076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e8550 0x55b6f36e69c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc0178e0 tx=0x7fc2dc041d80 comp rx=0 tx=0).stop 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4239081076 shutdown_connections 2024-04-27T03:30:33.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/4239081076 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1c0036900 0x7fc1c0038dc0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/4239081076 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e8550 0x55b6f36e69c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4239081076 >> 172.21.15.165:0/4239081076 conn(0x55b6f37f36f0 msgr2=0x55b6f37f3af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4239081076 shutdown_connections 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/4239081076 wait complete. 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.084+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b6f37ee640 msgr2=0x55b6f37f0a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b6f37ee640 0x55b6f37f0a90 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b6f36ec4c0 msgr2=0x55b6f37e7e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b6f36ec4c0 0x55b6f37e7e90 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b6f37ea700 msgr2=0x55b6f37ecaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b6f37ea700 0x55b6f37ecaf0 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1cc036900 msgr2=0x7fc1cc038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc1cc036900 0x7fc1cc038dc0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8054000 tx=0x7fc2d804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 msgr2=0x55b6f37e0c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 --2- 172.21.15.165:0/2257198306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b6f36e89b0 0x55b6f37e0c50 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00b700 tx=0x7fc2dc000bc0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e7fe6640 1 -- 172.21.15.165:0/2257198306 reap_dead start 2024-04-27T03:30:33.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 shutdown_connections 2024-04-27T03:30:33.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 >> 172.21.15.165:0/2257198306 conn(0x55b6f36e2340 msgr2=0x55b6f37d3200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 shutdown_connections 2024-04-27T03:30:33.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.085+0000 7fc2e9cdf800 1 -- 172.21.15.165:0/2257198306 wait complete. 2024-04-27T03:30:33.102 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:33.102 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-04-27T03:30:33.102 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-04-27T03:30:33.102 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-04-27T03:30:33.145 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:33.146 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:33.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.140+0000 7f2543e98800 1 Processor -- start 2024-04-27T03:30:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.140+0000 7f2543e98800 1 -- start start 2024-04-27T03:30:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.140+0000 7f2543e98800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55829dd237c0 0x55829dd23be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.140+0000 7f2543e98800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55829dbe3d40 con 0x55829dd24120 2024-04-27T03:30:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.140+0000 7f2543e98800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55829dbb64d0 con 0x55829dd237c0 2024-04-27T03:30:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.141+0000 7f2541997640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55829dd237c0 0x55829dd23be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.141+0000 7f2541997640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55829dd237c0 0x55829dd23be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52530/0 (socket says 172.21.15.165:52530) 2024-04-27T03:30:33.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.141+0000 7f2541196640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55829dd24120 0x55829dd245d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53758/0 (socket says 172.21.15.165:53758) 2024-04-27T03:30:33.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.141+0000 7f2541997640 1 -- 172.21.15.165:0/3752603048 learned_addr learned my addr 172.21.15.165:0/3752603048 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.141+0000 7f2541997640 1 -- 172.21.15.165:0/3752603048 >> v1:172.21.15.89:6789/0 conn(0x55829dd24120 legacy=0x55829dd245d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.141+0000 7f2541997640 1 -- 172.21.15.165:0/3752603048 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55829dbe2590 con 0x55829dd237c0 2024-04-27T03:30:33.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.143+0000 7f2541997640 1 --2- 172.21.15.165:0/3752603048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829dd23be0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f253400a520 tx=0x7f253401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ebeb0c8779821d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.143+0000 7f2540995640 1 -- 172.21.15.165:0/3752603048 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2534057070 con 0x55829dd237c0 2024-04-27T03:30:33.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.143+0000 7f2540995640 1 -- 172.21.15.165:0/3752603048 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2534052570 con 0x55829dd237c0 2024-04-27T03:30:33.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.143+0000 7f2540995640 1 -- 172.21.15.165:0/3752603048 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2534052870 con 0x55829dd237c0 2024-04-27T03:30:33.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.143+0000 7f2543e98800 1 -- 172.21.15.165:0/3752603048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 msgr2=0x55829dd23be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.143+0000 7f2543e98800 1 --2- 172.21.15.165:0/3752603048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829dd23be0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f253400a520 tx=0x7f253401d010 comp rx=0 tx=0).stop 2024-04-27T03:30:33.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.143+0000 7f2543e98800 1 -- 172.21.15.165:0/3752603048 shutdown_connections 2024-04-27T03:30:33.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.143+0000 7f2543e98800 1 --2- 172.21.15.165:0/3752603048 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829dd23be0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.143+0000 7f2543e98800 1 -- 172.21.15.165:0/3752603048 >> 172.21.15.165:0/3752603048 conn(0x55829dd1f320 msgr2=0x55829dd21740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.143+0000 7f2543e98800 1 -- 172.21.15.165:0/3752603048 shutdown_connections 2024-04-27T03:30:33.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.143+0000 7f2543e98800 1 -- 172.21.15.165:0/3752603048 wait complete. 2024-04-27T03:30:33.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.155+0000 7f2543e98800 1 Processor -- start 2024-04-27T03:30:33.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.155+0000 7f2543e98800 1 -- start start 2024-04-27T03:30:33.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.155+0000 7f2543e98800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829de18400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.155+0000 7f2543e98800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55829dc703c0 con 0x55829dd237c0 2024-04-27T03:30:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.155+0000 7f2541997640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829de18400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.156+0000 7f2541997640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829de18400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52534/0 (socket says 172.21.15.165:52534) 2024-04-27T03:30:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.156+0000 7f2541997640 1 -- 172.21.15.165:0/1760987931 learned_addr learned my addr 172.21.15.165:0/1760987931 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.156+0000 7f2541997640 1 -- 172.21.15.165:0/1760987931 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55829de189d0 con 0x55829dd237c0 2024-04-27T03:30:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.156+0000 7f2541997640 1 --2- 172.21.15.165:0/1760987931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829de18400 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2534055040 tx=0x7f253404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f120e50788c18c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.156+0000 7f242fe07640 1 -- 172.21.15.165:0/1760987931 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2534057040 con 0x55829dd237c0 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.156+0000 7f242fe07640 1 -- 172.21.15.165:0/1760987931 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2534052870 con 0x55829dd237c0 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.156+0000 7f242fe07640 1 -- 172.21.15.165:0/1760987931 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2534052b90 con 0x55829dd237c0 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2543e98800 1 -- 172.21.15.165:0/1760987931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 msgr2=0x55829de18400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2543e98800 1 --2- 172.21.15.165:0/1760987931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829de18400 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f2534055040 tx=0x7f253404c000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2543e98800 1 -- 172.21.15.165:0/1760987931 shutdown_connections 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2543e98800 1 --2- 172.21.15.165:0/1760987931 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829de18400 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2543e98800 1 -- 172.21.15.165:0/1760987931 >> 172.21.15.165:0/1760987931 conn(0x55829dd1f320 msgr2=0x55829de13d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2543e98800 1 -- 172.21.15.165:0/1760987931 shutdown_connections 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2543e98800 1 -- 172.21.15.165:0/1760987931 wait complete. 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2543e98800 1 Processor -- start 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2543e98800 1 -- start start 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2543e98800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829de1fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2543e98800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55829dd24f20 con 0x55829dd237c0 2024-04-27T03:30:33.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2541997640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829de1fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2541997640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829de1fe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52536/0 (socket says 172.21.15.165:52536) 2024-04-27T03:30:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.157+0000 7f2541997640 1 -- 172.21.15.165:0/3689385451 learned_addr learned my addr 172.21.15.165:0/3689385451 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.158+0000 7f2541997640 1 -- 172.21.15.165:0/3689385451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55829de20350 con 0x55829dd237c0 2024-04-27T03:30:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.158+0000 7f2541997640 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829de1fe10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f25340524f0 tx=0x7f2534003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.158+0000 7f242e604640 1 -- 172.21.15.165:0/3689385451 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2534073070 con 0x55829dd237c0 2024-04-27T03:30:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.158+0000 7f242e604640 1 -- 172.21.15.165:0/3689385451 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f253405d570 con 0x55829dd237c0 2024-04-27T03:30:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.158+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55829de205e0 con 0x55829dd237c0 2024-04-27T03:30:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.158+0000 7f242e604640 1 -- 172.21.15.165:0/3689385451 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2534057070 con 0x55829dd237c0 2024-04-27T03:30:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.158+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55829de23290 con 0x55829dd237c0 2024-04-27T03:30:33.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.159+0000 7f242e604640 1 -- 172.21.15.165:0/3689385451 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2534076070 con 0x55829dd237c0 2024-04-27T03:30:33.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.159+0000 7f242e604640 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2428036900 0x7f2428038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.159+0000 7f242e604640 1 -- 172.21.15.165:0/3689385451 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f253408dca0 con 0x55829dd237c0 2024-04-27T03:30:33.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.159+0000 7f2541196640 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2428036900 0x7f2428038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.159+0000 7f2543e98800 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55829dd24120 0x55829de27050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.159+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de27590 con 0x55829dd24120 2024-04-27T03:30:33.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.159+0000 7f2542198640 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55829dd24120 0x55829de27050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.160+0000 7f2541196640 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2428036900 0x7f2428038dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2538000f60 tx=0x7f2538001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.160+0000 7f2542198640 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55829dd24120 0x55829de27050 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.160+0000 7f2542198640 1 -- 172.21.15.165:0/3689385451 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829dd24120 2024-04-27T03:30:33.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.160+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de24600 con 0x55829dd24120 2024-04-27T03:30:33.194 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.160+0000 7f2542198640 1 -- 172.21.15.165:0/3689385451 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829dd24120 2024-04-27T03:30:33.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.160+0000 7f2543e98800 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55829de298c0 0x55829de2bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.160+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de2c3a0 con 0x55829de298c0 2024-04-27T03:30:33.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.160+0000 7f2541997640 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55829de298c0 0x55829de2bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.161+0000 7f2541997640 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55829de298c0 0x55829de2bcb0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.161+0000 7f2541997640 1 -- 172.21.15.165:0/3689385451 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2534075030 con 0x55829de298c0 2024-04-27T03:30:33.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.161+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de28870 con 0x55829dd24120 2024-04-27T03:30:33.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.161+0000 7f2542198640 1 -- 172.21.15.165:0/3689385451 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829dd24120 2024-04-27T03:30:33.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.161+0000 7f2543e98800 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55829de2d850 0x55829de2fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.161+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de30400 con 0x55829de2d850 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.161+0000 7f2541196640 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55829de2d850 0x55829de2fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.162+0000 7f2541196640 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55829de2d850 0x55829de2fca0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.162+0000 7f2541196640 1 -- 172.21.15.165:0/3689385451 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f253800f690 con 0x55829de2d850 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.162+0000 7f2543e98800 1 Processor -- start 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.162+0000 7f2543e98800 1 -- start start 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.162+0000 7f2543e98800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de3e860 0x55829de3ec80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.162+0000 7f2543e98800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55829dc166b0 con 0x55829de3e860 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.163+0000 7f2541997640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de3e860 0x55829de3ec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.163+0000 7f2541997640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de3e860 0x55829de3ec80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52544/0 (socket says 172.21.15.165:52544) 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.163+0000 7f2541997640 1 -- 172.21.15.165:0/1822074925 learned_addr learned my addr 172.21.15.165:0/1822074925 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.163+0000 7f2541997640 1 -- 172.21.15.165:0/1822074925 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55829de3f1c0 con 0x55829de3e860 2024-04-27T03:30:33.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.163+0000 7f2541997640 1 --2- 172.21.15.165:0/1822074925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de3e860 0x55829de3ec80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2534052c70 tx=0x7f2534066530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.163+0000 7f24277fe640 1 -- 172.21.15.165:0/1822074925 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2534091020 con 0x55829de3e860 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.163+0000 7f24277fe640 1 -- 172.21.15.165:0/1822074925 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2534006db0 con 0x55829de3e860 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.163+0000 7f2543e98800 1 -- 172.21.15.165:0/1822074925 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55829de3fc30 con 0x55829de3e860 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.163+0000 7f24277fe640 1 -- 172.21.15.165:0/1822074925 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2534077ba0 con 0x55829de3e860 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.163+0000 7f2543e98800 1 -- 172.21.15.165:0/1822074925 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55829de40130 con 0x55829de3e860 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.164+0000 7f24277fe640 1 -- 172.21.15.165:0/1822074925 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2534007e00 con 0x55829de3e860 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.164+0000 7f24277fe640 1 --2- 172.21.15.165:0/1822074925 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f241c036900 0x7f241c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.164+0000 7f24277fe640 1 -- 172.21.15.165:0/1822074925 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2534002000 con 0x55829de3e860 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.164+0000 7f2541196640 1 --2- 172.21.15.165:0/1822074925 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f241c036900 0x7f241c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.164+0000 7f2543e98800 1 Processor -- start 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.165+0000 7f2543e98800 1 -- start start 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.165+0000 7f2541196640 1 --2- 172.21.15.165:0/1822074925 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f241c036900 0x7f241c038dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55829de27e70 tx=0x7f253800f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.165+0000 7f2543e98800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 0x55829de4be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.165+0000 7f2543e98800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55829de18c90 con 0x55829de4ba30 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.165+0000 7f2541997640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 0x55829de4be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.165+0000 7f2541997640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 0x55829de4be50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52560/0 (socket says 172.21.15.165:52560) 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.165+0000 7f2541997640 1 -- 172.21.15.165:0/455401368 learned_addr learned my addr 172.21.15.165:0/455401368 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.165+0000 7f2541997640 1 -- 172.21.15.165:0/455401368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55829de4c420 con 0x55829de4ba30 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.165+0000 7f2541997640 1 --2- 172.21.15.165:0/455401368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 0x55829de4be50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f253405de90 tx=0x7f25340965e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2c09072359823bd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.165+0000 7f24257fa640 1 -- 172.21.15.165:0/455401368 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f253405cda0 con 0x55829de4ba30 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.165+0000 7f24257fa640 1 -- 172.21.15.165:0/455401368 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f25340a6070 con 0x55829de4ba30 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.165+0000 7f24257fa640 1 -- 172.21.15.165:0/455401368 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2534076360 con 0x55829de4ba30 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.166+0000 7f2543e98800 1 -- 172.21.15.165:0/455401368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 msgr2=0x55829de4be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.166+0000 7f2543e98800 1 --2- 172.21.15.165:0/455401368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 0x55829de4be50 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f253405de90 tx=0x7f25340965e0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.166+0000 7f2543e98800 1 -- 172.21.15.165:0/455401368 shutdown_connections 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.166+0000 7f2543e98800 1 --2- 172.21.15.165:0/455401368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 0x55829de4be50 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.166+0000 7f2543e98800 1 -- 172.21.15.165:0/455401368 >> 172.21.15.165:0/455401368 conn(0x55829de47d00 msgr2=0x55829de45d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.166+0000 7f2543e98800 1 -- 172.21.15.165:0/455401368 shutdown_connections 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.166+0000 7f2543e98800 1 -- 172.21.15.165:0/455401368 wait complete. 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.166+0000 7f2543e98800 1 Processor -- start 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.166+0000 7f2543e98800 1 -- start start 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.166+0000 7f2543e98800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 0x55829de52d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.166+0000 7f2543e98800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55829de4ca80 con 0x55829de4ba30 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.167+0000 7f2541997640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 0x55829de52d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.167+0000 7f2541997640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 0x55829de52d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52562/0 (socket says 172.21.15.165:52562) 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.167+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 learned_addr learned my addr 172.21.15.165:0/2758101987 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.167+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55829de532b0 con 0x55829de4ba30 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.167+0000 7f2541997640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 0x55829de52d70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2534089fb0 tx=0x7f2534052000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.167+0000 7f24137fe640 1 -- 172.21.15.165:0/2758101987 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2534089480 con 0x55829de4ba30 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.167+0000 7f24137fe640 1 -- 172.21.15.165:0/2758101987 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2534089620 con 0x55829de4ba30 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.167+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55829de532b0 con 0x55829de4ba30 2024-04-27T03:30:33.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.167+0000 7f24137fe640 1 -- 172.21.15.165:0/2758101987 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2534096ce0 con 0x55829de4ba30 2024-04-27T03:30:33.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.167+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55829de532b0 con 0x55829de4ba30 2024-04-27T03:30:33.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.168+0000 7f24137fe640 1 -- 172.21.15.165:0/2758101987 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2534089810 con 0x55829de4ba30 2024-04-27T03:30:33.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.168+0000 7f24137fe640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f240c036950 0x7f240c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.168+0000 7f24137fe640 1 -- 172.21.15.165:0/2758101987 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f25340acb20 con 0x55829de4ba30 2024-04-27T03:30:33.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.168+0000 7f2541196640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f240c036950 0x7f240c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.168+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.168+0000 7f2543e98800 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55829de5e4a0 0x55829de60960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.168+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de610b0 con 0x55829de5e4a0 2024-04-27T03:30:33.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2542198640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55829de5e4a0 0x55829de60960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2541196640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f240c036950 0x7f240c038e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55829de30800 tx=0x7f2538010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2542198640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55829de5e4a0 0x55829de60960 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2543e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2543e98800 20 realm 2024-04-27T03:30:33.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de614c0 con 0x55829de5e4a0 2024-04-27T03:30:33.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2543e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2543e98800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:33.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de61d90 con 0x55829de5e4a0 2024-04-27T03:30:33.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2543e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.169+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.170+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de5e4a0 2024-04-27T03:30:33.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.170+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.170+0000 7f2543e98800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:33.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.170+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.170+0000 7f2543e98800 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55829de62c20 0x55829de65010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.170+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de65770 con 0x55829de62c20 2024-04-27T03:30:33.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.170+0000 7f2541997640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55829de62c20 0x55829de65010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.170+0000 7f2541997640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55829de62c20 0x55829de65010 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.170+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.170+0000 7f2543e98800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:33.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.170+0000 7f2543e98800 20 searching for the correct realm 2024-04-27T03:30:33.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.170+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de67150 con 0x55829de62c20 2024-04-27T03:30:33.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.171+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.171+0000 7f2541997640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2534096020 0x7f253406a1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.205 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.171+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de67150 con 0x7f2534096020 2024-04-27T03:30:33.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.171+0000 7f2541196640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2534096020 0x7f253406a1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.171+0000 7f2541196640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2534096020 0x7f253406a1c0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.172+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.172+0000 7f2541196640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f253800f070 0x7f253802e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.172+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de67150 con 0x7f253800f070 2024-04-27T03:30:33.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.172+0000 7f2542198640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f253800f070 0x7f253802e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.172+0000 7f2542198640 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f253800f070 0x7f253802e910 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.172+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.172+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de5e4a0 2024-04-27T03:30:33.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.173+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.173+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de5e4a0 2024-04-27T03:30:33.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.173+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.173+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x7f253800f070 2024-04-27T03:30:33.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.173+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.173+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x7f253800f070 2024-04-27T03:30:33.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.173+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.173+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x7f2534096020 2024-04-27T03:30:33.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.174+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.174+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de62c20 2024-04-27T03:30:33.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.174+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.174+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de5e4a0 2024-04-27T03:30:33.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.174+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.174+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x7f253800f070 2024-04-27T03:30:33.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.174+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.174+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de5e4a0 2024-04-27T03:30:33.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.174+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.174+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x7f2534096020 2024-04-27T03:30:33.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.175+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.175+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de62c20 2024-04-27T03:30:33.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.175+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.175+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de62c20 2024-04-27T03:30:33.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.175+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.175+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x7f253800f070 2024-04-27T03:30:33.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.176+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.176+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de62c20 2024-04-27T03:30:33.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.176+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.176+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de62c20 2024-04-27T03:30:33.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.176+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.176+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de5e4a0 2024-04-27T03:30:33.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.176+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.176+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de5e4a0 2024-04-27T03:30:33.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.176+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.177+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x7f253800f070 2024-04-27T03:30:33.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.177+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.177+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de62c20 2024-04-27T03:30:33.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.177+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.177+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x7f2534096020 2024-04-27T03:30:33.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.177+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.177+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de5e4a0 2024-04-27T03:30:33.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.177+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.177+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x7f253800f070 2024-04-27T03:30:33.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.178+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.178+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de62c20 2024-04-27T03:30:33.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.178+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.178+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x7f253800f070 2024-04-27T03:30:33.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.178+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.178+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x7f2534096020 2024-04-27T03:30:33.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.178+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.179+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de62c20 2024-04-27T03:30:33.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.179+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.179+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x7f253800f070 2024-04-27T03:30:33.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.179+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.179+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de5e4a0 2024-04-27T03:30:33.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.179+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.179+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f253c038040 con 0x7f2534096020 2024-04-27T03:30:33.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:33.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:33.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:33.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:33.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:33.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:33.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de62130 con 0x55829de5e4a0 2024-04-27T03:30:33.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de66120 con 0x55829de5e4a0 2024-04-27T03:30:33.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:33.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de66120 con 0x7f253800f070 2024-04-27T03:30:33.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:33.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 zone default found 2024-04-27T03:30:33.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 4 Realm: () 2024-04-27T03:30:33.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:33.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:33.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:33.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 zonegroup default 2024-04-27T03:30:33.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.180+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de66540 con 0x7f253800f070 2024-04-27T03:30:33.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2543e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de66960 con 0x55829de62c20 2024-04-27T03:30:33.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2543e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de66d80 con 0x55829de62c20 2024-04-27T03:30:33.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2543e98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2543e98800 20 started sync module instance, tier type = 2024-04-27T03:30:33.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2543e98800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:33.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.181+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de684a0 con 0x7f2534096020 2024-04-27T03:30:33.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.182+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.182+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94019483240512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de69110 con 0x7f2534096020 2024-04-27T03:30:33.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.182+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de69ad0 con 0x55829de5e4a0 2024-04-27T03:30:33.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.183+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94019483240512] v25'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.183+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.183+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94019483244064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de69ef0 con 0x55829de5e4a0 2024-04-27T03:30:33.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.183+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6a8b0 con 0x55829de62c20 2024-04-27T03:30:33.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.185+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94019483244064] v25'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.185+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.185+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94019483245120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de69e60 con 0x55829de62c20 2024-04-27T03:30:33.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.185+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6b770 con 0x7f2534096020 2024-04-27T03:30:33.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.186+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94019483245120] v25'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.186+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.186+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94019483249824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6c010 con 0x7f2534096020 2024-04-27T03:30:33.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.186+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6c9d0 con 0x55829de5e4a0 2024-04-27T03:30:33.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.187+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94019483249824] v25'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.187+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.187+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94019483256096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6cdf0 con 0x55829de5e4a0 2024-04-27T03:30:33.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.187+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6d7b0 con 0x7f253800f070 2024-04-27T03:30:33.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.188+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.188+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94019483259648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6d720 con 0x7f253800f070 2024-04-27T03:30:33.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.188+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6e780 con 0x7f253800f070 2024-04-27T03:30:33.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.188+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94019483256096] v25'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.189+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94019483259648] v25'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.189+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.189+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94019483261856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6c940 con 0x7f253800f070 2024-04-27T03:30:33.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.189+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6ebf0 con 0x7f253800f070 2024-04-27T03:30:33.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.190+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94019483261856] v25'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.190+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'81 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.190+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94019483263600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6ebf0 con 0x7f253800f070 2024-04-27T03:30:33.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.190+0000 7f2543e98800 20 add_watcher() i=0 2024-04-27T03:30:33.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.190+0000 7f2543e98800 20 add_watcher() i=1 2024-04-27T03:30:33.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.190+0000 7f2543e98800 20 add_watcher() i=2 2024-04-27T03:30:33.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.190+0000 7f2543e98800 20 add_watcher() i=3 2024-04-27T03:30:33.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.190+0000 7f2543e98800 20 add_watcher() i=4 2024-04-27T03:30:33.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.190+0000 7f2543e98800 20 add_watcher() i=5 2024-04-27T03:30:33.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.190+0000 7f2543e98800 20 add_watcher() i=6 2024-04-27T03:30:33.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.193+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94019483263600] v25'82 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.193+0000 7f2543e98800 20 add_watcher() i=7 2024-04-27T03:30:33.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.193+0000 7f2543e98800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:33.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.193+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829de72050 con 0x55829de5e4a0 2024-04-27T03:30:33.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.193+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.193+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94019483265296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6f570 con 0x55829de5e4a0 2024-04-27T03:30:33.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.194+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94019483265296] v25'94 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.194+0000 7f23f37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:33.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.194+0000 7f2543e98800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:33.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.194+0000 7f2543e98800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:33.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.194+0000 7f2543e98800 5 note: GC not initialized 2024-04-27T03:30:33.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.195+0000 7f23f27fc640 20 reqs_thread_entry: start 2024-04-27T03:30:33.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.195+0000 7f2543e98800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:33.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.195+0000 7f2543e98800 20 Filter name: none 2024-04-27T03:30:33.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.195+0000 7f2363fff640 20 reqs_thread_entry: start 2024-04-27T03:30:33.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.195+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.195+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829dea4e80 con 0x55829de5e4a0 2024-04-27T03:30:33.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.195+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.195+0000 7f2543e98800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:33.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.195+0000 7f2543e98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.195+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829dea5870 con 0x55829de5e4a0 2024-04-27T03:30:33.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.196+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.196+0000 7f2543e98800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:33.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.196+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55829dea6880 con 0x7f2534096020 2024-04-27T03:30:33.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.196+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.197+0000 7f23f37fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:33.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.197+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94019483265296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de7b720 con 0x55829de5e4a0 2024-04-27T03:30:33.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.198+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94019483265296] v25'95 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.198+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94019483240512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6e8e0 con 0x7f2534096020 2024-04-27T03:30:33.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.199+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94019483240512] v25'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.199+0000 7f2543e98800 20 remove_watcher() i=0 2024-04-27T03:30:33.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.199+0000 7f2543e98800 2 removed watcher, disabling cache 2024-04-27T03:30:33.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.199+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94019483244064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de68840 con 0x55829de5e4a0 2024-04-27T03:30:33.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.200+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94019483244064] v25'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.200+0000 7f2543e98800 20 remove_watcher() i=1 2024-04-27T03:30:33.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.200+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94019483245120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de68840 con 0x55829de62c20 2024-04-27T03:30:33.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.201+0000 7f2541997640 1 -- 172.21.15.165:0/2758101987 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94019483245120] v25'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2534093020 con 0x55829de62c20 2024-04-27T03:30:33.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.201+0000 7f2543e98800 20 remove_watcher() i=2 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.201+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94019483249824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6e8e0 con 0x7f2534096020 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.202+0000 7f2541196640 1 -- 172.21.15.165:0/2758101987 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94019483249824] v25'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2538037040 con 0x7f2534096020 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.202+0000 7f2543e98800 20 remove_watcher() i=3 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.202+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94019483256096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6aca0 con 0x55829de5e4a0 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.203+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94019483256096] v25'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x55829de5e4a0 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.203+0000 7f2543e98800 20 remove_watcher() i=4 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.203+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94019483259648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6c520 con 0x7f253800f070 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.204+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94019483259648] v25'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.204+0000 7f2543e98800 20 remove_watcher() i=5 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.204+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94019483261856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6c520 con 0x7f253800f070 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.205+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94019483261856] v25'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.205+0000 7f2543e98800 20 remove_watcher() i=6 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.205+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94019483263600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55829de6dba0 con 0x7f253800f070 2024-04-27T03:30:33.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94019483263600] v25'83 uv81 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f253c00a4a0 con 0x7f253800f070 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 20 remove_watcher() i=7 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f253800f070 msgr2=0x7f253802e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f253800f070 0x7f253802e910 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55829de5e4a0 msgr2=0x55829de60960 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55829de5e4a0 0x55829de60960 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2534096020 msgr2=0x7f253406a1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2534096020 0x7f253406a1c0 crc :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55829de62c20 msgr2=0x55829de65010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55829de62c20 0x55829de65010 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f240c036950 msgr2=0x7f240c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f240c036950 0x7f240c038e10 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x55829de30800 tx=0x7f2538010000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 msgr2=0x55829de52d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 --2- 172.21.15.165:0/2758101987 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de4ba30 0x55829de52d70 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7f2534089fb0 tx=0x7f2534052000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 reap_dead start 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2542198640 1 -- 172.21.15.165:0/2758101987 reap_dead start 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 shutdown_connections 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.206+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 >> 172.21.15.165:0/2758101987 conn(0x55829de47d00 msgr2=0x55829de45360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 shutdown_connections 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/2758101987 wait complete. 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/1822074925 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f241c036900 msgr2=0x7f241c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 --2- 172.21.15.165:0/1822074925 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f241c036900 0x7f241c038dc0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x55829de27e70 tx=0x7f253800f000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/1822074925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de3e860 msgr2=0x55829de3ec80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 --2- 172.21.15.165:0/1822074925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de3e860 0x55829de3ec80 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f2534052c70 tx=0x7f2534066530 comp rx=0 tx=0).stop 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/1822074925 shutdown_connections 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 --2- 172.21.15.165:0/1822074925 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f241c036900 0x7f241c038dc0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 --2- 172.21.15.165:0/1822074925 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829de3e860 0x55829de3ec80 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/1822074925 >> 172.21.15.165:0/1822074925 conn(0x55829de32900 msgr2=0x55829de32d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/1822074925 shutdown_connections 2024-04-27T03:30:33.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/1822074925 wait complete. 2024-04-27T03:30:33.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55829de2d850 msgr2=0x55829de2fca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55829de2d850 0x55829de2fca0 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55829dd24120 msgr2=0x55829de27050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55829dd24120 0x55829de27050 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55829de298c0 msgr2=0x55829de2bcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55829de298c0 0x55829de2bcb0 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2428036900 msgr2=0x7f2428038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2428036900 0x7f2428038dc0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f2538000f60 tx=0x7f2538001370 comp rx=0 tx=0).stop 2024-04-27T03:30:33.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 msgr2=0x55829de1fe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 --2- 172.21.15.165:0/3689385451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55829dd237c0 0x55829de1fe10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f25340524f0 tx=0x7f2534003020 comp rx=0 tx=0).stop 2024-04-27T03:30:33.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2542198640 1 -- 172.21.15.165:0/3689385451 reap_dead start 2024-04-27T03:30:33.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 shutdown_connections 2024-04-27T03:30:33.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 >> 172.21.15.165:0/3689385451 conn(0x55829dd1f320 msgr2=0x55829de12520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.207+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 shutdown_connections 2024-04-27T03:30:33.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.208+0000 7f2543e98800 1 -- 172.21.15.165:0/3689385451 wait complete. 2024-04-27T03:30:33.257 INFO:teuthology.orchestra.run.smithi165.stdout:[] 2024-04-27T03:30:33.257 DEBUG:tasks.util.rgw: json result: [] 2024-04-27T03:30:33.257 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-04-27T03:30:33.257 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-04-27T03:30:33.257 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-04-27T03:30:33.340 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:33.340 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:33.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.335+0000 7fc5c8d3e800 1 Processor -- start 2024-04-27T03:30:33.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.335+0000 7fc5c8d3e800 1 -- start start 2024-04-27T03:30:33.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.336+0000 7fc5c8d3e800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x558d931eb4a0 0x558d931ed890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.336+0000 7fc5c8d3e800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558d930a5d40 con 0x558d931e7990 2024-04-27T03:30:33.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.336+0000 7fc5c8d3e800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x558d930784d0 con 0x558d931eb4a0 2024-04-27T03:30:33.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.336+0000 7fc5bc23e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x558d931eb4a0 0x558d931ed890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.336+0000 7fc5bc23e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x558d931eb4a0 0x558d931ed890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52566/0 (socket says 172.21.15.165:52566) 2024-04-27T03:30:33.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.336+0000 7fc5c4a3f640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x558d931e7990 0x558d931e9d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:53766/0 (socket says 172.21.15.165:53766) 2024-04-27T03:30:33.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.336+0000 7fc5bc23e640 1 -- 172.21.15.165:0/4126266944 learned_addr learned my addr 172.21.15.165:0/4126266944 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.336+0000 7fc5bc23e640 1 -- 172.21.15.165:0/4126266944 >> v1:172.21.15.89:6789/0 conn(0x558d931e7990 legacy=0x558d931e9d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.336+0000 7fc5bc23e640 1 -- 172.21.15.165:0/4126266944 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d930a4590 con 0x558d931eb4a0 2024-04-27T03:30:33.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.338+0000 7fc5bc23e640 1 --2- 172.21.15.165:0/4126266944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931eb4a0 0x558d931ed890 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00a520 tx=0x7fc5ac01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52d6a8174a11f8dd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.338+0000 7fc5bffff640 1 -- 172.21.15.165:0/4126266944 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5ac057070 con 0x558d931eb4a0 2024-04-27T03:30:33.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.338+0000 7fc5bffff640 1 -- 172.21.15.165:0/4126266944 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5ac052570 con 0x558d931eb4a0 2024-04-27T03:30:33.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.338+0000 7fc5bffff640 1 -- 172.21.15.165:0/4126266944 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5ac04ae60 con 0x558d931eb4a0 2024-04-27T03:30:33.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.339+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/4126266944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931eb4a0 msgr2=0x558d931ed890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.339+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/4126266944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931eb4a0 0x558d931ed890 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00a520 tx=0x7fc5ac01d010 comp rx=0 tx=0).stop 2024-04-27T03:30:33.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.339+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/4126266944 shutdown_connections 2024-04-27T03:30:33.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.339+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/4126266944 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931eb4a0 0x558d931ed890 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.339+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/4126266944 >> 172.21.15.165:0/4126266944 conn(0x558d931e1320 msgr2=0x558d931e3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.339+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/4126266944 shutdown_connections 2024-04-27T03:30:33.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.339+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/4126266944 wait complete. 2024-04-27T03:30:33.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.351+0000 7fc5c8d3e800 1 Processor -- start 2024-04-27T03:30:33.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.351+0000 7fc5c8d3e800 1 -- start start 2024-04-27T03:30:33.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.351+0000 7fc5c8d3e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 0x558d932d80c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.351+0000 7fc5c8d3e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558d931323c0 con 0x558d931e7990 2024-04-27T03:30:33.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.351+0000 7fc5c4a3f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 0x558d932d80c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.351+0000 7fc5c4a3f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 0x558d932d80c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52578/0 (socket says 172.21.15.165:52578) 2024-04-27T03:30:33.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.351+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/2994876125 learned_addr learned my addr 172.21.15.165:0/2994876125 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.351+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/2994876125 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d932d8690 con 0x558d931e7990 2024-04-27T03:30:33.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.352+0000 7fc5c4a3f640 1 --2- 172.21.15.165:0/2994876125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 0x558d932d80c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4001e10 tx=0x7fc5b400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5dc0f67d57eadf8b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.352+0000 7fc4b2e0d640 1 -- 172.21.15.165:0/2994876125 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5b400e5c0 con 0x558d931e7990 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.352+0000 7fc4b2e0d640 1 -- 172.21.15.165:0/2994876125 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b4006070 con 0x558d931e7990 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.352+0000 7fc4b2e0d640 1 -- 172.21.15.165:0/2994876125 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5b400e930 con 0x558d931e7990 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.352+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/2994876125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 msgr2=0x558d932d80c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.352+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/2994876125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 0x558d932d80c0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4001e10 tx=0x7fc5b400b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.352+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/2994876125 shutdown_connections 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.352+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/2994876125 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 0x558d932d80c0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.352+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/2994876125 >> 172.21.15.165:0/2994876125 conn(0x558d931e1320 msgr2=0x558d932d39e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.353+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/2994876125 shutdown_connections 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.353+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/2994876125 wait complete. 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.353+0000 7fc5c8d3e800 1 Processor -- start 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.353+0000 7fc5c8d3e800 1 -- start start 2024-04-27T03:30:33.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.353+0000 7fc5c8d3e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 0x558d932dfc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.353+0000 7fc5c8d3e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558d931f00e0 con 0x558d931e7990 2024-04-27T03:30:33.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.353+0000 7fc5c4a3f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 0x558d932dfc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.353+0000 7fc5c4a3f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 0x558d932dfc30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52594/0 (socket says 172.21.15.165:52594) 2024-04-27T03:30:33.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.353+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/210499868 learned_addr learned my addr 172.21.15.165:0/210499868 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.353+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/210499868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d932e0170 con 0x558d931e7990 2024-04-27T03:30:33.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.354+0000 7fc5c4a3f640 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 0x558d932dfc30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400b700 tx=0x7fc5b4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.354+0000 7fc4b160a640 1 -- 172.21.15.165:0/210499868 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5b400f8d0 con 0x558d931e7990 2024-04-27T03:30:33.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.354+0000 7fc4b160a640 1 -- 172.21.15.165:0/210499868 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b400fa70 con 0x558d931e7990 2024-04-27T03:30:33.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.354+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d932e0400 con 0x558d931e7990 2024-04-27T03:30:33.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.354+0000 7fc4b160a640 1 -- 172.21.15.165:0/210499868 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5b400fd50 con 0x558d931e7990 2024-04-27T03:30:33.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.354+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d932e30b0 con 0x558d931e7990 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.355+0000 7fc4b160a640 1 -- 172.21.15.165:0/210499868 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc5b4017070 con 0x558d931e7990 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.355+0000 7fc4b160a640 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc4ac036900 0x7fc4ac038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.355+0000 7fc4b160a640 1 -- 172.21.15.165:0/210499868 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fc5b4015020 con 0x558d931e7990 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.355+0000 7fc5bc23e640 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc4ac036900 0x7fc4ac038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.355+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558d931eb4a0 0x558d932e6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.355+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d932e73b0 con 0x558d931eb4a0 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.355+0000 7fc5c5240640 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558d931eb4a0 0x558d932e6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.355+0000 7fc5bc23e640 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc4ac036900 0x7fc4ac038dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac054000 tx=0x7fc5ac04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.355+0000 7fc5c5240640 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558d931eb4a0 0x558d932e6e70 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.356+0000 7fc5c5240640 1 -- 172.21.15.165:0/210499868 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d931eb4a0 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.356+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d932e4440 con 0x558d931eb4a0 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.356+0000 7fc5c5240640 1 -- 172.21.15.165:0/210499868 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d931eb4a0 2024-04-27T03:30:33.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.356+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558d932e96e0 0x558d932ebad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.356+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d932ec1c0 con 0x558d932e96e0 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.356+0000 7fc5c4a3f640 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558d932e96e0 0x558d932ebad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.356+0000 7fc5c4a3f640 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558d932e96e0 0x558d932ebad0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.357+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/210499868 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc5b4046040 con 0x558d932e96e0 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.357+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d932e8690 con 0x558d931eb4a0 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.357+0000 7fc5c5240640 1 -- 172.21.15.165:0/210499868 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d931eb4a0 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.357+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558d932ed670 0x558d932efac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.357+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d932f0220 con 0x558d932ed670 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.357+0000 7fc5bc23e640 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558d932ed670 0x558d932efac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.358+0000 7fc5bc23e640 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558d932ed670 0x558d932efac0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.358+0000 7fc5bc23e640 1 -- 172.21.15.165:0/210499868 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fc5ac05a040 con 0x558d932ed670 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.358+0000 7fc5c8d3e800 1 Processor -- start 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.358+0000 7fc5c8d3e800 1 -- start start 2024-04-27T03:30:33.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.358+0000 7fc5c8d3e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d932fe680 0x558d932feaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.358+0000 7fc5c8d3e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558d930d86b0 con 0x558d932fe680 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.359+0000 7fc5c4a3f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d932fe680 0x558d932feaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.359+0000 7fc5c4a3f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d932fe680 0x558d932feaa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52602/0 (socket says 172.21.15.165:52602) 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.359+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/1774995404 learned_addr learned my addr 172.21.15.165:0/1774995404 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.359+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/1774995404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d932fefe0 con 0x558d932fe680 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.359+0000 7fc5c4a3f640 1 --2- 172.21.15.165:0/1774995404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d932fe680 0x558d932feaa0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc5b404b4d0 tx=0x7fc5b4017e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.359+0000 7fc4aa7fc640 1 -- 172.21.15.165:0/1774995404 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5b401a7c0 con 0x558d932fe680 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.359+0000 7fc4aa7fc640 1 -- 172.21.15.165:0/1774995404 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b4044b00 con 0x558d932fe680 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.359+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/1774995404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d932ffa50 con 0x558d932fe680 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.359+0000 7fc4aa7fc640 1 -- 172.21.15.165:0/1774995404 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5b4044de0 con 0x558d932fe680 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.359+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/1774995404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d932fff50 con 0x558d932fe680 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.360+0000 7fc4aa7fc640 1 -- 172.21.15.165:0/1774995404 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc5b4019860 con 0x558d932fe680 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.360+0000 7fc4aa7fc640 1 --2- 172.21.15.165:0/1774995404 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc49c036950 0x7fc49c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.360+0000 7fc4aa7fc640 1 -- 172.21.15.165:0/1774995404 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fc5b403cae0 con 0x558d932fe680 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.360+0000 7fc5bc23e640 1 --2- 172.21.15.165:0/1774995404 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc49c036950 0x7fc49c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.360+0000 7fc5c8d3e800 1 Processor -- start 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.360+0000 7fc5c8d3e800 1 -- start start 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.360+0000 7fc5bc23e640 1 --2- 172.21.15.165:0/1774995404 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc49c036950 0x7fc49c038e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x558d932e7c90 tx=0x7fc5ac005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.361+0000 7fc5c8d3e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 0x558d9330bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.361+0000 7fc5c8d3e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558d932d8a10 con 0x558d9330b890 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.361+0000 7fc5c4a3f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 0x558d9330bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.361+0000 7fc5c4a3f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 0x558d9330bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52614/0 (socket says 172.21.15.165:52614) 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.361+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/3437028772 learned_addr learned my addr 172.21.15.165:0/3437028772 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.361+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/3437028772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d9330c280 con 0x558d9330b890 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.361+0000 7fc5c4a3f640 1 --2- 172.21.15.165:0/3437028772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 0x558d9330bcb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4019760 tx=0x7fc5b403f8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9bc92660f6f3ec8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.361+0000 7fc49bfff640 1 -- 172.21.15.165:0/3437028772 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5b4055040 con 0x558d9330b890 2024-04-27T03:30:33.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.361+0000 7fc49bfff640 1 -- 172.21.15.165:0/3437028772 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b4044f80 con 0x558d9330b890 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.361+0000 7fc49bfff640 1 -- 172.21.15.165:0/3437028772 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5b4026390 con 0x558d9330b890 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/3437028772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 msgr2=0x558d9330bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/3437028772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 0x558d9330bcb0 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4019760 tx=0x7fc5b403f8c0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/3437028772 shutdown_connections 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/3437028772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 0x558d9330bcb0 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/3437028772 >> 172.21.15.165:0/3437028772 conn(0x558d93307b60 msgr2=0x558d93305bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/3437028772 shutdown_connections 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/3437028772 wait complete. 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c8d3e800 1 Processor -- start 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c8d3e800 1 -- start start 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c8d3e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 0x558d93312bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c8d3e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558d9330c8e0 con 0x558d9330b890 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c4a3f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 0x558d93312bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c4a3f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 0x558d93312bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:52618/0 (socket says 172.21.15.165:52618) 2024-04-27T03:30:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.362+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 learned_addr learned my addr 172.21.15.165:0/176490702 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.363+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d93313110 con 0x558d9330b890 2024-04-27T03:30:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.363+0000 7fc5c4a3f640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 0x558d93312bd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc5b402c650 tx=0x7fc5b4033ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.363+0000 7fc49a7fc640 1 -- 172.21.15.165:0/176490702 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5b402c060 con 0x558d9330b890 2024-04-27T03:30:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.363+0000 7fc49a7fc640 1 -- 172.21.15.165:0/176490702 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc5b402c220 con 0x558d9330b890 2024-04-27T03:30:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.363+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558d93313110 con 0x558d9330b890 2024-04-27T03:30:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.363+0000 7fc49a7fc640 1 -- 172.21.15.165:0/176490702 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc5b404b5b0 con 0x558d9330b890 2024-04-27T03:30:33.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.363+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d93313110 con 0x558d9330b890 2024-04-27T03:30:33.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.364+0000 7fc49a7fc640 1 -- 172.21.15.165:0/176490702 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc5b406d020 con 0x558d9330b890 2024-04-27T03:30:33.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.364+0000 7fc49a7fc640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc490036950 0x7fc490038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.364+0000 7fc49a7fc640 1 -- 172.21.15.165:0/176490702 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fc5b406b020 con 0x558d9330b890 2024-04-27T03:30:33.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.364+0000 7fc5bc23e640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc490036950 0x7fc490038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.364+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.364+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558d9331e150 0x558d93320610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.364+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d93320d60 con 0x558d9331e150 2024-04-27T03:30:33.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.364+0000 7fc5c5240640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558d9331e150 0x558d93320610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.364+0000 7fc5bc23e640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc490036950 0x7fc490038e10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac055040 tx=0x7fc5ac04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.364+0000 7fc5c5240640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558d9331e150 0x558d93320610 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 20 realm 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d93321170 con 0x558d9331e150 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d93321a40 con 0x558d9331e150 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d93321de0 con 0x558d9331e150 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.365+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558d933228d0 0x558d93324cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.366+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d93325420 con 0x558d933228d0 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.366+0000 7fc5c4a3f640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558d933228d0 0x558d93324cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.366+0000 7fc5c4a3f640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558d933228d0 0x558d93324cc0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.366+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.366+0000 7fc5c8d3e800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.366+0000 7fc5c8d3e800 20 searching for the correct realm 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.366+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93326e00 con 0x558d933228d0 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.367+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.367+0000 7fc5c4a3f640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc5b4066c20 0x7fc5b402c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.367+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93326e00 con 0x7fc5b4066c20 2024-04-27T03:30:33.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.367+0000 7fc5bc23e640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc5b4066c20 0x7fc5b402c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.367+0000 7fc5bc23e640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc5b4066c20 0x7fc5b402c6d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.367+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.367+0000 7fc5bc23e640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc5ac074040 0x7fc5ac081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.367+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93326e00 con 0x7fc5ac074040 2024-04-27T03:30:33.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.368+0000 7fc5c5240640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc5ac074040 0x7fc5ac081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.368+0000 7fc5c5240640 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc5ac074040 0x7fc5ac081410 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.368+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.368+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93321de0 con 0x558d9331e150 2024-04-27T03:30:33.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.368+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.368+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93321de0 con 0x558d9331e150 2024-04-27T03:30:33.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.369+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.369+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93321de0 con 0x7fc5ac074040 2024-04-27T03:30:33.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.369+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.369+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93321de0 con 0x7fc5ac074040 2024-04-27T03:30:33.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.369+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.369+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93321de0 con 0x7fc5b4066c20 2024-04-27T03:30:33.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.369+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.369+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93321de0 con 0x558d933228d0 2024-04-27T03:30:33.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.370+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.370+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93321de0 con 0x558d9331e150 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.370+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.370+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93321de0 con 0x7fc5ac074040 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.370+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.370+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93321de0 con 0x558d9331e150 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.371+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.371+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558d93321de0 con 0x7fc5b4066c20 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.371+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.371+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x558d933228d0 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.371+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.371+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x558d933228d0 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.371+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.371+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x7fc5ac074040 2024-04-27T03:30:33.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.372+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.372+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x558d933228d0 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.372+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.372+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x558d933228d0 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.372+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.372+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x558d9331e150 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.372+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.372+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x558d9331e150 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.373+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.373+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x7fc5ac074040 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.373+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.373+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x558d933228d0 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.373+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.373+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x7fc5b4066c20 2024-04-27T03:30:33.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.374+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.374+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x558d9331e150 2024-04-27T03:30:33.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.374+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.374+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x7fc5ac074040 2024-04-27T03:30:33.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.374+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.374+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x558d933228d0 2024-04-27T03:30:33.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.375+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.375+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x7fc5ac074040 2024-04-27T03:30:33.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.375+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.375+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x7fc5b4066c20 2024-04-27T03:30:33.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.375+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.376+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x558d933228d0 2024-04-27T03:30:33.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.376+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.376+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x7fc5ac074040 2024-04-27T03:30:33.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.376+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.376+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5ac08e060 con 0x558d9331e150 2024-04-27T03:30:33.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.376+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.376+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fc5c0038040 con 0x7fc5b4066c20 2024-04-27T03:30:33.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:33.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:33.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:33.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:33.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:33.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d93325dd0 con 0x558d9331e150 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d933261c0 con 0x558d9331e150 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.377+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d933261c0 con 0x7fc5ac074040 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:33.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 20 zone default found 2024-04-27T03:30:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 4 Realm: () 2024-04-27T03:30:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 20 zonegroup default 2024-04-27T03:30:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d933265e0 con 0x7fc5ac074040 2024-04-27T03:30:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d93326a00 con 0x558d933228d0 2024-04-27T03:30:33.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d93328cb0 con 0x558d933228d0 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 20 started sync module instance, tier type = 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.378+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d93328150 con 0x7fc5b4066c20 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.379+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.379+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94066548311408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d93329500 con 0x7fc5b4066c20 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.380+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d93329d50 con 0x558d9331e150 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.380+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94066548311408] v25'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.381+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.381+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94066548314192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d93329db0 con 0x558d9331e150 2024-04-27T03:30:33.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.381+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332a8c0 con 0x558d933228d0 2024-04-27T03:30:33.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.382+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94066548314192] v25'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.382+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.382+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94066548319088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332acb0 con 0x558d933228d0 2024-04-27T03:30:33.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.382+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332b6f0 con 0x7fc5b4066c20 2024-04-27T03:30:33.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.383+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.383+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94066548322752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332bf90 con 0x7fc5b4066c20 2024-04-27T03:30:33.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.383+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332ca90 con 0x558d9331e150 2024-04-27T03:30:33.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.383+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94066548319088] v25'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.384+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.384+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94066548327808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332ce80 con 0x558d9331e150 2024-04-27T03:30:33.406 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.384+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332d8c0 con 0x7fc5ac074040 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.384+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94066548322752] v25'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.385+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.385+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94066548331408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332d880 con 0x7fc5ac074040 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.385+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332e2f0 con 0x7fc5ac074040 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.385+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94066548327808] v25'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.386+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94066548331408] v25'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.386+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.386+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94066548325088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332e2f0 con 0x7fc5ac074040 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.386+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332e760 con 0x7fc5ac074040 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.387+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'84 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.387+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94066548316320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332ebd0 con 0x7fc5ac074040 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.387+0000 7fc5c8d3e800 20 add_watcher() i=0 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.387+0000 7fc5c8d3e800 20 add_watcher() i=1 2024-04-27T03:30:33.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.387+0000 7fc5c8d3e800 20 add_watcher() i=2 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.387+0000 7fc5c8d3e800 20 add_watcher() i=3 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.387+0000 7fc5c8d3e800 20 add_watcher() i=4 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.387+0000 7fc5c8d3e800 20 add_watcher() i=5 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.387+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94066548325088] v25'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.387+0000 7fc5c8d3e800 20 add_watcher() i=6 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.390+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94066548316320] v25'85 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.390+0000 7fc5c8d3e800 20 add_watcher() i=7 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.390+0000 7fc5c8d3e800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.390+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d9332de20 con 0x558d9331e150 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.390+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.390+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94066548336416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332de20 con 0x558d9331e150 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.391+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94066548336416] v25'96 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.391+0000 7fc47afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:33.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc5c8d3e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc5c8d3e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc5c8d3e800 5 note: GC not initialized 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc479fe3640 20 reqs_thread_entry: start 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc5c8d3e800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc5c8d3e800 20 Filter name: none 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc3eb7fe640 20 reqs_thread_entry: start 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d933649f0 con 0x558d9331e150 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc5c8d3e800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc5c8d3e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.392+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d933653e0 con 0x558d9331e150 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.393+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.393+0000 7fc5c8d3e800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.393+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558d933663f0 con 0x7fc5b4066c20 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.393+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 foo.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 155+0+0 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.394+0000 7fc47afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.394+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94066548336416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332f590 con 0x558d9331e150 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.395+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94066548336416] v25'97 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.395+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94066548311408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332e720 con 0x7fc5b4066c20 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.396+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94066548311408] v25'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.396+0000 7fc5c8d3e800 20 remove_watcher() i=0 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.396+0000 7fc5c8d3e800 2 removed watcher, disabling cache 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.396+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94066548314192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332e720 con 0x558d9331e150 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.397+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94066548314192] v25'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.397+0000 7fc5c8d3e800 20 remove_watcher() i=1 2024-04-27T03:30:33.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.397+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94066548319088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d93329050 con 0x558d933228d0 2024-04-27T03:30:33.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.398+0000 7fc5c4a3f640 1 -- 172.21.15.165:0/176490702 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94066548319088] v25'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5b4063040 con 0x558d933228d0 2024-04-27T03:30:33.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.398+0000 7fc5c8d3e800 20 remove_watcher() i=2 2024-04-27T03:30:33.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.398+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94066548322752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332e720 con 0x7fc5b4066c20 2024-04-27T03:30:33.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.399+0000 7fc5bc23e640 1 -- 172.21.15.165:0/176490702 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94066548322752] v25'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5ac07f070 con 0x7fc5b4066c20 2024-04-27T03:30:33.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.399+0000 7fc5c8d3e800 20 remove_watcher() i=3 2024-04-27T03:30:33.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.399+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94066548327808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d93329050 con 0x558d9331e150 2024-04-27T03:30:33.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.400+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94066548327808] v25'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x558d9331e150 2024-04-27T03:30:33.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.401+0000 7fc5c8d3e800 20 remove_watcher() i=4 2024-04-27T03:30:33.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.401+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94066548331408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332e720 con 0x7fc5ac074040 2024-04-27T03:30:33.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.402+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94066548331408] v25'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.402+0000 7fc5c8d3e800 20 remove_watcher() i=5 2024-04-27T03:30:33.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.402+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94066548325088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d93329050 con 0x7fc5ac074040 2024-04-27T03:30:33.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.403+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94066548325088] v25'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.403+0000 7fc5c8d3e800 20 remove_watcher() i=6 2024-04-27T03:30:33.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.403+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94066548316320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558d9332e720 con 0x7fc5ac074040 2024-04-27T03:30:33.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94066548316320] v25'86 uv84 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc5c000a4a0 con 0x7fc5ac074040 2024-04-27T03:30:33.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 20 remove_watcher() i=7 2024-04-27T03:30:33.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc5ac074040 msgr2=0x7fc5ac081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc5ac074040 0x7fc5ac081410 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558d9331e150 msgr2=0x558d93320610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558d9331e150 0x558d93320610 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc5b4066c20 msgr2=0x7fc5b402c6d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc5b4066c20 0x7fc5b402c6d0 crc :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558d933228d0 msgr2=0x558d93324cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558d933228d0 0x558d93324cc0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc490036950 msgr2=0x7fc490038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc490036950 0x7fc490038e10 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac055040 tx=0x7fc5ac04f000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 msgr2=0x558d93312bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/176490702 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d9330b890 0x558d93312bd0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fc5b402c650 tx=0x7fc5b4033ea0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 reap_dead start 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c5240640 1 -- 172.21.15.165:0/176490702 reap_dead start 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 shutdown_connections 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 >> 172.21.15.165:0/176490702 conn(0x558d93307b60 msgr2=0x558d933051c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 shutdown_connections 2024-04-27T03:30:33.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/176490702 wait complete. 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/1774995404 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc49c036950 msgr2=0x7fc49c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/1774995404 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc49c036950 0x7fc49c038e10 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x558d932e7c90 tx=0x7fc5ac005ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/1774995404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d932fe680 msgr2=0x558d932feaa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/1774995404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d932fe680 0x558d932feaa0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc5b404b4d0 tx=0x7fc5b4017e10 comp rx=0 tx=0).stop 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/1774995404 shutdown_connections 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/1774995404 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc49c036950 0x7fc49c038e10 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/1774995404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d932fe680 0x558d932feaa0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.404+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/1774995404 >> 172.21.15.165:0/1774995404 conn(0x558d932f2720 msgr2=0x558d932f2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/1774995404 shutdown_connections 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/1774995404 wait complete. 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558d932ed670 msgr2=0x558d932efac0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558d932ed670 0x558d932efac0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558d931eb4a0 msgr2=0x558d932e6e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558d931eb4a0 0x558d932e6e70 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558d932e96e0 msgr2=0x558d932ebad0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558d932e96e0 0x558d932ebad0 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc4ac036900 msgr2=0x7fc4ac038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc4ac036900 0x7fc4ac038dc0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac054000 tx=0x7fc5ac04e000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 msgr2=0x558d932dfc30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 --2- 172.21.15.165:0/210499868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558d931e7990 0x558d932dfc30 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fc5b400b700 tx=0x7fc5b4000bc0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c5240640 1 -- 172.21.15.165:0/210499868 reap_dead start 2024-04-27T03:30:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 shutdown_connections 2024-04-27T03:30:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 >> 172.21.15.165:0/210499868 conn(0x558d931e1320 msgr2=0x558d932d21e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 shutdown_connections 2024-04-27T03:30:33.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.405+0000 7fc5c8d3e800 1 -- 172.21.15.165:0/210499868 wait complete. 2024-04-27T03:30:33.419 INFO:teuthology.orchestra.run.smithi165.stdout:[] 2024-04-27T03:30:33.419 DEBUG:tasks.util.rgw: json result: [] 2024-04-27T03:30:33.439 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:33.439 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list', '--uid', 'foo'] 2024-04-27T03:30:33.439 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list', '--uid', 'foo'] 2024-04-27T03:30:33.439 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list --uid foo 2024-04-27T03:30:33.502 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:33.502 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:33.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.497+0000 7f7604891800 1 Processor -- start 2024-04-27T03:30:33.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.497+0000 7f7604891800 1 -- start start 2024-04-27T03:30:33.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.498+0000 7f7604891800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x564559bd37a0 0x564559bd3bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.498+0000 7f7604891800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x564559a93d40 con 0x564559bd4100 2024-04-27T03:30:33.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.498+0000 7f7604891800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x564559a664d0 con 0x564559bd37a0 2024-04-27T03:30:33.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.498+0000 7f7602390640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x564559bd37a0 0x564559bd3bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.498+0000 7f7602390640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x564559bd37a0 0x564559bd3bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58234/0 (socket says 172.21.15.165:58234) 2024-04-27T03:30:33.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.498+0000 7f7602390640 1 -- 172.21.15.165:0/2380439034 learned_addr learned my addr 172.21.15.165:0/2380439034 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.498+0000 7f7602390640 1 -- 172.21.15.165:0/2380439034 >> v1:172.21.15.89:6789/0 conn(0x564559bd4100 legacy=0x564559bd45b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.498+0000 7f7602390640 1 -- 172.21.15.165:0/2380439034 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564559a92590 con 0x564559bd37a0 2024-04-27T03:30:33.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.500+0000 7f7602390640 1 --2- 172.21.15.165:0/2380439034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559bd3bc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f75f400a520 tx=0x7f75f401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6b5ddcce115d375 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.500+0000 7f7601b8f640 1 -- 172.21.15.165:0/2380439034 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4057070 con 0x564559bd37a0 2024-04-27T03:30:33.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.500+0000 7f7601b8f640 1 -- 172.21.15.165:0/2380439034 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f4052570 con 0x564559bd37a0 2024-04-27T03:30:33.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.500+0000 7f7601b8f640 1 -- 172.21.15.165:0/2380439034 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4052870 con 0x564559bd37a0 2024-04-27T03:30:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.501+0000 7f7604891800 1 -- 172.21.15.165:0/2380439034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 msgr2=0x564559bd3bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.501+0000 7f7604891800 1 --2- 172.21.15.165:0/2380439034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559bd3bc0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f75f400a520 tx=0x7f75f401d010 comp rx=0 tx=0).stop 2024-04-27T03:30:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.501+0000 7f7604891800 1 -- 172.21.15.165:0/2380439034 shutdown_connections 2024-04-27T03:30:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.501+0000 7f7604891800 1 --2- 172.21.15.165:0/2380439034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559bd3bc0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.501+0000 7f7604891800 1 -- 172.21.15.165:0/2380439034 >> 172.21.15.165:0/2380439034 conn(0x564559bcf300 msgr2=0x564559bd1720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.501+0000 7f7604891800 1 -- 172.21.15.165:0/2380439034 shutdown_connections 2024-04-27T03:30:33.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.501+0000 7f7604891800 1 -- 172.21.15.165:0/2380439034 wait complete. 2024-04-27T03:30:33.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.512+0000 7f7604891800 1 Processor -- start 2024-04-27T03:30:33.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.513+0000 7f7604891800 1 -- start start 2024-04-27T03:30:33.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.513+0000 7f7604891800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559cc8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.513+0000 7f7604891800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564559b203c0 con 0x564559bd37a0 2024-04-27T03:30:33.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.513+0000 7f7602390640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559cc8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.513+0000 7f7602390640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559cc8390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58240/0 (socket says 172.21.15.165:58240) 2024-04-27T03:30:33.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.513+0000 7f7602390640 1 -- 172.21.15.165:0/1540973553 learned_addr learned my addr 172.21.15.165:0/1540973553 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.513+0000 7f7602390640 1 -- 172.21.15.165:0/1540973553 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564559cc8960 con 0x564559bd37a0 2024-04-27T03:30:33.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.514+0000 7f7602390640 1 --2- 172.21.15.165:0/1540973553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559cc8390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f75f401d050 tx=0x7f75f404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbee8c2984b1222b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.514+0000 7f74f0608640 1 -- 172.21.15.165:0/1540973553 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4057040 con 0x564559bd37a0 2024-04-27T03:30:33.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.514+0000 7f74f0608640 1 -- 172.21.15.165:0/1540973553 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f4052850 con 0x564559bd37a0 2024-04-27T03:30:33.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.514+0000 7f74f0608640 1 -- 172.21.15.165:0/1540973553 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4052b70 con 0x564559bd37a0 2024-04-27T03:30:33.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.514+0000 7f7604891800 1 -- 172.21.15.165:0/1540973553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 msgr2=0x564559cc8390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.530 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.514+0000 7f7604891800 1 --2- 172.21.15.165:0/1540973553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559cc8390 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f75f401d050 tx=0x7f75f404c000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.514+0000 7f7604891800 1 -- 172.21.15.165:0/1540973553 shutdown_connections 2024-04-27T03:30:33.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.514+0000 7f7604891800 1 --2- 172.21.15.165:0/1540973553 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559cc8390 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.514+0000 7f7604891800 1 -- 172.21.15.165:0/1540973553 >> 172.21.15.165:0/1540973553 conn(0x564559bcf300 msgr2=0x564559cc3cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.515+0000 7f7604891800 1 -- 172.21.15.165:0/1540973553 shutdown_connections 2024-04-27T03:30:33.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.515+0000 7f7604891800 1 -- 172.21.15.165:0/1540973553 wait complete. 2024-04-27T03:30:33.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.515+0000 7f7604891800 1 Processor -- start 2024-04-27T03:30:33.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.515+0000 7f7604891800 1 -- start start 2024-04-27T03:30:33.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.515+0000 7f7604891800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559ccfdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.515+0000 7f7604891800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564559bd4f60 con 0x564559bd37a0 2024-04-27T03:30:33.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.515+0000 7f7602390640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559ccfdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.515+0000 7f7602390640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559ccfdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58250/0 (socket says 172.21.15.165:58250) 2024-04-27T03:30:33.531 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.515+0000 7f7602390640 1 -- 172.21.15.165:0/3595983450 learned_addr learned my addr 172.21.15.165:0/3595983450 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.515+0000 7f7602390640 1 -- 172.21.15.165:0/3595983450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564559cd0330 con 0x564559bd37a0 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.516+0000 7f7602390640 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559ccfdf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f75f401cfe0 tx=0x7f75f4003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.516+0000 7f74eee05640 1 -- 172.21.15.165:0/3595983450 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4073070 con 0x564559bd37a0 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.516+0000 7f74eee05640 1 -- 172.21.15.165:0/3595983450 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f405d650 con 0x564559bd37a0 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.516+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564559cd05c0 con 0x564559bd37a0 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.516+0000 7f74eee05640 1 -- 172.21.15.165:0/3595983450 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4057070 con 0x564559bd37a0 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.516+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564559cd3220 con 0x564559bd37a0 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.516+0000 7f74eee05640 1 -- 172.21.15.165:0/3595983450 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f75f4076070 con 0x564559bd37a0 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.517+0000 7f74eee05640 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74e8036900 0x7f74e8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.517+0000 7f74eee05640 1 -- 172.21.15.165:0/3595983450 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f75f408dde0 con 0x564559bd37a0 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.517+0000 7f75fbfff640 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74e8036900 0x7f74e8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.517+0000 7f7604891800 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564559bd4100 0x564559cd6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.517+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559cd7490 con 0x564559bd4100 2024-04-27T03:30:33.532 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.517+0000 7f7602b91640 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564559bd4100 0x564559cd6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.517+0000 7f75fbfff640 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74e8036900 0x7f74e8038dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f75ec001e10 tx=0x7f75ec001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.517+0000 7f7602b91640 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564559bd4100 0x564559cd6e70 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.518+0000 7f7602b91640 1 -- 172.21.15.165:0/3595983450 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559bd4100 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.518+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559cd7d70 con 0x564559bd4100 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.518+0000 7f7602b91640 1 -- 172.21.15.165:0/3595983450 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559bd4100 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.518+0000 7f7604891800 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564559cd9680 0x564559cdbb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.518+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559cdc2a0 con 0x564559cd9680 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.518+0000 7f7602390640 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564559cd9680 0x564559cdbb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.518+0000 7f7602390640 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564559cd9680 0x564559cdbb10 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.519+0000 7f7602390640 1 -- 172.21.15.165:0/3595983450 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f75f4075030 con 0x564559cd9680 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.519+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559cd85c0 con 0x564559bd4100 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.519+0000 7f7602b91640 1 -- 172.21.15.165:0/3595983450 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559bd4100 2024-04-27T03:30:33.533 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.519+0000 7f7604891800 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x564559cdd700 0x564559cdfb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.519+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559ce02d0 con 0x564559cdd700 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.519+0000 7f75fbfff640 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x564559cdd700 0x564559cdfb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.519+0000 7f75fbfff640 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x564559cdd700 0x564559cdfb70 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.520+0000 7f75fbfff640 1 -- 172.21.15.165:0/3595983450 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f75ec0106a0 con 0x564559cdd700 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.520+0000 7f7604891800 1 Processor -- start 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.520+0000 7f7604891800 1 -- start start 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.520+0000 7f7604891800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cee730 0x564559ceeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.520+0000 7f7604891800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564559ac66b0 con 0x564559cee730 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.520+0000 7f7602390640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cee730 0x564559ceeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.520+0000 7f7602390640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cee730 0x564559ceeb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58266/0 (socket says 172.21.15.165:58266) 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.520+0000 7f7602390640 1 -- 172.21.15.165:0/72152477 learned_addr learned my addr 172.21.15.165:0/72152477 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.521+0000 7f7602390640 1 -- 172.21.15.165:0/72152477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564559cef090 con 0x564559cee730 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.521+0000 7f7602390640 1 --2- 172.21.15.165:0/72152477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cee730 0x564559ceeb50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f75f405dfd0 tx=0x7f75f4066bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.521+0000 7f74e7fff640 1 -- 172.21.15.165:0/72152477 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4006830 con 0x564559cee730 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.521+0000 7f74e7fff640 1 -- 172.21.15.165:0/72152477 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f4090030 con 0x564559cee730 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.521+0000 7f7604891800 1 -- 172.21.15.165:0/72152477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564559cefb00 con 0x564559cee730 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.521+0000 7f74e7fff640 1 -- 172.21.15.165:0/72152477 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f40773b0 con 0x564559cee730 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.521+0000 7f7604891800 1 -- 172.21.15.165:0/72152477 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564559cf0000 con 0x564559cee730 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.522+0000 7f74e7fff640 1 -- 172.21.15.165:0/72152477 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f75f4077550 con 0x564559cee730 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.522+0000 7f74e7fff640 1 --2- 172.21.15.165:0/72152477 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74dc036950 0x7f74dc038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.522+0000 7f74e7fff640 1 -- 172.21.15.165:0/72152477 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f75f409a850 con 0x564559cee730 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.522+0000 7f75fbfff640 1 --2- 172.21.15.165:0/72152477 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74dc036950 0x7f74dc038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.522+0000 7f7604891800 1 Processor -- start 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.522+0000 7f7604891800 1 -- start start 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.522+0000 7f75fbfff640 1 --2- 172.21.15.165:0/72152477 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74dc036950 0x7f74dc038e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x564559cd44d0 tx=0x7f75ec006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.522+0000 7f7604891800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 0x564559cfbc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.522+0000 7f7604891800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564559cc8c20 con 0x564559cfb820 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.523+0000 7f7602390640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 0x564559cfbc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.523+0000 7f7602390640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 0x564559cfbc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58278/0 (socket says 172.21.15.165:58278) 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.523+0000 7f7602390640 1 -- 172.21.15.165:0/1831219694 learned_addr learned my addr 172.21.15.165:0/1831219694 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.523+0000 7f7602390640 1 -- 172.21.15.165:0/1831219694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564559cfc210 con 0x564559cfb820 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.523+0000 7f7602390640 1 --2- 172.21.15.165:0/1831219694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 0x564559cfbc40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f75f409bfd0 tx=0x7f75f406fdf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28092fe3c629eea0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.523+0000 7f74e5ffb640 1 -- 172.21.15.165:0/1831219694 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f40796b0 con 0x564559cfb820 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.523+0000 7f74e5ffb640 1 -- 172.21.15.165:0/1831219694 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f4089070 con 0x564559cfb820 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.523+0000 7f74e5ffb640 1 -- 172.21.15.165:0/1831219694 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4076360 con 0x564559cfb820 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.523+0000 7f7604891800 1 -- 172.21.15.165:0/1831219694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 msgr2=0x564559cfbc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.523+0000 7f7604891800 1 --2- 172.21.15.165:0/1831219694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 0x564559cfbc40 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f75f409bfd0 tx=0x7f75f406fdf0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7604891800 1 -- 172.21.15.165:0/1831219694 shutdown_connections 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7604891800 1 --2- 172.21.15.165:0/1831219694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 0x564559cfbc40 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7604891800 1 -- 172.21.15.165:0/1831219694 >> 172.21.15.165:0/1831219694 conn(0x564559cf7b80 msgr2=0x564559cf5be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7604891800 1 -- 172.21.15.165:0/1831219694 shutdown_connections 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7604891800 1 -- 172.21.15.165:0/1831219694 wait complete. 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7604891800 1 Processor -- start 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7604891800 1 -- start start 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7604891800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 0x564559d02bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7604891800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564559cfc8c0 con 0x564559cfb820 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7602390640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 0x564559d02bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7602390640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 0x564559d02bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58288/0 (socket says 172.21.15.165:58288) 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 learned_addr learned my addr 172.21.15.165:0/1021406174 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.524+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564559d03130 con 0x564559cfb820 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.525+0000 7f7602390640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 0x564559d02bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f75f407be00 tx=0x7f75f404bdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.525+0000 7f74d3fff640 1 -- 172.21.15.165:0/1021406174 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f406f410 con 0x564559cfb820 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.525+0000 7f74d3fff640 1 -- 172.21.15.165:0/1021406174 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f75f40b5070 con 0x564559cfb820 2024-04-27T03:30:33.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.525+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564559d03130 con 0x564559cfb820 2024-04-27T03:30:33.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.525+0000 7f74d3fff640 1 -- 172.21.15.165:0/1021406174 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f75f4079cc0 con 0x564559cfb820 2024-04-27T03:30:33.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.525+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564559d03130 con 0x564559cfb820 2024-04-27T03:30:33.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.526+0000 7f74d3fff640 1 -- 172.21.15.165:0/1021406174 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f75f40ac040 con 0x564559cfb820 2024-04-27T03:30:33.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.526+0000 7f74d3fff640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74cc036950 0x7f74cc038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.526+0000 7f74d3fff640 1 -- 172.21.15.165:0/1021406174 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f75f40aa020 con 0x564559cfb820 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.526+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.526+0000 7f75fbfff640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74cc036950 0x7f74cc038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.526+0000 7f7604891800 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564559d0e2d0 0x564559d10790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.526+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d10ee0 con 0x564559d0e2d0 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.526+0000 7f7602b91640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564559d0e2d0 0x564559d10790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.526+0000 7f75fbfff640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74cc036950 0x7f74cc038e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x564559ce06d0 tx=0x7f75ec011000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7602b91640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564559d0e2d0 0x564559d10790 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 20 realm 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d112f0 con 0x564559d0e2d0 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d11b90 con 0x564559d0e2d0 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d0e2d0 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564559d12c90 0x564559d15080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.527+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d155c0 con 0x564559d12c90 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.528+0000 7f7602390640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564559d12c90 0x564559d15080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.528+0000 7f7602390640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564559d12c90 0x564559d15080 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.528+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.528+0000 7f7604891800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.528+0000 7f7604891800 20 searching for the correct realm 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.528+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d16f70 con 0x564559d12c90 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.529+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.529+0000 7f7602390640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f75f4078c20 0x7f75f40ba8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.529+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d16f70 con 0x7f75f4078c20 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.529+0000 7f75fbfff640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f75f4078c20 0x7f75f40ba8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.529+0000 7f75fbfff640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f75f4078c20 0x7f75f40ba8f0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.529+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.529+0000 7f75fbfff640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f75ec0393c0 0x7f75ec039810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.529+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d16f70 con 0x7f75ec0393c0 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.530+0000 7f7602b91640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f75ec0393c0 0x7f75ec039810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.530+0000 7f7602b91640 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f75ec0393c0 0x7f75ec039810 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.530+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.530+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d0e2d0 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.530+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.530+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d0e2d0 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.531+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.531+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x7f75ec0393c0 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.531+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.531+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x7f75ec0393c0 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.531+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.531+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x7f75f4078c20 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.531+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.531+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d12c90 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.532+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.532+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d0e2d0 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.532+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.532+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x7f75ec0393c0 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.532+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.532+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d0e2d0 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.532+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.532+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x7f75f4078c20 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.533+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.533+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d12c90 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.533+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.533+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d12c90 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.533+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.533+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x7f75ec0393c0 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.533+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.533+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d12c90 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.534+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.534+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d12c90 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.534+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.534+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d0e2d0 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.534+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.534+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d0e2d0 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.534+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.534+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x7f75ec0393c0 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.535+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.535+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d12c90 2024-04-27T03:30:33.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.535+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.535+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x7f75f4078c20 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.535+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.535+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d0e2d0 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.535+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.535+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x7f75ec0393c0 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.536+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.536+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d12c90 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.536+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.536+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x7f75ec0393c0 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.536+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.536+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x7f75f4078c20 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.537+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.537+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d12c90 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.537+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.537+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x7f75ec0393c0 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.537+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.537+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d0e2d0 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f75fc038040 con 0x7f75f4078c20 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:33.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d11f30 con 0x564559d0e2d0 2024-04-27T03:30:33.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d15f40 con 0x564559d0e2d0 2024-04-27T03:30:33.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:33.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.538+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d15f40 con 0x7f75ec0393c0 2024-04-27T03:30:33.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 20 zone default found 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 4 Realm: () 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 20 zonegroup default 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d16360 con 0x7f75ec0393c0 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d167a0 con 0x564559d12c90 2024-04-27T03:30:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.539+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d18e20 con 0x564559d12c90 2024-04-27T03:30:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.540+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.540+0000 7f7604891800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.540+0000 7f7604891800 20 started sync module instance, tier type = 2024-04-27T03:30:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.540+0000 7f7604891800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.540+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d182c0 con 0x7f75f4078c20 2024-04-27T03:30:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.541+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.541+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94855859633680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d191c0 con 0x7f75f4078c20 2024-04-27T03:30:33.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.541+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d19a10 con 0x564559d0e2d0 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.542+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94855859633680] v25'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.542+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.542+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94855859639648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d19e30 con 0x564559d0e2d0 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.542+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1a7f0 con 0x564559d12c90 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.543+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94855859639648] v25'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.543+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.543+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94855859640704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d19da0 con 0x564559d12c90 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.543+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1b6b0 con 0x7f75f4078c20 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.544+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94855859640704] v25'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.544+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.544+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94855859646976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1ba50 con 0x7f75f4078c20 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.544+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1c470 con 0x564559d0e2d0 2024-04-27T03:30:33.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.545+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.545+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94855859650400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1c430 con 0x564559d0e2d0 2024-04-27T03:30:33.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.545+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94855859646976] v25'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.545+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1ac40 con 0x7f75ec0393c0 2024-04-27T03:30:33.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.546+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.546+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94855859645504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1d3b0 con 0x7f75ec0393c0 2024-04-27T03:30:33.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.546+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1de50 con 0x7f75ec0393c0 2024-04-27T03:30:33.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.546+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94855859650400] v25'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.547+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94855859645504] v25'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.548+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.548+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94855859656992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1cf40 con 0x7f75ec0393c0 2024-04-27T03:30:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.548+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1e4e0 con 0x7f75ec0393c0 2024-04-27T03:30:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.548+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94855859656992] v25'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.549+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'87 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.549+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94855859652624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1e4e0 con 0x7f75ec0393c0 2024-04-27T03:30:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.549+0000 7f7604891800 20 add_watcher() i=0 2024-04-27T03:30:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.549+0000 7f7604891800 20 add_watcher() i=1 2024-04-27T03:30:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.549+0000 7f7604891800 20 add_watcher() i=2 2024-04-27T03:30:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.549+0000 7f7604891800 20 add_watcher() i=3 2024-04-27T03:30:33.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.549+0000 7f7604891800 20 add_watcher() i=4 2024-04-27T03:30:33.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.549+0000 7f7604891800 20 add_watcher() i=5 2024-04-27T03:30:33.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.549+0000 7f7604891800 20 add_watcher() i=6 2024-04-27T03:30:33.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.550+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94855859652624] v25'88 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.550+0000 7f7604891800 20 add_watcher() i=7 2024-04-27T03:30:33.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.550+0000 7f7604891800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:33.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.550+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d21670 con 0x564559d0e2d0 2024-04-27T03:30:33.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.550+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.550+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94855859658800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1e900 con 0x564559d0e2d0 2024-04-27T03:30:33.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.551+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94855859658800] v25'98 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.551+0000 7f74b8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:33.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.551+0000 7f7604891800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:33.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.551+0000 7f7604891800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.551+0000 7f7604891800 5 note: GC not initialized 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.551+0000 7f74337fe640 20 reqs_thread_entry: start 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.551+0000 7f7604891800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.551+0000 7f7604891800 20 Filter name: none 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.552+0000 7f74317fa640 20 reqs_thread_entry: start 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.552+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.552+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d54580 con 0x564559d0e2d0 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.552+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.552+0000 7f7604891800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.552+0000 7f7604891800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.552+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d54f70 con 0x564559d0e2d0 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.552+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.552+0000 7f7604891800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.552+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x564559d55f80 con 0x7f75f4078c20 2024-04-27T03:30:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.553+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 foo.buckets [call out=183b] v0'0 uv2 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.553+0000 7f74b8fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.554+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94855859658800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d20a80 con 0x564559d0e2d0 2024-04-27T03:30:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.554+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94855859658800] v25'99 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.555+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94855859633680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1e030 con 0x7f75f4078c20 2024-04-27T03:30:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.555+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94855859633680] v25'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.555+0000 7f7604891800 20 remove_watcher() i=0 2024-04-27T03:30:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.555+0000 7f7604891800 2 removed watcher, disabling cache 2024-04-27T03:30:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.555+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94855859639648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1e030 con 0x564559d0e2d0 2024-04-27T03:30:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.556+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94855859639648] v25'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.556+0000 7f7604891800 20 remove_watcher() i=1 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.556+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94855859640704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d19560 con 0x564559d12c90 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.558+0000 7f7602390640 1 -- 172.21.15.165:0/1021406174 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94855859640704] v25'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75f4075030 con 0x564559d12c90 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.558+0000 7f7604891800 20 remove_watcher() i=2 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.558+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94855859646976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1e030 con 0x7f75f4078c20 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.559+0000 7f75fbfff640 1 -- 172.21.15.165:0/1021406174 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94855859646976] v25'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75ec036040 con 0x7f75f4078c20 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.559+0000 7f7604891800 20 remove_watcher() i=3 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.559+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94855859650400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1b200 con 0x564559d0e2d0 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.560+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94855859650400] v25'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x564559d0e2d0 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.560+0000 7f7604891800 20 remove_watcher() i=4 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.560+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94855859645504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1e030 con 0x7f75ec0393c0 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.561+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94855859645504] v25'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.561+0000 7f7604891800 20 remove_watcher() i=5 2024-04-27T03:30:33.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.561+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94855859656992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1ac40 con 0x7f75ec0393c0 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.562+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94855859656992] v25'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.562+0000 7f7604891800 20 remove_watcher() i=6 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.562+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94855859652624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x564559d1ac40 con 0x7f75ec0393c0 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94855859652624] v25'89 uv87 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f75fc00a4a0 con 0x7f75ec0393c0 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 20 remove_watcher() i=7 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f75ec0393c0 msgr2=0x7f75ec039810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f75ec0393c0 0x7f75ec039810 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564559d0e2d0 msgr2=0x564559d10790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564559d0e2d0 0x564559d10790 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f75f4078c20 msgr2=0x7f75f40ba8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f75f4078c20 0x7f75f40ba8f0 crc :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564559d12c90 msgr2=0x564559d15080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564559d12c90 0x564559d15080 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74cc036950 msgr2=0x7f74cc038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74cc036950 0x7f74cc038e10 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x564559ce06d0 tx=0x7f75ec011000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 msgr2=0x564559d02bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7602b91640 1 -- 172.21.15.165:0/1021406174 reap_dead start 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 0x564559d02bf0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f75f407be00 tx=0x7f75f404bdb0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 shutdown_connections 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 --2- 172.21.15.165:0/1021406174 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cfb820 0x564559d02bf0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 >> 172.21.15.165:0/1021406174 conn(0x564559cf7b80 msgr2=0x564559cf51e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 shutdown_connections 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/1021406174 wait complete. 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/72152477 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74dc036950 msgr2=0x7f74dc038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 --2- 172.21.15.165:0/72152477 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74dc036950 0x7f74dc038e10 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x564559cd44d0 tx=0x7f75ec006000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/72152477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cee730 msgr2=0x564559ceeb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 --2- 172.21.15.165:0/72152477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cee730 0x564559ceeb50 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f75f405dfd0 tx=0x7f75f4066bd0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/72152477 shutdown_connections 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 --2- 172.21.15.165:0/72152477 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74dc036950 0x7f74dc038e10 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 --2- 172.21.15.165:0/72152477 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559cee730 0x564559ceeb50 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/72152477 >> 172.21.15.165:0/72152477 conn(0x564559ce27d0 msgr2=0x564559ce2bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/72152477 shutdown_connections 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.563+0000 7f7604891800 1 -- 172.21.15.165:0/72152477 wait complete. 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x564559cdd700 msgr2=0x564559cdfb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x564559cdd700 0x564559cdfb70 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564559bd4100 msgr2=0x564559cd6e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564559bd4100 0x564559cd6e70 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564559cd9680 msgr2=0x564559cdbb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564559cd9680 0x564559cdbb10 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74e8036900 msgr2=0x7f74e8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f74e8036900 0x7f74e8038dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f75ec001e10 tx=0x7f75ec001390 comp rx=0 tx=0).stop 2024-04-27T03:30:33.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 msgr2=0x564559ccfdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 --2- 172.21.15.165:0/3595983450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564559bd37a0 0x564559ccfdf0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f75f401cfe0 tx=0x7f75f4003020 comp rx=0 tx=0).stop 2024-04-27T03:30:33.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7602b91640 1 -- 172.21.15.165:0/3595983450 reap_dead start 2024-04-27T03:30:33.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 shutdown_connections 2024-04-27T03:30:33.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 >> 172.21.15.165:0/3595983450 conn(0x564559bcf300 msgr2=0x564559cc24b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 shutdown_connections 2024-04-27T03:30:33.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.564+0000 7f7604891800 1 -- 172.21.15.165:0/3595983450 wait complete. 2024-04-27T03:30:33.583 INFO:teuthology.orchestra.run.smithi165.stdout:["myfoo"] 2024-04-27T03:30:33.583 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-04-27T03:30:33.587 INFO:tasks.radosgw_admin:RL: list_buckets foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:33.587 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'list'] 2024-04-27T03:30:33.588 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'list'] 2024-04-27T03:30:33.588 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket list 2024-04-27T03:30:33.664 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:33.664 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:33.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.659+0000 7f222472e800 1 Processor -- start 2024-04-27T03:30:33.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.659+0000 7f222472e800 1 -- start start 2024-04-27T03:30:33.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.659+0000 7f222472e800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b0e45023b0 0x55b0e45047a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.659+0000 7f222472e800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55b0e43bcd40 con 0x55b0e44fe8a0 2024-04-27T03:30:33.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.659+0000 7f222472e800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55b0e438f4d0 con 0x55b0e45023b0 2024-04-27T03:30:33.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.660+0000 7f2217c66640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b0e45023b0 0x55b0e45047a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.660+0000 7f2217c66640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b0e45023b0 0x55b0e45047a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58302/0 (socket says 172.21.15.165:58302) 2024-04-27T03:30:33.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.660+0000 7f2220467640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55b0e44fe8a0 0x55b0e4500c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35278/0 (socket says 172.21.15.165:35278) 2024-04-27T03:30:33.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.660+0000 7f2217c66640 1 -- 172.21.15.165:0/1176342152 learned_addr learned my addr 172.21.15.165:0/1176342152 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.660+0000 7f2217c66640 1 -- 172.21.15.165:0/1176342152 >> v1:172.21.15.89:6789/0 conn(0x55b0e44fe8a0 legacy=0x55b0e4500c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.660+0000 7f2217c66640 1 -- 172.21.15.165:0/1176342152 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e43bb590 con 0x55b0e45023b0 2024-04-27T03:30:33.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.662+0000 7f2217c66640 1 --2- 172.21.15.165:0/1176342152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e45023b0 0x55b0e45047a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f220800a520 tx=0x7f220801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38281e20c3045571 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.662+0000 7f221fc66640 1 -- 172.21.15.165:0/1176342152 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2208057070 con 0x55b0e45023b0 2024-04-27T03:30:33.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.662+0000 7f221fc66640 1 -- 172.21.15.165:0/1176342152 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2208052570 con 0x55b0e45023b0 2024-04-27T03:30:33.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.662+0000 7f221fc66640 1 -- 172.21.15.165:0/1176342152 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f220804ae60 con 0x55b0e45023b0 2024-04-27T03:30:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.663+0000 7f222472e800 1 -- 172.21.15.165:0/1176342152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e45023b0 msgr2=0x55b0e45047a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.663+0000 7f222472e800 1 --2- 172.21.15.165:0/1176342152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e45023b0 0x55b0e45047a0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f220800a520 tx=0x7f220801d010 comp rx=0 tx=0).stop 2024-04-27T03:30:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.663+0000 7f222472e800 1 -- 172.21.15.165:0/1176342152 shutdown_connections 2024-04-27T03:30:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.663+0000 7f222472e800 1 --2- 172.21.15.165:0/1176342152 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e45023b0 0x55b0e45047a0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.663+0000 7f222472e800 1 -- 172.21.15.165:0/1176342152 >> 172.21.15.165:0/1176342152 conn(0x55b0e44f8370 msgr2=0x55b0e44fa790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.663+0000 7f222472e800 1 -- 172.21.15.165:0/1176342152 shutdown_connections 2024-04-27T03:30:33.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.663+0000 7f222472e800 1 -- 172.21.15.165:0/1176342152 wait complete. 2024-04-27T03:30:33.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.674+0000 7f222472e800 1 Processor -- start 2024-04-27T03:30:33.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.674+0000 7f222472e800 1 -- start start 2024-04-27T03:30:33.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.675+0000 7f222472e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 0x55b0e44fe1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.675+0000 7f222472e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b0e44493b0 con 0x55b0e44fe8a0 2024-04-27T03:30:33.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.675+0000 7f2220467640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 0x55b0e44fe1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.675+0000 7f2220467640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 0x55b0e44fe1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58306/0 (socket says 172.21.15.165:58306) 2024-04-27T03:30:33.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.675+0000 7f2220467640 1 -- 172.21.15.165:0/3235705491 learned_addr learned my addr 172.21.15.165:0/3235705491 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.675+0000 7f2220467640 1 -- 172.21.15.165:0/3235705491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e44fc7a0 con 0x55b0e44fe8a0 2024-04-27T03:30:33.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.675+0000 7f2220467640 1 --2- 172.21.15.165:0/3235705491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 0x55b0e44fe1e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2210001e10 tx=0x7f221000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ebed6166ab9c3f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.675+0000 7f210e60c640 1 -- 172.21.15.165:0/3235705491 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221000e5c0 con 0x55b0e44fe8a0 2024-04-27T03:30:33.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.676+0000 7f210e60c640 1 -- 172.21.15.165:0/3235705491 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2210006070 con 0x55b0e44fe8a0 2024-04-27T03:30:33.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.676+0000 7f210e60c640 1 -- 172.21.15.165:0/3235705491 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221000e930 con 0x55b0e44fe8a0 2024-04-27T03:30:33.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.676+0000 7f222472e800 1 -- 172.21.15.165:0/3235705491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 msgr2=0x55b0e44fe1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.676+0000 7f222472e800 1 --2- 172.21.15.165:0/3235705491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 0x55b0e44fe1e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f2210001e10 tx=0x7f221000b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.676+0000 7f222472e800 1 -- 172.21.15.165:0/3235705491 shutdown_connections 2024-04-27T03:30:33.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.676+0000 7f222472e800 1 --2- 172.21.15.165:0/3235705491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 0x55b0e44fe1e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.676+0000 7f222472e800 1 -- 172.21.15.165:0/3235705491 >> 172.21.15.165:0/3235705491 conn(0x55b0e44f8370 msgr2=0x55b0e45ea980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.676+0000 7f222472e800 1 -- 172.21.15.165:0/3235705491 shutdown_connections 2024-04-27T03:30:33.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.676+0000 7f222472e800 1 -- 172.21.15.165:0/3235705491 wait complete. 2024-04-27T03:30:33.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.677+0000 7f222472e800 1 Processor -- start 2024-04-27T03:30:33.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.677+0000 7f222472e800 1 -- start start 2024-04-27T03:30:33.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.677+0000 7f222472e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 0x55b0e45f8db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.677+0000 7f222472e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b0e4504fe0 con 0x55b0e44fe8a0 2024-04-27T03:30:33.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.677+0000 7f2220467640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 0x55b0e45f8db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.677+0000 7f2220467640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 0x55b0e45f8db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58316/0 (socket says 172.21.15.165:58316) 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.677+0000 7f2220467640 1 -- 172.21.15.165:0/1978165020 learned_addr learned my addr 172.21.15.165:0/1978165020 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.677+0000 7f2220467640 1 -- 172.21.15.165:0/1978165020 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e45f92f0 con 0x55b0e44fe8a0 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.677+0000 7f2220467640 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 0x55b0e45f8db0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f221000b700 tx=0x7f2210000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.678+0000 7f210ce09640 1 -- 172.21.15.165:0/1978165020 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221000f8d0 con 0x55b0e44fe8a0 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.678+0000 7f210ce09640 1 -- 172.21.15.165:0/1978165020 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221000fa70 con 0x55b0e44fe8a0 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.678+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b0e45f9580 con 0x55b0e44fe8a0 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.678+0000 7f210ce09640 1 -- 172.21.15.165:0/1978165020 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221000fd50 con 0x55b0e44fe8a0 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.678+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e45fc230 con 0x55b0e44fe8a0 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.678+0000 7f210ce09640 1 -- 172.21.15.165:0/1978165020 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2210017070 con 0x55b0e44fe8a0 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.679+0000 7f210ce09640 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2108036900 0x7f2108038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.679+0000 7f210ce09640 1 -- 172.21.15.165:0/1978165020 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2210015020 con 0x55b0e44fe8a0 2024-04-27T03:30:33.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.679+0000 7f2217c66640 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2108036900 0x7f2108038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.679+0000 7f222472e800 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b0e45023b0 0x55b0e45ffff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.679+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e46005c0 con 0x55b0e45023b0 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.679+0000 7f2220c68640 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b0e45023b0 0x55b0e45ffff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.679+0000 7f2217c66640 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2108036900 0x7f2108038dc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2208054000 tx=0x7f220804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.679+0000 7f2220c68640 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b0e45023b0 0x55b0e45ffff0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.679+0000 7f2220c68640 1 -- 172.21.15.165:0/1978165020 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e45023b0 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.679+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e45fd5c0 con 0x55b0e45023b0 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.680+0000 7f2220c68640 1 -- 172.21.15.165:0/1978165020 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e45023b0 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.680+0000 7f222472e800 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b0e46028d0 0x55b0e4604cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.680+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4605360 con 0x55b0e46028d0 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.680+0000 7f2220467640 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b0e46028d0 0x55b0e4604cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.680+0000 7f2220467640 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b0e46028d0 0x55b0e4604cc0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.680+0000 7f2220467640 1 -- 172.21.15.165:0/1978165020 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2210046040 con 0x55b0e46028d0 2024-04-27T03:30:33.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.680+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4601880 con 0x55b0e45023b0 2024-04-27T03:30:33.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.681+0000 7f2220c68640 1 -- 172.21.15.165:0/1978165020 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e45023b0 2024-04-27T03:30:33.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.681+0000 7f222472e800 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b0e4606810 0x55b0e4608c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.681+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e46093c0 con 0x55b0e4606810 2024-04-27T03:30:33.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.681+0000 7f2217c66640 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b0e4606810 0x55b0e4608c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.681+0000 7f2217c66640 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b0e4606810 0x55b0e4608c60 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.682+0000 7f2217c66640 1 -- 172.21.15.165:0/1978165020 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f220805a040 con 0x55b0e4606810 2024-04-27T03:30:33.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.682+0000 7f222472e800 1 Processor -- start 2024-04-27T03:30:33.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.682+0000 7f222472e800 1 -- start start 2024-04-27T03:30:33.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.682+0000 7f222472e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4617820 0x55b0e4617c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.682+0000 7f222472e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b0e43ef6b0 con 0x55b0e4617820 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.682+0000 7f2220467640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4617820 0x55b0e4617c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.682+0000 7f2220467640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4617820 0x55b0e4617c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58318/0 (socket says 172.21.15.165:58318) 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.682+0000 7f2220467640 1 -- 172.21.15.165:0/2105362495 learned_addr learned my addr 172.21.15.165:0/2105362495 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.683+0000 7f2220467640 1 -- 172.21.15.165:0/2105362495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e4618180 con 0x55b0e4617820 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.683+0000 7f2220467640 1 --2- 172.21.15.165:0/2105362495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4617820 0x55b0e4617c40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f221004b4d0 tx=0x7f2210017d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.683+0000 7f2105ffb640 1 -- 172.21.15.165:0/2105362495 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221001ae10 con 0x55b0e4617820 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.683+0000 7f2105ffb640 1 -- 172.21.15.165:0/2105362495 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f221001a690 con 0x55b0e4617820 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.683+0000 7f222472e800 1 -- 172.21.15.165:0/2105362495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b0e4618bf0 con 0x55b0e4617820 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.683+0000 7f2105ffb640 1 -- 172.21.15.165:0/2105362495 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2210044b00 con 0x55b0e4617820 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.683+0000 7f222472e800 1 -- 172.21.15.165:0/2105362495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e46190f0 con 0x55b0e4617820 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.684+0000 7f2105ffb640 1 -- 172.21.15.165:0/2105362495 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2210044ca0 con 0x55b0e4617820 2024-04-27T03:30:33.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.684+0000 7f2105ffb640 1 --2- 172.21.15.165:0/2105362495 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f20f8036950 0x7f20f8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.684+0000 7f2105ffb640 1 -- 172.21.15.165:0/2105362495 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f221003cae0 con 0x55b0e4617820 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.684+0000 7f2217c66640 1 --2- 172.21.15.165:0/2105362495 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f20f8036950 0x7f20f8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.684+0000 7f222472e800 1 Processor -- start 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.684+0000 7f222472e800 1 -- start start 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.684+0000 7f2217c66640 1 --2- 172.21.15.165:0/2105362495 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f20f8036950 0x7f20f8038e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55b0e45fd4e0 tx=0x7f2208005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.684+0000 7f222472e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 0x55b0e4624d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.684+0000 7f222472e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b0e44fcb20 con 0x55b0e4624940 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.685+0000 7f2220467640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 0x55b0e4624d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.685+0000 7f2220467640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 0x55b0e4624d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58324/0 (socket says 172.21.15.165:58324) 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.685+0000 7f2220467640 1 -- 172.21.15.165:0/1703683659 learned_addr learned my addr 172.21.15.165:0/1703683659 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.685+0000 7f2220467640 1 -- 172.21.15.165:0/1703683659 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e4625330 con 0x55b0e4624940 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.685+0000 7f2220467640 1 --2- 172.21.15.165:0/1703683659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 0x55b0e4624d60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f221000eb60 tx=0x7f2210026ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235bb68f26ee99e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.685+0000 7f20ff7fe640 1 -- 172.21.15.165:0/1703683659 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2210060070 con 0x55b0e4624940 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.685+0000 7f20ff7fe640 1 -- 172.21.15.165:0/1703683659 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2210044f90 con 0x55b0e4624940 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.685+0000 7f20ff7fe640 1 -- 172.21.15.165:0/1703683659 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2210055040 con 0x55b0e4624940 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f222472e800 1 -- 172.21.15.165:0/1703683659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 msgr2=0x55b0e4624d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f222472e800 1 --2- 172.21.15.165:0/1703683659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 0x55b0e4624d60 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f221000eb60 tx=0x7f2210026ae0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f222472e800 1 -- 172.21.15.165:0/1703683659 shutdown_connections 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f222472e800 1 --2- 172.21.15.165:0/1703683659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 0x55b0e4624d60 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f222472e800 1 -- 172.21.15.165:0/1703683659 >> 172.21.15.165:0/1703683659 conn(0x55b0e4620ca0 msgr2=0x55b0e461ed00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f222472e800 1 -- 172.21.15.165:0/1703683659 shutdown_connections 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f222472e800 1 -- 172.21.15.165:0/1703683659 wait complete. 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f222472e800 1 Processor -- start 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f222472e800 1 -- start start 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f222472e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 0x55b0e462bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f222472e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b0e46259e0 con 0x55b0e4624940 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f2220467640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 0x55b0e462bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f2220467640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 0x55b0e462bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58336/0 (socket says 172.21.15.165:58336) 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.686+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 learned_addr learned my addr 172.21.15.165:0/3369931289 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.687+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0e462c290 con 0x55b0e4624940 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.687+0000 7f2220467640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 0x55b0e462bd50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2210026d90 tx=0x7f221001a970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.687+0000 7f20fdffb640 1 -- 172.21.15.165:0/3369931289 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2210060030 con 0x55b0e4624940 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.687+0000 7f20fdffb640 1 -- 172.21.15.165:0/3369931289 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2210019990 con 0x55b0e4624940 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.687+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b0e462c290 con 0x55b0e4624940 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.687+0000 7f20fdffb640 1 -- 172.21.15.165:0/3369931289 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f221004bc90 con 0x55b0e4624940 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.687+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0e462c290 con 0x55b0e4624940 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.688+0000 7f20fdffb640 1 -- 172.21.15.165:0/3369931289 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f221006e020 con 0x55b0e4624940 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.688+0000 7f20fdffb640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f20f0036950 0x7f20f0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.688+0000 7f222472e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.688+0000 7f20fdffb640 1 -- 172.21.15.165:0/3369931289 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f221006c020 con 0x55b0e4624940 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.688+0000 7f2217c66640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f20f0036950 0x7f20f0038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.688+0000 7f222472e800 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b0e4637340 0x55b0e46397e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.688+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4639f30 con 0x55b0e4637340 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.688+0000 7f2220c68640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b0e4637340 0x55b0e46397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f2220c68640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b0e4637340 0x55b0e46397e0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f2217c66640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f20f0036950 0x7f20f0038e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2208055040 tx=0x7f220804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f222472e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f222472e800 20 realm 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f222472e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463a340 con 0x55b0e4637340 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f222472e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f222472e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f222472e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463abe0 con 0x55b0e4637340 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f222472e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f222472e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463af80 con 0x55b0e4637340 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.689+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.690+0000 7f222472e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.690+0000 7f222472e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.690+0000 7f222472e800 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b0e463bce0 0x55b0e463e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.690+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463e610 con 0x55b0e463bce0 2024-04-27T03:30:33.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.690+0000 7f2220467640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b0e463bce0 0x55b0e463e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.690+0000 7f2220467640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b0e463bce0 0x55b0e463e0d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.690+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.690+0000 7f222472e800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.690+0000 7f222472e800 20 searching for the correct realm 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.690+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463ffc0 con 0x55b0e463bce0 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.691+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.691+0000 7f2220467640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f221000f070 0x7f221003c050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.691+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463ffc0 con 0x7f221000f070 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.691+0000 7f2217c66640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f221000f070 0x7f221003c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.691+0000 7f2217c66640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f221000f070 0x7f221003c050 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.691+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.691+0000 7f2217c66640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2208074040 0x7f2208081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.691+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463ffc0 con 0x7f2208074040 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.692+0000 7f2220c68640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2208074040 0x7f2208081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.692+0000 7f2220c68640 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2208074040 0x7f2208081410 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.692+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.692+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463af80 con 0x55b0e4637340 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.692+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.692+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463af80 con 0x55b0e4637340 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.693+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.693+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463af80 con 0x7f2208074040 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.693+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.693+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463af80 con 0x7f2208074040 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.693+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.693+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463af80 con 0x7f221000f070 2024-04-27T03:30:33.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.693+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.693+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463af80 con 0x55b0e463bce0 2024-04-27T03:30:33.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.694+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.694+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463af80 con 0x55b0e4637340 2024-04-27T03:30:33.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.694+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.694+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463af80 con 0x7f2208074040 2024-04-27T03:30:33.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.694+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.694+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463af80 con 0x55b0e4637340 2024-04-27T03:30:33.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.694+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.694+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463af80 con 0x7f221000f070 2024-04-27T03:30:33.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.695+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.695+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x55b0e463bce0 2024-04-27T03:30:33.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.695+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.695+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x55b0e463bce0 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.695+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.695+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x7f2208074040 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.695+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.695+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x55b0e463bce0 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.696+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.696+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x55b0e463bce0 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.696+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.696+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x55b0e4637340 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.696+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.696+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x55b0e4637340 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.696+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.696+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x7f2208074040 2024-04-27T03:30:33.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.697+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.697+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x55b0e463bce0 2024-04-27T03:30:33.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.697+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.697+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x7f221000f070 2024-04-27T03:30:33.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.697+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.697+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x55b0e4637340 2024-04-27T03:30:33.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.698+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.698+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x7f2208074040 2024-04-27T03:30:33.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.698+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.698+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x55b0e463bce0 2024-04-27T03:30:33.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.698+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.698+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x7f2208074040 2024-04-27T03:30:33.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.698+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.698+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x7f221000f070 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.699+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.699+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x55b0e463bce0 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.699+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.699+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x7f2208074040 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.699+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.699+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f220808e060 con 0x55b0e4637340 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.699+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.699+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2218038040 con 0x7f221000f070 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:33.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463ef90 con 0x55b0e4637340 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463f380 con 0x55b0e4637340 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.700+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463f380 con 0x7f2208074040 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 20 zone default found 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 4 Realm: () 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 20 zonegroup default 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463f7a0 con 0x7f2208074040 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463f7a0 con 0x55b0e463bce0 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4641e70 con 0x55b0e463bce0 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.701+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.702+0000 7f222472e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.702+0000 7f222472e800 20 started sync module instance, tier type = 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.702+0000 7f222472e800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.702+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4641310 con 0x7f221000f070 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.703+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.703+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94218234369792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4640f70 con 0x7f221000f070 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.703+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4642ce0 con 0x55b0e4637340 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.704+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94218234369792] v25'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.704+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.704+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94218234374192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4643100 con 0x55b0e4637340 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.704+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4643ac0 con 0x55b0e463bce0 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.705+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94218234374192] v25'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.705+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.705+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94218234375248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4643070 con 0x55b0e463bce0 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.705+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4644980 con 0x7f221000f070 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.706+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.706+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94218234381520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f221000f070 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.706+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94218234375248] v25'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.706+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e46457e0 con 0x55b0e4637340 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.707+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94218234381520] v25'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.707+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.707+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94218234379952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645bd0 con 0x55b0e4637340 2024-04-27T03:30:33.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.707+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e46465d0 con 0x7f2208074040 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.708+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94218234379952] v25'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.708+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.708+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94218234383728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4646590 con 0x7f2208074040 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.708+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4647010 con 0x7f2208074040 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.709+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94218234383728] v25'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.709+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.709+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94218234388704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e46471f0 con 0x7f2208074040 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.709+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4647a40 con 0x7f2208074040 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.710+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94218234388704] v25'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.710+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'90 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.710+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94218234392352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e46478b0 con 0x7f2208074040 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.710+0000 7f222472e800 20 add_watcher() i=0 2024-04-27T03:30:33.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.710+0000 7f222472e800 20 add_watcher() i=1 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.710+0000 7f222472e800 20 add_watcher() i=2 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.710+0000 7f222472e800 20 add_watcher() i=3 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.710+0000 7f222472e800 20 add_watcher() i=4 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.710+0000 7f222472e800 20 add_watcher() i=5 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.710+0000 7f222472e800 20 add_watcher() i=6 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.711+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94218234392352] v25'91 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.711+0000 7f222472e800 20 add_watcher() i=7 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.711+0000 7f222472e800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.711+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b0e464adf0 con 0x55b0e4637340 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.711+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.711+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94218234393408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4648310 con 0x55b0e4637340 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.712+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94218234393408] v25'100 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.712+0000 7f20d67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:33.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.712+0000 7f222472e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.712+0000 7f222472e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.713+0000 7f222472e800 5 note: GC not initialized 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.713+0000 7f20d57e2640 20 reqs_thread_entry: start 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.713+0000 7f222472e800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.713+0000 7f222472e800 20 Filter name: none 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.713+0000 7f2046ffd640 20 reqs_thread_entry: start 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.713+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:62 8.0 8:00000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e467d8e0 con 0x55b0e463bce0 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.714+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(62 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.714+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:63 8.10 8:08000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4643ac0 con 0x55b0e463bce0 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.714+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(63 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.714+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.8 8:10000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4643ac0 con 0x55b0e4637340 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.714+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(64 [pgnls start_epoch 25 out=49b] v25'2 uv2 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.714+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 8.18 8:18000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463ef90 con 0x55b0e4637340 2024-04-27T03:30:33.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.714+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 [pgnls start_epoch 25 out=49b] v25'11 uv11 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.714+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 8.4 8:20000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463ef90 con 0x7f221000f070 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.714+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.714+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 8.14 8:28000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e463bce0 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.715+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(67 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.715+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 8.c 8:30000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e4637340 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.715+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.715+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 8.1c 8:38000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e463bce0 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.715+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(69 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.715+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 8.2 8:40000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f221000f070 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.715+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 [pgnls start_epoch 25 out=49b] v25'2 uv2 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.715+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 8.12 8:48000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e463bce0 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.716+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(71 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.716+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 8.a 8:50000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e4637340 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.716+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(72 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.716+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 8.1a 8:58000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f2208074040 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.716+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.716+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 8.6 8:60000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f2208074040 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.717+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.717+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:75 8.16 8:68000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e463bce0 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.717+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(75 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.717+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 8.e 8:70000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f2208074040 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.717+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(76 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.717+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 8.1e 8:78000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f2208074040 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.717+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(77 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.717+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 8.1 8:80000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f2208074040 2024-04-27T03:30:33.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.718+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(78 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.718+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 8.11 8:88000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e463bce0 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.718+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(79 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.718+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:80 8.9 8:90000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f221000f070 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.718+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(80 [pgnls start_epoch 25 out=70b] v25'2 uv2 ondisk = 1) ==== 144+0+70 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.718+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 8.19 8:98000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e4637340 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.719+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(81 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.719+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:82 8.5 8:a0000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e4637340 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.719+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(82 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.719+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:83 8.15 8:a8000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e463bce0 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.719+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(83 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.719+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:84 8.d 8:b0000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f221000f070 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.719+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(84 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.719+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:85 8.1d 8:b8000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f2208074040 2024-04-27T03:30:33.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.720+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(85 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.720+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:86 8.3 8:c0000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e4637340 2024-04-27T03:30:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.720+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(86 [pgnls start_epoch 25 out=127b] v25'2 uv2 ondisk = 1) ==== 144+0+127 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.720+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:87 8.13 8:c8000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f2208074040 2024-04-27T03:30:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.720+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(87 [pgnls start_epoch 25 out=49b] v25'2 uv2 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.720+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:88 8.b 8:d0000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e463bce0 2024-04-27T03:30:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.720+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(88 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.720+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:89 8.1b 8:d8000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f221000f070 2024-04-27T03:30:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.721+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(89 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.721+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 8.7 8:e0000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f2208074040 2024-04-27T03:30:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.721+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(90 [pgnls start_epoch 25 out=49b] v25'1 uv1 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.721+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:91 8.17 8:e8000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e4637340 2024-04-27T03:30:33.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.721+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(91 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.721+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:92 8.f 8:f0000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x55b0e463bce0 2024-04-27T03:30:33.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.721+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(92 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.721+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 8.1f 8:f8000000:root:::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4645220 con 0x7f221000f070 2024-04-27T03:30:33.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.722+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(93 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.722+0000 7f222472e800 20 RGWRados::pool_iterate: got myfoo 2024-04-27T03:30:33.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.722+0000 7f222472e800 20 RGWRados::pool_iterate: got .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 2024-04-27T03:30:33.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.722+0000 7f20d67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:33.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.722+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94218234393408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4643ac0 con 0x55b0e4637340 2024-04-27T03:30:33.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.723+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94218234393408] v25'101 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.723+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94218234369792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463ef90 con 0x7f221000f070 2024-04-27T03:30:33.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.724+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94218234369792] v25'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.724+0000 7f222472e800 20 remove_watcher() i=0 2024-04-27T03:30:33.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.724+0000 7f222472e800 2 removed watcher, disabling cache 2024-04-27T03:30:33.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.724+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94218234374192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463ef90 con 0x55b0e4637340 2024-04-27T03:30:33.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.725+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94218234374192] v25'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.725+0000 7f222472e800 20 remove_watcher() i=1 2024-04-27T03:30:33.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.725+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94218234375248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e463ef30 con 0x55b0e463bce0 2024-04-27T03:30:33.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.726+0000 7f2220467640 1 -- 172.21.15.165:0/3369931289 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94218234375248] v25'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2210058040 con 0x55b0e463bce0 2024-04-27T03:30:33.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.726+0000 7f222472e800 20 remove_watcher() i=2 2024-04-27T03:30:33.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.727+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94218234381520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4641700 con 0x7f221000f070 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.727+0000 7f2217c66640 1 -- 172.21.15.165:0/3369931289 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94218234381520] v25'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f220807f070 con 0x7f221000f070 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.727+0000 7f222472e800 20 remove_watcher() i=3 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.727+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94218234379952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4641700 con 0x55b0e4637340 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.728+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94218234379952] v25'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x55b0e4637340 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.728+0000 7f222472e800 20 remove_watcher() i=4 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.729+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94218234383728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4641700 con 0x7f2208074040 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.729+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94218234383728] v25'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.729+0000 7f222472e800 20 remove_watcher() i=5 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.729+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94218234388704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e4644d70 con 0x7f2208074040 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.730+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94218234388704] v25'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.730+0000 7f222472e800 20 remove_watcher() i=6 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.730+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94218234392352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b0e46460e0 con 0x7f2208074040 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.731+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94218234392352] v25'92 uv90 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f221800a4a0 con 0x7f2208074040 2024-04-27T03:30:33.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.731+0000 7f222472e800 20 remove_watcher() i=7 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.731+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2208074040 msgr2=0x7f2208081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.731+0000 7f222472e800 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2208074040 0x7f2208081410 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.731+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b0e4637340 msgr2=0x55b0e46397e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.731+0000 7f222472e800 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b0e4637340 0x55b0e46397e0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.731+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f221000f070 msgr2=0x7f221003c050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.731+0000 7f222472e800 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f221000f070 0x7f221003c050 crc :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.731+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b0e463bce0 msgr2=0x55b0e463e0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.731+0000 7f222472e800 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b0e463bce0 0x55b0e463e0d0 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f20f0036950 msgr2=0x7f20f0038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f20f0036950 0x7f20f0038e10 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f2208055040 tx=0x7f220804f000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 msgr2=0x55b0e462bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f2220c68640 1 -- 172.21.15.165:0/3369931289 reap_dead start 2024-04-27T03:30:33.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 0x55b0e462bd50 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f2210026d90 tx=0x7f221001a970 comp rx=0 tx=0).stop 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 shutdown_connections 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 --2- 172.21.15.165:0/3369931289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4624940 0x55b0e462bd50 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 >> 172.21.15.165:0/3369931289 conn(0x55b0e4620ca0 msgr2=0x55b0e461e300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 shutdown_connections 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/3369931289 wait complete. 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/2105362495 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f20f8036950 msgr2=0x7f20f8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 --2- 172.21.15.165:0/2105362495 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f20f8036950 0x7f20f8038e10 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x55b0e45fd4e0 tx=0x7f2208005ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/2105362495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4617820 msgr2=0x55b0e4617c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 --2- 172.21.15.165:0/2105362495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4617820 0x55b0e4617c40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f221004b4d0 tx=0x7f2210017d50 comp rx=0 tx=0).stop 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/2105362495 shutdown_connections 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 --2- 172.21.15.165:0/2105362495 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f20f8036950 0x7f20f8038e10 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 --2- 172.21.15.165:0/2105362495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e4617820 0x55b0e4617c40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/2105362495 >> 172.21.15.165:0/2105362495 conn(0x55b0e460b8c0 msgr2=0x55b0e460bcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/2105362495 shutdown_connections 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/2105362495 wait complete. 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b0e4606810 msgr2=0x55b0e4608c60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b0e4606810 0x55b0e4608c60 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b0e45023b0 msgr2=0x55b0e45ffff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b0e45023b0 0x55b0e45ffff0 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b0e46028d0 msgr2=0x55b0e4604cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.732+0000 7f222472e800 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b0e46028d0 0x55b0e4604cc0 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.733+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2108036900 msgr2=0x7f2108038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.733+0000 7f222472e800 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2108036900 0x7f2108038dc0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f2208054000 tx=0x7f220804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.733+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 msgr2=0x55b0e45f8db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.733+0000 7f222472e800 1 --2- 172.21.15.165:0/1978165020 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b0e44fe8a0 0x55b0e45f8db0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f221000b700 tx=0x7f2210000bc0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.733+0000 7f2220c68640 1 -- 172.21.15.165:0/1978165020 reap_dead start 2024-04-27T03:30:33.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.733+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 shutdown_connections 2024-04-27T03:30:33.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.733+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 >> 172.21.15.165:0/1978165020 conn(0x55b0e44f8370 msgr2=0x55b0e44fd080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.733+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 shutdown_connections 2024-04-27T03:30:33.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.733+0000 7f222472e800 1 -- 172.21.15.165:0/1978165020 wait complete. 2024-04-27T03:30:33.756 INFO:teuthology.orchestra.run.smithi165.stdout:["myfoo"] 2024-04-27T03:30:33.756 DEBUG:tasks.util.rgw: json result: ['myfoo'] 2024-04-27T03:30:33.766 INFO:tasks.radosgw_admin:RL: create_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:33.779 INFO:tasks.radosgw_admin:RL: create_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:33.791 INFO:tasks.radosgw_admin:RL: create_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:33.835 INFO:tasks.radosgw_admin:RL: create_bucket myfoo5 foo bytes_out=0 bytes_in=237 failed=True 2024-04-27T03:30:33.866 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo2 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:33.898 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo3 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:33.926 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo4 foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:33.926 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T03:30:33.927 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T03:30:33.927 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-04-27T03:30:33.969 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:33.969 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:33.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.964+0000 7fb92dc36800 1 Processor -- start 2024-04-27T03:30:33.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.964+0000 7fb92dc36800 1 -- start start 2024-04-27T03:30:33.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.965+0000 7fb92dc36800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55f7015d61b0 0x55f7015d85a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.965+0000 7fb92dc36800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55f701491d40 con 0x55f7015d1d20 2024-04-27T03:30:33.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.965+0000 7fb92dc36800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55f7014644d0 con 0x55f7015d61b0 2024-04-27T03:30:33.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.965+0000 7fb92af34640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55f7015d61b0 0x55f7015d85a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.965+0000 7fb92b735640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55f7015d1d20 0x55f7015d2140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35290/0 (socket says 172.21.15.165:35290) 2024-04-27T03:30:33.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.965+0000 7fb92b735640 1 -- 172.21.15.165:0/1565178970 learned_addr learned my addr 172.21.15.165:0/1565178970 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:33.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.965+0000 7fb92af34640 1 -- 172.21.15.165:0/1565178970 >> v1:172.21.15.89:6789/0 conn(0x55f7015d1d20 legacy=0x55f7015d2140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.965+0000 7fb92af34640 1 -- 172.21.15.165:0/1565178970 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f701490590 con 0x55f7015d61b0 2024-04-27T03:30:33.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.967+0000 7fb92af34640 1 --2- 172.21.15.165:0/1565178970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d61b0 0x55f7015d85a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb92000a470 tx=0x7fb92001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46ed376fe3f16414 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.967+0000 7fb92a733640 1 -- 172.21.15.165:0/1565178970 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb920057070 con 0x55f7015d61b0 2024-04-27T03:30:33.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.967+0000 7fb92a733640 1 -- 172.21.15.165:0/1565178970 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb920052570 con 0x55f7015d61b0 2024-04-27T03:30:33.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.967+0000 7fb92a733640 1 -- 172.21.15.165:0/1565178970 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb920052870 con 0x55f7015d61b0 2024-04-27T03:30:33.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.968+0000 7fb92dc36800 1 -- 172.21.15.165:0/1565178970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d61b0 msgr2=0x55f7015d85a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.968+0000 7fb92dc36800 1 --2- 172.21.15.165:0/1565178970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d61b0 0x55f7015d85a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fb92000a470 tx=0x7fb92001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:33.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.968+0000 7fb92dc36800 1 -- 172.21.15.165:0/1565178970 shutdown_connections 2024-04-27T03:30:33.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.968+0000 7fb92dc36800 1 --2- 172.21.15.165:0/1565178970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d61b0 0x55f7015d85a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.968+0000 7fb92dc36800 1 -- 172.21.15.165:0/1565178970 >> 172.21.15.165:0/1565178970 conn(0x55f7015cd380 msgr2=0x55f7015cf7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.968+0000 7fb92dc36800 1 -- 172.21.15.165:0/1565178970 shutdown_connections 2024-04-27T03:30:33.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.968+0000 7fb92dc36800 1 -- 172.21.15.165:0/1565178970 wait complete. 2024-04-27T03:30:33.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.979+0000 7fb92dc36800 1 Processor -- start 2024-04-27T03:30:33.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.979+0000 7fb92dc36800 1 -- start start 2024-04-27T03:30:33.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.980+0000 7fb92dc36800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 0x55f7016c1ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.980+0000 7fb92dc36800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f70151e3c0 con 0x55f7015d1d20 2024-04-27T03:30:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.980+0000 7fb92b735640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 0x55f7016c1ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.980+0000 7fb92b735640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 0x55f7016c1ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58358/0 (socket says 172.21.15.165:58358) 2024-04-27T03:30:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.980+0000 7fb92b735640 1 -- 172.21.15.165:0/304327719 learned_addr learned my addr 172.21.15.165:0/304327719 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.980+0000 7fb92b735640 1 -- 172.21.15.165:0/304327719 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7016c24b0 con 0x55f7015d1d20 2024-04-27T03:30:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.980+0000 7fb92b735640 1 --2- 172.21.15.165:0/304327719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 0x55f7016c1ee0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb91c001e10 tx=0x7fb91c00b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4064913413921b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.981+0000 7fb81960a640 1 -- 172.21.15.165:0/304327719 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c012020 con 0x55f7015d1d20 2024-04-27T03:30:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.981+0000 7fb81960a640 1 -- 172.21.15.165:0/304327719 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c00e5a0 con 0x55f7015d1d20 2024-04-27T03:30:33.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.981+0000 7fb81960a640 1 -- 172.21.15.165:0/304327719 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c00e8e0 con 0x55f7015d1d20 2024-04-27T03:30:33.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.981+0000 7fb92dc36800 1 -- 172.21.15.165:0/304327719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 msgr2=0x55f7016c1ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:33.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.981+0000 7fb92dc36800 1 --2- 172.21.15.165:0/304327719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 0x55f7016c1ee0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fb91c001e10 tx=0x7fb91c00b780 comp rx=0 tx=0).stop 2024-04-27T03:30:33.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.981+0000 7fb92dc36800 1 -- 172.21.15.165:0/304327719 shutdown_connections 2024-04-27T03:30:33.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.981+0000 7fb92dc36800 1 --2- 172.21.15.165:0/304327719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 0x55f7016c1ee0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:33.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.981+0000 7fb92dc36800 1 -- 172.21.15.165:0/304327719 >> 172.21.15.165:0/304327719 conn(0x55f7015cd380 msgr2=0x55f7016bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:33.994 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.981+0000 7fb92dc36800 1 -- 172.21.15.165:0/304327719 shutdown_connections 2024-04-27T03:30:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.981+0000 7fb92dc36800 1 -- 172.21.15.165:0/304327719 wait complete. 2024-04-27T03:30:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.982+0000 7fb92dc36800 1 Processor -- start 2024-04-27T03:30:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.982+0000 7fb92dc36800 1 -- start start 2024-04-27T03:30:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.982+0000 7fb92dc36800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 0x55f7016cdeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.982+0000 7fb92dc36800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7015d8e70 con 0x55f7015d1d20 2024-04-27T03:30:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.982+0000 7fb92b735640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 0x55f7016cdeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.982+0000 7fb92b735640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 0x55f7016cdeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58374/0 (socket says 172.21.15.165:58374) 2024-04-27T03:30:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.982+0000 7fb92b735640 1 -- 172.21.15.165:0/3509509269 learned_addr learned my addr 172.21.15.165:0/3509509269 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.982+0000 7fb92b735640 1 -- 172.21.15.165:0/3509509269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7016ce3f0 con 0x55f7015d1d20 2024-04-27T03:30:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.983+0000 7fb92b735640 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 0x55f7016cdeb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb91c001e10 tx=0x7fb91c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.983+0000 7fb817e07640 1 -- 172.21.15.165:0/3509509269 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c012030 con 0x55f7015d1d20 2024-04-27T03:30:33.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.983+0000 7fb817e07640 1 -- 172.21.15.165:0/3509509269 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c01a7a0 con 0x55f7015d1d20 2024-04-27T03:30:33.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.983+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f7016ce680 con 0x55f7015d1d20 2024-04-27T03:30:33.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.983+0000 7fb817e07640 1 -- 172.21.15.165:0/3509509269 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c01aac0 con 0x55f7015d1d20 2024-04-27T03:30:33.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.983+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7016d1330 con 0x55f7015d1d20 2024-04-27T03:30:33.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.984+0000 7fb817e07640 1 -- 172.21.15.165:0/3509509269 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fb91c015020 con 0x55f7015d1d20 2024-04-27T03:30:33.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.984+0000 7fb817e07640 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb810036950 0x7fb810038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.984+0000 7fb817e07640 1 -- 172.21.15.165:0/3509509269 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fb91c020030 con 0x55f7015d1d20 2024-04-27T03:30:33.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.984+0000 7fb92af34640 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb810036950 0x7fb810038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.984+0000 7fb92dc36800 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7015d61b0 0x55f7016d50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.984+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f7016d5630 con 0x55f7015d61b0 2024-04-27T03:30:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.984+0000 7fb92bf36640 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7015d61b0 0x55f7016d50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.984+0000 7fb92af34640 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb810036950 0x7fb810038e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb920054000 tx=0x7fb92004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.984+0000 7fb92bf36640 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7015d61b0 0x55f7016d50f0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.985+0000 7fb92bf36640 1 -- 172.21.15.165:0/3509509269 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f7015d61b0 2024-04-27T03:30:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.985+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f7016d26c0 con 0x55f7015d61b0 2024-04-27T03:30:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.985+0000 7fb92bf36640 1 -- 172.21.15.165:0/3509509269 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f7015d61b0 2024-04-27T03:30:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.985+0000 7fb92dc36800 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7016d7960 0x55f7016d9d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.985+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f7016da440 con 0x55f7016d7960 2024-04-27T03:30:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.985+0000 7fb92b735640 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7016d7960 0x55f7016d9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.985+0000 7fb92b735640 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7016d7960 0x55f7016d9d50 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.986+0000 7fb92b735640 1 -- 172.21.15.165:0/3509509269 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fb91c018950 con 0x55f7016d7960 2024-04-27T03:30:33.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.986+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f7016d6910 con 0x55f7015d61b0 2024-04-27T03:30:33.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.986+0000 7fb92bf36640 1 -- 172.21.15.165:0/3509509269 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f7015d61b0 2024-04-27T03:30:33.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.986+0000 7fb92dc36800 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55f7016db8a0 0x55f7016ddd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.986+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f7016de4c0 con 0x55f7016db8a0 2024-04-27T03:30:33.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.986+0000 7fb92af34640 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55f7016db8a0 0x55f7016ddd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.987+0000 7fb92af34640 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55f7016db8a0 0x55f7016ddd30 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.987+0000 7fb92af34640 1 -- 172.21.15.165:0/3509509269 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fb920052940 con 0x55f7016db8a0 2024-04-27T03:30:33.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.987+0000 7fb92dc36800 1 Processor -- start 2024-04-27T03:30:33.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.987+0000 7fb92dc36800 1 -- start start 2024-04-27T03:30:33.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.987+0000 7fb92dc36800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016ec8b0 0x55f7016eccd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:33.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.987+0000 7fb92dc36800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7014c46b0 con 0x55f7016ec8b0 2024-04-27T03:30:33.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.988+0000 7fb92b735640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016ec8b0 0x55f7016eccd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:33.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.988+0000 7fb92b735640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016ec8b0 0x55f7016eccd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58384/0 (socket says 172.21.15.165:58384) 2024-04-27T03:30:33.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.988+0000 7fb92b735640 1 -- 172.21.15.165:0/892426781 learned_addr learned my addr 172.21.15.165:0/892426781 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:33.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.988+0000 7fb92b735640 1 -- 172.21.15.165:0/892426781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7016ed210 con 0x55f7016ec8b0 2024-04-27T03:30:33.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.988+0000 7fb92b735640 1 --2- 172.21.15.165:0/892426781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016ec8b0 0x55f7016eccd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb91c01a6f0 tx=0x7fb91c017e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:33.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.988+0000 7fb815602640 1 -- 172.21.15.165:0/892426781 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c042040 con 0x55f7016ec8b0 2024-04-27T03:30:33.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.988+0000 7fb815602640 1 -- 172.21.15.165:0/892426781 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c04c1a0 con 0x55f7016ec8b0 2024-04-27T03:30:33.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.988+0000 7fb92dc36800 1 -- 172.21.15.165:0/892426781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f7016edc80 con 0x55f7016ec8b0 2024-04-27T03:30:33.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.988+0000 7fb815602640 1 -- 172.21.15.165:0/892426781 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c04c4c0 con 0x55f7016ec8b0 2024-04-27T03:30:33.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.988+0000 7fb92dc36800 1 -- 172.21.15.165:0/892426781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7016ee180 con 0x55f7016ec8b0 2024-04-27T03:30:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.989+0000 7fb815602640 1 -- 172.21.15.165:0/892426781 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fb91c044280 con 0x55f7016ec8b0 2024-04-27T03:30:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.989+0000 7fb815602640 1 --2- 172.21.15.165:0/892426781 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb804036a50 0x7fb804038f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.989+0000 7fb815602640 1 -- 172.21.15.165:0/892426781 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fb91c003c00 con 0x55f7016ec8b0 2024-04-27T03:30:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.989+0000 7fb92af34640 1 --2- 172.21.15.165:0/892426781 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb804036a50 0x7fb804038f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.989+0000 7fb92dc36800 1 Processor -- start 2024-04-27T03:30:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.989+0000 7fb92dc36800 1 -- start start 2024-04-27T03:30:34.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.990+0000 7fb92af34640 1 --2- 172.21.15.165:0/892426781 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb804036a50 0x7fb804038f10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55f7016d5f10 tx=0x7fb920005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.990+0000 7fb92dc36800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 0x55f7016f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.990+0000 7fb92dc36800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7016c2830 con 0x55f7016f9a50 2024-04-27T03:30:34.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.990+0000 7fb92b735640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 0x55f7016f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.990+0000 7fb92b735640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 0x55f7016f9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58398/0 (socket says 172.21.15.165:58398) 2024-04-27T03:30:34.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.990+0000 7fb92b735640 1 -- 172.21.15.165:0/2519508812 learned_addr learned my addr 172.21.15.165:0/2519508812 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.990+0000 7fb92b735640 1 -- 172.21.15.165:0/2519508812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7016fa440 con 0x55f7016f9a50 2024-04-27T03:30:34.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.990+0000 7fb92b735640 1 --2- 172.21.15.165:0/2519508812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 0x55f7016f9e70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb91c026040 tx=0x7fb91c034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3459dd331e60131 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.990+0000 7fb80effd640 1 -- 172.21.15.165:0/2519508812 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c05d070 con 0x55f7016f9a50 2024-04-27T03:30:34.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.990+0000 7fb80effd640 1 -- 172.21.15.165:0/2519508812 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c049710 con 0x55f7016f9a50 2024-04-27T03:30:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.990+0000 7fb80effd640 1 -- 172.21.15.165:0/2519508812 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c050070 con 0x55f7016f9a50 2024-04-27T03:30:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.991+0000 7fb92dc36800 1 -- 172.21.15.165:0/2519508812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 msgr2=0x55f7016f9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.991+0000 7fb92dc36800 1 --2- 172.21.15.165:0/2519508812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 0x55f7016f9e70 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fb91c026040 tx=0x7fb91c034670 comp rx=0 tx=0).stop 2024-04-27T03:30:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.991+0000 7fb92dc36800 1 -- 172.21.15.165:0/2519508812 shutdown_connections 2024-04-27T03:30:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.991+0000 7fb92dc36800 1 --2- 172.21.15.165:0/2519508812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 0x55f7016f9e70 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.991+0000 7fb92dc36800 1 -- 172.21.15.165:0/2519508812 >> 172.21.15.165:0/2519508812 conn(0x55f7016f5d70 msgr2=0x55f7016f3dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.991+0000 7fb92dc36800 1 -- 172.21.15.165:0/2519508812 shutdown_connections 2024-04-27T03:30:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.991+0000 7fb92dc36800 1 -- 172.21.15.165:0/2519508812 wait complete. 2024-04-27T03:30:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.991+0000 7fb92dc36800 1 Processor -- start 2024-04-27T03:30:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.991+0000 7fb92dc36800 1 -- start start 2024-04-27T03:30:34.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.991+0000 7fb92dc36800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 0x55f701700e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.991+0000 7fb92dc36800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7016fab60 con 0x55f7016f9a50 2024-04-27T03:30:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.992+0000 7fb92b735640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 0x55f701700e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.992+0000 7fb92b735640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 0x55f701700e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58402/0 (socket says 172.21.15.165:58402) 2024-04-27T03:30:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.992+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 learned_addr learned my addr 172.21.15.165:0/2857024199 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.992+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f701701390 con 0x55f7016f9a50 2024-04-27T03:30:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.992+0000 7fb92b735640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 0x55f701700e50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb91c034ae0 tx=0x7fb91c003880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.992+0000 7fb80d7fa640 1 -- 172.21.15.165:0/2857024199 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c05d040 con 0x55f7016f9a50 2024-04-27T03:30:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.992+0000 7fb80d7fa640 1 -- 172.21.15.165:0/2857024199 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fb91c034ba0 con 0x55f7016f9a50 2024-04-27T03:30:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.992+0000 7fb80d7fa640 1 -- 172.21.15.165:0/2857024199 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fb91c053040 con 0x55f7016f9a50 2024-04-27T03:30:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.992+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f701701390 con 0x55f7016f9a50 2024-04-27T03:30:34.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.992+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f701701390 con 0x55f7016f9a50 2024-04-27T03:30:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.993+0000 7fb80d7fa640 1 -- 172.21.15.165:0/2857024199 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fb91c06b020 con 0x55f7016f9a50 2024-04-27T03:30:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.993+0000 7fb80d7fa640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb7f4036950 0x7fb7f4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.993+0000 7fb80d7fa640 1 -- 172.21.15.165:0/2857024199 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fb91c069020 con 0x55f7016f9a50 2024-04-27T03:30:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.993+0000 7fb92af34640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb7f4036950 0x7fb7f4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.993+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.993+0000 7fb92dc36800 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f70170c510 0x55f70170e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.993+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f70170f120 con 0x55f70170c510 2024-04-27T03:30:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92bf36640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f70170c510 0x55f70170e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92af34640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb7f4036950 0x7fb7f4038e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb920052fd0 tx=0x7fb92004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92bf36640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f70170c510 0x55f70170e9d0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f70170c510 2024-04-27T03:30:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92dc36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92dc36800 20 realm 2024-04-27T03:30:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f70170f530 con 0x55f70170c510 2024-04-27T03:30:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f70170c510 2024-04-27T03:30:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92dc36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92dc36800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.994+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f70170fdd0 con 0x55f70170c510 2024-04-27T03:30:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.995+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f70170c510 2024-04-27T03:30:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.995+0000 7fb92dc36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.995+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.995+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701710170 con 0x55f70170c510 2024-04-27T03:30:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.995+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f70170c510 2024-04-27T03:30:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.995+0000 7fb92dc36800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.995+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.995+0000 7fb92dc36800 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f701710ed0 0x55f7017132c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.995+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701713840 con 0x55f701710ed0 2024-04-27T03:30:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.995+0000 7fb92b735640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f701710ed0 0x55f7017132c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.995+0000 7fb92b735640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f701710ed0 0x55f7017132c0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.996+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.996+0000 7fb92dc36800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.996+0000 7fb92dc36800 20 searching for the correct realm 2024-04-27T03:30:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.996+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f7017151c0 con 0x55f701710ed0 2024-04-27T03:30:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.996+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.996+0000 7fb92b735640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fb91c017050 0x7fb91c0332e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.996+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f7017151c0 con 0x7fb91c017050 2024-04-27T03:30:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.996+0000 7fb92af34640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fb91c017050 0x7fb91c0332e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.996+0000 7fb92af34640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fb91c017050 0x7fb91c0332e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.997+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb920081070 con 0x7fb91c017050 2024-04-27T03:30:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.997+0000 7fb92af34640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fb920076040 0x7fb920083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.997+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f7017151c0 con 0x7fb920076040 2024-04-27T03:30:34.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.997+0000 7fb92bf36640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fb920076040 0x7fb920083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.997+0000 7fb92bf36640 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fb920076040 0x7fb920083410 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.998+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fb92400a4a0 con 0x7fb920076040 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.998+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f701710170 con 0x55f70170c510 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.998+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f70170c510 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.998+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f701710170 con 0x55f70170c510 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.998+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f70170c510 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.998+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f701710170 con 0x7fb920076040 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.998+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb92400a4a0 con 0x7fb920076040 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.998+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f701710170 con 0x7fb920076040 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.999+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb92400a4a0 con 0x7fb920076040 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.999+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f701710170 con 0x7fb91c017050 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.999+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb920081070 con 0x7fb91c017050 2024-04-27T03:30:34.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.999+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f701710170 con 0x55f701710ed0 2024-04-27T03:30:34.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.999+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.999+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f701710170 con 0x55f70170c510 2024-04-27T03:30:34.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.999+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f70170c510 2024-04-27T03:30:34.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:33.999+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f701710170 con 0x7fb920076040 2024-04-27T03:30:34.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.000+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb92400a4a0 con 0x7fb920076040 2024-04-27T03:30:34.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.000+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f701710170 con 0x55f70170c510 2024-04-27T03:30:34.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.000+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f70170c510 2024-04-27T03:30:34.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.000+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55f701710170 con 0x7fb91c017050 2024-04-27T03:30:34.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.000+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fb920081070 con 0x7fb91c017050 2024-04-27T03:30:34.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.000+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x55f701710ed0 2024-04-27T03:30:34.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.000+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.000+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x55f701710ed0 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.001+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.001+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x7fb920076040 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.001+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb92400a4a0 con 0x7fb920076040 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.001+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x55f701710ed0 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.001+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.001+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x55f701710ed0 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.001+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.001+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x55f70170c510 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.002+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f70170c510 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.002+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x55f70170c510 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.002+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fb92400a4a0 con 0x55f70170c510 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.002+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x7fb920076040 2024-04-27T03:30:34.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.002+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb924033040 con 0x7fb920076040 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.002+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x55f701710ed0 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.002+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.002+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x7fb91c017050 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.003+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb920081070 con 0x7fb91c017050 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.003+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x55f70170c510 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.003+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.003+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x7fb920076040 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.003+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.003+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x55f701710ed0 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.003+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.003+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x7fb920076040 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.004+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.004+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x7fb91c017050 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.004+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb920081070 con 0x7fb91c017050 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.004+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb920090060 con 0x55f701710ed0 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.004+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.004+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb91c08a040 con 0x7fb920076040 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.004+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.004+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb91c08a040 con 0x55f70170c510 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fb91c08a040 con 0x7fb91c017050 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fb920081070 con 0x7fb91c017050 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:34.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701714190 con 0x55f70170c510 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701714580 con 0x55f70170c510 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.005+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701714580 con 0x7fb920076040 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 20 zone default found 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 4 Realm: () 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 20 zonegroup default 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f7017149a0 con 0x7fb920076040 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701714dc0 con 0x55f701710ed0 2024-04-27T03:30:34.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.006+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701717070 con 0x55f701710ed0 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.007+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.007+0000 7fb92dc36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.007+0000 7fb92dc36800 20 started sync module instance, tier type = 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.007+0000 7fb92dc36800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.007+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f701716510 con 0x7fb91c017050 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.008+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb920081070 con 0x7fb91c017050 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.008+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94519369490528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f701717410 con 0x7fb91c017050 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.008+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f701717dd0 con 0x55f70170c510 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.009+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94519369490528] v25'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb920081070 con 0x7fb91c017050 2024-04-27T03:30:34.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.009+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.009+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94519369496864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f7017181f0 con 0x55f70170c510 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.009+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f701718bf0 con 0x55f701710ed0 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.010+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94519369496864] v25'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.010+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.010+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94519369497920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f701718160 con 0x55f701710ed0 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.010+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f701719ab0 con 0x7fb91c017050 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.011+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94519369497920] v25'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.011+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb920081070 con 0x7fb91c017050 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.011+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94519369504256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f701719e50 con 0x7fb91c017050 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.011+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70171a810 con 0x55f70170c510 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.012+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94519369504256] v25'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb920081070 con 0x7fb91c017050 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.013+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.013+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94519369507680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70171a780 con 0x55f70170c510 2024-04-27T03:30:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.013+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70171b6d0 con 0x7fb920076040 2024-04-27T03:30:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.014+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94519369507680] v25'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.014+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.014+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94519369509808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70171b6d0 con 0x7fb920076040 2024-04-27T03:30:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.014+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70171c0e0 con 0x7fb920076040 2024-04-27T03:30:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.014+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94519369509808] v25'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.015+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.015+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94519369511456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70171c2c0 con 0x7fb920076040 2024-04-27T03:30:34.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.015+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70171cb10 con 0x7fb920076040 2024-04-27T03:30:34.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.016+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94519369511456] v25'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.016+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'93 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.016+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94519369514992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70171c960 con 0x7fb920076040 2024-04-27T03:30:34.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.016+0000 7fb92dc36800 20 add_watcher() i=0 2024-04-27T03:30:34.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.016+0000 7fb92dc36800 20 add_watcher() i=1 2024-04-27T03:30:34.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.016+0000 7fb92dc36800 20 add_watcher() i=2 2024-04-27T03:30:34.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.016+0000 7fb92dc36800 20 add_watcher() i=3 2024-04-27T03:30:34.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.016+0000 7fb92dc36800 20 add_watcher() i=4 2024-04-27T03:30:34.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.016+0000 7fb92dc36800 20 add_watcher() i=5 2024-04-27T03:30:34.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.016+0000 7fb92dc36800 20 add_watcher() i=6 2024-04-27T03:30:34.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.018+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94519369514992] v25'94 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.018+0000 7fb92dc36800 20 add_watcher() i=7 2024-04-27T03:30:34.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.018+0000 7fb92dc36800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:34.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.018+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f70171fea0 con 0x55f70170c510 2024-04-27T03:30:34.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.019+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.019+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94519369516048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70171d3c0 con 0x55f70170c510 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.020+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94519369516048] v25'102 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.020+0000 7fb7e1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.020+0000 7fb92dc36800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.020+0000 7fb92dc36800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.020+0000 7fb92dc36800 5 note: GC not initialized 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.020+0000 7fb7e0fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.020+0000 7fb92dc36800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.020+0000 7fb92dc36800 20 Filter name: none 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.020+0000 7fb75a7fc640 20 reqs_thread_entry: start 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.020+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.020+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701753250 con 0x7fb91c017050 2024-04-27T03:30:34.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.021+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fb920081070 con 0x7fb91c017050 2024-04-27T03:30:34.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.021+0000 7fb92dc36800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:34.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.021+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.021+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701753ae0 con 0x55f70170c510 2024-04-27T03:30:34.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.021+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(63 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.021+0000 7fb92dc36800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:34.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.021+0000 7fb92dc36800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T03:30:34.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.021+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.021+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f7017525f0 con 0x7fb91c017050 2024-04-27T03:30:34.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fb920081070 con 0x7fb91c017050 2024-04-27T03:30:34.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:34.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701755110 con 0x55f70170c510 2024-04-27T03:30:34.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:34.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T03:30:34.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 4.2f 4:f7b16f5a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701756b10 con 0x55f70170c510 2024-04-27T03:30:34.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 4.16 4:6b5ddb47:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f7017596d0 con 0x55f701710ed0 2024-04-27T03:30:34.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:68 4.7 4:e2bbac01:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f70175c310 con 0x7fb920076040 2024-04-27T03:30:34.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 4.3e 4:7e0d8a50:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f70175ec00 con 0x7fb91c017050 2024-04-27T03:30:34.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:70 4.19 4:9b87498d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f70175c310 con 0x55f701710ed0 2024-04-27T03:30:34.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 4.2 4:42daa0e0:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701763b20 con 0x55f70170c510 2024-04-27T03:30:34.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 4.1b 4:d9d9fd94:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701763b60 con 0x7fb91c017050 2024-04-27T03:30:34.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:73 4.e 4:70dfcb91:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701768f50 con 0x7fb91c017050 2024-04-27T03:30:34.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.022+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:74 4.3d 4:bd27934b:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f70176bde0 con 0x55f701710ed0 2024-04-27T03:30:34.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(69 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fb920095040 con 0x7fb91c017050 2024-04-27T03:30:34.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(72 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fb920095040 con 0x7fb91c017050 2024-04-27T03:30:34.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:75 4.5 4:a1c15390:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f701768f50 con 0x55f701710ed0 2024-04-27T03:30:34.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(68 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:76 4.1a 4:59a985c3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55f70176c750 con 0x7fb91c017050 2024-04-27T03:30:34.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(73 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fb920095040 con 0x7fb91c017050 2024-04-27T03:30:34.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(71 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(67 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(70 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(74 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(76 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fb920095040 con 0x7fb91c017050 2024-04-27T03:30:34.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.023+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(75 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.024+0000 7fb7e1fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.024+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94519369516048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f701768f50 con 0x55f70170c510 2024-04-27T03:30:34.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.025+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94519369516048] v25'103 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.025+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94519369490528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70173b990 con 0x7fb91c017050 2024-04-27T03:30:34.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.026+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94519369490528] v25'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb920095040 con 0x7fb91c017050 2024-04-27T03:30:34.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.026+0000 7fb92dc36800 20 remove_watcher() i=0 2024-04-27T03:30:34.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.026+0000 7fb92dc36800 2 removed watcher, disabling cache 2024-04-27T03:30:34.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.026+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94519369496864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70173b990 con 0x55f70170c510 2024-04-27T03:30:34.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.027+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94519369496864] v25'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.027+0000 7fb92dc36800 20 remove_watcher() i=1 2024-04-27T03:30:34.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.027+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94519369497920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70173b990 con 0x55f701710ed0 2024-04-27T03:30:34.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.028+0000 7fb92b735640 1 -- 172.21.15.165:0/2857024199 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94519369497920] v25'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb91c03d070 con 0x55f701710ed0 2024-04-27T03:30:34.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.028+0000 7fb92dc36800 20 remove_watcher() i=2 2024-04-27T03:30:34.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.028+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94519369504256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70173b990 con 0x7fb91c017050 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.029+0000 7fb92af34640 1 -- 172.21.15.165:0/2857024199 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94519369504256] v25'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb920095040 con 0x7fb91c017050 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.029+0000 7fb92dc36800 20 remove_watcher() i=3 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.029+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94519369507680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70173b990 con 0x55f70170c510 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.030+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94519369507680] v25'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x55f70170c510 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.030+0000 7fb92dc36800 20 remove_watcher() i=4 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.030+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94519369509808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70173b990 con 0x7fb920076040 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.031+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94519369509808] v25'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.031+0000 7fb92dc36800 20 remove_watcher() i=5 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.031+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94519369511456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70173b990 con 0x7fb920076040 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.032+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94519369511456] v25'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.032+0000 7fb92dc36800 20 remove_watcher() i=6 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.032+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94519369514992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55f70173b990 con 0x7fb920076040 2024-04-27T03:30:34.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.033+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94519369514992] v25'95 uv93 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fb924031040 con 0x7fb920076040 2024-04-27T03:30:34.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.033+0000 7fb92dc36800 20 remove_watcher() i=7 2024-04-27T03:30:34.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.033+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fb920076040 msgr2=0x7fb920083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.033+0000 7fb92dc36800 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fb920076040 0x7fb920083410 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.033+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f70170c510 msgr2=0x55f70170e9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.033+0000 7fb92dc36800 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f70170c510 0x55f70170e9d0 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.033+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fb91c017050 msgr2=0x7fb91c0332e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.033+0000 7fb92dc36800 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fb91c017050 0x7fb91c0332e0 crc :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.033+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f701710ed0 msgr2=0x55f7017132c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.033+0000 7fb92dc36800 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f701710ed0 0x55f7017132c0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb7f4036950 msgr2=0x7fb7f4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb7f4036950 0x7fb7f4038e10 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fb920052fd0 tx=0x7fb92004f000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 msgr2=0x55f701700e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92bf36640 1 -- 172.21.15.165:0/2857024199 reap_dead start 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 0x55f701700e50 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fb91c034ae0 tx=0x7fb91c003880 comp rx=0 tx=0).stop 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 shutdown_connections 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 --2- 172.21.15.165:0/2857024199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016f9a50 0x55f701700e50 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 >> 172.21.15.165:0/2857024199 conn(0x55f7016f5d70 msgr2=0x55f7016f33d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 shutdown_connections 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/2857024199 wait complete. 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/892426781 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb804036a50 msgr2=0x7fb804038f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 --2- 172.21.15.165:0/892426781 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb804036a50 0x7fb804038f10 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x55f7016d5f10 tx=0x7fb920005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/892426781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016ec8b0 msgr2=0x55f7016eccd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 --2- 172.21.15.165:0/892426781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016ec8b0 0x55f7016eccd0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fb91c01a6f0 tx=0x7fb91c017e80 comp rx=0 tx=0).stop 2024-04-27T03:30:34.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/892426781 shutdown_connections 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 --2- 172.21.15.165:0/892426781 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb804036a50 0x7fb804038f10 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 --2- 172.21.15.165:0/892426781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7016ec8b0 0x55f7016eccd0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/892426781 >> 172.21.15.165:0/892426781 conn(0x55f7016e0950 msgr2=0x55f7016e0d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/892426781 shutdown_connections 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/892426781 wait complete. 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55f7016db8a0 msgr2=0x55f7016ddd30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55f7016db8a0 0x55f7016ddd30 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7015d61b0 msgr2=0x55f7016d50f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7015d61b0 0x55f7016d50f0 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7016d7960 msgr2=0x55f7016d9d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.034+0000 7fb92dc36800 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7016d7960 0x55f7016d9d50 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.035+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb810036950 msgr2=0x7fb810038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.035+0000 7fb92dc36800 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb810036950 0x7fb810038e10 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fb920054000 tx=0x7fb92004e000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.035+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 msgr2=0x55f7016cdeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.035+0000 7fb92dc36800 1 --2- 172.21.15.165:0/3509509269 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7015d1d20 0x55f7016cdeb0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb91c001e10 tx=0x7fb91c000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.035+0000 7fb92bf36640 1 -- 172.21.15.165:0/3509509269 reap_dead start 2024-04-27T03:30:34.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.035+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 shutdown_connections 2024-04-27T03:30:34.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.035+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 >> 172.21.15.165:0/3509509269 conn(0x55f7015cd380 msgr2=0x55f7016bc080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.035+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 shutdown_connections 2024-04-27T03:30:34.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.035+0000 7fb92dc36800 1 -- 172.21.15.165:0/3509509269 wait complete. 2024-04-27T03:30:34.062 INFO:teuthology.orchestra.run.smithi165.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"fd642a41-253a-48d4-9473-6eca68174247","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1","marker":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-04-27T03:30:33.428997Z","creation_time":"2024-04-27T03:30:33.422040Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}} 2024-04-27T03:30:34.062 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'fd642a41-253a-48d4-9473-6eca68174247', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1', 'marker': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T03:30:33.428997Z', 'creation_time': '2024-04-27T03:30:33.422040Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T03:30:34.062 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--uid', 'foo'] 2024-04-27T03:30:34.062 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--uid', 'foo'] 2024-04-27T03:30:34.063 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --uid foo 2024-04-27T03:30:34.145 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:34.146 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:34.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.140+0000 7f33abc30800 1 Processor -- start 2024-04-27T03:30:34.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.141+0000 7f33abc30800 1 -- start start 2024-04-27T03:30:34.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.141+0000 7f33abc30800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560afd68b330 0x560afd68d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.141+0000 7f33abc30800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x560afd545d40 con 0x560afd686f90 2024-04-27T03:30:34.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.141+0000 7f33abc30800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x560afd5184d0 con 0x560afd68b330 2024-04-27T03:30:34.147 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.141+0000 7f33a668a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560afd68b330 0x560afd68d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.141+0000 7f33a6e8b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x560afd686f90 0x560afd6873b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35300/0 (socket says 172.21.15.165:35300) 2024-04-27T03:30:34.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.141+0000 7f33a6e8b640 1 -- 172.21.15.165:0/242809781 learned_addr learned my addr 172.21.15.165:0/242809781 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:34.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.141+0000 7f33a668a640 1 -- 172.21.15.165:0/242809781 >> v1:172.21.15.89:6789/0 conn(0x560afd686f90 legacy=0x560afd6873b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.141+0000 7f33a668a640 1 -- 172.21.15.165:0/242809781 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x560afd544590 con 0x560afd68b330 2024-04-27T03:30:34.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.143+0000 7f33a668a640 1 --2- 172.21.15.165:0/242809781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd68b330 0x560afd68d720 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f339c00a470 tx=0x7f339c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81073915fe4f1241 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.143+0000 7f33a5e89640 1 -- 172.21.15.165:0/242809781 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f339c057070 con 0x560afd68b330 2024-04-27T03:30:34.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.143+0000 7f33a5e89640 1 -- 172.21.15.165:0/242809781 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339c052570 con 0x560afd68b330 2024-04-27T03:30:34.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.143+0000 7f33a5e89640 1 -- 172.21.15.165:0/242809781 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f339c052870 con 0x560afd68b330 2024-04-27T03:30:34.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.144+0000 7f33abc30800 1 -- 172.21.15.165:0/242809781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd68b330 msgr2=0x560afd68d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.144+0000 7f33abc30800 1 --2- 172.21.15.165:0/242809781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd68b330 0x560afd68d720 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f339c00a470 tx=0x7f339c01ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:34.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.144+0000 7f33abc30800 1 -- 172.21.15.165:0/242809781 shutdown_connections 2024-04-27T03:30:34.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.144+0000 7f33abc30800 1 --2- 172.21.15.165:0/242809781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd68b330 0x560afd68d720 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.144+0000 7f33abc30800 1 -- 172.21.15.165:0/242809781 >> 172.21.15.165:0/242809781 conn(0x560afd682530 msgr2=0x560afd684950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.144+0000 7f33abc30800 1 -- 172.21.15.165:0/242809781 shutdown_connections 2024-04-27T03:30:34.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.144+0000 7f33abc30800 1 -- 172.21.15.165:0/242809781 wait complete. 2024-04-27T03:30:34.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.156+0000 7f33abc30800 1 Processor -- start 2024-04-27T03:30:34.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.156+0000 7f33abc30800 1 -- start start 2024-04-27T03:30:34.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.156+0000 7f33abc30800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 0x560afd7780f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.156+0000 7f33abc30800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560afd5d23c0 con 0x560afd686f90 2024-04-27T03:30:34.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.156+0000 7f33a6e8b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 0x560afd7780f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.156+0000 7f33a6e8b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 0x560afd7780f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58420/0 (socket says 172.21.15.165:58420) 2024-04-27T03:30:34.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.156+0000 7f33a6e8b640 1 -- 172.21.15.165:0/2252721376 learned_addr learned my addr 172.21.15.165:0/2252721376 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.156+0000 7f33a6e8b640 1 -- 172.21.15.165:0/2252721376 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560afd7786c0 con 0x560afd686f90 2024-04-27T03:30:34.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.157+0000 7f33a6e8b640 1 --2- 172.21.15.165:0/2252721376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 0x560afd7780f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3398001e10 tx=0x7f339800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece2bd8c008d3397 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.157+0000 7f3294e09640 1 -- 172.21.15.165:0/2252721376 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3398012020 con 0x560afd686f90 2024-04-27T03:30:34.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.157+0000 7f3294e09640 1 -- 172.21.15.165:0/2252721376 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339800e5a0 con 0x560afd686f90 2024-04-27T03:30:34.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.157+0000 7f3294e09640 1 -- 172.21.15.165:0/2252721376 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f339800e8e0 con 0x560afd686f90 2024-04-27T03:30:34.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.157+0000 7f33abc30800 1 -- 172.21.15.165:0/2252721376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 msgr2=0x560afd7780f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.157+0000 7f33abc30800 1 --2- 172.21.15.165:0/2252721376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 0x560afd7780f0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f3398001e10 tx=0x7f339800b780 comp rx=0 tx=0).stop 2024-04-27T03:30:34.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.158+0000 7f33abc30800 1 -- 172.21.15.165:0/2252721376 shutdown_connections 2024-04-27T03:30:34.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.158+0000 7f33abc30800 1 --2- 172.21.15.165:0/2252721376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 0x560afd7780f0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.158+0000 7f33abc30800 1 -- 172.21.15.165:0/2252721376 >> 172.21.15.165:0/2252721376 conn(0x560afd682530 msgr2=0x560afd773a10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.158+0000 7f33abc30800 1 -- 172.21.15.165:0/2252721376 shutdown_connections 2024-04-27T03:30:34.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.158+0000 7f33abc30800 1 -- 172.21.15.165:0/2252721376 wait complete. 2024-04-27T03:30:34.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.158+0000 7f33abc30800 1 Processor -- start 2024-04-27T03:30:34.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.158+0000 7f33abc30800 1 -- start start 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.158+0000 7f33abc30800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 0x560afd77fca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.158+0000 7f33abc30800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560afd68dff0 con 0x560afd686f90 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.158+0000 7f33a6e8b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 0x560afd77fca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.158+0000 7f33a6e8b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 0x560afd77fca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58424/0 (socket says 172.21.15.165:58424) 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.158+0000 7f33a6e8b640 1 -- 172.21.15.165:0/1849633306 learned_addr learned my addr 172.21.15.165:0/1849633306 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.159+0000 7f33a6e8b640 1 -- 172.21.15.165:0/1849633306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560afd7801e0 con 0x560afd686f90 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.159+0000 7f33a6e8b640 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 0x560afd77fca0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f339800b7c0 tx=0x7f3398000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.159+0000 7f3293606640 1 -- 172.21.15.165:0/1849633306 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3398012030 con 0x560afd686f90 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.159+0000 7f3293606640 1 -- 172.21.15.165:0/1849633306 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3398001470 con 0x560afd686f90 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.159+0000 7f3293606640 1 -- 172.21.15.165:0/1849633306 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f339801a8d0 con 0x560afd686f90 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.159+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560afd780470 con 0x560afd686f90 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.159+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560afd783120 con 0x560afd686f90 2024-04-27T03:30:34.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.160+0000 7f3293606640 1 -- 172.21.15.165:0/1849633306 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f3398015020 con 0x560afd686f90 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.160+0000 7f3293606640 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f328c036950 0x7f328c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.160+0000 7f3293606640 1 -- 172.21.15.165:0/1849633306 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f3398020030 con 0x560afd686f90 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.160+0000 7f33a668a640 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f328c036950 0x7f328c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.160+0000 7f33abc30800 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560afd68b330 0x560afd786ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.160+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd787420 con 0x560afd68b330 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.160+0000 7f33a768c640 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560afd68b330 0x560afd786ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.160+0000 7f33a668a640 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f328c036950 0x7f328c038e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f339c054000 tx=0x7f339c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.161+0000 7f33a768c640 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560afd68b330 0x560afd786ee0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.161+0000 7f33a768c640 1 -- 172.21.15.165:0/1849633306 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd68b330 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.161+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7844b0 con 0x560afd68b330 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.161+0000 7f33a768c640 1 -- 172.21.15.165:0/1849633306 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd68b330 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.161+0000 7f33abc30800 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560afd789750 0x560afd78bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.161+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd78c210 con 0x560afd789750 2024-04-27T03:30:34.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.161+0000 7f33a6e8b640 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560afd789750 0x560afd78bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.161+0000 7f33a6e8b640 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560afd789750 0x560afd78bb40 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.162+0000 7f33a6e8b640 1 -- 172.21.15.165:0/1849633306 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f339802dcf0 con 0x560afd789750 2024-04-27T03:30:34.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.162+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd788700 con 0x560afd68b330 2024-04-27T03:30:34.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.162+0000 7f33a768c640 1 -- 172.21.15.165:0/1849633306 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd68b330 2024-04-27T03:30:34.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.162+0000 7f33abc30800 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560afd78d650 0x560afd78fae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.162+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd790270 con 0x560afd78d650 2024-04-27T03:30:34.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.162+0000 7f33a668a640 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560afd78d650 0x560afd78fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.163+0000 7f33a668a640 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560afd78d650 0x560afd78fae0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.163+0000 7f33a668a640 1 -- 172.21.15.165:0/1849633306 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f339c052a40 con 0x560afd78d650 2024-04-27T03:30:34.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.163+0000 7f33abc30800 1 Processor -- start 2024-04-27T03:30:34.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.163+0000 7f33abc30800 1 -- start start 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.164+0000 7f33abc30800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd5fcc00 0x560afd5fb070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.164+0000 7f33abc30800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560afd5786b0 con 0x560afd5fcc00 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.164+0000 7f33a6e8b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd5fcc00 0x560afd5fb070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.164+0000 7f33a6e8b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd5fcc00 0x560afd5fb070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58440/0 (socket says 172.21.15.165:58440) 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.164+0000 7f33a6e8b640 1 -- 172.21.15.165:0/1838199881 learned_addr learned my addr 172.21.15.165:0/1838199881 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.164+0000 7f33a6e8b640 1 -- 172.21.15.165:0/1838199881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560afd5fb5b0 con 0x560afd5fcc00 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.164+0000 7f33a6e8b640 1 --2- 172.21.15.165:0/1838199881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd5fcc00 0x560afd5fb070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f339804ae70 tx=0x7f3398045f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.164+0000 7f3290e01640 1 -- 172.21.15.165:0/1838199881 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f339804c0c0 con 0x560afd5fcc00 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.164+0000 7f3290e01640 1 -- 172.21.15.165:0/1838199881 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339803b070 con 0x560afd5fcc00 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.164+0000 7f33abc30800 1 -- 172.21.15.165:0/1838199881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560afd5fb840 con 0x560afd5fcc00 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.164+0000 7f3290e01640 1 -- 172.21.15.165:0/1838199881 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f339804c390 con 0x560afd5fcc00 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.164+0000 7f33abc30800 1 -- 172.21.15.165:0/1838199881 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560afd5fb840 con 0x560afd5fcc00 2024-04-27T03:30:34.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.165+0000 7f3290e01640 1 -- 172.21.15.165:0/1838199881 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f3398041040 con 0x560afd5fcc00 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.165+0000 7f3290e01640 1 --2- 172.21.15.165:0/1838199881 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3280036950 0x7f3280038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.165+0000 7f3290e01640 1 -- 172.21.15.165:0/1838199881 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f3398049d50 con 0x560afd5fcc00 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.165+0000 7f33a668a640 1 --2- 172.21.15.165:0/1838199881 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3280036950 0x7f3280038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.165+0000 7f33abc30800 1 Processor -- start 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.166+0000 7f33abc30800 1 -- start start 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.166+0000 7f33a668a640 1 --2- 172.21.15.165:0/1838199881 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3280036950 0x7f3280038e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x560afd7878d0 tx=0x7f339c005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.166+0000 7f33abc30800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 0x560afd7adc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.166+0000 7f33abc30800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560afd778a40 con 0x560afd7ad870 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.166+0000 7f33a6e8b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 0x560afd7adc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.166+0000 7f33a6e8b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 0x560afd7adc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58444/0 (socket says 172.21.15.165:58444) 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.166+0000 7f33a6e8b640 1 -- 172.21.15.165:0/1183926212 learned_addr learned my addr 172.21.15.165:0/1183926212 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.166+0000 7f33a6e8b640 1 -- 172.21.15.165:0/1183926212 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560afd7ae260 con 0x560afd7ad870 2024-04-27T03:30:34.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.166+0000 7f33a6e8b640 1 --2- 172.21.15.165:0/1183926212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 0x560afd7adc90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f339803b040 tx=0x7f339801bdc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c3be05dcd1f53c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.166+0000 7f328a7fc640 1 -- 172.21.15.165:0/1183926212 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3398044440 con 0x560afd7ad870 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.166+0000 7f328a7fc640 1 -- 172.21.15.165:0/1183926212 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339805c030 con 0x560afd7ad870 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.166+0000 7f328a7fc640 1 -- 172.21.15.165:0/1183926212 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3398024420 con 0x560afd7ad870 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.167+0000 7f33abc30800 1 -- 172.21.15.165:0/1183926212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 msgr2=0x560afd7adc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.167+0000 7f33abc30800 1 --2- 172.21.15.165:0/1183926212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 0x560afd7adc90 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f339803b040 tx=0x7f339801bdc0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.167+0000 7f33abc30800 1 -- 172.21.15.165:0/1183926212 shutdown_connections 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.167+0000 7f33abc30800 1 --2- 172.21.15.165:0/1183926212 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 0x560afd7adc90 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.167+0000 7f33abc30800 1 -- 172.21.15.165:0/1183926212 >> 172.21.15.165:0/1183926212 conn(0x560afd7a9b40 msgr2=0x560afd7a7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.167+0000 7f33abc30800 1 -- 172.21.15.165:0/1183926212 shutdown_connections 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.167+0000 7f33abc30800 1 -- 172.21.15.165:0/1183926212 wait complete. 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.167+0000 7f33abc30800 1 Processor -- start 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.167+0000 7f33abc30800 1 -- start start 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.167+0000 7f33abc30800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 0x560afd7b4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.167+0000 7f33abc30800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560afd7ae930 con 0x560afd7ad870 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.168+0000 7f33a6e8b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 0x560afd7b4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.168+0000 7f33a6e8b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 0x560afd7b4cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58446/0 (socket says 172.21.15.165:58446) 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.168+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 learned_addr learned my addr 172.21.15.165:0/633358869 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.168+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560afd7b5210 con 0x560afd7ad870 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.168+0000 7f33a6e8b640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 0x560afd7b4cd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f339800bcd0 tx=0x7f3398034e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.168+0000 7f3288ff9640 1 -- 172.21.15.165:0/633358869 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3398071070 con 0x560afd7ad870 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.168+0000 7f3288ff9640 1 -- 172.21.15.165:0/633358869 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f339801c0e0 con 0x560afd7ad870 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.168+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560afd7b5210 con 0x560afd7ad870 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.168+0000 7f3288ff9640 1 -- 172.21.15.165:0/633358869 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3398024420 con 0x560afd7ad870 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.168+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560afd7b5210 con 0x560afd7ad870 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.169+0000 7f3288ff9640 1 -- 172.21.15.165:0/633358869 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f3398074070 con 0x560afd7ad870 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.169+0000 7f3288ff9640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3270036900 0x7f3270038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.169+0000 7f3288ff9640 1 -- 172.21.15.165:0/633358869 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f3398069070 con 0x560afd7ad870 2024-04-27T03:30:34.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.169+0000 7f33a668a640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3270036900 0x7f3270038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.169+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.169+0000 7f33abc30800 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560afd7c0310 0x560afd7c27d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.169+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c2f20 con 0x560afd7c0310 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33a768c640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560afd7c0310 0x560afd7c27d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33a668a640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3270036900 0x7f3270038dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f339c052fd0 tx=0x7f339c04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33a768c640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560afd7c0310 0x560afd7c27d0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33abc30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33abc30800 20 realm 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3330 con 0x560afd7c0310 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33abc30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33abc30800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3c00 con 0x560afd7c0310 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.170+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33abc30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3fa0 con 0x560afd7c0310 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33abc30800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33abc30800 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560afd7c4a90 0x560afd7c6e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c75c0 con 0x560afd7c4a90 2024-04-27T03:30:34.188 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33a6e8b640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560afd7c4a90 0x560afd7c6e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33a6e8b640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560afd7c4a90 0x560afd7c6e80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33abc30800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33abc30800 20 searching for the correct realm 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.171+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c8f90 con 0x560afd7c4a90 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.172+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.172+0000 7f33a6e8b640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3398076060 0x7f339802d590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.172+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c8f90 con 0x7f3398076060 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.172+0000 7f33a668a640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3398076060 0x7f339802d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.172+0000 7f33a668a640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3398076060 0x7f339802d590 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.173+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.173+0000 7f33a668a640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f339c076040 0x7f339c083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.173+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c8f90 con 0x7f339c076040 2024-04-27T03:30:34.189 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.173+0000 7f33a768c640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f339c076040 0x7f339c083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.173+0000 7f33a768c640 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f339c076040 0x7f339c083410 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.174+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.174+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3fa0 con 0x560afd7c0310 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.174+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.174+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3fa0 con 0x560afd7c0310 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.174+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.174+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3fa0 con 0x7f339c076040 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.174+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.174+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3fa0 con 0x7f339c076040 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.174+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.174+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3fa0 con 0x7f3398076060 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.175+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.175+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3fa0 con 0x560afd7c4a90 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.175+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.175+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3fa0 con 0x560afd7c0310 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.175+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.175+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3fa0 con 0x7f339c076040 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.175+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.176+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3fa0 con 0x560afd7c0310 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.176+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.176+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c3fa0 con 0x7f3398076060 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.176+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.176+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x560afd7c4a90 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.176+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.176+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x560afd7c4a90 2024-04-27T03:30:34.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.176+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.176+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x7f339c076040 2024-04-27T03:30:34.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.177+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.177+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x560afd7c4a90 2024-04-27T03:30:34.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.177+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.177+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x560afd7c4a90 2024-04-27T03:30:34.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.177+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.177+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x560afd7c0310 2024-04-27T03:30:34.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.177+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.177+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x560afd7c0310 2024-04-27T03:30:34.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.178+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.178+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x7f339c076040 2024-04-27T03:30:34.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.178+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.178+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x560afd7c4a90 2024-04-27T03:30:34.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.178+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.178+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x7f3398076060 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.178+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.178+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x560afd7c0310 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.178+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.178+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x7f339c076040 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.179+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.179+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x560afd7c4a90 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.179+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.179+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x7f339c076040 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.179+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.180+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x7f3398076060 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.180+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.180+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x560afd7c4a90 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.180+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.208 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.180+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x7f339c076040 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.180+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.180+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f339c090060 con 0x560afd7c0310 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.180+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.180+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f33a0038040 con 0x7f3398076060 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.209 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c7f60 con 0x560afd7c0310 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c8350 con 0x560afd7c0310 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.181+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c8350 con 0x7f339c076040 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 20 zone default found 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 4 Realm: () 2024-04-27T03:30:34.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:34.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:34.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:34.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 20 zonegroup default 2024-04-27T03:30:34.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c8770 con 0x7f339c076040 2024-04-27T03:30:34.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7c8770 con 0x560afd7c4a90 2024-04-27T03:30:34.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7cae40 con 0x560afd7c4a90 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 20 started sync module instance, tier type = 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.182+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7ca2e0 con 0x7f3398076060 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.183+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.183+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94605202466512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7cb6e0 con 0x7f3398076060 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.183+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7cbf30 con 0x560afd7c0310 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.184+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94605202466512] v25'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.185+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.185+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94605202471552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7cc350 con 0x560afd7c0310 2024-04-27T03:30:34.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.185+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7ccba0 con 0x560afd7c4a90 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.186+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.186+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94605202474320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7cd440 con 0x560afd7c4a90 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.186+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7cdf30 con 0x7f3398076060 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.186+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94605202471552] v25'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.187+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94605202474320] v25'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.187+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.187+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94605202479616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7ce320 con 0x7f3398076060 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.187+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7cebb0 con 0x560afd7c0310 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.188+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94605202479616] v25'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.188+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.188+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94605202472608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7cefa0 con 0x560afd7c0310 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.188+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7cf9a0 con 0x7f339c076040 2024-04-27T03:30:34.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.189+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.189+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94605202486448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7cf910 con 0x7f339c076040 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.189+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7d0910 con 0x7f339c076040 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.189+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94605202472608] v25'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.190+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94605202486448] v25'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.190+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.190+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94605202488720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7d0460 con 0x7f339c076040 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.190+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7d0e20 con 0x7f339c076040 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.191+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94605202488720] v25'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.192+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'96 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.192+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94605202482880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7d0e20 con 0x7f339c076040 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.192+0000 7f33abc30800 20 add_watcher() i=0 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.192+0000 7f33abc30800 20 add_watcher() i=1 2024-04-27T03:30:34.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.192+0000 7f33abc30800 20 add_watcher() i=2 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.192+0000 7f33abc30800 20 add_watcher() i=3 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.192+0000 7f33abc30800 20 add_watcher() i=4 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.192+0000 7f33abc30800 20 add_watcher() i=5 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.192+0000 7f33abc30800 20 add_watcher() i=6 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.193+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94605202482880] v25'97 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.193+0000 7f33abc30800 20 add_watcher() i=7 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.193+0000 7f33abc30800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.193+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd7d4140 con 0x560afd7c0310 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.193+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.193+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94605202492128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7d13b0 con 0x560afd7c0310 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.194+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94605202492128] v25'104 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.194+0000 7f325d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.194+0000 7f33abc30800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:34.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.194+0000 7f33abc30800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.194+0000 7f33abc30800 5 note: GC not initialized 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.194+0000 7f31d7fff640 20 reqs_thread_entry: start 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.195+0000 7f33abc30800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.195+0000 7f33abc30800 20 Filter name: none 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.195+0000 7f31d5ffb640 20 reqs_thread_entry: start 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.195+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.195+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd807010 con 0x560afd7c0310 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.195+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.195+0000 7f33abc30800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.195+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.195+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd807a00 con 0x560afd7c0310 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.196+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.196+0000 7f33abc30800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:34.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.196+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd808a10 con 0x7f3398076060 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.196+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.196+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.196+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd8062e0 con 0x7f3398076060 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.196+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(65 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.196+0000 7f33abc30800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.196+0000 7f33abc30800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.196+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd808a10 con 0x560afd7c0310 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 4.2f 4:f7b16f5a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd808410 con 0x560afd7c0310 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 4.16 4:6b5ddb47:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd80d470 con 0x560afd7c4a90 2024-04-27T03:30:34.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 4.7 4:e2bbac01:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd80fd20 con 0x7f339c076040 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 4.3e 4:7e0d8a50:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd812660 con 0x7f3398076060 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 4.19 4:9b87498d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd80fd20 con 0x560afd7c4a90 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 4.2 4:42daa0e0:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd8174e0 con 0x560afd7c0310 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:73 4.1b 4:d9d9fd94:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd80fd20 con 0x7f3398076060 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 4.e 4:70dfcb91:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd81c940 con 0x7f3398076060 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:75 4.3d 4:bd27934b:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd81f800 con 0x560afd7c4a90 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:76 4.5 4:a1c15390:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd822670 con 0x560afd7c4a90 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(67 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:77 4.1a 4:59a985c3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560afd825530 con 0x7f3398076060 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(69 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(72 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.5 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(68 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.197+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(71 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.198+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(70 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.198+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(75 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.198+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(73 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.198+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(74 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.198+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(76 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.198+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(77 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.199+0000 7f325d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.199+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:78 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94605202492128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd81c940 con 0x560afd7c0310 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.199+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(78 data_loggenerations_metadata [watch unwatch cookie 94605202492128] v25'105 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.199+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:79 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94605202466512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7d0ae0 con 0x7f3398076060 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.200+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(79 notify.0 [watch unwatch cookie 94605202466512] v25'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.200+0000 7f33abc30800 20 remove_watcher() i=0 2024-04-27T03:30:34.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.200+0000 7f33abc30800 2 removed watcher, disabling cache 2024-04-27T03:30:34.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.200+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94605202471552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7ca6d0 con 0x560afd7c0310 2024-04-27T03:30:34.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.201+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(80 notify.1 [watch unwatch cookie 94605202471552] v25'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.202+0000 7f33abc30800 20 remove_watcher() i=1 2024-04-27T03:30:34.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.202+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:81 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94605202474320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7d0ae0 con 0x560afd7c4a90 2024-04-27T03:30:34.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.203+0000 7f33a6e8b640 1 -- 172.21.15.165:0/633358869 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(81 notify.2 [watch unwatch cookie 94605202474320] v25'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3398073030 con 0x560afd7c4a90 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.203+0000 7f33abc30800 20 remove_watcher() i=2 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.203+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:82 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94605202479616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7cba80 con 0x7f3398076060 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.204+0000 7f33a668a640 1 -- 172.21.15.165:0/633358869 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(82 notify.3 [watch unwatch cookie 94605202479616] v25'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f339c081070 con 0x7f3398076060 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.204+0000 7f33abc30800 20 remove_watcher() i=3 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.204+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:83 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94605202472608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7ca6d0 con 0x560afd7c0310 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.205+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(83 notify.4 [watch unwatch cookie 94605202472608] v25'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x560afd7c0310 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.205+0000 7f33abc30800 20 remove_watcher() i=4 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.205+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:84 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94605202486448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7d0ae0 con 0x7f339c076040 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.206+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(84 notify.5 [watch unwatch cookie 94605202486448] v25'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.206+0000 7f33abc30800 20 remove_watcher() i=5 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.206+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:85 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94605202488720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7ca6d0 con 0x7f339c076040 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.207+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(85 notify.6 [watch unwatch cookie 94605202488720] v25'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.207+0000 7f33abc30800 20 remove_watcher() i=6 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.207+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:86 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94605202482880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560afd7cf4b0 con 0x7f339c076040 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(86 notify.7 [watch unwatch cookie 94605202482880] v25'98 uv96 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f33a000a4a0 con 0x7f339c076040 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 20 remove_watcher() i=7 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f339c076040 msgr2=0x7f339c083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f339c076040 0x7f339c083410 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560afd7c0310 msgr2=0x560afd7c27d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560afd7c0310 0x560afd7c27d0 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3398076060 msgr2=0x7f339802d590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3398076060 0x7f339802d590 crc :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560afd7c4a90 msgr2=0x560afd7c6e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560afd7c4a90 0x560afd7c6e80 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3270036900 msgr2=0x7f3270038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3270036900 0x7f3270038dc0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f339c052fd0 tx=0x7f339c04f000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 msgr2=0x560afd7b4cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33a768c640 1 -- 172.21.15.165:0/633358869 reap_dead start 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 0x560afd7b4cd0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f339800bcd0 tx=0x7f3398034e70 comp rx=0 tx=0).stop 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 shutdown_connections 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 --2- 172.21.15.165:0/633358869 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd7ad870 0x560afd7b4cd0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.208+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 >> 172.21.15.165:0/633358869 conn(0x560afd7a9b40 msgr2=0x560afd7a71a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 shutdown_connections 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/633358869 wait complete. 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1838199881 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3280036950 msgr2=0x7f3280038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 --2- 172.21.15.165:0/1838199881 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3280036950 0x7f3280038e10 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x560afd7878d0 tx=0x7f339c005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1838199881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd5fcc00 msgr2=0x560afd5fb070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 --2- 172.21.15.165:0/1838199881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd5fcc00 0x560afd5fb070 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f339804ae70 tx=0x7f3398045f80 comp rx=0 tx=0).stop 2024-04-27T03:30:34.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1838199881 shutdown_connections 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 --2- 172.21.15.165:0/1838199881 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3280036950 0x7f3280038e10 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 --2- 172.21.15.165:0/1838199881 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd5fcc00 0x560afd5fb070 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1838199881 >> 172.21.15.165:0/1838199881 conn(0x560afd792730 msgr2=0x560afd792b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1838199881 shutdown_connections 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1838199881 wait complete. 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560afd78d650 msgr2=0x560afd78fae0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560afd78d650 0x560afd78fae0 crc :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560afd68b330 msgr2=0x560afd786ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560afd68b330 0x560afd786ee0 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560afd789750 msgr2=0x560afd78bb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560afd789750 0x560afd78bb40 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f328c036950 msgr2=0x7f328c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f328c036950 0x7f328c038e10 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f339c054000 tx=0x7f339c04e000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 msgr2=0x560afd77fca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 --2- 172.21.15.165:0/1849633306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560afd686f90 0x560afd77fca0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f339800b7c0 tx=0x7f3398000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33a768c640 1 -- 172.21.15.165:0/1849633306 reap_dead start 2024-04-27T03:30:34.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 shutdown_connections 2024-04-27T03:30:34.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 >> 172.21.15.165:0/1849633306 conn(0x560afd682530 msgr2=0x560afd772210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 shutdown_connections 2024-04-27T03:30:34.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.209+0000 7f33abc30800 1 -- 172.21.15.165:0/1849633306 wait complete. 2024-04-27T03:30:34.230 INFO:teuthology.orchestra.run.smithi165.stdout:[{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"fd642a41-253a-48d4-9473-6eca68174247","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1","marker":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-04-27T03:30:33.428997Z","creation_time":"2024-04-27T03:30:33.422040Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}}] 2024-04-27T03:30:34.230 DEBUG:tasks.util.rgw: json result: [{'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'fd642a41-253a-48d4-9473-6eca68174247', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1', 'marker': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T03:30:33.428997Z', 'creation_time': '2024-04-27T03:30:33.422040Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2024-04-27T03:30:34.236 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-27T03:30:34.236 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T03:30:34.236 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T03:30:34.236 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-04-27T03:30:34.315 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:34.315 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:34.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.310+0000 7f521873f800 1 Processor -- start 2024-04-27T03:30:34.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.310+0000 7f521873f800 1 -- start start 2024-04-27T03:30:34.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.310+0000 7f521873f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a494ca3170 0x55a494ca5630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.310+0000 7f521873f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a494b5dd40 con 0x55a494c12a50 2024-04-27T03:30:34.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.310+0000 7f521873f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55a494b304d0 con 0x55a494ca3170 2024-04-27T03:30:34.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.311+0000 7f520bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a494ca3170 0x55a494ca5630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.311+0000 7f521449b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55a494c12a50 0x55a494ca1a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35306/0 (socket says 172.21.15.165:35306) 2024-04-27T03:30:34.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.311+0000 7f520bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a494ca3170 0x55a494ca5630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58448/0 (socket says 172.21.15.165:58448) 2024-04-27T03:30:34.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.311+0000 7f521449b640 1 -- 172.21.15.165:0/3355717187 learned_addr learned my addr 172.21.15.165:0/3355717187 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:34.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.311+0000 7f520bfff640 1 -- 172.21.15.165:0/3355717187 >> v1:172.21.15.89:6789/0 conn(0x55a494c12a50 legacy=0x55a494ca1a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.311+0000 7f520bfff640 1 -- 172.21.15.165:0/3355717187 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a494b5c590 con 0x55a494ca3170 2024-04-27T03:30:34.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.313+0000 7f520bfff640 1 --2- 172.21.15.165:0/3355717187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494ca3170 0x55a494ca5630 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00a470 tx=0x7f51fc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2d4a61fe296be5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.313+0000 7f5213c9a640 1 -- 172.21.15.165:0/3355717187 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51fc057070 con 0x55a494ca3170 2024-04-27T03:30:34.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.313+0000 7f5213c9a640 1 -- 172.21.15.165:0/3355717187 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f51fc052570 con 0x55a494ca3170 2024-04-27T03:30:34.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.313+0000 7f5213c9a640 1 -- 172.21.15.165:0/3355717187 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f51fc052870 con 0x55a494ca3170 2024-04-27T03:30:34.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.314+0000 7f521873f800 1 -- 172.21.15.165:0/3355717187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494ca3170 msgr2=0x55a494ca5630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.314+0000 7f521873f800 1 --2- 172.21.15.165:0/3355717187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494ca3170 0x55a494ca5630 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f51fc00a470 tx=0x7f51fc01ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:34.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.314+0000 7f521873f800 1 -- 172.21.15.165:0/3355717187 shutdown_connections 2024-04-27T03:30:34.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.314+0000 7f521873f800 1 --2- 172.21.15.165:0/3355717187 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494ca3170 0x55a494ca5630 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.314+0000 7f521873f800 1 -- 172.21.15.165:0/3355717187 >> 172.21.15.165:0/3355717187 conn(0x55a494c99320 msgr2=0x55a494c9b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.314+0000 7f521873f800 1 -- 172.21.15.165:0/3355717187 shutdown_connections 2024-04-27T03:30:34.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.314+0000 7f521873f800 1 -- 172.21.15.165:0/3355717187 wait complete. 2024-04-27T03:30:34.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.325+0000 7f521873f800 1 Processor -- start 2024-04-27T03:30:34.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.325+0000 7f521873f800 1 -- start start 2024-04-27T03:30:34.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.325+0000 7f521873f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 0x55a494d90120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.326+0000 7f521873f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a494bea3c0 con 0x55a494c12a50 2024-04-27T03:30:34.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.326+0000 7f521449b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 0x55a494d90120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.326+0000 7f521449b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 0x55a494d90120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58450/0 (socket says 172.21.15.165:58450) 2024-04-27T03:30:34.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.326+0000 7f521449b640 1 -- 172.21.15.165:0/564824921 learned_addr learned my addr 172.21.15.165:0/564824921 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.326+0000 7f521449b640 1 -- 172.21.15.165:0/564824921 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a494d906f0 con 0x55a494c12a50 2024-04-27T03:30:34.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.326+0000 7f521449b640 1 --2- 172.21.15.165:0/564824921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 0x55a494d90120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5204001de0 tx=0x7f520400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562906aae646cd68 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.326+0000 7f510260c640 1 -- 172.21.15.165:0/564824921 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5204012020 con 0x55a494c12a50 2024-04-27T03:30:34.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.326+0000 7f510260c640 1 -- 172.21.15.165:0/564824921 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520400e5a0 con 0x55a494c12a50 2024-04-27T03:30:34.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.326+0000 7f510260c640 1 -- 172.21.15.165:0/564824921 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f520400e8c0 con 0x55a494c12a50 2024-04-27T03:30:34.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.327+0000 7f521873f800 1 -- 172.21.15.165:0/564824921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 msgr2=0x55a494d90120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.327+0000 7f521873f800 1 --2- 172.21.15.165:0/564824921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 0x55a494d90120 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f5204001de0 tx=0x7f520400b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.327+0000 7f521873f800 1 -- 172.21.15.165:0/564824921 shutdown_connections 2024-04-27T03:30:34.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.327+0000 7f521873f800 1 --2- 172.21.15.165:0/564824921 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 0x55a494d90120 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.327+0000 7f521873f800 1 -- 172.21.15.165:0/564824921 >> 172.21.15.165:0/564824921 conn(0x55a494c99320 msgr2=0x55a494d8ba40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.327+0000 7f521873f800 1 -- 172.21.15.165:0/564824921 shutdown_connections 2024-04-27T03:30:34.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.327+0000 7f521873f800 1 -- 172.21.15.165:0/564824921 wait complete. 2024-04-27T03:30:34.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.327+0000 7f521873f800 1 Processor -- start 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.327+0000 7f521873f800 1 -- start start 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.328+0000 7f521873f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 0x55a494c9f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.328+0000 7f521873f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a494ca5fe0 con 0x55a494c12a50 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.328+0000 7f521449b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 0x55a494c9f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.328+0000 7f521449b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 0x55a494c9f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58452/0 (socket says 172.21.15.165:58452) 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.328+0000 7f521449b640 1 -- 172.21.15.165:0/977200307 learned_addr learned my addr 172.21.15.165:0/977200307 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.328+0000 7f521449b640 1 -- 172.21.15.165:0/977200307 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a494c9da40 con 0x55a494c12a50 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.328+0000 7f521449b640 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 0x55a494c9f350 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f520400c010 tx=0x7f5204000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.328+0000 7f5100e09640 1 -- 172.21.15.165:0/977200307 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5204012070 con 0x55a494c12a50 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.328+0000 7f5100e09640 1 -- 172.21.15.165:0/977200307 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520401a7a0 con 0x55a494c12a50 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.328+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a494c9dcd0 con 0x55a494c12a50 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.328+0000 7f5100e09640 1 -- 172.21.15.165:0/977200307 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f520401aa80 con 0x55a494c12a50 2024-04-27T03:30:34.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.328+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a494c9e1f0 con 0x55a494c12a50 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.329+0000 7f5100e09640 1 -- 172.21.15.165:0/977200307 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f5204015020 con 0x55a494c12a50 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.329+0000 7f5100e09640 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50fc036900 0x7f50fc038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.329+0000 7f5100e09640 1 -- 172.21.15.165:0/977200307 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f5204020070 con 0x55a494c12a50 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.329+0000 7f520bfff640 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50fc036900 0x7f50fc038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.329+0000 7f521873f800 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a494ca3170 0x55a494da10c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.329+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494da1690 con 0x55a494ca3170 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.330+0000 7f5214c9c640 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a494ca3170 0x55a494da10c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.330+0000 7f520bfff640 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50fc036900 0x7f50fc038dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f51fc054000 tx=0x7f51fc04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.330+0000 7f5214c9c640 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a494ca3170 0x55a494da10c0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.330+0000 7f5214c9c640 1 -- 172.21.15.165:0/977200307 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494ca3170 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.330+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494da1f80 con 0x55a494ca3170 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.330+0000 7f5214c9c640 1 -- 172.21.15.165:0/977200307 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494ca3170 2024-04-27T03:30:34.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.330+0000 7f521873f800 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a494da39b0 0x55a494da5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.330+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494da6600 con 0x55a494da39b0 2024-04-27T03:30:34.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.331+0000 7f521449b640 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a494da39b0 0x55a494da5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.331+0000 7f521449b640 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a494da39b0 0x55a494da5e70 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.331+0000 7f521449b640 1 -- 172.21.15.165:0/977200307 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f520402b070 con 0x55a494da39b0 2024-04-27T03:30:34.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.331+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494da2800 con 0x55a494ca3170 2024-04-27T03:30:34.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.331+0000 7f5214c9c640 1 -- 172.21.15.165:0/977200307 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494ca3170 2024-04-27T03:30:34.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.331+0000 7f521873f800 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55a494da7b00 0x55a494da9f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.331+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494daa6d0 con 0x55a494da7b00 2024-04-27T03:30:34.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.332+0000 7f520bfff640 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55a494da7b00 0x55a494da9f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.332+0000 7f520bfff640 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55a494da7b00 0x55a494da9f70 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.332+0000 7f520bfff640 1 -- 172.21.15.165:0/977200307 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f51fc052a40 con 0x55a494da7b00 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.332+0000 7f521873f800 1 Processor -- start 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.333+0000 7f521873f800 1 -- start start 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.333+0000 7f521873f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494db8a20 0x55a494db8e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.333+0000 7f521873f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a494c9f890 con 0x55a494db8a20 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.333+0000 7f521449b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494db8a20 0x55a494db8e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.333+0000 7f521449b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494db8a20 0x55a494db8e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58456/0 (socket says 172.21.15.165:58456) 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.333+0000 7f521449b640 1 -- 172.21.15.165:0/614585031 learned_addr learned my addr 172.21.15.165:0/614585031 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.333+0000 7f521449b640 1 -- 172.21.15.165:0/614585031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a494db9380 con 0x55a494db8a20 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.333+0000 7f521449b640 1 --2- 172.21.15.165:0/614585031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494db8a20 0x55a494db8e40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f520404aa00 tx=0x7f520402caa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.333+0000 7f50f9ffb640 1 -- 172.21.15.165:0/614585031 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52040492f0 con 0x55a494db8a20 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.333+0000 7f50f9ffb640 1 -- 172.21.15.165:0/614585031 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5204049490 con 0x55a494db8a20 2024-04-27T03:30:34.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.333+0000 7f521873f800 1 -- 172.21.15.165:0/614585031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a494db9df0 con 0x55a494db8a20 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.334+0000 7f50f9ffb640 1 -- 172.21.15.165:0/614585031 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5204044060 con 0x55a494db8a20 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.334+0000 7f521873f800 1 -- 172.21.15.165:0/614585031 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a494dba2f0 con 0x55a494db8a20 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.334+0000 7f50f9ffb640 1 -- 172.21.15.165:0/614585031 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f5204044200 con 0x55a494db8a20 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.334+0000 7f50f9ffb640 1 --2- 172.21.15.165:0/614585031 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50ec036950 0x7f50ec038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.334+0000 7f50f9ffb640 1 -- 172.21.15.165:0/614585031 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f520404dd00 con 0x55a494db8a20 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.334+0000 7f520bfff640 1 --2- 172.21.15.165:0/614585031 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50ec036950 0x7f50ec038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.335+0000 7f521873f800 1 Processor -- start 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.335+0000 7f521873f800 1 -- start start 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.335+0000 7f520bfff640 1 --2- 172.21.15.165:0/614585031 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50ec036950 0x7f50ec038e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55a494c9e610 tx=0x7f51fc005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.335+0000 7f521873f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 0x55a494dc6010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.335+0000 7f521873f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a494b906b0 con 0x55a494dc5bf0 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.335+0000 7f521449b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 0x55a494dc6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.335+0000 7f521449b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 0x55a494dc6010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58466/0 (socket says 172.21.15.165:58466) 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.335+0000 7f521449b640 1 -- 172.21.15.165:0/1689049529 learned_addr learned my addr 172.21.15.165:0/1689049529 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.335+0000 7f521449b640 1 -- 172.21.15.165:0/1689049529 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a494dc65e0 con 0x55a494dc5bf0 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.335+0000 7f521449b640 1 --2- 172.21.15.165:0/1689049529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 0x55a494dc6010 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f520400e570 tx=0x7f520402e810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70e4728c08a87c83 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.336+0000 7f50f37fe640 1 -- 172.21.15.165:0/1689049529 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5204024d60 con 0x55a494dc5bf0 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.336+0000 7f50f37fe640 1 -- 172.21.15.165:0/1689049529 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f520405b070 con 0x55a494dc5bf0 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.336+0000 7f50f37fe640 1 -- 172.21.15.165:0/1689049529 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5204044ce0 con 0x55a494dc5bf0 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.336+0000 7f521873f800 1 -- 172.21.15.165:0/1689049529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 msgr2=0x55a494dc6010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.336+0000 7f521873f800 1 --2- 172.21.15.165:0/1689049529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 0x55a494dc6010 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f520400e570 tx=0x7f520402e810 comp rx=0 tx=0).stop 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.336+0000 7f521873f800 1 -- 172.21.15.165:0/1689049529 shutdown_connections 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.336+0000 7f521873f800 1 --2- 172.21.15.165:0/1689049529 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 0x55a494dc6010 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.336+0000 7f521873f800 1 -- 172.21.15.165:0/1689049529 >> 172.21.15.165:0/1689049529 conn(0x55a494dc1ec0 msgr2=0x55a494dbff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.336+0000 7f521873f800 1 -- 172.21.15.165:0/1689049529 shutdown_connections 2024-04-27T03:30:34.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.336+0000 7f521873f800 1 -- 172.21.15.165:0/1689049529 wait complete. 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.337+0000 7f521873f800 1 Processor -- start 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.337+0000 7f521873f800 1 -- start start 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.337+0000 7f521873f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 0x55a494dcce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.337+0000 7f521873f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a494dc6b30 con 0x55a494dc5bf0 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.337+0000 7f521449b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 0x55a494dcce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.337+0000 7f521449b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 0x55a494dcce30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58478/0 (socket says 172.21.15.165:58478) 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.337+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 learned_addr learned my addr 172.21.15.165:0/3344095880 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.337+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a494dcd370 con 0x55a494dc5bf0 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.337+0000 7f521449b640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 0x55a494dcce30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f520401ef20 tx=0x7f5204025cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.338+0000 7f50f1ffb640 1 -- 172.21.15.165:0/3344095880 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5204044e10 con 0x55a494dc5bf0 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.338+0000 7f50f1ffb640 1 -- 172.21.15.165:0/3344095880 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52040163b0 con 0x55a494dc5bf0 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.338+0000 7f50f1ffb640 1 -- 172.21.15.165:0/3344095880 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5204016690 con 0x55a494dc5bf0 2024-04-27T03:30:34.355 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.338+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a494dcd370 con 0x55a494dc5bf0 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.338+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a494dcd370 con 0x55a494dc5bf0 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.339+0000 7f50f1ffb640 1 -- 172.21.15.165:0/3344095880 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f5204055070 con 0x55a494dc5bf0 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.339+0000 7f50f1ffb640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50e4036950 0x7f50e4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.339+0000 7f50f1ffb640 1 -- 172.21.15.165:0/3344095880 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f5204052030 con 0x55a494dc5bf0 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.339+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.339+0000 7f520bfff640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50e4036950 0x7f50e4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.339+0000 7f521873f800 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a494dd83d0 0x55a494dda890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.339+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddafe0 con 0x55a494dd83d0 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.339+0000 7f5214c9c640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a494dd83d0 0x55a494dda890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.339+0000 7f5214c9c640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a494dd83d0 0x55a494dda890 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.339+0000 7f520bfff640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50e4036950 0x7f50e4038e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f51fc052fd0 tx=0x7f51fc04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 20 realm 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddb3f0 con 0x55a494dd83d0 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddbc90 con 0x55a494dd83d0 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddc030 con 0x55a494dd83d0 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a494ddcd90 0x55a494ddf180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddf6c0 con 0x55a494ddcd90 2024-04-27T03:30:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.340+0000 7f521449b640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a494ddcd90 0x55a494ddf180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.341+0000 7f521449b640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a494ddcd90 0x55a494ddf180 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.341+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.341+0000 7f521873f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.341+0000 7f521873f800 20 searching for the correct realm 2024-04-27T03:30:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.341+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494de1070 con 0x55a494ddcd90 2024-04-27T03:30:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.341+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.341+0000 7f521449b640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f5204017050 0x7f520404c610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.341+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494de1070 con 0x7f5204017050 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.342+0000 7f520bfff640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f5204017050 0x7f520404c610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.342+0000 7f520bfff640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f5204017050 0x7f520404c610 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.342+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f51fc081070 con 0x7f5204017050 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.342+0000 7f520bfff640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f51fc076040 0x7f51fc083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.342+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494de1070 con 0x7f51fc076040 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.342+0000 7f5214c9c640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f51fc076040 0x7f51fc083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.343+0000 7f5214c9c640 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f51fc076040 0x7f51fc083410 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.343+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.343+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddc030 con 0x55a494dd83d0 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.343+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.343+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddc030 con 0x55a494dd83d0 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.343+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.343+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddc030 con 0x7f51fc076040 2024-04-27T03:30:34.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.344+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.344+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddc030 con 0x7f51fc076040 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.344+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.344+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddc030 con 0x7f5204017050 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.344+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f51fc081070 con 0x7f5204017050 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.344+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddc030 con 0x55a494ddcd90 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.344+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.344+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddc030 con 0x55a494dd83d0 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.344+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.344+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddc030 con 0x7f51fc076040 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.345+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.345+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddc030 con 0x55a494dd83d0 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.345+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.345+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55a494ddc030 con 0x7f5204017050 2024-04-27T03:30:34.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.346+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f51fc081070 con 0x7f5204017050 2024-04-27T03:30:34.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.346+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x55a494ddcd90 2024-04-27T03:30:34.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.346+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.346+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x55a494ddcd90 2024-04-27T03:30:34.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.346+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.346+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x7f51fc076040 2024-04-27T03:30:34.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.346+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.346+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x55a494ddcd90 2024-04-27T03:30:34.375 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.347+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.347+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x55a494ddcd90 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.347+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.347+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x55a494dd83d0 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.347+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.347+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x55a494dd83d0 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.348+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.348+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x7f51fc076040 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.348+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.348+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x55a494ddcd90 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.348+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.348+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x7f5204017050 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.348+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f51fc081070 con 0x7f5204017050 2024-04-27T03:30:34.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.348+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x55a494dd83d0 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.349+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.349+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x7f51fc076040 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.349+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.349+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x55a494ddcd90 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.349+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.349+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x7f51fc076040 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.349+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.349+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x7f5204017050 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.350+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f51fc081070 con 0x7f5204017050 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.350+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x55a494ddcd90 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.350+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.350+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x7f51fc076040 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.350+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.350+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f51fc090060 con 0x55a494dd83d0 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f520c038040 con 0x7f5204017050 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f51fc081070 con 0x7f5204017050 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494de0040 con 0x55a494dd83d0 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494de0430 con 0x55a494dd83d0 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.351+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494de0430 con 0x7f51fc076040 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 20 zone default found 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 4 Realm: () 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:34.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 20 zonegroup default 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494de0850 con 0x7f51fc076040 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494de0850 con 0x55a494ddcd90 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.352+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494de2f20 con 0x55a494ddcd90 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.353+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.353+0000 7f521873f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.353+0000 7f521873f800 20 started sync module instance, tier type = 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.353+0000 7f521873f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.353+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de23c0 con 0x7f5204017050 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.354+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f51fc081070 con 0x7f5204017050 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.354+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94165360584624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de3770 con 0x7f5204017050 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.354+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de3fc0 con 0x55a494dd83d0 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.355+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94165360584624] v25'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f51fc081070 con 0x7f5204017050 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.355+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.355+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94165360589584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de43e0 con 0x55a494dd83d0 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.355+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de4c30 con 0x55a494ddcd90 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.356+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94165360589584] v25'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.356+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.356+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94165360591120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de4fd0 con 0x55a494ddcd90 2024-04-27T03:30:34.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.356+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de59d0 con 0x7f5204017050 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.357+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94165360591120] v25'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.357+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f51fc081070 con 0x7f5204017050 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.357+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94165360592272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de5df0 con 0x7f5204017050 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.357+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de67f0 con 0x55a494dd83d0 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.358+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94165360592272] v25'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f51fc081070 con 0x7f5204017050 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.358+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.358+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94165360596192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de6be0 con 0x55a494dd83d0 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.358+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de75a0 con 0x7f51fc076040 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.359+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.359+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94165360596192] v25'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.359+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94165360603376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de7510 con 0x7f51fc076040 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.359+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de6300 con 0x7f51fc076040 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.360+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94165360603376] v25'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.360+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.360+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94165360599808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de8010 con 0x7f51fc076040 2024-04-27T03:30:34.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.360+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de89d0 con 0x7f51fc076040 2024-04-27T03:30:34.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.361+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94165360599808] v25'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.361+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'99 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.361+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94165360605552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de89d0 con 0x7f51fc076040 2024-04-27T03:30:34.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.361+0000 7f521873f800 20 add_watcher() i=0 2024-04-27T03:30:34.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.361+0000 7f521873f800 20 add_watcher() i=1 2024-04-27T03:30:34.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.361+0000 7f521873f800 20 add_watcher() i=2 2024-04-27T03:30:34.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.361+0000 7f521873f800 20 add_watcher() i=3 2024-04-27T03:30:34.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.361+0000 7f521873f800 20 add_watcher() i=4 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.361+0000 7f521873f800 20 add_watcher() i=5 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.361+0000 7f521873f800 20 add_watcher() i=6 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.362+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94165360605552] v25'100 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.362+0000 7f521873f800 20 add_watcher() i=7 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.362+0000 7f521873f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.362+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494debd10 con 0x55a494dd83d0 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.362+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.362+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94165360621360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494de8b40 con 0x55a494dd83d0 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.363+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94165360621360] v25'106 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f50ca7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f521873f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f521873f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f521873f800 5 note: GC not initialized 2024-04-27T03:30:34.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f50c97e2640 20 reqs_thread_entry: start 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f521873f800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f521873f800 20 Filter name: none 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f503affd640 20 reqs_thread_entry: start 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e1f100 con 0x7f5204017050 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f51fc081070 con 0x7f5204017050 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f521873f800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.364+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e1f990 con 0x55a494dd83d0 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(63 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f521873f800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f521873f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e1e4c0 con 0x7f5204017050 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f51fc081070 con 0x7f5204017050 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f521873f800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f521873f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e21010 con 0x55a494dd83d0 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f521873f800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f521873f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.365+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 4.2f 4:f7b16f5a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e20a40 con 0x55a494dd83d0 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 4.16 4:6b5ddb47:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e25930 con 0x55a494ddcd90 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:68 4.7 4:e2bbac01:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e281a0 con 0x7f51fc076040 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 4.3e 4:7e0d8a50:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e2aa10 con 0x7f5204017050 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:70 4.19 4:9b87498d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e2d280 con 0x55a494ddcd90 2024-04-27T03:30:34.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 4.2 4:42daa0e0:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e2fab0 con 0x55a494dd83d0 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 4.1b 4:d9d9fd94:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e32500 con 0x7f5204017050 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:73 4.e 4:70dfcb91:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e32540 con 0x7f5204017050 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:74 4.3d 4:bd27934b:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e37490 con 0x55a494ddcd90 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:75 4.5 4:a1c15390:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e39ee0 con 0x55a494ddcd90 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:76 4.1a 4:59a985c3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55a494e3cda0 con 0x7f5204017050 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(70 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(71 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.5 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(68 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f51fc095040 con 0x7f5204017050 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(74 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(69 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f51fc095040 con 0x7f5204017050 2024-04-27T03:30:34.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(75 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(73 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f51fc095040 con 0x7f5204017050 2024-04-27T03:30:34.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.366+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(76 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f51fc095040 con 0x7f5204017050 2024-04-27T03:30:34.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.367+0000 7f50ca7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.367+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94165360621360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494e3cda0 con 0x55a494dd83d0 2024-04-27T03:30:34.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.368+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94165360621360] v25'107 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.368+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94165360584624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494e3cda0 con 0x7f5204017050 2024-04-27T03:30:34.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.369+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94165360584624] v25'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f51fc095040 con 0x7f5204017050 2024-04-27T03:30:34.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.369+0000 7f521873f800 20 remove_watcher() i=0 2024-04-27T03:30:34.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.369+0000 7f521873f800 2 removed watcher, disabling cache 2024-04-27T03:30:34.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.369+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94165360589584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494e3cda0 con 0x55a494dd83d0 2024-04-27T03:30:34.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.370+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94165360589584] v25'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.370+0000 7f521873f800 20 remove_watcher() i=1 2024-04-27T03:30:34.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.370+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94165360591120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494e3cda0 con 0x55a494ddcd90 2024-04-27T03:30:34.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.371+0000 7f521449b640 1 -- 172.21.15.165:0/3344095880 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94165360591120] v25'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520402b070 con 0x55a494ddcd90 2024-04-27T03:30:34.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.371+0000 7f521873f800 20 remove_watcher() i=2 2024-04-27T03:30:34.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.371+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94165360592272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494e3cda0 con 0x7f5204017050 2024-04-27T03:30:34.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.372+0000 7f520bfff640 1 -- 172.21.15.165:0/3344095880 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94165360592272] v25'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f51fc095040 con 0x7f5204017050 2024-04-27T03:30:34.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.372+0000 7f521873f800 20 remove_watcher() i=3 2024-04-27T03:30:34.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.372+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94165360596192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494e3cda0 con 0x55a494dd83d0 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.373+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94165360596192] v25'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x55a494dd83d0 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.373+0000 7f521873f800 20 remove_watcher() i=4 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.373+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94165360603376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494e3cda0 con 0x7f51fc076040 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.374+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94165360603376] v25'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.374+0000 7f521873f800 20 remove_watcher() i=5 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.374+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94165360599808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494e3cda0 con 0x7f51fc076040 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.375+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94165360599808] v25'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.375+0000 7f521873f800 20 remove_watcher() i=6 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.375+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94165360605552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55a494e3cda0 con 0x7f51fc076040 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94165360605552] v25'101 uv99 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f520c00a4a0 con 0x7f51fc076040 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 20 remove_watcher() i=7 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f51fc076040 msgr2=0x7f51fc083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f51fc076040 0x7f51fc083410 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a494dd83d0 msgr2=0x55a494dda890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a494dd83d0 0x55a494dda890 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f5204017050 msgr2=0x7f520404c610 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f5204017050 0x7f520404c610 crc :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a494ddcd90 msgr2=0x55a494ddf180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a494ddcd90 0x55a494ddf180 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50e4036950 msgr2=0x7f50e4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50e4036950 0x7f50e4038e10 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f51fc052fd0 tx=0x7f51fc04f000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 msgr2=0x55a494dcce30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 0x55a494dcce30 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f520401ef20 tx=0x7f5204025cf0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f5214c9c640 1 -- 172.21.15.165:0/3344095880 reap_dead start 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 shutdown_connections 2024-04-27T03:30:34.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 --2- 172.21.15.165:0/3344095880 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494dc5bf0 0x55a494dcce30 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 >> 172.21.15.165:0/3344095880 conn(0x55a494dc1ec0 msgr2=0x55a494dbf520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 shutdown_connections 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.376+0000 7f521873f800 1 -- 172.21.15.165:0/3344095880 wait complete. 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/614585031 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50ec036950 msgr2=0x7f50ec038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 --2- 172.21.15.165:0/614585031 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50ec036950 0x7f50ec038e10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x55a494c9e610 tx=0x7f51fc005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/614585031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494db8a20 msgr2=0x55a494db8e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 --2- 172.21.15.165:0/614585031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494db8a20 0x55a494db8e40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f520404aa00 tx=0x7f520402caa0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/614585031 shutdown_connections 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 --2- 172.21.15.165:0/614585031 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50ec036950 0x7f50ec038e10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 --2- 172.21.15.165:0/614585031 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494db8a20 0x55a494db8e40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/614585031 >> 172.21.15.165:0/614585031 conn(0x55a494dacac0 msgr2=0x55a494dacec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/614585031 shutdown_connections 2024-04-27T03:30:34.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/614585031 wait complete. 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55a494da7b00 msgr2=0x55a494da9f70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55a494da7b00 0x55a494da9f70 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a494ca3170 msgr2=0x55a494da10c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a494ca3170 0x55a494da10c0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a494da39b0 msgr2=0x55a494da5e70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a494da39b0 0x55a494da5e70 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50fc036900 msgr2=0x7f50fc038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f50fc036900 0x7f50fc038dc0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f51fc054000 tx=0x7f51fc04e000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 msgr2=0x55a494c9f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 --2- 172.21.15.165:0/977200307 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a494c12a50 0x55a494c9f350 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f520400c010 tx=0x7f5204000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f5214c9c640 1 -- 172.21.15.165:0/977200307 reap_dead start 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 shutdown_connections 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 >> 172.21.15.165:0/977200307 conn(0x55a494c99320 msgr2=0x55a494d8a240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 shutdown_connections 2024-04-27T03:30:34.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.377+0000 7f521873f800 1 -- 172.21.15.165:0/977200307 wait complete. 2024-04-27T03:30:34.398 INFO:teuthology.orchestra.run.smithi165.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"fd642a41-253a-48d4-9473-6eca68174247","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1","marker":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-04-27T03:30:33.428997Z","creation_time":"2024-04-27T03:30:33.422040Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{"rgw.main":{"size":3,"size_actual":4096,"size_utilized":3,"size_kb":1,"size_kb_actual":4,"size_kb_utilized":1,"num_objects":1}},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}} 2024-04-27T03:30:34.399 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'fd642a41-253a-48d4-9473-6eca68174247', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1', 'marker': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T03:30:33.428997Z', 'creation_time': '2024-04-27T03:30:33.422040Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T03:30:34.399 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-04-27T03:30:34.399 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'stats', '--uid', 'foo', '--sync-stats'] 2024-04-27T03:30:34.399 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user stats --uid foo --sync-stats 2024-04-27T03:30:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:34.482 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:34.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.477+0000 7f93a543f800 1 Processor -- start 2024-04-27T03:30:34.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.477+0000 7f93a543f800 1 -- start start 2024-04-27T03:30:34.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.477+0000 7f93a543f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x557c78ee7350 0x557c78ee9740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.478+0000 7f93a543f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557c78da1d40 con 0x557c78ee2fb0 2024-04-27T03:30:34.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.478+0000 7f93a543f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x557c78d744d0 con 0x557c78ee7350 2024-04-27T03:30:34.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.478+0000 7f939a73d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x557c78ee7350 0x557c78ee9740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.478+0000 7f939a73d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x557c78ee7350 0x557c78ee9740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58486/0 (socket says 172.21.15.165:58486) 2024-04-27T03:30:34.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.478+0000 7f939a73d640 1 -- 172.21.15.165:0/2597059894 learned_addr learned my addr 172.21.15.165:0/2597059894 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.478+0000 7f93a2f3e640 1 --1- 172.21.15.165:0/2597059894 >> v1:172.21.15.89:6789/0 conn(0x557c78ee2fb0 0x557c78ee33d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35310/0 (socket says 172.21.15.165:35310) 2024-04-27T03:30:34.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.478+0000 7f939a73d640 1 -- 172.21.15.165:0/2597059894 >> v1:172.21.15.89:6789/0 conn(0x557c78ee2fb0 legacy=0x557c78ee33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.478+0000 7f939a73d640 1 -- 172.21.15.165:0/2597059894 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c78da0590 con 0x557c78ee7350 2024-04-27T03:30:34.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.480+0000 7f939a73d640 1 --2- 172.21.15.165:0/2597059894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee7350 0x557c78ee9740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f938c00a520 tx=0x7f938c01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5592796eb0a6d72 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.480+0000 7f93a273d640 1 -- 172.21.15.165:0/2597059894 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f938c057070 con 0x557c78ee7350 2024-04-27T03:30:34.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.480+0000 7f93a273d640 1 -- 172.21.15.165:0/2597059894 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f938c052570 con 0x557c78ee7350 2024-04-27T03:30:34.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.480+0000 7f93a273d640 1 -- 172.21.15.165:0/2597059894 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f938c04ae60 con 0x557c78ee7350 2024-04-27T03:30:34.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.481+0000 7f93a543f800 1 -- 172.21.15.165:0/2597059894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee7350 msgr2=0x557c78ee9740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.481+0000 7f93a543f800 1 --2- 172.21.15.165:0/2597059894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee7350 0x557c78ee9740 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f938c00a520 tx=0x7f938c01d010 comp rx=0 tx=0).stop 2024-04-27T03:30:34.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.481+0000 7f93a543f800 1 -- 172.21.15.165:0/2597059894 shutdown_connections 2024-04-27T03:30:34.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.481+0000 7f93a543f800 1 --2- 172.21.15.165:0/2597059894 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee7350 0x557c78ee9740 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.481+0000 7f93a543f800 1 -- 172.21.15.165:0/2597059894 >> 172.21.15.165:0/2597059894 conn(0x557c78ede550 msgr2=0x557c78ee0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.481+0000 7f93a543f800 1 -- 172.21.15.165:0/2597059894 shutdown_connections 2024-04-27T03:30:34.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.481+0000 7f93a543f800 1 -- 172.21.15.165:0/2597059894 wait complete. 2024-04-27T03:30:34.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.492+0000 7f93a543f800 1 Processor -- start 2024-04-27T03:30:34.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.492+0000 7f93a543f800 1 -- start start 2024-04-27T03:30:34.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.493+0000 7f93a543f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 0x557c78fd4110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.493+0000 7f93a543f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557c78e2e3b0 con 0x557c78ee2fb0 2024-04-27T03:30:34.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.493+0000 7f93a2f3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 0x557c78fd4110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.493+0000 7f93a2f3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 0x557c78fd4110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58492/0 (socket says 172.21.15.165:58492) 2024-04-27T03:30:34.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.493+0000 7f93a2f3e640 1 -- 172.21.15.165:0/3648258772 learned_addr learned my addr 172.21.15.165:0/3648258772 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.493+0000 7f93a2f3e640 1 -- 172.21.15.165:0/3648258772 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c78fd46e0 con 0x557c78ee2fb0 2024-04-27T03:30:34.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.493+0000 7f93a2f3e640 1 --2- 172.21.15.165:0/3648258772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 0x557c78fd4110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f939400b230 tx=0x7f939400b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9168d5782d8c7d62 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.493+0000 7f9290e09640 1 -- 172.21.15.165:0/3648258772 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f939400e5c0 con 0x557c78ee2fb0 2024-04-27T03:30:34.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.493+0000 7f9290e09640 1 -- 172.21.15.165:0/3648258772 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9394006070 con 0x557c78ee2fb0 2024-04-27T03:30:34.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.493+0000 7f9290e09640 1 -- 172.21.15.165:0/3648258772 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f939400e890 con 0x557c78ee2fb0 2024-04-27T03:30:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.494+0000 7f93a543f800 1 -- 172.21.15.165:0/3648258772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 msgr2=0x557c78fd4110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.494+0000 7f93a543f800 1 --2- 172.21.15.165:0/3648258772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 0x557c78fd4110 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f939400b230 tx=0x7f939400b6f0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.494+0000 7f93a543f800 1 -- 172.21.15.165:0/3648258772 shutdown_connections 2024-04-27T03:30:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.494+0000 7f93a543f800 1 --2- 172.21.15.165:0/3648258772 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 0x557c78fd4110 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.494+0000 7f93a543f800 1 -- 172.21.15.165:0/3648258772 >> 172.21.15.165:0/3648258772 conn(0x557c78ede550 msgr2=0x557c78fcfa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.494+0000 7f93a543f800 1 -- 172.21.15.165:0/3648258772 shutdown_connections 2024-04-27T03:30:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.494+0000 7f93a543f800 1 -- 172.21.15.165:0/3648258772 wait complete. 2024-04-27T03:30:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.495+0000 7f93a543f800 1 Processor -- start 2024-04-27T03:30:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.495+0000 7f93a543f800 1 -- start start 2024-04-27T03:30:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.495+0000 7f93a543f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 0x557c78e58970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.495+0000 7f93a543f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557c78eea010 con 0x557c78ee2fb0 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.495+0000 7f93a2f3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 0x557c78e58970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.495+0000 7f93a2f3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 0x557c78e58970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58496/0 (socket says 172.21.15.165:58496) 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.495+0000 7f93a2f3e640 1 -- 172.21.15.165:0/3390187362 learned_addr learned my addr 172.21.15.165:0/3390187362 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.495+0000 7f93a2f3e640 1 -- 172.21.15.165:0/3390187362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c78e58eb0 con 0x557c78ee2fb0 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.495+0000 7f93a2f3e640 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 0x557c78e58970 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9394000fd0 tx=0x7f9394000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.496+0000 7f928f606640 1 -- 172.21.15.165:0/3390187362 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f939400f8d0 con 0x557c78ee2fb0 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.496+0000 7f928f606640 1 -- 172.21.15.165:0/3390187362 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939400fa90 con 0x557c78ee2fb0 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.496+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c78e570b0 con 0x557c78ee2fb0 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.496+0000 7f928f606640 1 -- 172.21.15.165:0/3390187362 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f939400fdb0 con 0x557c78ee2fb0 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.496+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c78e575d0 con 0x557c78ee2fb0 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.496+0000 7f928f606640 1 -- 172.21.15.165:0/3390187362 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9394017070 con 0x557c78ee2fb0 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.497+0000 7f928f606640 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9288036900 0x7f9288038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.497+0000 7f928f606640 1 -- 172.21.15.165:0/3390187362 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f9394015020 con 0x557c78ee2fb0 2024-04-27T03:30:34.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.497+0000 7f939a73d640 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9288036900 0x7f9288038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.497+0000 7f93a543f800 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557c78ee7350 0x557c78fe5100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.497+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c78fe5680 con 0x557c78ee7350 2024-04-27T03:30:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.497+0000 7f93a373f640 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557c78ee7350 0x557c78fe5100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.497+0000 7f939a73d640 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9288036900 0x7f9288038dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f938c054000 tx=0x7f938c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.497+0000 7f93a373f640 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557c78ee7350 0x557c78fe5100 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.497+0000 7f93a373f640 1 -- 172.21.15.165:0/3390187362 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c78ee7350 2024-04-27T03:30:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.498+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c78e57ad0 con 0x557c78ee7350 2024-04-27T03:30:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.498+0000 7f93a373f640 1 -- 172.21.15.165:0/3390187362 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c78ee7350 2024-04-27T03:30:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.498+0000 7f93a543f800 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557c78fe79b0 0x557c78fe9da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.498+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c78fea490 con 0x557c78fe79b0 2024-04-27T03:30:34.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.498+0000 7f93a2f3e640 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557c78fe79b0 0x557c78fe9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.498+0000 7f93a2f3e640 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557c78fe79b0 0x557c78fe9da0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.498+0000 7f93a2f3e640 1 -- 172.21.15.165:0/3390187362 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f939402e070 con 0x557c78fe79b0 2024-04-27T03:30:34.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.498+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c78fe6960 con 0x557c78ee7350 2024-04-27T03:30:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.499+0000 7f93a373f640 1 -- 172.21.15.165:0/3390187362 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c78ee7350 2024-04-27T03:30:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.499+0000 7f93a543f800 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x557c78feb940 0x557c78fedd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.499+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c78fee4f0 con 0x557c78feb940 2024-04-27T03:30:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.499+0000 7f939a73d640 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x557c78feb940 0x557c78fedd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.499+0000 7f939a73d640 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x557c78feb940 0x557c78fedd90 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.500+0000 7f939a73d640 1 -- 172.21.15.165:0/3390187362 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f938c05a040 con 0x557c78feb940 2024-04-27T03:30:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.500+0000 7f93a543f800 1 Processor -- start 2024-04-27T03:30:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.500+0000 7f93a543f800 1 -- start start 2024-04-27T03:30:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.500+0000 7f93a543f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ffc950 0x557c78ffcd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.500+0000 7f93a543f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557c78dd46b0 con 0x557c78ffc950 2024-04-27T03:30:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.500+0000 7f93a2f3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ffc950 0x557c78ffcd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.500+0000 7f93a2f3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ffc950 0x557c78ffcd70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58504/0 (socket says 172.21.15.165:58504) 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.500+0000 7f93a2f3e640 1 -- 172.21.15.165:0/1500497900 learned_addr learned my addr 172.21.15.165:0/1500497900 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.501+0000 7f93a2f3e640 1 -- 172.21.15.165:0/1500497900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c78ffd2b0 con 0x557c78ffc950 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.501+0000 7f93a2f3e640 1 --2- 172.21.15.165:0/1500497900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ffc950 0x557c78ffcd70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f939400ff70 tx=0x7f939404cba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.501+0000 7f928ce01640 1 -- 172.21.15.165:0/1500497900 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f939404ccd0 con 0x557c78ffc950 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.501+0000 7f928ce01640 1 -- 172.21.15.165:0/1500497900 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939401a480 con 0x557c78ffc950 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.501+0000 7f93a543f800 1 -- 172.21.15.165:0/1500497900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c78ffdd20 con 0x557c78ffc950 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.501+0000 7f928ce01640 1 -- 172.21.15.165:0/1500497900 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f939401a760 con 0x557c78ffc950 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.501+0000 7f93a543f800 1 -- 172.21.15.165:0/1500497900 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c78ffe220 con 0x557c78ffc950 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.502+0000 7f928ce01640 1 -- 172.21.15.165:0/1500497900 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9394040900 con 0x557c78ffc950 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.502+0000 7f928ce01640 1 --2- 172.21.15.165:0/1500497900 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f927c036900 0x7f927c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.502+0000 7f928ce01640 1 -- 172.21.15.165:0/1500497900 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f939403cae0 con 0x557c78ffc950 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.502+0000 7f939a73d640 1 --2- 172.21.15.165:0/1500497900 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f927c036900 0x7f927c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.502+0000 7f93a543f800 1 Processor -- start 2024-04-27T03:30:34.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.502+0000 7f93a543f800 1 -- start start 2024-04-27T03:30:34.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.502+0000 7f939a73d640 1 --2- 172.21.15.165:0/1500497900 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f927c036900 0x7f927c038dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x557c78fe5f60 tx=0x7f938c005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.502+0000 7f93a543f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 0x557c79009e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.502+0000 7f93a543f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557c78fd4a60 con 0x557c79009a10 2024-04-27T03:30:34.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.503+0000 7f93a2f3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 0x557c79009e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.503+0000 7f93a2f3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 0x557c79009e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58516/0 (socket says 172.21.15.165:58516) 2024-04-27T03:30:34.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.503+0000 7f93a2f3e640 1 -- 172.21.15.165:0/1345221396 learned_addr learned my addr 172.21.15.165:0/1345221396 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.503+0000 7f93a2f3e640 1 -- 172.21.15.165:0/1345221396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c7900a400 con 0x557c79009a10 2024-04-27T03:30:34.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.503+0000 7f93a2f3e640 1 --2- 172.21.15.165:0/1345221396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 0x557c79009e30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f939401e0c0 tx=0x7f939403f2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504e0e6ea7f41069 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.503+0000 7f92867fc640 1 -- 172.21.15.165:0/1345221396 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f939405b020 con 0x557c79009a10 2024-04-27T03:30:34.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.503+0000 7f92867fc640 1 -- 172.21.15.165:0/1345221396 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f939403ce40 con 0x557c79009a10 2024-04-27T03:30:34.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.503+0000 7f92867fc640 1 -- 172.21.15.165:0/1345221396 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9394031d10 con 0x557c79009a10 2024-04-27T03:30:34.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a543f800 1 -- 172.21.15.165:0/1345221396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 msgr2=0x557c79009e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a543f800 1 --2- 172.21.15.165:0/1345221396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 0x557c79009e30 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f939401e0c0 tx=0x7f939403f2a0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a543f800 1 -- 172.21.15.165:0/1345221396 shutdown_connections 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a543f800 1 --2- 172.21.15.165:0/1345221396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 0x557c79009e30 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a543f800 1 -- 172.21.15.165:0/1345221396 >> 172.21.15.165:0/1345221396 conn(0x557c79005d70 msgr2=0x557c79003dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a543f800 1 -- 172.21.15.165:0/1345221396 shutdown_connections 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a543f800 1 -- 172.21.15.165:0/1345221396 wait complete. 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a543f800 1 Processor -- start 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a543f800 1 -- start start 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a543f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 0x557c79010de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a543f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557c7900aab0 con 0x557c79009a10 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a2f3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 0x557c79010de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a2f3e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 0x557c79010de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58518/0 (socket says 172.21.15.165:58518) 2024-04-27T03:30:34.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.504+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 learned_addr learned my addr 172.21.15.165:0/2322324071 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.505+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557c79011320 con 0x557c79009a10 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.505+0000 7f93a2f3e640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 0x557c79010de0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9394005bb0 tx=0x7f939401e0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.505+0000 7f9284ff9640 1 -- 172.21.15.165:0/2322324071 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9394070070 con 0x557c79009a10 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.505+0000 7f9284ff9640 1 -- 172.21.15.165:0/2322324071 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9394026bb0 con 0x557c79009a10 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.505+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x557c79011320 con 0x557c79009a10 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.505+0000 7f9284ff9640 1 -- 172.21.15.165:0/2322324071 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f939401ccb0 con 0x557c79009a10 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.505+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557c79011320 con 0x557c79009a10 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.506+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.506+0000 7f9284ff9640 1 -- 172.21.15.165:0/2322324071 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9394073070 con 0x557c79009a10 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.506+0000 7f9284ff9640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f926c036950 0x7f926c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.506+0000 7f9284ff9640 1 -- 172.21.15.165:0/2322324071 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f9394068070 con 0x557c79009a10 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.506+0000 7f939a73d640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f926c036950 0x7f926c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a543f800 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557c7901c510 0x557c7901e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c7901f120 con 0x557c7901c510 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a373f640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557c7901c510 0x557c7901e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f939a73d640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f926c036950 0x7f926c038e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x557c78fee8f0 tx=0x7f938c04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a373f640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557c7901c510 0x557c7901e9d0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c7901c510 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a543f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a543f800 20 realm 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c7901f580 con 0x557c7901c510 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c7901c510 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a543f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a543f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.507+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c7901fe20 con 0x557c7901c510 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c7901c510 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a543f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c790201c0 con 0x557c7901c510 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c7901c510 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a543f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a543f800 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557c79020f20 0x557c79023310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c79023890 con 0x557c79020f20 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a2f3e640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557c79020f20 0x557c79023310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a2f3e640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557c79020f20 0x557c79023310 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a543f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:34.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.508+0000 7f93a543f800 20 searching for the correct realm 2024-04-27T03:30:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.509+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c79025210 con 0x557c79020f20 2024-04-27T03:30:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.509+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.509+0000 7f93a2f3e640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9394075060 0x7f939403c4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.509+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c79025210 con 0x7f9394075060 2024-04-27T03:30:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.509+0000 7f939a73d640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9394075060 0x7f939403c4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.509+0000 7f939a73d640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9394075060 0x7f939403c4f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.510+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.510+0000 7f939a73d640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f938c074040 0x7f938c081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.510+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c79025210 con 0x7f938c074040 2024-04-27T03:30:34.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.510+0000 7f93a373f640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f938c074040 0x7f938c081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.511+0000 7f93a373f640 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f938c074040 0x7f938c081410 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.511+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f939c00a4a0 con 0x7f938c074040 2024-04-27T03:30:34.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.511+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c790201c0 con 0x557c7901c510 2024-04-27T03:30:34.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.511+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c7901c510 2024-04-27T03:30:34.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.511+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c790201c0 con 0x557c7901c510 2024-04-27T03:30:34.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.511+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c7901c510 2024-04-27T03:30:34.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.511+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c790201c0 con 0x7f938c074040 2024-04-27T03:30:34.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.511+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c00a4a0 con 0x7f938c074040 2024-04-27T03:30:34.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.511+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c790201c0 con 0x7f938c074040 2024-04-27T03:30:34.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.512+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c00a4a0 con 0x7f938c074040 2024-04-27T03:30:34.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.512+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c790201c0 con 0x7f9394075060 2024-04-27T03:30:34.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.512+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.512+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c790201c0 con 0x557c79020f20 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.512+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.512+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c790201c0 con 0x557c7901c510 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.512+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c7901c510 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.512+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c790201c0 con 0x7f938c074040 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.513+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c00a4a0 con 0x7f938c074040 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.513+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c790201c0 con 0x557c7901c510 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.513+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c7901c510 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.513+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x557c790201c0 con 0x7f9394075060 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.513+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.513+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x557c79020f20 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.513+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.513+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x557c79020f20 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.514+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.514+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x7f938c074040 2024-04-27T03:30:34.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.514+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c00a4a0 con 0x7f938c074040 2024-04-27T03:30:34.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.514+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x557c79020f20 2024-04-27T03:30:34.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.514+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.514+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x557c79020f20 2024-04-27T03:30:34.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.514+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.514+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x557c7901c510 2024-04-27T03:30:34.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.515+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c7901c510 2024-04-27T03:30:34.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.515+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x557c7901c510 2024-04-27T03:30:34.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.515+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f939c00a4a0 con 0x557c7901c510 2024-04-27T03:30:34.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.515+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x7f938c074040 2024-04-27T03:30:34.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.515+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c033040 con 0x7f938c074040 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.515+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x557c79020f20 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.516+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.516+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x7f9394075060 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.516+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.516+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x557c7901c510 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.516+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.516+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x7f938c074040 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.516+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.516+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x557c79020f20 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.517+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.517+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x7f938c074040 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.517+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.517+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x7f9394075060 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.518+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.518+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x557c79020f20 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.518+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.518+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x7f938c074040 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.518+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.518+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x557c7901c510 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.518+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.518+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f938c08e060 con 0x7f9394075060 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:34.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:34.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:34.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c790241e0 con 0x557c7901c510 2024-04-27T03:30:34.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c790245d0 con 0x557c7901c510 2024-04-27T03:30:34.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.519+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c790245d0 con 0x7f938c074040 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 20 zone default found 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 4 Realm: () 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 20 zonegroup default 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c790249f0 con 0x7f938c074040 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c79024e10 con 0x557c79020f20 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.520+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c790270c0 con 0x557c79020f20 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.521+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.521+0000 7f93a543f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.521+0000 7f93a543f800 20 started sync module instance, tier type = 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.521+0000 7f93a543f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.521+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c79026560 con 0x7f9394075060 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.522+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.522+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93993094506880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c790261c0 con 0x7f9394075060 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.522+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c79027e00 con 0x557c7901c510 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.523+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93993094506880] v25'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.523+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.523+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93993094509664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c790281f0 con 0x557c7901c510 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.523+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c79028c20 con 0x557c79020f20 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.524+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93993094509664] v25'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.524+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.524+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93993094510720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c79029010 con 0x557c79020f20 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.524+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c79029a30 con 0x7f9394075060 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.525+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93993094510720] v25'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.525+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.525+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93993094514480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7902a2d0 con 0x7f9394075060 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.525+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7902ac90 con 0x557c7901c510 2024-04-27T03:30:34.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.526+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.526+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93993094523328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7902b490 con 0x557c7901c510 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.526+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7902bf80 con 0x7f938c074040 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.526+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93993094514480] v25'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.527+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.527+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93993094527568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7902bf40 con 0x7f938c074040 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.527+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7902c990 con 0x7f938c074040 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.527+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93993094523328] v25'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.528+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93993094527568] v25'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.528+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.528+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93993094520352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7902c4e0 con 0x7f938c074040 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.528+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7902cea0 con 0x7f938c074040 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.529+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93993094520352] v25'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.529+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'102 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.529+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93993094524640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7902cea0 con 0x7f938c074040 2024-04-27T03:30:34.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.529+0000 7f93a543f800 20 add_watcher() i=0 2024-04-27T03:30:34.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.529+0000 7f93a543f800 20 add_watcher() i=1 2024-04-27T03:30:34.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.529+0000 7f93a543f800 20 add_watcher() i=2 2024-04-27T03:30:34.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.529+0000 7f93a543f800 20 add_watcher() i=3 2024-04-27T03:30:34.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.529+0000 7f93a543f800 20 add_watcher() i=4 2024-04-27T03:30:34.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.529+0000 7f93a543f800 20 add_watcher() i=5 2024-04-27T03:30:34.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.529+0000 7f93a543f800 20 add_watcher() i=6 2024-04-27T03:30:34.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.530+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93993094524640] v25'103 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.530+0000 7f93a543f800 20 add_watcher() i=7 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.530+0000 7f93a543f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.530+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c790301c0 con 0x557c7901c510 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.531+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.531+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93993094531936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7902d430 con 0x557c7901c510 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.532+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93993094531936] v25'108 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.532+0000 7f92597e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.532+0000 7f93a543f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.532+0000 7f93a543f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.532+0000 7f93a543f800 5 note: GC not initialized 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.532+0000 7f91d3fff640 20 reqs_thread_entry: start 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.532+0000 7f93a543f800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.532+0000 7f93a543f800 20 Filter name: none 2024-04-27T03:30:34.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.532+0000 7f91d1ffb640 20 reqs_thread_entry: start 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.532+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.532+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c79063020 con 0x557c7901c510 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.533+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.533+0000 7f93a543f800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.533+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c79064570 con 0x7f9394075060 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.533+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv14 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.533+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.533+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c79064570 con 0x557c7901c510 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.533+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.533+0000 7f93a543f800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.533+0000 7f93a543f800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 4.2f 4:f7b16f5a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c79066070 con 0x557c7901c510 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 4.16 4:6b5ddb47:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c79068d00 con 0x557c79020f20 2024-04-27T03:30:34.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:67 4.7 4:e2bbac01:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c7906b660 con 0x7f938c074040 2024-04-27T03:30:34.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 4.3e 4:7e0d8a50:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c7906dfc0 con 0x7f9394075060 2024-04-27T03:30:34.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 4.19 4:9b87498d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c79070b90 con 0x557c79020f20 2024-04-27T03:30:34.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 4.2 4:42daa0e0:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c79073500 con 0x557c7901c510 2024-04-27T03:30:34.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 4.1b 4:d9d9fd94:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c79076070 con 0x7f9394075060 2024-04-27T03:30:34.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 4.e 4:70dfcb91:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c790788e0 con 0x7f9394075060 2024-04-27T03:30:34.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 4.3d 4:bd27934b:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c7907b4d0 con 0x557c79020f20 2024-04-27T03:30:34.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:74 4.5 4:a1c15390:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x557c79020f20 2024-04-27T03:30:34.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:75 4.1a 4:59a985c3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c79080fb0 con 0x7f9394075060 2024-04-27T03:30:34.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(70 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.5 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(68 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(69 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(67 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(71 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(73 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(74 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(75 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.534+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(72 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.535+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:76 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x7f9394075060 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.536+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(76 foo.buckets [call] v25'15 uv15 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.536+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:77 8.9 8:96d98480:users.uid::foo.buckets:head [call user.complete_stats_sync in=37b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x7f9394075060 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.537+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(77 foo.buckets [call] v25'16 uv16 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.537+0000 7f93a543f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.537+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:78 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x557c7901c510 2024-04-27T03:30:34.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.537+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(78 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.537+0000 7f93a543f800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.537+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:79 8.9 8:96d98480:users.uid::foo.buckets:head [call user.get_header in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x7f9394075060 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.538+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(79 foo.buckets [call out=58b] v0'0 uv16 ondisk = 0) ==== 155+0+58 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.538+0000 7f92597e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.538+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93993094531936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x557c7901c510 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.539+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(80 data_loggenerations_metadata [watch unwatch cookie 93993094531936] v25'109 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.539+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:81 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93993094506880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x7f9394075060 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.540+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(81 notify.0 [watch unwatch cookie 93993094506880] v25'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.540+0000 7f93a543f800 20 remove_watcher() i=0 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.540+0000 7f93a543f800 2 removed watcher, disabling cache 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.540+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:82 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93993094509664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x557c7901c510 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.542+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(82 notify.1 [watch unwatch cookie 93993094509664] v25'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.542+0000 7f93a543f800 20 remove_watcher() i=1 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.542+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:83 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93993094510720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x557c79020f20 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.543+0000 7f93a2f3e640 1 -- 172.21.15.165:0/2322324071 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(83 notify.2 [watch unwatch cookie 93993094510720] v25'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9394072030 con 0x557c79020f20 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.543+0000 7f93a543f800 20 remove_watcher() i=2 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.543+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:84 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93993094514480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x7f9394075060 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.544+0000 7f939a73d640 1 -- 172.21.15.165:0/2322324071 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(84 notify.3 [watch unwatch cookie 93993094514480] v25'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f938c07f070 con 0x7f9394075060 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.544+0000 7f93a543f800 20 remove_watcher() i=3 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.544+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:85 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93993094523328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x557c7901c510 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.545+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(85 notify.4 [watch unwatch cookie 93993094523328] v25'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x557c7901c510 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.545+0000 7f93a543f800 20 remove_watcher() i=4 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.545+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:86 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93993094527568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x7f938c074040 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.545+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(86 notify.5 [watch unwatch cookie 93993094527568] v25'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.546+0000 7f93a543f800 20 remove_watcher() i=5 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.546+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:87 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93993094520352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x7f938c074040 2024-04-27T03:30:34.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.547+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(87 notify.6 [watch unwatch cookie 93993094520352] v25'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.547+0000 7f93a543f800 20 remove_watcher() i=6 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.547+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93993094524640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x557c7907e230 con 0x7f938c074040 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(88 notify.7 [watch unwatch cookie 93993094524640] v25'104 uv102 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f939c031040 con 0x7f938c074040 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 20 remove_watcher() i=7 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f938c074040 msgr2=0x7f938c081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f938c074040 0x7f938c081410 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557c7901c510 msgr2=0x557c7901e9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557c7901c510 0x557c7901e9d0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9394075060 msgr2=0x7f939403c4f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9394075060 0x7f939403c4f0 crc :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557c79020f20 msgr2=0x557c79023310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557c79020f20 0x557c79023310 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f926c036950 msgr2=0x7f926c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f926c036950 0x7f926c038e10 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x557c78fee8f0 tx=0x7f938c04f000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 msgr2=0x557c79010de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a373f640 1 -- 172.21.15.165:0/2322324071 reap_dead start 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 0x557c79010de0 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f9394005bb0 tx=0x7f939401e0f0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 shutdown_connections 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 --2- 172.21.15.165:0/2322324071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c79009a10 0x557c79010de0 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 >> 172.21.15.165:0/2322324071 conn(0x557c79005d70 msgr2=0x557c790033d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 shutdown_connections 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/2322324071 wait complete. 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/1500497900 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f927c036900 msgr2=0x7f927c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 --2- 172.21.15.165:0/1500497900 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f927c036900 0x7f927c038dc0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x557c78fe5f60 tx=0x7f938c005ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/1500497900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ffc950 msgr2=0x557c78ffcd70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 --2- 172.21.15.165:0/1500497900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ffc950 0x557c78ffcd70 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f939400ff70 tx=0x7f939404cba0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/1500497900 shutdown_connections 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 --2- 172.21.15.165:0/1500497900 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f927c036900 0x7f927c038dc0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 --2- 172.21.15.165:0/1500497900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ffc950 0x557c78ffcd70 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/1500497900 >> 172.21.15.165:0/1500497900 conn(0x557c78ff09f0 msgr2=0x557c78ff0df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/1500497900 shutdown_connections 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.548+0000 7f93a543f800 1 -- 172.21.15.165:0/1500497900 wait complete. 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x557c78feb940 msgr2=0x557c78fedd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x557c78feb940 0x557c78fedd90 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557c78ee7350 msgr2=0x557c78fe5100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557c78ee7350 0x557c78fe5100 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557c78fe79b0 msgr2=0x557c78fe9da0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557c78fe79b0 0x557c78fe9da0 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9288036900 msgr2=0x7f9288038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9288036900 0x7f9288038dc0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f938c054000 tx=0x7f938c04e000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 msgr2=0x557c78e58970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 --2- 172.21.15.165:0/3390187362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557c78ee2fb0 0x557c78e58970 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9394000fd0 tx=0x7f9394000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a373f640 1 -- 172.21.15.165:0/3390187362 reap_dead start 2024-04-27T03:30:34.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 shutdown_connections 2024-04-27T03:30:34.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 >> 172.21.15.165:0/3390187362 conn(0x557c78ede550 msgr2=0x557c78fce230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 shutdown_connections 2024-04-27T03:30:34.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.549+0000 7f93a543f800 1 -- 172.21.15.165:0/3390187362 wait complete. 2024-04-27T03:30:34.570 INFO:teuthology.orchestra.run.smithi165.stdout:{"stats":{"size":3,"size_actual":4096,"size_kb":1,"size_kb_actual":4,"num_objects":1},"last_stats_sync":"2024-04-27T03:30:34.537268Z","last_stats_update":"2024-04-27T03:30:34.535973Z"} 2024-04-27T03:30:34.570 DEBUG:tasks.util.rgw: json result: {'stats': {'size': 3, 'size_actual': 4096, 'size_kb': 1, 'size_kb_actual': 4, 'num_objects': 1}, 'last_stats_sync': '2024-04-27T03:30:34.537268Z', 'last_stats_update': '2024-04-27T03:30:34.535973Z'} 2024-04-27T03:30:34.575 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:34.575 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-04-27T03:30:34.576 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'foo', '--bucket', 'myfoo'] 2024-04-27T03:30:34.576 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid foo --bucket myfoo 2024-04-27T03:30:34.620 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:34.620 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:34.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.615+0000 7f1549a98800 1 Processor -- start 2024-04-27T03:30:34.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.615+0000 7f1549a98800 1 -- start start 2024-04-27T03:30:34.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.616+0000 7f1549a98800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b022d643e0 0x55b022d667d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.616+0000 7f1549a98800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55b022c1ed40 con 0x55b022d608d0 2024-04-27T03:30:34.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.616+0000 7f1549a98800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55b022bf14d0 con 0x55b022d643e0 2024-04-27T03:30:34.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.616+0000 7f153ed96640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b022d643e0 0x55b022d667d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.616+0000 7f153ed96640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55b022d643e0 0x55b022d667d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58522/0 (socket says 172.21.15.165:58522) 2024-04-27T03:30:34.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.616+0000 7f153ed96640 1 -- 172.21.15.165:0/1632415110 learned_addr learned my addr 172.21.15.165:0/1632415110 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.616+0000 7f1547597640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55b022d608d0 0x55b022d62cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35314/0 (socket says 172.21.15.165:35314) 2024-04-27T03:30:34.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.616+0000 7f153ed96640 1 -- 172.21.15.165:0/1632415110 >> v1:172.21.15.89:6789/0 conn(0x55b022d608d0 legacy=0x55b022d62cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.616+0000 7f153ed96640 1 -- 172.21.15.165:0/1632415110 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b022c1d590 con 0x55b022d643e0 2024-04-27T03:30:34.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.616+0000 7f1546d96640 1 -- 172.21.15.165:0/1632415110 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 59149282 0 0) 0x55b022c1ed40 con 0x55b022d608d0 2024-04-27T03:30:34.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.618+0000 7f153ed96640 1 --2- 172.21.15.165:0/1632415110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d643e0 0x55b022d667d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f153000a520 tx=0x7f153001d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a912c69d7b16122 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.618+0000 7f1546d96640 1 -- 172.21.15.165:0/1632415110 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1530057070 con 0x55b022d643e0 2024-04-27T03:30:34.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.618+0000 7f1546d96640 1 -- 172.21.15.165:0/1632415110 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1530052570 con 0x55b022d643e0 2024-04-27T03:30:34.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.618+0000 7f1546d96640 1 -- 172.21.15.165:0/1632415110 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f153004ae60 con 0x55b022d643e0 2024-04-27T03:30:34.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.619+0000 7f1549a98800 1 -- 172.21.15.165:0/1632415110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d643e0 msgr2=0x55b022d667d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.619+0000 7f1549a98800 1 --2- 172.21.15.165:0/1632415110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d643e0 0x55b022d667d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f153000a520 tx=0x7f153001d010 comp rx=0 tx=0).stop 2024-04-27T03:30:34.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.619+0000 7f1549a98800 1 -- 172.21.15.165:0/1632415110 shutdown_connections 2024-04-27T03:30:34.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.619+0000 7f1549a98800 1 --2- 172.21.15.165:0/1632415110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d643e0 0x55b022d667d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.619+0000 7f1549a98800 1 -- 172.21.15.165:0/1632415110 >> 172.21.15.165:0/1632415110 conn(0x55b022d5a3a0 msgr2=0x55b022d5c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.619+0000 7f1549a98800 1 -- 172.21.15.165:0/1632415110 shutdown_connections 2024-04-27T03:30:34.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.619+0000 7f1549a98800 1 -- 172.21.15.165:0/1632415110 wait complete. 2024-04-27T03:30:34.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.630+0000 7f1549a98800 1 Processor -- start 2024-04-27T03:30:34.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.631+0000 7f1549a98800 1 -- start start 2024-04-27T03:30:34.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.631+0000 7f1549a98800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 0x55b022e50f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.631+0000 7f1549a98800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b022cab3b0 con 0x55b022d608d0 2024-04-27T03:30:34.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.631+0000 7f1547597640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 0x55b022e50f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.631+0000 7f1547597640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 0x55b022e50f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58524/0 (socket says 172.21.15.165:58524) 2024-04-27T03:30:34.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.631+0000 7f1547597640 1 -- 172.21.15.165:0/2941116295 learned_addr learned my addr 172.21.15.165:0/2941116295 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.631+0000 7f1547597640 1 -- 172.21.15.165:0/2941116295 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b022e51500 con 0x55b022d608d0 2024-04-27T03:30:34.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.631+0000 7f1547597640 1 --2- 172.21.15.165:0/2941116295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 0x55b022e50f30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f153800b210 tx=0x7f153800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e20b41ad1d7e3e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.631+0000 7f143560a640 1 -- 172.21.15.165:0/2941116295 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538008070 con 0x55b022d608d0 2024-04-27T03:30:34.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.632+0000 7f143560a640 1 -- 172.21.15.165:0/2941116295 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153800d500 con 0x55b022d608d0 2024-04-27T03:30:34.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.632+0000 7f143560a640 1 -- 172.21.15.165:0/2941116295 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f153800d840 con 0x55b022d608d0 2024-04-27T03:30:34.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.632+0000 7f1549a98800 1 -- 172.21.15.165:0/2941116295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 msgr2=0x55b022e50f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.632+0000 7f1549a98800 1 --2- 172.21.15.165:0/2941116295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 0x55b022e50f30 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f153800b210 tx=0x7f153800b6d0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.632+0000 7f1549a98800 1 -- 172.21.15.165:0/2941116295 shutdown_connections 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.632+0000 7f1549a98800 1 --2- 172.21.15.165:0/2941116295 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 0x55b022e50f30 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.632+0000 7f1549a98800 1 -- 172.21.15.165:0/2941116295 >> 172.21.15.165:0/2941116295 conn(0x55b022d5a3a0 msgr2=0x55b022e4c850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.632+0000 7f1549a98800 1 -- 172.21.15.165:0/2941116295 shutdown_connections 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.632+0000 7f1549a98800 1 -- 172.21.15.165:0/2941116295 wait complete. 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.633+0000 7f1549a98800 1 Processor -- start 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.633+0000 7f1549a98800 1 -- start start 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.633+0000 7f1549a98800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 0x55b022e58ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.633+0000 7f1549a98800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b022d67010 con 0x55b022d608d0 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.633+0000 7f1547597640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 0x55b022e58ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.633+0000 7f1547597640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 0x55b022e58ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58532/0 (socket says 172.21.15.165:58532) 2024-04-27T03:30:34.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.633+0000 7f1547597640 1 -- 172.21.15.165:0/301687472 learned_addr learned my addr 172.21.15.165:0/301687472 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.633+0000 7f1547597640 1 -- 172.21.15.165:0/301687472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b022e58ff0 con 0x55b022d608d0 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.634+0000 7f1547597640 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 0x55b022e58ab0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1538001e60 tx=0x7f1538000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.634+0000 7f1433e07640 1 -- 172.21.15.165:0/301687472 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538019870 con 0x55b022d608d0 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.634+0000 7f1433e07640 1 -- 172.21.15.165:0/301687472 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1538019a10 con 0x55b022d608d0 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.634+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b022e59280 con 0x55b022d608d0 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.634+0000 7f1433e07640 1 -- 172.21.15.165:0/301687472 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538019cf0 con 0x55b022d608d0 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.634+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b022e5bf30 con 0x55b022d608d0 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.635+0000 7f1433e07640 1 -- 172.21.15.165:0/301687472 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f1538017040 con 0x55b022d608d0 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.635+0000 7f1433e07640 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f142c036950 0x7f142c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.635+0000 7f1433e07640 1 -- 172.21.15.165:0/301687472 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f1538015020 con 0x55b022d608d0 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.635+0000 7f153ed96640 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f142c036950 0x7f142c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.635+0000 7f1549a98800 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b022d643e0 0x55b022e5fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.635+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022e60270 con 0x55b022d643e0 2024-04-27T03:30:34.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.635+0000 7f1547d98640 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b022d643e0 0x55b022e5fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.635+0000 7f153ed96640 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f142c036950 0x7f142c038e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1530054000 tx=0x7f153004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.635+0000 7f1547d98640 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b022d643e0 0x55b022e5fcf0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.636+0000 7f1547d98640 1 -- 172.21.15.165:0/301687472 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022d643e0 2024-04-27T03:30:34.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.636+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022e5d2c0 con 0x55b022d643e0 2024-04-27T03:30:34.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.636+0000 7f1547d98640 1 -- 172.21.15.165:0/301687472 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022d643e0 2024-04-27T03:30:34.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.636+0000 7f1549a98800 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b022e625a0 0x55b022e64990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.636+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022e65080 con 0x55b022e625a0 2024-04-27T03:30:34.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.636+0000 7f1547597640 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b022e625a0 0x55b022e64990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.636+0000 7f1547597640 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b022e625a0 0x55b022e64990 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.637+0000 7f1547597640 1 -- 172.21.15.165:0/301687472 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f153802d070 con 0x55b022e625a0 2024-04-27T03:30:34.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.637+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022e61550 con 0x55b022d643e0 2024-04-27T03:30:34.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.637+0000 7f1547d98640 1 -- 172.21.15.165:0/301687472 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022d643e0 2024-04-27T03:30:34.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.637+0000 7f1549a98800 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b022e66530 0x55b022e68980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.637+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022e690e0 con 0x55b022e66530 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.637+0000 7f153ed96640 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b022e66530 0x55b022e68980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.638+0000 7f153ed96640 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b022e66530 0x55b022e68980 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.638+0000 7f153ed96640 1 -- 172.21.15.165:0/301687472 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f153005a040 con 0x55b022e66530 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.638+0000 7f1549a98800 1 Processor -- start 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.638+0000 7f1549a98800 1 -- start start 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.638+0000 7f1549a98800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022e77540 0x55b022e77960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.638+0000 7f1549a98800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b022c516b0 con 0x55b022e77540 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.638+0000 7f1547597640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022e77540 0x55b022e77960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.638+0000 7f1547597640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022e77540 0x55b022e77960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58534/0 (socket says 172.21.15.165:58534) 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.638+0000 7f1547597640 1 -- 172.21.15.165:0/2347902013 learned_addr learned my addr 172.21.15.165:0/2347902013 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.639+0000 7f1547597640 1 -- 172.21.15.165:0/2347902013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b022e77ea0 con 0x55b022e77540 2024-04-27T03:30:34.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.639+0000 7f1547597640 1 --2- 172.21.15.165:0/2347902013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022e77540 0x55b022e77960 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f153804b370 tx=0x7f153800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.639+0000 7f1431602640 1 -- 172.21.15.165:0/2347902013 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538044b00 con 0x55b022e77540 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.639+0000 7f1431602640 1 -- 172.21.15.165:0/2347902013 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1538044ca0 con 0x55b022e77540 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.639+0000 7f1549a98800 1 -- 172.21.15.165:0/2347902013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b022e78910 con 0x55b022e77540 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.639+0000 7f1431602640 1 -- 172.21.15.165:0/2347902013 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538040850 con 0x55b022e77540 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.639+0000 7f1549a98800 1 -- 172.21.15.165:0/2347902013 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b022e78e10 con 0x55b022e77540 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.640+0000 7f1431602640 1 -- 172.21.15.165:0/2347902013 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f15380409f0 con 0x55b022e77540 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.640+0000 7f1431602640 1 --2- 172.21.15.165:0/2347902013 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1420036900 0x7f1420038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.640+0000 7f1431602640 1 -- 172.21.15.165:0/2347902013 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f153803caa0 con 0x55b022e77540 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.640+0000 7f153ed96640 1 --2- 172.21.15.165:0/2347902013 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1420036900 0x7f1420038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.640+0000 7f1549a98800 1 Processor -- start 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.640+0000 7f1549a98800 1 -- start start 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.640+0000 7f153ed96640 1 --2- 172.21.15.165:0/2347902013 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1420036900 0x7f1420038dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55b022e60b50 tx=0x7f1530005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.640+0000 7f1549a98800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 0x55b022d602e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.640+0000 7f1549a98800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b022e51880 con 0x55b022d5fec0 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.641+0000 7f1547597640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 0x55b022d602e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.641+0000 7f1547597640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 0x55b022d602e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58546/0 (socket says 172.21.15.165:58546) 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.641+0000 7f1547597640 1 -- 172.21.15.165:0/3085413423 learned_addr learned my addr 172.21.15.165:0/3085413423 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.641+0000 7f1547597640 1 -- 172.21.15.165:0/3085413423 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b022d5e6c0 con 0x55b022d5fec0 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.641+0000 7f1547597640 1 --2- 172.21.15.165:0/3085413423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 0x55b022d602e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1538040fd0 tx=0x7f1538044270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ee53559b890e88c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.641+0000 7f142affd640 1 -- 172.21.15.165:0/3085413423 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f153805c020 con 0x55b022d5fec0 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.641+0000 7f142affd640 1 -- 172.21.15.165:0/3085413423 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f153803ce40 con 0x55b022d5fec0 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.641+0000 7f142affd640 1 -- 172.21.15.165:0/3085413423 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538030d50 con 0x55b022d5fec0 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1549a98800 1 -- 172.21.15.165:0/3085413423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 msgr2=0x55b022d602e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1549a98800 1 --2- 172.21.15.165:0/3085413423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 0x55b022d602e0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1538040fd0 tx=0x7f1538044270 comp rx=0 tx=0).stop 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1549a98800 1 -- 172.21.15.165:0/3085413423 shutdown_connections 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1549a98800 1 --2- 172.21.15.165:0/3085413423 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 0x55b022d602e0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1549a98800 1 -- 172.21.15.165:0/3085413423 >> 172.21.15.165:0/3085413423 conn(0x55b022e80ba0 msgr2=0x55b022e7ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1549a98800 1 -- 172.21.15.165:0/3085413423 shutdown_connections 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1549a98800 1 -- 172.21.15.165:0/3085413423 wait complete. 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1549a98800 1 Processor -- start 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1549a98800 1 -- start start 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1549a98800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 0x55b022e8de00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1549a98800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55b022d5ead0 con 0x55b022d5fec0 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1547597640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 0x55b022e8de00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1547597640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 0x55b022e8de00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58562/0 (socket says 172.21.15.165:58562) 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.642+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 learned_addr learned my addr 172.21.15.165:0/599441724 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.643+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b022e8e340 con 0x55b022d5fec0 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.643+0000 7f1547597640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 0x55b022e8de00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f153804bf30 tx=0x7f153802b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.643+0000 7f14297fa640 1 -- 172.21.15.165:0/599441724 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538071070 con 0x55b022d5fec0 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.643+0000 7f14297fa640 1 -- 172.21.15.165:0/599441724 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1538025390 con 0x55b022d5fec0 2024-04-27T03:30:34.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.643+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55b022e8e340 con 0x55b022d5fec0 2024-04-27T03:30:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.643+0000 7f14297fa640 1 -- 172.21.15.165:0/599441724 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1538026430 con 0x55b022d5fec0 2024-04-27T03:30:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.643+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b022e8e340 con 0x55b022d5fec0 2024-04-27T03:30:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.644+0000 7f14297fa640 1 -- 172.21.15.165:0/599441724 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f1538074070 con 0x55b022d5fec0 2024-04-27T03:30:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.644+0000 7f14297fa640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1410036950 0x7f1410038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.644+0000 7f14297fa640 1 -- 172.21.15.165:0/599441724 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f1538069070 con 0x55b022d5fec0 2024-04-27T03:30:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.644+0000 7f153ed96640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1410036950 0x7f1410038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.644+0000 7f153ed96640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1410036950 0x7f1410038e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1530000c00 tx=0x7f153004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.644+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.644+0000 7f1549a98800 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b022e994c0 0x55b022e9b980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.644+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9c0d0 con 0x55b022e994c0 2024-04-27T03:30:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.644+0000 7f1547d98640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b022e994c0 0x55b022e9b980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1547d98640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b022e994c0 0x55b022e9b980 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022e994c0 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1549a98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1549a98800 20 realm 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9c4e0 con 0x55b022e994c0 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022e994c0 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1549a98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1549a98800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9cdb0 con 0x55b022e994c0 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022e994c0 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1549a98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.645+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9d150 con 0x55b022e994c0 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.646+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022e994c0 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.646+0000 7f1549a98800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.646+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.646+0000 7f1549a98800 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b022e9deb0 0x55b022ea02a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.646+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea08b0 con 0x55b022e9deb0 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.646+0000 7f1547597640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b022e9deb0 0x55b022ea02a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.646+0000 7f1547597640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b022e9deb0 0x55b022ea02a0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.646+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.646+0000 7f1549a98800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.646+0000 7f1549a98800 20 searching for the correct realm 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.646+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea21e0 con 0x55b022e9deb0 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.647+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.647+0000 7f1547597640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1538076060 0x7f1538025620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.647+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea21e0 con 0x7f1538076060 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.647+0000 7f153ed96640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1538076060 0x7f1538025620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.647+0000 7f153ed96640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1538076060 0x7f1538025620 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.648+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.648+0000 7f153ed96640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1530074040 0x7f1530081410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.648+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea21e0 con 0x7f1530074040 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.648+0000 7f1547d98640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1530074040 0x7f1530081410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.648+0000 7f1547d98640 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1530074040 0x7f1530081410 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.648+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f154000a4a0 con 0x7f1530074040 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.648+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9d150 con 0x55b022e994c0 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.649+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022e994c0 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.649+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9d150 con 0x55b022e994c0 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.649+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022e994c0 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.649+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9d150 con 0x7f1530074040 2024-04-27T03:30:34.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.649+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f154000a4a0 con 0x7f1530074040 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.649+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9d150 con 0x7f1530074040 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.649+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f154000a4a0 con 0x7f1530074040 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.649+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9d150 con 0x7f1538076060 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.649+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.650+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9d150 con 0x55b022e9deb0 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.650+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.650+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9d150 con 0x55b022e994c0 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.650+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022e994c0 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.650+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9d150 con 0x7f1530074040 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.650+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f154000a4a0 con 0x7f1530074040 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.650+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9d150 con 0x55b022e994c0 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.651+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022e994c0 2024-04-27T03:30:34.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.651+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55b022e9d150 con 0x7f1538076060 2024-04-27T03:30:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.651+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.651+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x55b022e9deb0 2024-04-27T03:30:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.651+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.651+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x55b022e9deb0 2024-04-27T03:30:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.651+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.651+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x7f1530074040 2024-04-27T03:30:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.652+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f154000a4a0 con 0x7f1530074040 2024-04-27T03:30:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.652+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x55b022e9deb0 2024-04-27T03:30:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.652+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.652+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x55b022e9deb0 2024-04-27T03:30:34.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.652+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.652+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x55b022e994c0 2024-04-27T03:30:34.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.652+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022e994c0 2024-04-27T03:30:34.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.652+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x55b022e994c0 2024-04-27T03:30:34.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.652+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f154000a4a0 con 0x55b022e994c0 2024-04-27T03:30:34.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.652+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x7f1530074040 2024-04-27T03:30:34.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.653+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1540033040 con 0x7f1530074040 2024-04-27T03:30:34.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.653+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x55b022e9deb0 2024-04-27T03:30:34.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.653+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.653+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x7f1538076060 2024-04-27T03:30:34.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.653+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.653+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x55b022e994c0 2024-04-27T03:30:34.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.653+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.654+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x7f1530074040 2024-04-27T03:30:34.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.654+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.654+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x55b022e9deb0 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.654+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.654+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x7f1530074040 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.654+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.654+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x7f1538076060 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.654+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.654+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x55b022e9deb0 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.655+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.655+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x7f1530074040 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.655+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.655+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x55b022e994c0 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f153008e060 con 0x7f1538076060 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea1170 con 0x55b022e994c0 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.686 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea1560 con 0x55b022e994c0 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.656+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea1560 con 0x7f1530074040 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 20 zone default found 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 4 Realm: () 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 20 zonegroup default 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea0cf0 con 0x7f1530074040 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea4040 con 0x55b022e9deb0 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea43e0 con 0x55b022e9deb0 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 20 started sync module instance, tier type = 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:34.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.657+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea34e0 con 0x7f1538076060 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.658+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.658+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94214988378160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea4780 con 0x7f1538076060 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.658+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea4fd0 con 0x55b022e994c0 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.659+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94214988378160] v25'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.660+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.660+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94214988385056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea4f40 con 0x55b022e994c0 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.660+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea5e90 con 0x55b022e9deb0 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.661+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.661+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94214988388832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea6730 con 0x55b022e9deb0 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.661+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea71e0 con 0x7f1538076060 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.661+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94214988385056] v25'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.662+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94214988388832] v25'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.689 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.662+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.662+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94214988387264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea75d0 con 0x7f1538076060 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.662+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea7e80 con 0x55b022e994c0 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.663+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.663+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94214988396912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea8720 con 0x55b022e994c0 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.663+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea9200 con 0x7f1530074040 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.663+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94214988387264] v25'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.664+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.664+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94214988396912] v25'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.664+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94214988401936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea9200 con 0x7f1530074040 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.664+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea8270 con 0x7f1530074040 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.665+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94214988401936] v25'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.665+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.665+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94214988393664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea9760 con 0x7f1530074040 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.665+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022eaa030 con 0x7f1530074040 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.667+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94214988393664] v25'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.667+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'105 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.667+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94214988399136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea9c10 con 0x7f1530074040 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.667+0000 7f1549a98800 20 add_watcher() i=0 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.667+0000 7f1549a98800 20 add_watcher() i=1 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.667+0000 7f1549a98800 20 add_watcher() i=2 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.667+0000 7f1549a98800 20 add_watcher() i=3 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.667+0000 7f1549a98800 20 add_watcher() i=4 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.667+0000 7f1549a98800 20 add_watcher() i=5 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.667+0000 7f1549a98800 20 add_watcher() i=6 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.667+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94214988399136] v25'106 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.668+0000 7f1549a98800 20 add_watcher() i=7 2024-04-27T03:30:34.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.668+0000 7f1549a98800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.668+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ead450 con 0x55b022e994c0 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.668+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.668+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94214988405040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022eaa950 con 0x55b022e994c0 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.669+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94214988405040] v25'110 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.669+0000 7f13fdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.669+0000 7f1549a98800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.669+0000 7f1549a98800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.669+0000 7f1549a98800 5 note: GC not initialized 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.669+0000 7f13fcfe1640 20 reqs_thread_entry: start 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.669+0000 7f1549a98800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.669+0000 7f1549a98800 20 Filter name: none 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f13767fc640 20 reqs_thread_entry: start 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:34.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ee0770 con 0x55b022e994c0 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ee2850 con 0x7f1538076060 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv1 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:34.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 : miss 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.670+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ee40d0 con 0x55b022e994c0 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.671+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.671+0000 7f1549a98800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.671+0000 7f1549a98800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.671+0000 7f1549a98800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.671+0000 7f1549a98800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.671+0000 7f1549a98800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.671+0000 7f1549a98800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.671+0000 7f1549a98800 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.671+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ee77f0 con 0x7f1538076060 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.672+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(65 foo.buckets [call] v25'17 uv17 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.672+0000 7f1549a98800 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.672+0000 7f1549a98800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.672+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ee6990 con 0x7f1538076060 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.672+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 233+0+288 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.672+0000 7f1549a98800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.672+0000 7f1549a98800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.672+0000 7f1549a98800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.672+0000 7f1549a98800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.672+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ee7550 con 0x7f1538076060 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.673+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v25'18 uv18 ondisk = 0) ==== 317+0+0 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.673+0000 7f1549a98800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.673+0000 7f1549a98800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.673+0000 7f1549a98800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T03:30:34.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.673+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94214988656608 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55b022ee9090 con 0x7f1530074040 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.674+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== watch-notify(notify (1) cookie 94214988393664 notify 107374182401 ret 0) ==== 380+0+0 (crc 0 0 0) 0x55b022ee44c0 con 0x7f1530074040 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.674+0000 7f142affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182401 cookie 94214988393664 notifier 4469 bl.length()=338 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.674+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(68 notify.6 [notify cookie 94214988656608 out=8b] v0'0 uv105 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.674+0000 7f142affd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.674+0000 7f142affd640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.674+0000 7f142affd640 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f140c003ea0 con 0x7f1530074040 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.674+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.674+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== watch-notify(notify_complete (2) cookie 94214988656608 notify 107374182401 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55b022ee3640 con 0x7f1530074040 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.675+0000 7f13fdfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.675+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94214988405040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ee6fe0 con 0x55b022e994c0 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.676+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94214988405040] v25'111 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.676+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94214988378160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea9900 con 0x7f1538076060 2024-04-27T03:30:34.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.677+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94214988378160] v25'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.677+0000 7f1549a98800 20 remove_watcher() i=0 2024-04-27T03:30:34.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.677+0000 7f1549a98800 2 removed watcher, disabling cache 2024-04-27T03:30:34.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.677+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94214988385056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea3030 con 0x55b022e994c0 2024-04-27T03:30:34.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.678+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94214988385056] v25'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.678+0000 7f1549a98800 20 remove_watcher() i=1 2024-04-27T03:30:34.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.678+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94214988388832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea3030 con 0x55b022e9deb0 2024-04-27T03:30:34.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.679+0000 7f1547597640 1 -- 172.21.15.165:0/599441724 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94214988388832] v25'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1538073030 con 0x55b022e9deb0 2024-04-27T03:30:34.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.679+0000 7f1549a98800 20 remove_watcher() i=2 2024-04-27T03:30:34.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.679+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94214988387264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea3030 con 0x7f1538076060 2024-04-27T03:30:34.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.680+0000 7f153ed96640 1 -- 172.21.15.165:0/599441724 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94214988387264] v25'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f153007f070 con 0x7f1538076060 2024-04-27T03:30:34.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.680+0000 7f1549a98800 20 remove_watcher() i=3 2024-04-27T03:30:34.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.680+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94214988396912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea53c0 con 0x55b022e994c0 2024-04-27T03:30:34.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.681+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94214988396912] v25'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x55b022e994c0 2024-04-27T03:30:34.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.681+0000 7f1549a98800 20 remove_watcher() i=4 2024-04-27T03:30:34.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.681+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94214988401936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea3030 con 0x7f1530074040 2024-04-27T03:30:34.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.682+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94214988401936] v25'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.682+0000 7f1549a98800 20 remove_watcher() i=5 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.682+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94214988393664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea53c0 con 0x7f1530074040 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.683+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94214988393664] v25'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.683+0000 7f1549a98800 20 remove_watcher() i=6 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.683+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94214988399136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55b022ea3030 con 0x7f1530074040 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94214988399136] v25'107 uv105 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1540031040 con 0x7f1530074040 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 20 remove_watcher() i=7 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1530074040 msgr2=0x7f1530081410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1530074040 0x7f1530081410 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b022e994c0 msgr2=0x55b022e9b980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b022e994c0 0x55b022e9b980 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1538076060 msgr2=0x7f1538025620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.703 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1538076060 0x7f1538025620 crc :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b022e9deb0 msgr2=0x55b022ea02a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b022e9deb0 0x55b022ea02a0 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1410036950 msgr2=0x7f1410038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1410036950 0x7f1410038e10 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f1530000c00 tx=0x7f153004f000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 msgr2=0x55b022e8de00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 0x55b022e8de00 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f153804bf30 tx=0x7f153802b0c0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1547d98640 1 -- 172.21.15.165:0/599441724 reap_dead start 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 shutdown_connections 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 --2- 172.21.15.165:0/599441724 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d5fec0 0x55b022e8de00 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 >> 172.21.15.165:0/599441724 conn(0x55b022e80ba0 msgr2=0x55b022e7e200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 shutdown_connections 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.684+0000 7f1549a98800 1 -- 172.21.15.165:0/599441724 wait complete. 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/2347902013 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1420036900 msgr2=0x7f1420038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 --2- 172.21.15.165:0/2347902013 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1420036900 0x7f1420038dc0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x55b022e60b50 tx=0x7f1530005ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/2347902013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022e77540 msgr2=0x55b022e77960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 --2- 172.21.15.165:0/2347902013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022e77540 0x55b022e77960 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f153804b370 tx=0x7f153800ed80 comp rx=0 tx=0).stop 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/2347902013 shutdown_connections 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 --2- 172.21.15.165:0/2347902013 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1420036900 0x7f1420038dc0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 --2- 172.21.15.165:0/2347902013 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022e77540 0x55b022e77960 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/2347902013 >> 172.21.15.165:0/2347902013 conn(0x55b022e6b5e0 msgr2=0x55b022e6b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/2347902013 shutdown_connections 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/2347902013 wait complete. 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b022e66530 msgr2=0x55b022e68980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55b022e66530 0x55b022e68980 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b022d643e0 msgr2=0x55b022e5fcf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55b022d643e0 0x55b022e5fcf0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b022e625a0 msgr2=0x55b022e64990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55b022e625a0 0x55b022e64990 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f142c036950 msgr2=0x7f142c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f142c036950 0x7f142c038e10 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f1530054000 tx=0x7f153004e000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 msgr2=0x55b022e58ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 --2- 172.21.15.165:0/301687472 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55b022d608d0 0x55b022e58ab0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f1538001e60 tx=0x7f1538000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1547d98640 1 -- 172.21.15.165:0/301687472 reap_dead start 2024-04-27T03:30:34.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 shutdown_connections 2024-04-27T03:30:34.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 >> 172.21.15.165:0/301687472 conn(0x55b022d5a3a0 msgr2=0x55b022e4b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 shutdown_connections 2024-04-27T03:30:34.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.685+0000 7f1549a98800 1 -- 172.21.15.165:0/301687472 wait complete. 2024-04-27T03:30:34.706 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-27T03:30:34.707 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-27T03:30:34.707 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-04-27T03:30:34.759 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:34.759 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:34.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.754+0000 7ff54520e800 1 Processor -- start 2024-04-27T03:30:34.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.754+0000 7ff54520e800 1 -- start start 2024-04-27T03:30:34.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.754+0000 7ff54520e800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x556202196af0 0x5562021950a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.754+0000 7ff54520e800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5562020dfd40 con 0x5562021955e0 2024-04-27T03:30:34.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.754+0000 7ff54520e800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5562020b24d0 con 0x556202196af0 2024-04-27T03:30:34.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.754+0000 7ff542d0d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x556202196af0 0x5562021950a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.754+0000 7ff54250c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5562021955e0 0x556202228e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35316/0 (socket says 172.21.15.165:35316) 2024-04-27T03:30:34.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.754+0000 7ff54250c640 1 -- 172.21.15.165:0/3158323677 learned_addr learned my addr 172.21.15.165:0/3158323677 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:34.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.755+0000 7ff542d0d640 1 -- 172.21.15.165:0/3158323677 >> v1:172.21.15.89:6789/0 conn(0x5562021955e0 legacy=0x556202228e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.755+0000 7ff542d0d640 1 -- 172.21.15.165:0/3158323677 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562020de590 con 0x556202196af0 2024-04-27T03:30:34.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.756+0000 7ff542d0d640 1 --2- 172.21.15.165:0/3158323677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202196af0 0x5562021950a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff53400a470 tx=0x7ff53401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dd47276b5fd102f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.757+0000 7ff541d0b640 1 -- 172.21.15.165:0/3158323677 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff534057070 con 0x556202196af0 2024-04-27T03:30:34.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.757+0000 7ff541d0b640 1 -- 172.21.15.165:0/3158323677 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff534052570 con 0x556202196af0 2024-04-27T03:30:34.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.757+0000 7ff541d0b640 1 -- 172.21.15.165:0/3158323677 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff534052870 con 0x556202196af0 2024-04-27T03:30:34.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.757+0000 7ff54520e800 1 -- 172.21.15.165:0/3158323677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202196af0 msgr2=0x5562021950a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.757+0000 7ff54520e800 1 --2- 172.21.15.165:0/3158323677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202196af0 0x5562021950a0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7ff53400a470 tx=0x7ff53401ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:34.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.757+0000 7ff54520e800 1 -- 172.21.15.165:0/3158323677 shutdown_connections 2024-04-27T03:30:34.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.757+0000 7ff54520e800 1 --2- 172.21.15.165:0/3158323677 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202196af0 0x5562021950a0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.757+0000 7ff54520e800 1 -- 172.21.15.165:0/3158323677 >> 172.21.15.165:0/3158323677 conn(0x55620221c580 msgr2=0x55620221e9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.757+0000 7ff54520e800 1 -- 172.21.15.165:0/3158323677 shutdown_connections 2024-04-27T03:30:34.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.757+0000 7ff54520e800 1 -- 172.21.15.165:0/3158323677 wait complete. 2024-04-27T03:30:34.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.769+0000 7ff54520e800 1 Processor -- start 2024-04-27T03:30:34.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.769+0000 7ff54520e800 1 -- start start 2024-04-27T03:30:34.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.769+0000 7ff54520e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 0x556202314940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.769+0000 7ff54520e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55620216c3e0 con 0x5562021955e0 2024-04-27T03:30:34.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.769+0000 7ff542d0d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 0x556202314940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.769+0000 7ff542d0d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 0x556202314940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58568/0 (socket says 172.21.15.165:58568) 2024-04-27T03:30:34.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.769+0000 7ff542d0d640 1 -- 172.21.15.165:0/47577763 learned_addr learned my addr 172.21.15.165:0/47577763 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.770+0000 7ff542d0d640 1 -- 172.21.15.165:0/47577763 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556202314f10 con 0x5562021955e0 2024-04-27T03:30:34.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.770+0000 7ff542d0d640 1 --2- 172.21.15.165:0/47577763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 0x556202314940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff534000c00 tx=0x7ff53404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4670511e55fb1b85 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.770+0000 7ff430e09640 1 -- 172.21.15.165:0/47577763 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff534057070 con 0x5562021955e0 2024-04-27T03:30:34.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.770+0000 7ff430e09640 1 -- 172.21.15.165:0/47577763 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff534063070 con 0x5562021955e0 2024-04-27T03:30:34.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.770+0000 7ff430e09640 1 -- 172.21.15.165:0/47577763 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff534052720 con 0x5562021955e0 2024-04-27T03:30:34.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff54520e800 1 -- 172.21.15.165:0/47577763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 msgr2=0x556202314940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff54520e800 1 --2- 172.21.15.165:0/47577763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 0x556202314940 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7ff534000c00 tx=0x7ff53404c000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff54520e800 1 -- 172.21.15.165:0/47577763 shutdown_connections 2024-04-27T03:30:34.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff54520e800 1 --2- 172.21.15.165:0/47577763 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 0x556202314940 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff54520e800 1 -- 172.21.15.165:0/47577763 >> 172.21.15.165:0/47577763 conn(0x55620221c580 msgr2=0x556202310260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff54520e800 1 -- 172.21.15.165:0/47577763 shutdown_connections 2024-04-27T03:30:34.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff54520e800 1 -- 172.21.15.165:0/47577763 wait complete. 2024-04-27T03:30:34.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff54520e800 1 Processor -- start 2024-04-27T03:30:34.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff54520e800 1 -- start start 2024-04-27T03:30:34.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff54520e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 0x55620231c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff54520e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55620222c860 con 0x5562021955e0 2024-04-27T03:30:34.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff542d0d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 0x55620231c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff542d0d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 0x55620231c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58574/0 (socket says 172.21.15.165:58574) 2024-04-27T03:30:34.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.771+0000 7ff542d0d640 1 -- 172.21.15.165:0/3633380701 learned_addr learned my addr 172.21.15.165:0/3633380701 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.772+0000 7ff542d0d640 1 -- 172.21.15.165:0/3633380701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620231c920 con 0x5562021955e0 2024-04-27T03:30:34.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.772+0000 7ff542d0d640 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 0x55620231c3e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff534000c00 tx=0x7ff534002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.772+0000 7ff42f606640 1 -- 172.21.15.165:0/3633380701 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff534052c70 con 0x5562021955e0 2024-04-27T03:30:34.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.772+0000 7ff42f606640 1 -- 172.21.15.165:0/3633380701 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff53406c070 con 0x5562021955e0 2024-04-27T03:30:34.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.772+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55620231cbb0 con 0x5562021955e0 2024-04-27T03:30:34.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.772+0000 7ff42f606640 1 -- 172.21.15.165:0/3633380701 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff53405e480 con 0x5562021955e0 2024-04-27T03:30:34.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.772+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620231f860 con 0x5562021955e0 2024-04-27T03:30:34.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.773+0000 7ff42f606640 1 -- 172.21.15.165:0/3633380701 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ff53406e030 con 0x5562021955e0 2024-04-27T03:30:34.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.773+0000 7ff42f606640 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff428036950 0x7ff428038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.773+0000 7ff42f606640 1 -- 172.21.15.165:0/3633380701 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ff53408e100 con 0x5562021955e0 2024-04-27T03:30:34.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.773+0000 7ff54250c640 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff428036950 0x7ff428038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.773+0000 7ff54520e800 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556202196af0 0x556202323620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.773+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556202323b60 con 0x556202196af0 2024-04-27T03:30:34.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.773+0000 7ff54350e640 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556202196af0 0x556202323620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.774+0000 7ff54250c640 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff428036950 0x7ff428038e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff538000f60 tx=0x7ff5380012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.774+0000 7ff54350e640 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556202196af0 0x556202323620 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.774+0000 7ff54350e640 1 -- 172.21.15.165:0/3633380701 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff53c00a4a0 con 0x556202196af0 2024-04-27T03:30:34.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.774+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556202320bf0 con 0x556202196af0 2024-04-27T03:30:34.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.774+0000 7ff54350e640 1 -- 172.21.15.165:0/3633380701 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff53c00a4a0 con 0x556202196af0 2024-04-27T03:30:34.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.774+0000 7ff54520e800 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556202325eb0 0x5562023282a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.774+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556202328940 con 0x556202325eb0 2024-04-27T03:30:34.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.774+0000 7ff542d0d640 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556202325eb0 0x5562023282a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.775+0000 7ff542d0d640 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556202325eb0 0x5562023282a0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.775+0000 7ff542d0d640 1 -- 172.21.15.165:0/3633380701 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ff534070410 con 0x556202325eb0 2024-04-27T03:30:34.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.775+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556202324e60 con 0x556202196af0 2024-04-27T03:30:34.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.775+0000 7ff54350e640 1 -- 172.21.15.165:0/3633380701 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff53c00a4a0 con 0x556202196af0 2024-04-27T03:30:34.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.775+0000 7ff54520e800 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x556202329df0 0x55620232c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.775+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55620232c9c0 con 0x556202329df0 2024-04-27T03:30:34.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.776+0000 7ff54250c640 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x556202329df0 0x55620232c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.776+0000 7ff54250c640 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x556202329df0 0x55620232c260 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.776+0000 7ff54250c640 1 -- 172.21.15.165:0/3633380701 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ff538009050 con 0x556202329df0 2024-04-27T03:30:34.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.776+0000 7ff54520e800 1 Processor -- start 2024-04-27T03:30:34.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.777+0000 7ff54520e800 1 -- start start 2024-04-27T03:30:34.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.777+0000 7ff54520e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55620233adf0 0x55620233b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.777+0000 7ff54520e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5562021126b0 con 0x55620233adf0 2024-04-27T03:30:34.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.777+0000 7ff542d0d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55620233adf0 0x55620233b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.777+0000 7ff542d0d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55620233adf0 0x55620233b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58576/0 (socket says 172.21.15.165:58576) 2024-04-27T03:30:34.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.777+0000 7ff542d0d640 1 -- 172.21.15.165:0/459345467 learned_addr learned my addr 172.21.15.165:0/459345467 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.777+0000 7ff542d0d640 1 -- 172.21.15.165:0/459345467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620233b750 con 0x55620233adf0 2024-04-27T03:30:34.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.777+0000 7ff542d0d640 1 --2- 172.21.15.165:0/459345467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55620233adf0 0x55620233b210 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff53406fb70 tx=0x7ff534067f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.777+0000 7ff42ce01640 1 -- 172.21.15.165:0/459345467 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff534094070 con 0x55620233adf0 2024-04-27T03:30:34.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.777+0000 7ff42ce01640 1 -- 172.21.15.165:0/459345467 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff534058040 con 0x55620233adf0 2024-04-27T03:30:34.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.777+0000 7ff54520e800 1 -- 172.21.15.165:0/459345467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55620233c1c0 con 0x55620233adf0 2024-04-27T03:30:34.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.778+0000 7ff42ce01640 1 -- 172.21.15.165:0/459345467 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff534084040 con 0x55620233adf0 2024-04-27T03:30:34.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.778+0000 7ff54520e800 1 -- 172.21.15.165:0/459345467 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620233c6c0 con 0x55620233adf0 2024-04-27T03:30:34.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.778+0000 7ff42ce01640 1 -- 172.21.15.165:0/459345467 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ff534097070 con 0x55620233adf0 2024-04-27T03:30:34.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.778+0000 7ff42ce01640 1 --2- 172.21.15.165:0/459345467 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff41c036900 0x7ff41c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.778+0000 7ff42ce01640 1 -- 172.21.15.165:0/459345467 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ff53409aaf0 con 0x55620233adf0 2024-04-27T03:30:34.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.779+0000 7ff54250c640 1 --2- 172.21.15.165:0/459345467 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff41c036900 0x7ff41c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.779+0000 7ff54520e800 1 Processor -- start 2024-04-27T03:30:34.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.779+0000 7ff54520e800 1 -- start start 2024-04-27T03:30:34.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.779+0000 7ff54250c640 1 --2- 172.21.15.165:0/459345467 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff41c036900 0x7ff41c038dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff53800bf50 tx=0x7ff53800f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.779+0000 7ff54520e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 0x556202348410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.779+0000 7ff54520e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5562023151d0 con 0x556202347ff0 2024-04-27T03:30:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.779+0000 7ff542d0d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 0x556202348410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.779+0000 7ff542d0d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 0x556202348410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58590/0 (socket says 172.21.15.165:58590) 2024-04-27T03:30:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.779+0000 7ff542d0d640 1 -- 172.21.15.165:0/2223310826 learned_addr learned my addr 172.21.15.165:0/2223310826 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.780+0000 7ff542d0d640 1 -- 172.21.15.165:0/2223310826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5562023489e0 con 0x556202347ff0 2024-04-27T03:30:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.780+0000 7ff542d0d640 1 --2- 172.21.15.165:0/2223310826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 0x556202348410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff53408e0b0 tx=0x7ff53409afb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=807d6889d0c0e396 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.780+0000 7ff4267fc640 1 -- 172.21.15.165:0/2223310826 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff534054080 con 0x556202347ff0 2024-04-27T03:30:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.780+0000 7ff4267fc640 1 -- 172.21.15.165:0/2223310826 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff534079cc0 con 0x556202347ff0 2024-04-27T03:30:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.780+0000 7ff4267fc640 1 -- 172.21.15.165:0/2223310826 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff53408ca30 con 0x556202347ff0 2024-04-27T03:30:34.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.780+0000 7ff54520e800 1 -- 172.21.15.165:0/2223310826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 msgr2=0x556202348410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.780+0000 7ff54520e800 1 --2- 172.21.15.165:0/2223310826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 0x556202348410 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7ff53408e0b0 tx=0x7ff53409afb0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.780+0000 7ff54520e800 1 -- 172.21.15.165:0/2223310826 shutdown_connections 2024-04-27T03:30:34.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.780+0000 7ff54520e800 1 --2- 172.21.15.165:0/2223310826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 0x556202348410 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.780+0000 7ff54520e800 1 -- 172.21.15.165:0/2223310826 >> 172.21.15.165:0/2223310826 conn(0x5562023442c0 msgr2=0x556202342320 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.780+0000 7ff54520e800 1 -- 172.21.15.165:0/2223310826 shutdown_connections 2024-04-27T03:30:34.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.780+0000 7ff54520e800 1 -- 172.21.15.165:0/2223310826 wait complete. 2024-04-27T03:30:34.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.781+0000 7ff54520e800 1 Processor -- start 2024-04-27T03:30:34.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.781+0000 7ff54520e800 1 -- start start 2024-04-27T03:30:34.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.781+0000 7ff54520e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 0x55620234f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.781+0000 7ff54520e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556202349040 con 0x556202347ff0 2024-04-27T03:30:34.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.781+0000 7ff542d0d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 0x55620234f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.781+0000 7ff542d0d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 0x55620234f340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58600/0 (socket says 172.21.15.165:58600) 2024-04-27T03:30:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.781+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 learned_addr learned my addr 172.21.15.165:0/1091404481 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.781+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55620234f880 con 0x556202347ff0 2024-04-27T03:30:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.781+0000 7ff542d0d640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 0x55620234f340 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff53400a010 tx=0x7ff53406af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.782+0000 7ff424ff9640 1 -- 172.21.15.165:0/1091404481 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff53404f830 con 0x556202347ff0 2024-04-27T03:30:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.782+0000 7ff424ff9640 1 -- 172.21.15.165:0/1091404481 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ff5340795d0 con 0x556202347ff0 2024-04-27T03:30:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.782+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55620234f880 con 0x556202347ff0 2024-04-27T03:30:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.782+0000 7ff424ff9640 1 -- 172.21.15.165:0/1091404481 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ff5340798d0 con 0x556202347ff0 2024-04-27T03:30:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.782+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55620234f880 con 0x556202347ff0 2024-04-27T03:30:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.782+0000 7ff424ff9640 1 -- 172.21.15.165:0/1091404481 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ff53404f2a0 con 0x556202347ff0 2024-04-27T03:30:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.782+0000 7ff424ff9640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff40c036950 0x7ff40c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.783+0000 7ff424ff9640 1 -- 172.21.15.165:0/1091404481 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ff5340c4aa0 con 0x556202347ff0 2024-04-27T03:30:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.783+0000 7ff54250c640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff40c036950 0x7ff40c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.783+0000 7ff54250c640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff40c036950 0x7ff40c038e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff538001b60 tx=0x7ff538010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.783+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.783+0000 7ff54520e800 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55620235ab10 0x55620235cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.783+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55620235d720 con 0x55620235ab10 2024-04-27T03:30:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.783+0000 7ff54350e640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55620235ab10 0x55620235cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54350e640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55620235ab10 0x55620235cfd0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff53c00a4a0 con 0x55620235ab10 2024-04-27T03:30:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54520e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54520e800 20 realm 2024-04-27T03:30:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55620235db30 con 0x55620235ab10 2024-04-27T03:30:34.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff53c00a4a0 con 0x55620235ab10 2024-04-27T03:30:34.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54520e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54520e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:34.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55620235e400 con 0x55620235ab10 2024-04-27T03:30:34.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff53c00a4a0 con 0x55620235ab10 2024-04-27T03:30:34.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54520e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.784+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235ab10 2024-04-27T03:30:34.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.785+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ff53c00a4a0 con 0x55620235ab10 2024-04-27T03:30:34.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.785+0000 7ff54520e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.785+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.785+0000 7ff54520e800 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55620235f500 0x5562023618f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.785+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556202361f00 con 0x55620235f500 2024-04-27T03:30:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.785+0000 7ff542d0d640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55620235f500 0x5562023618f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.785+0000 7ff542d0d640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55620235f500 0x5562023618f0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.785+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.785+0000 7ff54520e800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.785+0000 7ff54520e800 20 searching for the correct realm 2024-04-27T03:30:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.785+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5562023637c0 con 0x55620235f500 2024-04-27T03:30:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.786+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.786+0000 7ff542d0d640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ff5340b1040 0x7ff534084940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.786+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5562023637c0 con 0x7ff5340b1040 2024-04-27T03:30:34.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.786+0000 7ff54250c640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ff5340b1040 0x7ff534084940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.786+0000 7ff54250c640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ff5340b1040 0x7ff534084940 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.786+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff538009050 con 0x7ff5340b1040 2024-04-27T03:30:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.786+0000 7ff54250c640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ff538038040 0x7ff53801bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.786+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5562023637c0 con 0x7ff538038040 2024-04-27T03:30:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.787+0000 7ff54350e640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ff538038040 0x7ff53801bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.787+0000 7ff54350e640 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ff538038040 0x7ff53801bab0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.787+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ff53c00a4a0 con 0x7ff538038040 2024-04-27T03:30:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.787+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235ab10 2024-04-27T03:30:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.788+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c00a4a0 con 0x55620235ab10 2024-04-27T03:30:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.788+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235ab10 2024-04-27T03:30:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.788+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c00a4a0 con 0x55620235ab10 2024-04-27T03:30:34.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.788+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff538038040 2024-04-27T03:30:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.788+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c00a4a0 con 0x7ff538038040 2024-04-27T03:30:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.788+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff538038040 2024-04-27T03:30:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.788+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c00a4a0 con 0x7ff538038040 2024-04-27T03:30:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.788+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff5340b1040 2024-04-27T03:30:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.789+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff538009050 con 0x7ff5340b1040 2024-04-27T03:30:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.789+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235f500 2024-04-27T03:30:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.789+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.789+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235ab10 2024-04-27T03:30:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.789+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c00a4a0 con 0x55620235ab10 2024-04-27T03:30:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.789+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff538038040 2024-04-27T03:30:34.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.790+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c00a4a0 con 0x7ff538038040 2024-04-27T03:30:34.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.790+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235ab10 2024-04-27T03:30:34.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.790+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c00a4a0 con 0x55620235ab10 2024-04-27T03:30:34.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.790+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff5340b1040 2024-04-27T03:30:34.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.790+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ff538009050 con 0x7ff5340b1040 2024-04-27T03:30:34.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.790+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235f500 2024-04-27T03:30:34.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.790+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.790+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235f500 2024-04-27T03:30:34.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.790+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.791+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff538038040 2024-04-27T03:30:34.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.791+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c00a4a0 con 0x7ff538038040 2024-04-27T03:30:34.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.791+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235f500 2024-04-27T03:30:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.791+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.791+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235f500 2024-04-27T03:30:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.791+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.791+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235ab10 2024-04-27T03:30:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.791+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c00a4a0 con 0x55620235ab10 2024-04-27T03:30:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.792+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235ab10 2024-04-27T03:30:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.792+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ff53c00a4a0 con 0x55620235ab10 2024-04-27T03:30:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.792+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff538038040 2024-04-27T03:30:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.792+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c033040 con 0x7ff538038040 2024-04-27T03:30:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.792+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235f500 2024-04-27T03:30:34.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.792+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.792+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff5340b1040 2024-04-27T03:30:34.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.793+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff538009050 con 0x7ff5340b1040 2024-04-27T03:30:34.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.793+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235ab10 2024-04-27T03:30:34.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.793+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.793+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff538038040 2024-04-27T03:30:34.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.793+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.793+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235f500 2024-04-27T03:30:34.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.793+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.793+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff538038040 2024-04-27T03:30:34.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.794+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.794+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff5340b1040 2024-04-27T03:30:34.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.794+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff538009050 con 0x7ff5340b1040 2024-04-27T03:30:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.794+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235f500 2024-04-27T03:30:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.794+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.794+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff538038040 2024-04-27T03:30:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.794+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.794+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x55620235ab10 2024-04-27T03:30:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.794+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55620235e7a0 con 0x7ff5340b1040 2024-04-27T03:30:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ff538045040 con 0x7ff5340b1040 2024-04-27T03:30:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:34.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556202362750 con 0x55620235ab10 2024-04-27T03:30:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556202362af0 con 0x55620235ab10 2024-04-27T03:30:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.795+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556202362af0 con 0x7ff538038040 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 20 zone default found 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 4 Realm: () 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 20 zonegroup default 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556202363c40 con 0x7ff538038040 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556202365620 con 0x55620235f500 2024-04-27T03:30:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5562023659c0 con 0x55620235f500 2024-04-27T03:30:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 20 started sync module instance, tier type = 2024-04-27T03:30:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:34.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.796+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202364ac0 con 0x7ff5340b1040 2024-04-27T03:30:34.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.797+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff538045040 con 0x7ff5340b1040 2024-04-27T03:30:34.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.797+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93879432267280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202365d60 con 0x7ff5340b1040 2024-04-27T03:30:34.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.797+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202366720 con 0x55620235ab10 2024-04-27T03:30:34.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.798+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93879432267280] v25'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff538045040 con 0x7ff5340b1040 2024-04-27T03:30:34.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.799+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.799+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93879432274544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202366b40 con 0x55620235ab10 2024-04-27T03:30:34.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.799+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202367500 con 0x55620235f500 2024-04-27T03:30:34.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.800+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93879432274544] v25'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.800+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.800+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93879432278096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5562023678a0 con 0x55620235f500 2024-04-27T03:30:34.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.800+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202368310 con 0x7ff5340b1040 2024-04-27T03:30:34.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.801+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93879432278096] v25'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.801+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff538045040 con 0x7ff5340b1040 2024-04-27T03:30:34.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.801+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93879432275600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202368bb0 con 0x7ff5340b1040 2024-04-27T03:30:34.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.801+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202369570 con 0x55620235ab10 2024-04-27T03:30:34.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.802+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.802+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93879432286912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202369e10 con 0x55620235ab10 2024-04-27T03:30:34.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.802+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55620236a8c0 con 0x7ff538038040 2024-04-27T03:30:34.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.803+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93879432275600] v25'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff538045040 con 0x7ff5340b1040 2024-04-27T03:30:34.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.803+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.803+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93879432283904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55620236a880 con 0x7ff538038040 2024-04-27T03:30:34.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.803+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55620236b2d0 con 0x7ff538038040 2024-04-27T03:30:34.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.803+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93879432286912] v25'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.804+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93879432283904] v25'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.804+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.804+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93879432291280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55620236a7f0 con 0x7ff538038040 2024-04-27T03:30:34.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.804+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55620236b780 con 0x7ff538038040 2024-04-27T03:30:34.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.805+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93879432291280] v25'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.805+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'108 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.805+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93879432293920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55620236b6f0 con 0x7ff538038040 2024-04-27T03:30:34.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.805+0000 7ff54520e800 20 add_watcher() i=0 2024-04-27T03:30:34.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.805+0000 7ff54520e800 20 add_watcher() i=1 2024-04-27T03:30:34.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.805+0000 7ff54520e800 20 add_watcher() i=2 2024-04-27T03:30:34.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.805+0000 7ff54520e800 20 add_watcher() i=3 2024-04-27T03:30:34.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.805+0000 7ff54520e800 20 add_watcher() i=4 2024-04-27T03:30:34.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.805+0000 7ff54520e800 20 add_watcher() i=5 2024-04-27T03:30:34.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.805+0000 7ff54520e800 20 add_watcher() i=6 2024-04-27T03:30:34.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.806+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93879432293920] v25'109 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.806+0000 7ff54520e800 20 add_watcher() i=7 2024-04-27T03:30:34.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.806+0000 7ff54520e800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:34.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.806+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55620236eaf0 con 0x55620235ab10 2024-04-27T03:30:34.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.806+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.806+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93879432295440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55620236bfb0 con 0x55620235ab10 2024-04-27T03:30:34.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.807+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93879432295440] v25'112 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.808+0000 7ff3f97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.808+0000 7ff54520e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:34.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.808+0000 7ff54520e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:34.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.808+0000 7ff54520e800 5 note: GC not initialized 2024-04-27T03:30:34.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.808+0000 7ff373fff640 20 reqs_thread_entry: start 2024-04-27T03:30:34.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.808+0000 7ff54520e800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:34.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.808+0000 7ff54520e800 20 Filter name: none 2024-04-27T03:30:34.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.808+0000 7ff371ffb640 20 reqs_thread_entry: start 2024-04-27T03:30:34.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.808+0000 7ff54520e800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T03:30:34.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.808+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.808+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5562023a1e60 con 0x7ff538038040 2024-04-27T03:30:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54520e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54520e800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-27T03:30:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54520e800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:30:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54520e800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-27T03:30:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54520e800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5562023a0600 con 0x7ff538038040 2024-04-27T03:30:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54520e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54520e800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-27T03:30:34.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54520e800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T03:30:34.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54520e800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T03:30:34.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54520e800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T03:30:34.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.809+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5562023a0ef0 con 0x7ff538038040 2024-04-27T03:30:34.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.810+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~380] v25'1 uv1 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.810+0000 7ff54520e800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-27T03:30:34.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.810+0000 7ff54520e800 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:30:34.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.810+0000 7ff54520e800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-27T03:30:34.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.810+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 93879432514112 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5562023a5900 con 0x7ff538038040 2024-04-27T03:30:34.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.810+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== watch-notify(notify (1) cookie 93879432283904 notify 107374182402 ret 0) ==== 598+0+0 (crc 0 0 0) 0x5562023a1290 con 0x7ff538038040 2024-04-27T03:30:34.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.810+0000 7ff4267fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182402 cookie 93879432283904 notifier 4476 bl.length()=556 2024-04-27T03:30:34.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.810+0000 7ff4267fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-27T03:30:34.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.810+0000 7ff4267fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:30:34.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.810+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(65 notify.5 [notify cookie 93879432514112 out=8b] v0'0 uv108 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.810+0000 7ff4267fc640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7ff408003ea0 con 0x7ff538038040 2024-04-27T03:30:34.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.811+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.811+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== watch-notify(notify_complete (2) cookie 93879432514112 notify 107374182402 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5562023a18b0 con 0x7ff538038040 2024-04-27T03:30:34.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.811+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5562023a3430 con 0x7ff538038040 2024-04-27T03:30:34.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v25'3 uv3 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff54520e800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T03:30:34.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff54520e800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T03:30:34.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff54520e800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T03:30:34.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 93879432513984 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5562023a5900 con 0x55620235f500 2024-04-27T03:30:34.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== watch-notify(notify (1) cookie 93879432278096 notify 107374182402 ret 0) ==== 219+0+0 (crc 0 0 0) 0x55620233c1c0 con 0x55620235f500 2024-04-27T03:30:34.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff425ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182402 cookie 93879432278096 notifier 4476 bl.length()=177 2024-04-27T03:30:34.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff425ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T03:30:34.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff425ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T03:30:34.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff425ffb640 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7ff378003400 con 0x55620235f500 2024-04-27T03:30:34.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(68 notify.2 [notify cookie 93879432513984 out=8b] v0'0 uv108 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== watch-notify(notify_complete (2) cookie 93879432513984 notify 107374182402 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55620233b750 con 0x55620235f500 2024-04-27T03:30:34.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.812+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.839 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"fud","display_name":"Fud","email":"","suspended":0,"max_buckets":1,"subusers":[],"keys":[{"user":"fud","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-04-27T03:30:34.810281Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:34.810264Z","tags":[],"group_ids":[]} 2024-04-27T03:30:34.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.813+0000 7ff3f97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.813+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93879432295440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55620236e6d0 con 0x55620235ab10 2024-04-27T03:30:34.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.814+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 93879432295440] v25'113 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.814+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93879432267280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55620236b3e0 con 0x7ff5340b1040 2024-04-27T03:30:34.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.815+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 93879432267280] v25'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff538045040 con 0x7ff5340b1040 2024-04-27T03:30:34.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.815+0000 7ff54520e800 20 remove_watcher() i=0 2024-04-27T03:30:34.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.815+0000 7ff54520e800 2 removed watcher, disabling cache 2024-04-27T03:30:34.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.815+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93879432274544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202364610 con 0x55620235ab10 2024-04-27T03:30:34.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.816+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 93879432274544] v25'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.816+0000 7ff54520e800 20 remove_watcher() i=1 2024-04-27T03:30:34.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.816+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93879432278096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202364610 con 0x55620235f500 2024-04-27T03:30:34.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.817+0000 7ff542d0d640 1 -- 172.21.15.165:0/1091404481 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 93879432278096] v25'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff5340c0020 con 0x55620235f500 2024-04-27T03:30:34.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.817+0000 7ff54520e800 20 remove_watcher() i=2 2024-04-27T03:30:34.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.817+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93879432275600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202367050 con 0x7ff5340b1040 2024-04-27T03:30:34.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.818+0000 7ff54250c640 1 -- 172.21.15.165:0/1091404481 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 93879432275600] v25'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff538045040 con 0x7ff5340b1040 2024-04-27T03:30:34.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.818+0000 7ff54520e800 20 remove_watcher() i=3 2024-04-27T03:30:34.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.818+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93879432286912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202364610 con 0x55620235ab10 2024-04-27T03:30:34.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.819+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 93879432286912] v25'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x55620235ab10 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.819+0000 7ff54520e800 20 remove_watcher() i=4 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.819+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93879432283904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202364610 con 0x7ff538038040 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.820+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 93879432283904] v25'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.820+0000 7ff54520e800 20 remove_watcher() i=5 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.820+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93879432291280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556202368700 con 0x7ff538038040 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.821+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 93879432291280] v25'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.821+0000 7ff54520e800 20 remove_watcher() i=6 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.821+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93879432293920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55620236a3d0 con 0x7ff538038040 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 93879432293920] v25'110 uv108 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ff53c031040 con 0x7ff538038040 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 20 remove_watcher() i=7 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ff538038040 msgr2=0x7ff53801bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ff538038040 0x7ff53801bab0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55620235ab10 msgr2=0x55620235cfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55620235ab10 0x55620235cfd0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ff5340b1040 msgr2=0x7ff534084940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ff5340b1040 0x7ff534084940 crc :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55620235f500 msgr2=0x5562023618f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55620235f500 0x5562023618f0 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff40c036950 msgr2=0x7ff40c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff40c036950 0x7ff40c038e10 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7ff538001b60 tx=0x7ff538010000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 msgr2=0x55620234f340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54350e640 1 -- 172.21.15.165:0/1091404481 reap_dead start 2024-04-27T03:30:34.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.822+0000 7ff54520e800 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 0x55620234f340 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7ff53400a010 tx=0x7ff53406af70 comp rx=0 tx=0).stop 2024-04-27T03:30:34.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 shutdown_connections 2024-04-27T03:30:34.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 --2- 172.21.15.165:0/1091404481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556202347ff0 0x55620234f340 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 >> 172.21.15.165:0/1091404481 conn(0x5562023442c0 msgr2=0x556202341920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 shutdown_connections 2024-04-27T03:30:34.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/1091404481 wait complete. 2024-04-27T03:30:34.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/459345467 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff41c036900 msgr2=0x7ff41c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 --2- 172.21.15.165:0/459345467 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff41c036900 0x7ff41c038dc0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7ff53800bf50 tx=0x7ff53800f000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/459345467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55620233adf0 msgr2=0x55620233b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 --2- 172.21.15.165:0/459345467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55620233adf0 0x55620233b210 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7ff53406fb70 tx=0x7ff534067f20 comp rx=0 tx=0).stop 2024-04-27T03:30:34.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/459345467 shutdown_connections 2024-04-27T03:30:34.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 --2- 172.21.15.165:0/459345467 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff41c036900 0x7ff41c038dc0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 --2- 172.21.15.165:0/459345467 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55620233adf0 0x55620233b210 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/459345467 >> 172.21.15.165:0/459345467 conn(0x55620232ee90 msgr2=0x55620232f290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/459345467 shutdown_connections 2024-04-27T03:30:34.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/459345467 wait complete. 2024-04-27T03:30:34.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x556202329df0 msgr2=0x55620232c260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x556202329df0 0x55620232c260 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556202196af0 msgr2=0x556202323620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556202196af0 0x556202323620 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556202325eb0 msgr2=0x5562023282a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556202325eb0 0x5562023282a0 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff428036950 msgr2=0x7ff428038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff428036950 0x7ff428038e10 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7ff538000f60 tx=0x7ff5380012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 msgr2=0x55620231c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 --2- 172.21.15.165:0/3633380701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5562021955e0 0x55620231c3e0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7ff534000c00 tx=0x7ff534002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54350e640 1 -- 172.21.15.165:0/3633380701 reap_dead start 2024-04-27T03:30:34.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 shutdown_connections 2024-04-27T03:30:34.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 >> 172.21.15.165:0/3633380701 conn(0x55620221c580 msgr2=0x55620230ea60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.823+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 shutdown_connections 2024-04-27T03:30:34.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.824+0000 7ff54520e800 1 -- 172.21.15.165:0/3633380701 wait complete. 2024-04-27T03:30:34.852 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-04-27T03:30:34.810281Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:34.810264Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:34.857 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-27T03:30:34.861 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:34.861 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:myfoo'] 2024-04-27T03:30:34.861 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:myfoo'] 2024-04-27T03:30:34.862 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:myfoo 2024-04-27T03:30:34.903 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:34.903 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:34.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.898+0000 7f092b133800 1 Processor -- start 2024-04-27T03:30:34.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.899+0000 7f092b133800 1 -- start start 2024-04-27T03:30:34.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.899+0000 7f092b133800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x556827034970 0x556827036d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.899+0000 7f092b133800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x556826ef2d40 con 0x5568270372a0 2024-04-27T03:30:34.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.899+0000 7f092b133800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x556826ec54d0 con 0x556827034970 2024-04-27T03:30:34.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.899+0000 7f0926dd8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x556827034970 0x556827036d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.899+0000 7f09265d7640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5568270372a0 0x556827039690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35324/0 (socket says 172.21.15.165:35324) 2024-04-27T03:30:34.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.899+0000 7f0926dd8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x556827034970 0x556827036d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58602/0 (socket says 172.21.15.165:58602) 2024-04-27T03:30:34.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.899+0000 7f09265d7640 1 -- 172.21.15.165:0/1026977744 learned_addr learned my addr 172.21.15.165:0/1026977744 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:34.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.899+0000 7f0926dd8640 1 -- 172.21.15.165:0/1026977744 >> v1:172.21.15.89:6789/0 conn(0x5568270372a0 legacy=0x556827039690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.899+0000 7f0926dd8640 1 -- 172.21.15.165:0/1026977744 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x556826ef1590 con 0x556827034970 2024-04-27T03:30:34.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.901+0000 7f0926dd8640 1 --2- 172.21.15.165:0/1026977744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x556827036d60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f092000a470 tx=0x7f092001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f642e2456936c7f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.901+0000 7f0925dd6640 1 -- 172.21.15.165:0/1026977744 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0920057070 con 0x556827034970 2024-04-27T03:30:34.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.901+0000 7f0925dd6640 1 -- 172.21.15.165:0/1026977744 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0920052570 con 0x556827034970 2024-04-27T03:30:34.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.901+0000 7f0925dd6640 1 -- 172.21.15.165:0/1026977744 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0920052870 con 0x556827034970 2024-04-27T03:30:34.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.902+0000 7f092b133800 1 -- 172.21.15.165:0/1026977744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 msgr2=0x556827036d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.902+0000 7f092b133800 1 --2- 172.21.15.165:0/1026977744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x556827036d60 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7f092000a470 tx=0x7f092001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:34.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.902+0000 7f092b133800 1 -- 172.21.15.165:0/1026977744 shutdown_connections 2024-04-27T03:30:34.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.902+0000 7f092b133800 1 --2- 172.21.15.165:0/1026977744 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x556827036d60 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.902+0000 7f092b133800 1 -- 172.21.15.165:0/1026977744 >> 172.21.15.165:0/1026977744 conn(0x55682702e320 msgr2=0x556827030740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.902+0000 7f092b133800 1 -- 172.21.15.165:0/1026977744 shutdown_connections 2024-04-27T03:30:34.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.902+0000 7f092b133800 1 -- 172.21.15.165:0/1026977744 wait complete. 2024-04-27T03:30:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.914+0000 7f092b133800 1 Processor -- start 2024-04-27T03:30:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.914+0000 7f092b133800 1 -- start start 2024-04-27T03:30:34.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.914+0000 7f092b133800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x556827129af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.914+0000 7f092b133800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556826f7f3c0 con 0x556827034970 2024-04-27T03:30:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.914+0000 7f0926dd8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x556827129af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.914+0000 7f0926dd8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x556827129af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58604/0 (socket says 172.21.15.165:58604) 2024-04-27T03:30:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.914+0000 7f0926dd8640 1 -- 172.21.15.165:0/2634940044 learned_addr learned my addr 172.21.15.165:0/2634940044 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.914+0000 7f0926dd8640 1 -- 172.21.15.165:0/2634940044 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682712a0c0 con 0x556827034970 2024-04-27T03:30:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.915+0000 7f0926dd8640 1 --2- 172.21.15.165:0/2634940044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x556827129af0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0920055040 tx=0x7f092004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=755adce6eb4fd065 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.915+0000 7f0814e09640 1 -- 172.21.15.165:0/2634940044 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0920057070 con 0x556827034970 2024-04-27T03:30:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.915+0000 7f0814e09640 1 -- 172.21.15.165:0/2634940044 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0920063070 con 0x556827034970 2024-04-27T03:30:34.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.915+0000 7f0814e09640 1 -- 172.21.15.165:0/2634940044 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0920052720 con 0x556827034970 2024-04-27T03:30:34.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f092b133800 1 -- 172.21.15.165:0/2634940044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 msgr2=0x556827129af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f092b133800 1 --2- 172.21.15.165:0/2634940044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x556827129af0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7f0920055040 tx=0x7f092004c000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f092b133800 1 -- 172.21.15.165:0/2634940044 shutdown_connections 2024-04-27T03:30:34.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f092b133800 1 --2- 172.21.15.165:0/2634940044 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x556827129af0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f092b133800 1 -- 172.21.15.165:0/2634940044 >> 172.21.15.165:0/2634940044 conn(0x55682702e320 msgr2=0x5568270328c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f092b133800 1 -- 172.21.15.165:0/2634940044 shutdown_connections 2024-04-27T03:30:34.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f092b133800 1 -- 172.21.15.165:0/2634940044 wait complete. 2024-04-27T03:30:34.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f092b133800 1 Processor -- start 2024-04-27T03:30:34.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f092b133800 1 -- start start 2024-04-27T03:30:34.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f092b133800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x55682712ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f092b133800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55682703d0c0 con 0x556827034970 2024-04-27T03:30:34.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f0926dd8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x55682712ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f0926dd8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x55682712ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58606/0 (socket says 172.21.15.165:58606) 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f0926dd8640 1 -- 172.21.15.165:0/1335687106 learned_addr learned my addr 172.21.15.165:0/1335687106 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.916+0000 7f0926dd8640 1 -- 172.21.15.165:0/1335687106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682712f250 con 0x556827034970 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.917+0000 7f0926dd8640 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x55682712ed10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0920060040 tx=0x7f0920002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.917+0000 7f0813606640 1 -- 172.21.15.165:0/1335687106 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f092000a2d0 con 0x556827034970 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.917+0000 7f0813606640 1 -- 172.21.15.165:0/1335687106 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092006c070 con 0x556827034970 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.917+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55682712f4e0 con 0x556827034970 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.917+0000 7f0813606640 1 -- 172.21.15.165:0/1335687106 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f092005e5d0 con 0x556827034970 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.917+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556827132190 con 0x556827034970 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.918+0000 7f0813606640 1 -- 172.21.15.165:0/1335687106 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f092005e770 con 0x556827034970 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.918+0000 7f0813606640 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f080c036950 0x7f080c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.918+0000 7f0813606640 1 -- 172.21.15.165:0/1335687106 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f092008e830 con 0x556827034970 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.918+0000 7f09265d7640 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f080c036950 0x7f080c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.918+0000 7f092b133800 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568270372a0 0x556827135f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.918+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568271364d0 con 0x5568270372a0 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.918+0000 7f09275d9640 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568270372a0 0x556827135f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.918+0000 7f09265d7640 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f080c036950 0x7f080c038e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0910000f60 tx=0x7f09100012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.918+0000 7f09275d9640 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568270372a0 0x556827135f50 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.919+0000 7f09275d9640 1 -- 172.21.15.165:0/1335687106 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f091800a4a0 con 0x5568270372a0 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.919+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556827133520 con 0x5568270372a0 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.919+0000 7f09275d9640 1 -- 172.21.15.165:0/1335687106 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f091800a4a0 con 0x5568270372a0 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.919+0000 7f092b133800 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556827138850 0x55682713ac40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.919+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55682713b330 con 0x556827138850 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.919+0000 7f0926dd8640 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556827138850 0x55682713ac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.919+0000 7f0926dd8640 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556827138850 0x55682713ac40 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.920+0000 7f0926dd8640 1 -- 172.21.15.165:0/1335687106 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0920071990 con 0x556827138850 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.920+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556827137800 con 0x5568270372a0 2024-04-27T03:30:34.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.920+0000 7f09275d9640 1 -- 172.21.15.165:0/1335687106 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f091800a4a0 con 0x5568270372a0 2024-04-27T03:30:34.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.920+0000 7f092b133800 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55682713c7e0 0x55682713ec30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.920+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55682713f390 con 0x55682713c7e0 2024-04-27T03:30:34.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.920+0000 7f09265d7640 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55682713c7e0 0x55682713ec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.920+0000 7f09265d7640 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55682713c7e0 0x55682713ec30 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.921+0000 7f09265d7640 1 -- 172.21.15.165:0/1335687106 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0910009050 con 0x55682713c7e0 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.921+0000 7f092b133800 1 Processor -- start 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.921+0000 7f092b133800 1 -- start start 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.921+0000 7f092b133800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682714d7f0 0x55682714dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.921+0000 7f092b133800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556826f256b0 con 0x55682714d7f0 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.921+0000 7f0926dd8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682714d7f0 0x55682714dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.921+0000 7f0926dd8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682714d7f0 0x55682714dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58622/0 (socket says 172.21.15.165:58622) 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.921+0000 7f0926dd8640 1 -- 172.21.15.165:0/1747968985 learned_addr learned my addr 172.21.15.165:0/1747968985 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.922+0000 7f0926dd8640 1 -- 172.21.15.165:0/1747968985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682714e150 con 0x55682714d7f0 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.922+0000 7f0926dd8640 1 --2- 172.21.15.165:0/1747968985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682714d7f0 0x55682714dc10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f092008e010 tx=0x7f0920009640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.922+0000 7f0810e01640 1 -- 172.21.15.165:0/1747968985 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0920092020 con 0x55682714d7f0 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.922+0000 7f0810e01640 1 -- 172.21.15.165:0/1747968985 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0920058040 con 0x55682714d7f0 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.922+0000 7f092b133800 1 -- 172.21.15.165:0/1747968985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55682714ebc0 con 0x55682714d7f0 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.922+0000 7f0810e01640 1 -- 172.21.15.165:0/1747968985 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f092008eb90 con 0x55682714d7f0 2024-04-27T03:30:34.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.922+0000 7f092b133800 1 -- 172.21.15.165:0/1747968985 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55682714f0c0 con 0x55682714d7f0 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.923+0000 7f0810e01640 1 -- 172.21.15.165:0/1747968985 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0920095020 con 0x55682714d7f0 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.923+0000 7f0810e01640 1 --2- 172.21.15.165:0/1747968985 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0800036900 0x7f0800038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.923+0000 7f0810e01640 1 -- 172.21.15.165:0/1747968985 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f092008b070 con 0x55682714d7f0 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.923+0000 7f09265d7640 1 --2- 172.21.15.165:0/1747968985 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0800036900 0x7f0800038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.923+0000 7f092b133800 1 Processor -- start 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.923+0000 7f092b133800 1 -- start start 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.923+0000 7f092b133800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 0x55682715ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.923+0000 7f09265d7640 1 --2- 172.21.15.165:0/1747968985 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0800036900 0x7f0800038dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f091000bf50 tx=0x7f091000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.923+0000 7f092b133800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55682712a2f0 con 0x55682715a910 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.924+0000 7f0926dd8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 0x55682715ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.924+0000 7f0926dd8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 0x55682715ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58628/0 (socket says 172.21.15.165:58628) 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.924+0000 7f0926dd8640 1 -- 172.21.15.165:0/3932116094 learned_addr learned my addr 172.21.15.165:0/3932116094 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.924+0000 7f0926dd8640 1 -- 172.21.15.165:0/3932116094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55682715b2e0 con 0x55682715a910 2024-04-27T03:30:34.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.924+0000 7f0926dd8640 1 --2- 172.21.15.165:0/3932116094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 0x55682715ad10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0920071c80 tx=0x7f092007bd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53e13b898093eee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.924+0000 7f080a7fc640 1 -- 172.21.15.165:0/3932116094 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f092008e0b0 con 0x55682715a910 2024-04-27T03:30:34.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.924+0000 7f080a7fc640 1 -- 172.21.15.165:0/3932116094 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f092007b380 con 0x55682715a910 2024-04-27T03:30:34.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.924+0000 7f080a7fc640 1 -- 172.21.15.165:0/3932116094 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f092008e250 con 0x55682715a910 2024-04-27T03:30:34.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f092b133800 1 -- 172.21.15.165:0/3932116094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 msgr2=0x55682715ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f092b133800 1 --2- 172.21.15.165:0/3932116094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 0x55682715ad10 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f0920071c80 tx=0x7f092007bd50 comp rx=0 tx=0).stop 2024-04-27T03:30:34.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f092b133800 1 -- 172.21.15.165:0/3932116094 shutdown_connections 2024-04-27T03:30:34.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f092b133800 1 --2- 172.21.15.165:0/3932116094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 0x55682715ad10 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f092b133800 1 -- 172.21.15.165:0/3932116094 >> 172.21.15.165:0/3932116094 conn(0x556827156be0 msgr2=0x556827154c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f092b133800 1 -- 172.21.15.165:0/3932116094 shutdown_connections 2024-04-27T03:30:34.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f092b133800 1 -- 172.21.15.165:0/3932116094 wait complete. 2024-04-27T03:30:34.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f092b133800 1 Processor -- start 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f092b133800 1 -- start start 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f092b133800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 0x556827161cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f092b133800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55682715b9d0 con 0x55682715a910 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f0926dd8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 0x556827161cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f0926dd8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 0x556827161cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58636/0 (socket says 172.21.15.165:58636) 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.925+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 learned_addr learned my addr 172.21.15.165:0/54675812 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.926+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556827162200 con 0x55682715a910 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.926+0000 7f0926dd8640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 0x556827161cc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f092009a5c0 tx=0x7f0920079f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.926+0000 7f0808ff9640 1 -- 172.21.15.165:0/54675812 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f09200963b0 con 0x55682715a910 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.926+0000 7f0808ff9640 1 -- 172.21.15.165:0/54675812 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0920096550 con 0x55682715a910 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.926+0000 7f0808ff9640 1 -- 172.21.15.165:0/54675812 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f092008b890 con 0x55682715a910 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.926+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556827162200 con 0x55682715a910 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.926+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556827162200 con 0x55682715a910 2024-04-27T03:30:34.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.927+0000 7f0808ff9640 1 -- 172.21.15.165:0/54675812 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f09200bd060 con 0x55682715a910 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.927+0000 7f0808ff9640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f07f0036900 0x7f07f0038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.927+0000 7f0808ff9640 1 -- 172.21.15.165:0/54675812 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f09200b7b30 con 0x55682715a910 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.927+0000 7f092b133800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.927+0000 7f09265d7640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f07f0036900 0x7f07f0038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.927+0000 7f092b133800 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55682716d380 0x55682716f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.927+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55682716ff90 con 0x55682716d380 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.927+0000 7f09275d9640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55682716d380 0x55682716f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f09265d7640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f07f0036900 0x7f07f0038dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0910001b60 tx=0x7f0910010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f09275d9640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55682716d380 0x55682716f840 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f091800a4a0 con 0x55682716d380 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f092b133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f092b133800 20 realm 2024-04-27T03:30:34.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f092b133800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568271703a0 con 0x55682716d380 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f091800a4a0 con 0x55682716d380 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f092b133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f092b133800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f092b133800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556827170c40 con 0x55682716d380 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f091800a4a0 con 0x55682716d380 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f092b133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f092b133800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.928+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x55682716d380 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.929+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f091800a4a0 con 0x55682716d380 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.929+0000 7f092b133800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.929+0000 7f092b133800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.929+0000 7f092b133800 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556827171d40 0x556827174130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.929+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568271746b0 con 0x556827171d40 2024-04-27T03:30:34.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.929+0000 7f0926dd8640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556827171d40 0x556827174130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.929+0000 7f0926dd8640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556827171d40 0x556827174130 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.929+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.929+0000 7f092b133800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:34.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.929+0000 7f092b133800 20 searching for the correct realm 2024-04-27T03:30:34.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.929+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827176030 con 0x556827171d40 2024-04-27T03:30:34.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.930+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.930+0000 7f0926dd8640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0920097020 0x7f092009a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.930+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827176030 con 0x7f0920097020 2024-04-27T03:30:34.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.930+0000 7f09265d7640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0920097020 0x7f092009a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.930+0000 7f09265d7640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0920097020 0x7f092009a640 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.930+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.930+0000 7f09265d7640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0910037040 0x7f0910036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.930+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827176030 con 0x7f0910037040 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.931+0000 7f09275d9640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0910037040 0x7f0910036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.931+0000 7f09275d9640 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0910037040 0x7f0910036410 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.931+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f091800a4a0 con 0x7f0910037040 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.931+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x55682716d380 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.932+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f091800a4a0 con 0x55682716d380 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.932+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x55682716d380 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.932+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f091800a4a0 con 0x55682716d380 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.932+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0910037040 2024-04-27T03:30:34.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.932+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f091800a4a0 con 0x7f0910037040 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.932+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0910037040 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.932+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f091800a4a0 con 0x7f0910037040 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.932+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0920097020 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.932+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.933+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x556827171d40 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.933+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.933+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x55682716d380 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.933+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f091800a4a0 con 0x55682716d380 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.933+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0910037040 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.934+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f091800a4a0 con 0x7f0910037040 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.934+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x55682716d380 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.934+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f091800a4a0 con 0x55682716d380 2024-04-27T03:30:34.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.934+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0920097020 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.934+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.934+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x556827171d40 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.934+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.934+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x556827171d40 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.935+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.935+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0910037040 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.935+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f091800a4a0 con 0x7f0910037040 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.935+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x556827171d40 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.935+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.935+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x556827171d40 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.935+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.935+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x55682716d380 2024-04-27T03:30:34.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.935+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f091800a4a0 con 0x55682716d380 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.935+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x55682716d380 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.936+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f091800a4a0 con 0x55682716d380 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.936+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0910037040 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.936+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0918033040 con 0x7f0910037040 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.936+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x556827171d40 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.936+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.936+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0920097020 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.936+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.936+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x55682716d380 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.937+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.937+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0910037040 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.937+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.937+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x556827171d40 2024-04-27T03:30:34.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.937+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.937+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0910037040 2024-04-27T03:30:34.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.937+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.937+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0920097020 2024-04-27T03:30:34.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.938+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.938+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x556827171d40 2024-04-27T03:30:34.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.938+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.938+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0910037040 2024-04-27T03:30:34.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.938+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.938+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x55682716d380 2024-04-27T03:30:34.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.938+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.938+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x556827170fe0 con 0x7f0920097020 2024-04-27T03:30:34.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:34.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:34.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:34.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556827175000 con 0x55682716d380 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568271753f0 con 0x55682716d380 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.939+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568271753f0 con 0x7f0910037040 2024-04-27T03:30:34.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 20 zone default found 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 4 Realm: () 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 20 zonegroup default 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556827175810 con 0x7f0910037040 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556827175810 con 0x556827171d40 2024-04-27T03:30:34.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x556827177ee0 con 0x556827171d40 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 20 started sync module instance, tier type = 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.940+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556827177380 con 0x7f0920097020 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.941+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.941+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93905820809120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556827176fe0 con 0x7f0920097020 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.941+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556827178c60 con 0x55682716d380 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.942+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93905820809120] v25'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.943+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.943+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93905820811904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556827179050 con 0x55682716d380 2024-04-27T03:30:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.943+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556827179ab0 con 0x556827171d40 2024-04-27T03:30:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.944+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93905820811904] v25'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.944+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.944+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93905820816736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556827179ea0 con 0x556827171d40 2024-04-27T03:30:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.944+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717a8e0 con 0x7f0920097020 2024-04-27T03:30:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.945+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.945+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93905820822736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717b5a0 con 0x7f0920097020 2024-04-27T03:30:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.945+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717c010 con 0x55682716d380 2024-04-27T03:30:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.945+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93905820816736] v25'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.946+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.946+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93905820826336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717c400 con 0x55682716d380 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.946+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717ce40 con 0x7f0910037040 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.946+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93905820822736] v25'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.947+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93905820826336] v25'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.947+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.947+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93905820812960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717ce00 con 0x7f0910037040 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.947+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717d6e0 con 0x7f0910037040 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.948+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93905820812960] v25'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.948+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.948+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93905820829968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717ce00 con 0x7f0910037040 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.948+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717dd00 con 0x7f0910037040 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.949+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'111 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.949+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93905820833872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717e120 con 0x7f0910037040 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.949+0000 7f092b133800 20 add_watcher() i=0 2024-04-27T03:30:34.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.949+0000 7f092b133800 20 add_watcher() i=1 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.949+0000 7f092b133800 20 add_watcher() i=2 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.949+0000 7f092b133800 20 add_watcher() i=3 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.949+0000 7f092b133800 20 add_watcher() i=4 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.949+0000 7f092b133800 20 add_watcher() i=5 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.949+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 93905820829968] v25'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.949+0000 7f092b133800 20 add_watcher() i=6 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.950+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93905820833872] v25'112 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.950+0000 7f092b133800 20 add_watcher() i=7 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.950+0000 7f092b133800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.950+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55682717e120 con 0x55682716d380 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.950+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.950+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93905820834928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717e090 con 0x55682716d380 2024-04-27T03:30:34.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.951+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93905820834928] v25'114 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.951+0000 7f07dd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.951+0000 7f092b133800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.951+0000 7f092b133800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.951+0000 7f092b133800 5 note: GC not initialized 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.952+0000 7f0757fff640 20 reqs_thread_entry: start 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.952+0000 7f092b133800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.952+0000 7f092b133800 20 Filter name: none 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.952+0000 7f0755ffb640 20 reqs_thread_entry: start 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.952+0000 7f092b133800 20 rados->read ofs=0 len=0 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.952+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568271b3e90 con 0x7f0920097020 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.952+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(62 myfoo [call out=48b,stat out=16b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv18 ondisk = 0) ==== 275+0+304 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.952+0000 7f092b133800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:34.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.953+0000 7f07dd7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.953+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93905820834928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568271b1df0 con 0x55682716d380 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.954+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 93905820834928] v25'115 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.954+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93905820809120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568271b1df0 con 0x7f0920097020 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.955+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 93905820809120] v25'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.955+0000 7f092b133800 20 remove_watcher() i=0 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.955+0000 7f092b133800 2 removed watcher, disabling cache 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.955+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93905820811904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568271777a0 con 0x55682716d380 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.956+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 93905820811904] v25'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.956+0000 7f092b133800 20 remove_watcher() i=1 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.956+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93905820816736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568271b1df0 con 0x556827171d40 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.957+0000 7f0926dd8640 1 -- 172.21.15.165:0/54675812 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 93905820816736] v25'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0920071990 con 0x556827171d40 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.957+0000 7f092b133800 20 remove_watcher() i=2 2024-04-27T03:30:34.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.957+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93905820822736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x556827178280 con 0x7f0920097020 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.958+0000 7f09265d7640 1 -- 172.21.15.165:0/54675812 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 93905820822736] v25'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0910009050 con 0x7f0920097020 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.958+0000 7f092b133800 20 remove_watcher() i=3 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.958+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93905820826336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568271b1df0 con 0x55682716d380 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.959+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 93905820826336] v25'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x55682716d380 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.959+0000 7f092b133800 20 remove_watcher() i=4 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.959+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93905820812960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717acd0 con 0x7f0910037040 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.960+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 93905820812960] v25'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.960+0000 7f092b133800 20 remove_watcher() i=5 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.960+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93905820829968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568271b1df0 con 0x7f0910037040 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.961+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 93905820829968] v25'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.961+0000 7f092b133800 20 remove_watcher() i=6 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.961+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93905820833872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55682717acd0 con 0x7f0910037040 2024-04-27T03:30:34.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 93905820833872] v25'113 uv111 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0918031040 con 0x7f0910037040 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 20 remove_watcher() i=7 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0910037040 msgr2=0x7f0910036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0910037040 0x7f0910036410 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55682716d380 msgr2=0x55682716f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55682716d380 0x55682716f840 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0920097020 msgr2=0x7f092009a640 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0920097020 0x7f092009a640 crc :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556827171d40 msgr2=0x556827174130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556827171d40 0x556827174130 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f07f0036900 msgr2=0x7f07f0038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f07f0036900 0x7f07f0038dc0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f0910001b60 tx=0x7f0910010000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 msgr2=0x556827161cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 0x556827161cc0 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f092009a5c0 tx=0x7f0920079f60 comp rx=0 tx=0).stop 2024-04-27T03:30:34.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f09275d9640 1 -- 172.21.15.165:0/54675812 reap_dead start 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 shutdown_connections 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.962+0000 7f092b133800 1 --2- 172.21.15.165:0/54675812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682715a910 0x556827161cc0 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 >> 172.21.15.165:0/54675812 conn(0x556827156be0 msgr2=0x556827154240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 shutdown_connections 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/54675812 wait complete. 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1747968985 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0800036900 msgr2=0x7f0800038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 --2- 172.21.15.165:0/1747968985 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0800036900 0x7f0800038dc0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f091000bf50 tx=0x7f091000f000 comp rx=0 tx=0).stop 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1747968985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682714d7f0 msgr2=0x55682714dc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 --2- 172.21.15.165:0/1747968985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682714d7f0 0x55682714dc10 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f092008e010 tx=0x7f0920009640 comp rx=0 tx=0).stop 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1747968985 shutdown_connections 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 --2- 172.21.15.165:0/1747968985 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0800036900 0x7f0800038dc0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 --2- 172.21.15.165:0/1747968985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55682714d7f0 0x55682714dc10 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1747968985 >> 172.21.15.165:0/1747968985 conn(0x556827141890 msgr2=0x556827141c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1747968985 shutdown_connections 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1747968985 wait complete. 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55682713c7e0 msgr2=0x55682713ec30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55682713c7e0 0x55682713ec30 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568270372a0 msgr2=0x556827135f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568270372a0 0x556827135f50 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556827138850 msgr2=0x55682713ac40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556827138850 0x55682713ac40 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f080c036950 msgr2=0x7f080c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f080c036950 0x7f080c038e10 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f0910000f60 tx=0x7f09100012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 msgr2=0x55682712ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 --2- 172.21.15.165:0/1335687106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556827034970 0x55682712ed10 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7f0920060040 tx=0x7f0920002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:34.983 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f09275d9640 1 -- 172.21.15.165:0/1335687106 reap_dead start 2024-04-27T03:30:34.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 shutdown_connections 2024-04-27T03:30:34.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.963+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 >> 172.21.15.165:0/1335687106 conn(0x55682702e320 msgr2=0x556827032890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:34.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.964+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 shutdown_connections 2024-04-27T03:30:34.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:34.964+0000 7f092b133800 1 -- 172.21.15.165:0/1335687106 wait complete. 2024-04-27T03:30:34.985 INFO:teuthology.orchestra.run.smithi165.stdout:{"key":"bucket:myfoo","ver":{"tag":"_M9eK2p6xF8XWFb12xOuDLa-","ver":2},"mtime":"2024-04-27T03:30:34.673783Z","data":{"bucket":{"name":"myfoo","marker":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1","bucket_id":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1","tenant":"","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""}},"owner":"foo","creation_time":"2024-04-27T03:30:33.422040Z","linked":false,"has_bucket_info":false}} 2024-04-27T03:30:34.985 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:myfoo', 'ver': {'tag': '_M9eK2p6xF8XWFb12xOuDLa-', 'ver': 2}, 'mtime': '2024-04-27T03:30:34.673783Z', 'data': {'bucket': {'name': 'myfoo', 'marker': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1', 'bucket_id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1', 'tenant': '', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'foo', 'creation_time': '2024-04-27T03:30:33.422040Z', 'linked': False, 'has_bucket_info': False}} 2024-04-27T03:30:34.985 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1'] 2024-04-27T03:30:34.986 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'fud', '--bucket', 'myfoo', '--bucket-id', '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1'] 2024-04-27T03:30:34.986 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid fud --bucket myfoo --bucket-id 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 2024-04-27T03:30:35.074 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:35.074 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:35.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.069+0000 7f615d839800 1 Processor -- start 2024-04-27T03:30:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.069+0000 7f615d839800 1 -- start start 2024-04-27T03:30:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.069+0000 7f615d839800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562a1edb2200 0x562a1ee419e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.069+0000 7f615d839800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562a1ecfdd40 con 0x562a1ee41f80 2024-04-27T03:30:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.069+0000 7f615d839800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x562a1ecd04d0 con 0x562a1edb2200 2024-04-27T03:30:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.070+0000 7f615846e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562a1edb2200 0x562a1ee419e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.070+0000 7f615846e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562a1edb2200 0x562a1ee419e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58640/0 (socket says 172.21.15.165:58640) 2024-04-27T03:30:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.070+0000 7f614fc6d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x562a1ee41f80 0x562a1ee44420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35326/0 (socket says 172.21.15.165:35326) 2024-04-27T03:30:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.070+0000 7f615846e640 1 -- 172.21.15.165:0/1178173482 learned_addr learned my addr 172.21.15.165:0/1178173482 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.070+0000 7f615846e640 1 -- 172.21.15.165:0/1178173482 >> v1:172.21.15.89:6789/0 conn(0x562a1ee41f80 legacy=0x562a1ee44420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.070+0000 7f615846e640 1 -- 172.21.15.165:0/1178173482 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1ecfc590 con 0x562a1edb2200 2024-04-27T03:30:35.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.072+0000 7f615846e640 1 --2- 172.21.15.165:0/1178173482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ee419e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f614800a520 tx=0x7f614801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a167ececd64a349d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.072+0000 7f6157c6d640 1 -- 172.21.15.165:0/1178173482 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6148057070 con 0x562a1edb2200 2024-04-27T03:30:35.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.072+0000 7f6157c6d640 1 -- 172.21.15.165:0/1178173482 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6148052570 con 0x562a1edb2200 2024-04-27T03:30:35.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.072+0000 7f6157c6d640 1 -- 172.21.15.165:0/1178173482 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f614804ae60 con 0x562a1edb2200 2024-04-27T03:30:35.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.072+0000 7f615d839800 1 -- 172.21.15.165:0/1178173482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 msgr2=0x562a1ee419e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.072+0000 7f615d839800 1 --2- 172.21.15.165:0/1178173482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ee419e0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f614800a520 tx=0x7f614801d010 comp rx=0 tx=0).stop 2024-04-27T03:30:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.072+0000 7f615d839800 1 -- 172.21.15.165:0/1178173482 shutdown_connections 2024-04-27T03:30:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.072+0000 7f615d839800 1 --2- 172.21.15.165:0/1178173482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ee419e0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.072+0000 7f615d839800 1 -- 172.21.15.165:0/1178173482 >> 172.21.15.165:0/1178173482 conn(0x562a1ee393a0 msgr2=0x562a1ee3b7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.073+0000 7f615d839800 1 -- 172.21.15.165:0/1178173482 shutdown_connections 2024-04-27T03:30:35.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.073+0000 7f615d839800 1 -- 172.21.15.165:0/1178173482 wait complete. 2024-04-27T03:30:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.084+0000 7f615d839800 1 Processor -- start 2024-04-27T03:30:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.084+0000 7f615d839800 1 -- start start 2024-04-27T03:30:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.084+0000 7f615d839800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ee3f310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.084+0000 7f615d839800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562a1ed8a3e0 con 0x562a1edb2200 2024-04-27T03:30:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.085+0000 7f615846e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ee3f310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.085+0000 7f615846e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ee3f310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58650/0 (socket says 172.21.15.165:58650) 2024-04-27T03:30:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.085+0000 7f615846e640 1 -- 172.21.15.165:0/4154569649 learned_addr learned my addr 172.21.15.165:0/4154569649 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.085+0000 7f615846e640 1 -- 172.21.15.165:0/4154569649 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1ee3d8d0 con 0x562a1edb2200 2024-04-27T03:30:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.085+0000 7f615846e640 1 --2- 172.21.15.165:0/4154569649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ee3f310 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6148055040 tx=0x7f614804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57f04976610e408e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.085+0000 7f604660c640 1 -- 172.21.15.165:0/4154569649 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6148057040 con 0x562a1edb2200 2024-04-27T03:30:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.085+0000 7f604660c640 1 -- 172.21.15.165:0/4154569649 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6148052720 con 0x562a1edb2200 2024-04-27T03:30:35.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.085+0000 7f604660c640 1 -- 172.21.15.165:0/4154569649 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6148052a40 con 0x562a1edb2200 2024-04-27T03:30:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.086+0000 7f615d839800 1 -- 172.21.15.165:0/4154569649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 msgr2=0x562a1ee3f310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.086+0000 7f615d839800 1 --2- 172.21.15.165:0/4154569649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ee3f310 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f6148055040 tx=0x7f614804c000 comp rx=0 tx=0).stop 2024-04-27T03:30:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.086+0000 7f615d839800 1 -- 172.21.15.165:0/4154569649 shutdown_connections 2024-04-27T03:30:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.086+0000 7f615d839800 1 --2- 172.21.15.165:0/4154569649 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ee3f310 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.086+0000 7f615d839800 1 -- 172.21.15.165:0/4154569649 >> 172.21.15.165:0/4154569649 conn(0x562a1ee393a0 msgr2=0x562a1ef2bae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.086+0000 7f615d839800 1 -- 172.21.15.165:0/4154569649 shutdown_connections 2024-04-27T03:30:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.086+0000 7f615d839800 1 -- 172.21.15.165:0/4154569649 wait complete. 2024-04-27T03:30:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.086+0000 7f615d839800 1 Processor -- start 2024-04-27T03:30:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.086+0000 7f615d839800 1 -- start start 2024-04-27T03:30:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.087+0000 7f615d839800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ef39ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.087+0000 7f615d839800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562a1ee45e90 con 0x562a1edb2200 2024-04-27T03:30:35.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.087+0000 7f615846e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ef39ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.087+0000 7f615846e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ef39ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58652/0 (socket says 172.21.15.165:58652) 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.087+0000 7f615846e640 1 -- 172.21.15.165:0/1413683939 learned_addr learned my addr 172.21.15.165:0/1413683939 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.087+0000 7f615846e640 1 -- 172.21.15.165:0/1413683939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1ef3a430 con 0x562a1edb2200 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.087+0000 7f615846e640 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ef39ef0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6148052540 tx=0x7f6148003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.087+0000 7f6044e09640 1 -- 172.21.15.165:0/1413683939 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6148073070 con 0x562a1edb2200 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.087+0000 7f6044e09640 1 -- 172.21.15.165:0/1413683939 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f614805d440 con 0x562a1edb2200 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.087+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562a1ef3a6c0 con 0x562a1edb2200 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.087+0000 7f6044e09640 1 -- 172.21.15.165:0/1413683939 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6148057070 con 0x562a1edb2200 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.088+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a1ef3d370 con 0x562a1edb2200 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.088+0000 7f6044e09640 1 -- 172.21.15.165:0/1413683939 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6148076070 con 0x562a1edb2200 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.088+0000 7f6044e09640 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6040036950 0x7f6040038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.088+0000 7f6044e09640 1 -- 172.21.15.165:0/1413683939 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f614808d9e0 con 0x562a1edb2200 2024-04-27T03:30:35.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.089+0000 7f614fc6d640 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6040036950 0x7f6040038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.089+0000 7f615d839800 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562a1ee41f80 0x562a1ef41130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.089+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef41670 con 0x562a1ee41f80 2024-04-27T03:30:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.089+0000 7f6158c6f640 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562a1ee41f80 0x562a1ef41130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.089+0000 7f614fc6d640 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6040036950 0x7f6040038e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6140000f60 tx=0x7f6140001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.089+0000 7f6158c6f640 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562a1ee41f80 0x562a1ef41130 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.089+0000 7f6158c6f640 1 -- 172.21.15.165:0/1413683939 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ee41f80 2024-04-27T03:30:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.089+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef3e700 con 0x562a1ee41f80 2024-04-27T03:30:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.090+0000 7f6158c6f640 1 -- 172.21.15.165:0/1413683939 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ee41f80 2024-04-27T03:30:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.090+0000 7f615d839800 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562a1ef439d0 0x562a1ef45dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.090+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef464b0 con 0x562a1ef439d0 2024-04-27T03:30:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.090+0000 7f615846e640 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562a1ef439d0 0x562a1ef45dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.090+0000 7f615846e640 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562a1ef439d0 0x562a1ef45dc0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.090+0000 7f615846e640 1 -- 172.21.15.165:0/1413683939 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f6148075030 con 0x562a1ef439d0 2024-04-27T03:30:35.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.090+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef42980 con 0x562a1ee41f80 2024-04-27T03:30:35.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.091+0000 7f6158c6f640 1 -- 172.21.15.165:0/1413683939 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ee41f80 2024-04-27T03:30:35.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.091+0000 7f615d839800 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562a1ef47960 0x562a1ef49db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.091+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef4a510 con 0x562a1ef47960 2024-04-27T03:30:35.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.091+0000 7f614fc6d640 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562a1ef47960 0x562a1ef49db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.091+0000 7f614fc6d640 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562a1ef47960 0x562a1ef49db0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.092+0000 7f614fc6d640 1 -- 172.21.15.165:0/1413683939 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f614000f690 con 0x562a1ef47960 2024-04-27T03:30:35.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.092+0000 7f615d839800 1 Processor -- start 2024-04-27T03:30:35.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.092+0000 7f615d839800 1 -- start start 2024-04-27T03:30:35.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.092+0000 7f615d839800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef58900 0x562a1ef58d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.092+0000 7f615d839800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562a1ed306b0 con 0x562a1ef58900 2024-04-27T03:30:35.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.092+0000 7f615846e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef58900 0x562a1ef58d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.092+0000 7f615846e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef58900 0x562a1ef58d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58658/0 (socket says 172.21.15.165:58658) 2024-04-27T03:30:35.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.092+0000 7f615846e640 1 -- 172.21.15.165:0/3648205482 learned_addr learned my addr 172.21.15.165:0/3648205482 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.093+0000 7f615846e640 1 -- 172.21.15.165:0/3648205482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1ef59260 con 0x562a1ef58900 2024-04-27T03:30:35.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.093+0000 7f615846e640 1 --2- 172.21.15.165:0/3648205482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef58900 0x562a1ef58d20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6148076b10 tx=0x7f61480776a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.093+0000 7f603dffb640 1 -- 172.21.15.165:0/3648205482 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6148083040 con 0x562a1ef58900 2024-04-27T03:30:35.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.093+0000 7f603dffb640 1 -- 172.21.15.165:0/3648205482 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6148054dd0 con 0x562a1ef58900 2024-04-27T03:30:35.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.093+0000 7f615d839800 1 -- 172.21.15.165:0/3648205482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562a1ef59cd0 con 0x562a1ef58900 2024-04-27T03:30:35.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.093+0000 7f603dffb640 1 -- 172.21.15.165:0/3648205482 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f614805cbb0 con 0x562a1ef58900 2024-04-27T03:30:35.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.093+0000 7f615d839800 1 -- 172.21.15.165:0/3648205482 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a1ef5a1d0 con 0x562a1ef58900 2024-04-27T03:30:35.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.094+0000 7f603dffb640 1 -- 172.21.15.165:0/3648205482 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f614805d720 con 0x562a1ef58900 2024-04-27T03:30:35.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.094+0000 7f603dffb640 1 --2- 172.21.15.165:0/3648205482 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6030036900 0x7f6030038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.094+0000 7f603dffb640 1 -- 172.21.15.165:0/3648205482 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f6148098ab0 con 0x562a1ef58900 2024-04-27T03:30:35.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.094+0000 7f614fc6d640 1 --2- 172.21.15.165:0/3648205482 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6030036900 0x7f6030038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.094+0000 7f615d839800 1 Processor -- start 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.094+0000 7f615d839800 1 -- start start 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.094+0000 7f614fc6d640 1 --2- 172.21.15.165:0/3648205482 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6030036900 0x7f6030038dc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x562a1ef41f80 tx=0x7f614000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.094+0000 7f615d839800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 0x562a1ef65eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.094+0000 7f615d839800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562a1ee3db90 con 0x562a1ef65a90 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.095+0000 7f615846e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 0x562a1ef65eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.095+0000 7f615846e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 0x562a1ef65eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58674/0 (socket says 172.21.15.165:58674) 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.095+0000 7f615846e640 1 -- 172.21.15.165:0/1225378522 learned_addr learned my addr 172.21.15.165:0/1225378522 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.095+0000 7f615846e640 1 -- 172.21.15.165:0/1225378522 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1ef66480 con 0x562a1ef65a90 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.095+0000 7f615846e640 1 --2- 172.21.15.165:0/1225378522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 0x562a1ef65eb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f614809e040 tx=0x7f61480888f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94db3df6381770fd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.095+0000 7f60377fe640 1 -- 172.21.15.165:0/1225378522 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f61480a5030 con 0x562a1ef65a90 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.095+0000 7f60377fe640 1 -- 172.21.15.165:0/1225378522 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6148098d80 con 0x562a1ef65a90 2024-04-27T03:30:35.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.095+0000 7f60377fe640 1 -- 172.21.15.165:0/1225378522 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f614805c890 con 0x562a1ef65a90 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.096+0000 7f615d839800 1 -- 172.21.15.165:0/1225378522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 msgr2=0x562a1ef65eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.096+0000 7f615d839800 1 --2- 172.21.15.165:0/1225378522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 0x562a1ef65eb0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f614809e040 tx=0x7f61480888f0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.096+0000 7f615d839800 1 -- 172.21.15.165:0/1225378522 shutdown_connections 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.096+0000 7f615d839800 1 --2- 172.21.15.165:0/1225378522 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 0x562a1ef65eb0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.096+0000 7f615d839800 1 -- 172.21.15.165:0/1225378522 >> 172.21.15.165:0/1225378522 conn(0x562a1ef61da0 msgr2=0x562a1ef5fe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.096+0000 7f615d839800 1 -- 172.21.15.165:0/1225378522 shutdown_connections 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.096+0000 7f615d839800 1 -- 172.21.15.165:0/1225378522 wait complete. 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.096+0000 7f615d839800 1 Processor -- start 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.096+0000 7f615d839800 1 -- start start 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.096+0000 7f615d839800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 0x562a1ef6cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.096+0000 7f615d839800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562a1ef66b10 con 0x562a1ef65a90 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.097+0000 7f615846e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 0x562a1ef6cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.097+0000 7f615846e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 0x562a1ef6cee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58680/0 (socket says 172.21.15.165:58680) 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.097+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 learned_addr learned my addr 172.21.15.165:0/1618222567 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.097+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a1ef6d420 con 0x562a1ef65a90 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.097+0000 7f615846e640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 0x562a1ef6cee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f614808d9b0 tx=0x7f6148096f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.098+0000 7f6035ffb640 1 -- 172.21.15.165:0/1618222567 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f614807f210 con 0x562a1ef65a90 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.098+0000 7f6035ffb640 1 -- 172.21.15.165:0/1618222567 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f61480bc070 con 0x562a1ef65a90 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.098+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562a1ef6d420 con 0x562a1ef65a90 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.098+0000 7f6035ffb640 1 -- 172.21.15.165:0/1618222567 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6148066d70 con 0x562a1ef65a90 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.098+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a1ef6d420 con 0x562a1ef65a90 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.098+0000 7f6035ffb640 1 -- 172.21.15.165:0/1618222567 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6148061b20 con 0x562a1ef65a90 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.099+0000 7f6035ffb640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6028036900 0x7f6028038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.099+0000 7f6035ffb640 1 -- 172.21.15.165:0/1618222567 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f61480a9070 con 0x562a1ef65a90 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.099+0000 7f614fc6d640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6028036900 0x7f6028038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.099+0000 7f614fc6d640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6028036900 0x7f6028038dc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x562a1ef4a910 tx=0x7f6140010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.099+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.099+0000 7f615d839800 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562a1ef78450 0x562a1ef7a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.099+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7b060 con 0x562a1ef78450 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.099+0000 7f6158c6f640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562a1ef78450 0x562a1ef7a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f6158c6f640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562a1ef78450 0x562a1ef7a910 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ef78450 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f615d839800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f615d839800 20 realm 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7b470 con 0x562a1ef78450 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ef78450 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f615d839800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f615d839800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:35.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7bd40 con 0x562a1ef78450 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ef78450 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f615d839800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.100+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef78450 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.101+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ef78450 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.101+0000 7f615d839800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.101+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.101+0000 7f615d839800 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562a1ef7ce40 0x562a1ef7f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.101+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7f840 con 0x562a1ef7ce40 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.101+0000 7f615846e640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562a1ef7ce40 0x562a1ef7f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.101+0000 7f615846e640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562a1ef7ce40 0x562a1ef7f230 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.101+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.101+0000 7f615d839800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.101+0000 7f615d839800 20 searching for the correct realm 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.101+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef81170 con 0x562a1ef7ce40 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.102+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.102+0000 7f615846e640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6148097020 0x7f61480b8a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.102+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef81170 con 0x7f6148097020 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.102+0000 7f614fc6d640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6148097020 0x7f61480b8a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.102+0000 7f614fc6d640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6148097020 0x7f61480b8a30 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.102+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.102+0000 7f614fc6d640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f614000f070 0x7f614002e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.102+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef81170 con 0x7f614000f070 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.103+0000 7f6158c6f640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f614000f070 0x7f614002e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.103+0000 7f6158c6f640 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f614000f070 0x7f614002e910 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.103+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f615000a4a0 con 0x7f614000f070 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.103+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef78450 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.104+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ef78450 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.104+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef78450 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.104+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ef78450 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.104+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f614000f070 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.104+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f615000a4a0 con 0x7f614000f070 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.104+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f614000f070 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.104+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f615000a4a0 con 0x7f614000f070 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.104+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f6148097020 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.105+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.105+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef7ce40 2024-04-27T03:30:35.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.105+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.105+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef78450 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.105+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ef78450 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.105+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f614000f070 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.105+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f615000a4a0 con 0x7f614000f070 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.105+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef78450 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.106+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ef78450 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.106+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f6148097020 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.106+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.106+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef7ce40 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.106+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.106+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef7ce40 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.106+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.106+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f614000f070 2024-04-27T03:30:35.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.107+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f615000a4a0 con 0x7f614000f070 2024-04-27T03:30:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.107+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef7ce40 2024-04-27T03:30:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.107+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.107+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef7ce40 2024-04-27T03:30:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.107+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.107+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef78450 2024-04-27T03:30:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.107+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ef78450 2024-04-27T03:30:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.107+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef78450 2024-04-27T03:30:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.108+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f615000a4a0 con 0x562a1ef78450 2024-04-27T03:30:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.108+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f614000f070 2024-04-27T03:30:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.108+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6150033040 con 0x7f614000f070 2024-04-27T03:30:35.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.108+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef7ce40 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.108+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.108+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f6148097020 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.109+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.109+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef78450 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.109+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.109+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f614000f070 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.109+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.109+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef7ce40 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.109+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.109+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f614000f070 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.110+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.110+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f6148097020 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.110+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.110+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef7ce40 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.110+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.110+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f614000f070 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.110+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.110+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x562a1ef78450 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef7c0e0 con 0x7f6148097020 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:35.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef80100 con 0x562a1ef78450 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef804f0 con 0x562a1ef78450 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.111+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef804f0 con 0x7f614000f070 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615d839800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615d839800 20 zone default found 2024-04-27T03:30:35.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615d839800 4 Realm: () 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615d839800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615d839800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615d839800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615d839800 20 zonegroup default 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef815f0 con 0x7f614000f070 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615d839800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef82fd0 con 0x562a1ef7ce40 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.112+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.113+0000 7f615d839800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.113+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.113+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef83370 con 0x562a1ef7ce40 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.113+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.113+0000 7f615d839800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.113+0000 7f615d839800 20 started sync module instance, tier type = 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.113+0000 7f615d839800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.113+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef82470 con 0x7f6148097020 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.114+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.114+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94738908192704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef83710 con 0x7f6148097020 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.114+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef83fc0 con 0x562a1ef78450 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.115+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94738908192704] v25'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.115+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.115+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94738908199600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef843b0 con 0x562a1ef78450 2024-04-27T03:30:35.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.115+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef84dd0 con 0x562a1ef7ce40 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.116+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.116+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94738908203200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef85670 con 0x562a1ef7ce40 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.116+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef86150 con 0x7f6148097020 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.116+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94738908199600] v25'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.117+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94738908203200] v25'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.117+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.117+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94738908194960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef85670 con 0x7f6148097020 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.117+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef86a30 con 0x562a1ef78450 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.118+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94738908194960] v25'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.118+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.119+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94738908205504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef86e20 con 0x562a1ef78450 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.119+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef87860 con 0x7f614000f070 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.119+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94738908205504] v25'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.120+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.120+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94738908214064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef87820 con 0x7f614000f070 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.120+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef88270 con 0x7f614000f070 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.121+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94738908214064] v25'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.121+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.121+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94738908210496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef88270 con 0x7f614000f070 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.121+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef88c30 con 0x7f614000f070 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.122+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94738908210496] v25'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.122+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'114 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.122+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94738908217248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef88c30 con 0x7f614000f070 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.122+0000 7f615d839800 20 add_watcher() i=0 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.122+0000 7f615d839800 20 add_watcher() i=1 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.122+0000 7f615d839800 20 add_watcher() i=2 2024-04-27T03:30:35.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.122+0000 7f615d839800 20 add_watcher() i=3 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.122+0000 7f615d839800 20 add_watcher() i=4 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.122+0000 7f615d839800 20 add_watcher() i=5 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.122+0000 7f615d839800 20 add_watcher() i=6 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.123+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94738908217248] v25'115 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.123+0000 7f615d839800 20 add_watcher() i=7 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.123+0000 7f615d839800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.123+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef8c090 con 0x562a1ef78450 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.123+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.123+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94738908219728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef89550 con 0x562a1ef78450 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.124+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94738908219728] v25'116 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.124+0000 7f600e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.124+0000 7f615d839800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:35.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.124+0000 7f615d839800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.124+0000 7f615d839800 5 note: GC not initialized 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.125+0000 7f600d7e2640 20 reqs_thread_entry: start 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.125+0000 7f615d839800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.125+0000 7f615d839800 20 Filter name: none 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.125+0000 7f5f7effd640 20 reqs_thread_entry: start 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.125+0000 7f615d839800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.125+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.125+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1efbf3f0 con 0x7f614000f070 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(62 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 20 rados_obj.operate() r=0 bl.length=380 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:35.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1efc14d0 con 0x7f6148097020 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv18 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 : miss 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1efc2f90 con 0x562a1ef78450 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.126+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(64 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv2 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.127+0000 7f615d839800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.127+0000 7f615d839800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.127+0000 7f615d839800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:35.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.127+0000 7f615d839800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:35.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.127+0000 7f615d839800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:35.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.127+0000 7f615d839800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 2024-04-27T03:30:35.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.127+0000 7f615d839800 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-04-27T03:30:35.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.127+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1efc7520 con 0x7f6148097020 2024-04-27T03:30:35.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.128+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(65 foo.buckets [call] v25'19 uv17 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.128+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1efcd460 con 0x562a1ef78450 2024-04-27T03:30:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(66 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [delete,create,call,call,writefull 0~379,setxattr (123)] v25'3 uv3 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f615d839800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f615d839800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f615d839800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f615d839800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1, ofs0, ns] 2024-04-27T03:30:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [notify cookie 94738908457056 in=764b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1efc7950 con 0x562a1ef78450 2024-04-27T03:30:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify (1) cookie 94738908199600 notify 107374182418 ret 0) ==== 794+0+0 (crc 0 0 0) 0x562a1efc3380 con 0x562a1ef78450 2024-04-27T03:30:35.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f60377fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182418 cookie 94738908199600 notifier 4490 bl.length()=752 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(67 notify.1 [notify cookie 94738908457056 out=8b] v0'0 uv114 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f60377fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f60377fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f60377fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.129+0000 7f60377fe640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f601c003ea0 con 0x562a1ef78450 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.130+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== watch-notify(notify_complete (2) cookie 94738908457056 notify 107374182418 ret 0) ==== 42+0+48 (crc 0 0 0) 0x562a1efc24d0 con 0x562a1ef78450 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.130+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [notify-ack] v0'0 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.130+0000 7f615d839800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): orig_sources= new_sources= 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.130+0000 7f615d839800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): potential sources added=[] removed=[] 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.130+0000 7f615d839800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): orig_dests= new_dests= 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.130+0000 7f615d839800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): potential dests added=[] removed=[] 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.130+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1efc2860 con 0x7f614000f070 2024-04-27T03:30:35.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.131+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 fud.buckets [call] v25'1 uv1 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.131+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1efc2860 con 0x7f6148097020 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.132+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v25'20 uv20 ondisk = 0) ==== 275+0+0 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.132+0000 7f615d839800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.132+0000 7f615d839800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.132+0000 7f615d839800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.132+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94738908457056 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562a1efcc0f0 con 0x7f614000f070 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.132+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== watch-notify(notify (1) cookie 94738908210496 notify 107374182403 ret 0) ==== 356+0+0 (crc 0 0 0) 0x7f615001e040 con 0x7f614000f070 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.133+0000 7f60377fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182403 cookie 94738908210496 notifier 4490 bl.length()=314 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.133+0000 7f60377fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.133+0000 7f60377fe640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.133+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(71 notify.6 [notify cookie 94738908457056 out=8b] v0'0 uv114 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.133+0000 7f60377fe640 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f601c003ea0 con 0x7f614000f070 2024-04-27T03:30:35.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.133+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== watch-notify(notify_complete (2) cookie 94738908457056 notify 107374182403 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f6150027420 con 0x7f614000f070 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.133+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.134+0000 7f600e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.134+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94738908219728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1efc2860 con 0x562a1ef78450 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.135+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94738908219728] v25'117 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.135+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94738908192704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef88920 con 0x7f6148097020 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.136+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94738908192704] v25'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.136+0000 7f615d839800 20 remove_watcher() i=0 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.136+0000 7f615d839800 2 removed watcher, disabling cache 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.136+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94738908199600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef81fc0 con 0x562a1ef78450 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.137+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94738908199600] v25'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.137+0000 7f615d839800 20 remove_watcher() i=1 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.137+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94738908203200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef81fc0 con 0x562a1ef7ce40 2024-04-27T03:30:35.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.138+0000 7f615846e640 1 -- 172.21.15.165:0/1618222567 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94738908203200] v25'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f614809e070 con 0x562a1ef7ce40 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.138+0000 7f615d839800 20 remove_watcher() i=2 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.138+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94738908194960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef81fc0 con 0x7f6148097020 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.139+0000 7f614fc6d640 1 -- 172.21.15.165:0/1618222567 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94738908194960] v25'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6140037040 con 0x7f6148097020 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.139+0000 7f615d839800 20 remove_watcher() i=3 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.139+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94738908205504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef81fc0 con 0x562a1ef78450 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.140+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94738908205504] v25'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x562a1ef78450 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.140+0000 7f615d839800 20 remove_watcher() i=4 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.140+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94738908214064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef81fc0 con 0x7f614000f070 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.141+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94738908214064] v25'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.141+0000 7f615d839800 20 remove_watcher() i=5 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.141+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94738908210496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef87330 con 0x7f614000f070 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.142+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94738908210496] v25'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.142+0000 7f615d839800 20 remove_watcher() i=6 2024-04-27T03:30:35.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.142+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94738908217248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562a1ef81fc0 con 0x7f614000f070 2024-04-27T03:30:35.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94738908217248] v25'116 uv114 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6150031040 con 0x7f614000f070 2024-04-27T03:30:35.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 20 remove_watcher() i=7 2024-04-27T03:30:35.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f614000f070 msgr2=0x7f614002e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f614000f070 0x7f614002e910 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562a1ef78450 msgr2=0x562a1ef7a910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562a1ef78450 0x562a1ef7a910 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6148097020 msgr2=0x7f61480b8a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6148097020 0x7f61480b8a30 crc :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562a1ef7ce40 msgr2=0x562a1ef7f230 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562a1ef7ce40 0x562a1ef7f230 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6028036900 msgr2=0x7f6028038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6028036900 0x7f6028038dc0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x562a1ef4a910 tx=0x7f6140010000 comp rx=0 tx=0).stop 2024-04-27T03:30:35.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 msgr2=0x562a1ef6cee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f6158c6f640 1 -- 172.21.15.165:0/1618222567 reap_dead start 2024-04-27T03:30:35.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 0x562a1ef6cee0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f614808d9b0 tx=0x7f6148096f20 comp rx=0 tx=0).stop 2024-04-27T03:30:35.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 shutdown_connections 2024-04-27T03:30:35.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 --2- 172.21.15.165:0/1618222567 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef65a90 0x562a1ef6cee0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 >> 172.21.15.165:0/1618222567 conn(0x562a1ef61da0 msgr2=0x562a1ef5f400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 shutdown_connections 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.143+0000 7f615d839800 1 -- 172.21.15.165:0/1618222567 wait complete. 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/3648205482 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6030036900 msgr2=0x7f6030038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 --2- 172.21.15.165:0/3648205482 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6030036900 0x7f6030038dc0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x562a1ef41f80 tx=0x7f614000f000 comp rx=0 tx=0).stop 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/3648205482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef58900 msgr2=0x562a1ef58d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 --2- 172.21.15.165:0/3648205482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef58900 0x562a1ef58d20 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f6148076b10 tx=0x7f61480776a0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/3648205482 shutdown_connections 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 --2- 172.21.15.165:0/3648205482 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6030036900 0x7f6030038dc0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 --2- 172.21.15.165:0/3648205482 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1ef58900 0x562a1ef58d20 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/3648205482 >> 172.21.15.165:0/3648205482 conn(0x562a1ef4c9a0 msgr2=0x562a1ef4cda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/3648205482 shutdown_connections 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/3648205482 wait complete. 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562a1ef47960 msgr2=0x562a1ef49db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562a1ef47960 0x562a1ef49db0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562a1ee41f80 msgr2=0x562a1ef41130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562a1ee41f80 0x562a1ef41130 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562a1ef439d0 msgr2=0x562a1ef45dc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562a1ef439d0 0x562a1ef45dc0 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6040036950 msgr2=0x7f6040038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6040036950 0x7f6040038e10 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f6140000f60 tx=0x7f6140001370 comp rx=0 tx=0).stop 2024-04-27T03:30:35.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 msgr2=0x562a1ef39ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 --2- 172.21.15.165:0/1413683939 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562a1edb2200 0x562a1ef39ef0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f6148052540 tx=0x7f6148003020 comp rx=0 tx=0).stop 2024-04-27T03:30:35.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f6158c6f640 1 -- 172.21.15.165:0/1413683939 reap_dead start 2024-04-27T03:30:35.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 shutdown_connections 2024-04-27T03:30:35.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 >> 172.21.15.165:0/1413683939 conn(0x562a1ee393a0 msgr2=0x562a1ee3e0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 shutdown_connections 2024-04-27T03:30:35.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.144+0000 7f615d839800 1 -- 172.21.15.165:0/1413683939 wait complete. 2024-04-27T03:30:35.165 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-04-27T03:30:35.166 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-04-27T03:30:35.166 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-04-27T03:30:35.208 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:35.208 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:35.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.203+0000 7f10b97dc800 1 Processor -- start 2024-04-27T03:30:35.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.203+0000 7f10b97dc800 1 -- start start 2024-04-27T03:30:35.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.203+0000 7f10b97dc800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x558423ed7480 0x558423ed9870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.203+0000 7f10b97dc800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558423d91d40 con 0x558423ed3970 2024-04-27T03:30:35.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.203+0000 7f10b97dc800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x558423d644d0 con 0x558423ed7480 2024-04-27T03:30:35.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.204+0000 7f10aeada640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x558423ed7480 0x558423ed9870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.204+0000 7f10b72db640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x558423ed3970 0x558423ed5d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35334/0 (socket says 172.21.15.165:35334) 2024-04-27T03:30:35.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.204+0000 7f10b72db640 1 -- 172.21.15.165:0/3895312900 learned_addr learned my addr 172.21.15.165:0/3895312900 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:35.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.204+0000 7f10aeada640 1 -- 172.21.15.165:0/3895312900 >> v1:172.21.15.89:6789/0 conn(0x558423ed3970 legacy=0x558423ed5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.204+0000 7f10b6ada640 1 -- 172.21.15.165:0/3895312900 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 753654157 0 0) 0x558423d91d40 con 0x558423ed3970 2024-04-27T03:30:35.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.204+0000 7f10aeada640 1 -- 172.21.15.165:0/3895312900 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558423d90590 con 0x558423ed7480 2024-04-27T03:30:35.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.206+0000 7f10aeada640 1 --2- 172.21.15.165:0/3895312900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed7480 0x558423ed9870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f10a000a470 tx=0x7f10a001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b1847e041cf1307 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.206+0000 7f10b6ada640 1 -- 172.21.15.165:0/3895312900 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10a0057070 con 0x558423ed7480 2024-04-27T03:30:35.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.206+0000 7f10b6ada640 1 -- 172.21.15.165:0/3895312900 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a0052570 con 0x558423ed7480 2024-04-27T03:30:35.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.206+0000 7f10b6ada640 1 -- 172.21.15.165:0/3895312900 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10a0052870 con 0x558423ed7480 2024-04-27T03:30:35.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.206+0000 7f10b97dc800 1 -- 172.21.15.165:0/3895312900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed7480 msgr2=0x558423ed9870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.206+0000 7f10b97dc800 1 --2- 172.21.15.165:0/3895312900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed7480 0x558423ed9870 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f10a000a470 tx=0x7f10a001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:35.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.206+0000 7f10b97dc800 1 -- 172.21.15.165:0/3895312900 shutdown_connections 2024-04-27T03:30:35.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.206+0000 7f10b97dc800 1 --2- 172.21.15.165:0/3895312900 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed7480 0x558423ed9870 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.206+0000 7f10b97dc800 1 -- 172.21.15.165:0/3895312900 >> 172.21.15.165:0/3895312900 conn(0x558423ecd320 msgr2=0x558423ecf740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.206+0000 7f10b97dc800 1 -- 172.21.15.165:0/3895312900 shutdown_connections 2024-04-27T03:30:35.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.206+0000 7f10b97dc800 1 -- 172.21.15.165:0/3895312900 wait complete. 2024-04-27T03:30:35.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.218+0000 7f10b97dc800 1 Processor -- start 2024-04-27T03:30:35.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.218+0000 7f10b97dc800 1 -- start start 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.218+0000 7f10b97dc800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 0x558423ed32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.218+0000 7f10b97dc800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558423e1e3c0 con 0x558423ed3970 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.219+0000 7f10b72db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 0x558423ed32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.219+0000 7f10b72db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 0x558423ed32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58710/0 (socket says 172.21.15.165:58710) 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.219+0000 7f10b72db640 1 -- 172.21.15.165:0/3410533199 learned_addr learned my addr 172.21.15.165:0/3410533199 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.219+0000 7f10b72db640 1 -- 172.21.15.165:0/3410533199 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558423ed1870 con 0x558423ed3970 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.219+0000 7f10b72db640 1 --2- 172.21.15.165:0/3410533199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 0x558423ed32b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f10a800b210 tx=0x7f10a800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79ddb3ee2d2ad3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.219+0000 7f0fa560a640 1 -- 172.21.15.165:0/3410533199 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10a8008070 con 0x558423ed3970 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.219+0000 7f0fa560a640 1 -- 172.21.15.165:0/3410533199 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a800d500 con 0x558423ed3970 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.219+0000 7f0fa560a640 1 -- 172.21.15.165:0/3410533199 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10a800d840 con 0x558423ed3970 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.220+0000 7f10b97dc800 1 -- 172.21.15.165:0/3410533199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 msgr2=0x558423ed32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.220+0000 7f10b97dc800 1 --2- 172.21.15.165:0/3410533199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 0x558423ed32b0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f10a800b210 tx=0x7f10a800b6d0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.220+0000 7f10b97dc800 1 -- 172.21.15.165:0/3410533199 shutdown_connections 2024-04-27T03:30:35.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.220+0000 7f10b97dc800 1 --2- 172.21.15.165:0/3410533199 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 0x558423ed32b0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.220+0000 7f10b97dc800 1 -- 172.21.15.165:0/3410533199 >> 172.21.15.165:0/3410533199 conn(0x558423ecd320 msgr2=0x558423fbf870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.220+0000 7f10b97dc800 1 -- 172.21.15.165:0/3410533199 shutdown_connections 2024-04-27T03:30:35.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.220+0000 7f10b97dc800 1 -- 172.21.15.165:0/3410533199 wait complete. 2024-04-27T03:30:35.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.220+0000 7f10b97dc800 1 Processor -- start 2024-04-27T03:30:35.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.220+0000 7f10b97dc800 1 -- start start 2024-04-27T03:30:35.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.221+0000 7f10b97dc800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 0x558423fcdce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.221+0000 7f10b97dc800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558423edc0c0 con 0x558423ed3970 2024-04-27T03:30:35.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.221+0000 7f10b72db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 0x558423fcdce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.221+0000 7f10b72db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 0x558423fcdce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58726/0 (socket says 172.21.15.165:58726) 2024-04-27T03:30:35.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.221+0000 7f10b72db640 1 -- 172.21.15.165:0/902880311 learned_addr learned my addr 172.21.15.165:0/902880311 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.221+0000 7f10b72db640 1 -- 172.21.15.165:0/902880311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558423fce220 con 0x558423ed3970 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.221+0000 7f10b72db640 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 0x558423fcdce0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f10a800b210 tx=0x7f10a8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.221+0000 7f0fa3e07640 1 -- 172.21.15.165:0/902880311 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10a800f950 con 0x558423ed3970 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.222+0000 7f0fa3e07640 1 -- 172.21.15.165:0/902880311 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a800faf0 con 0x558423ed3970 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.222+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558423fce4b0 con 0x558423ed3970 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.222+0000 7f0fa3e07640 1 -- 172.21.15.165:0/902880311 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10a800fdd0 con 0x558423ed3970 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.222+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558423fd1160 con 0x558423ed3970 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.222+0000 7f0fa3e07640 1 -- 172.21.15.165:0/902880311 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f10a8017040 con 0x558423ed3970 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.222+0000 7f0fa3e07640 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f9c0368b0 0x7f0f9c038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.222+0000 7f0fa3e07640 1 -- 172.21.15.165:0/902880311 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f10a8015020 con 0x558423ed3970 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.223+0000 7f10aeada640 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f9c0368b0 0x7f0f9c038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.223+0000 7f10b97dc800 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558423ed7480 0x558423fd4f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.223+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558423fd5460 con 0x558423ed7480 2024-04-27T03:30:35.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.223+0000 7f10b7adc640 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558423ed7480 0x558423fd4f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.223+0000 7f10aeada640 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f9c0368b0 0x7f0f9c038d70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f10a0054000 tx=0x7f10a004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.223+0000 7f10b7adc640 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558423ed7480 0x558423fd4f20 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.223+0000 7f10b7adc640 1 -- 172.21.15.165:0/902880311 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f10b000a4a0 con 0x558423ed7480 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.223+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558423fd24f0 con 0x558423ed7480 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.224+0000 7f10b7adc640 1 -- 172.21.15.165:0/902880311 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f10b000a4a0 con 0x558423ed7480 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.224+0000 7f10b97dc800 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558423fd7770 0x558423fd9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.224+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558423fda200 con 0x558423fd7770 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.224+0000 7f10b72db640 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558423fd7770 0x558423fd9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.224+0000 7f10b72db640 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558423fd7770 0x558423fd9b60 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.224+0000 7f10b72db640 1 -- 172.21.15.165:0/902880311 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f10a802d070 con 0x558423fd7770 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.224+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558423fd6720 con 0x558423ed7480 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.225+0000 7f10b7adc640 1 -- 172.21.15.165:0/902880311 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f10b000a4a0 con 0x558423ed7480 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.225+0000 7f10b97dc800 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558423fdb6b0 0x558423fddb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.225+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558423fde260 con 0x558423fdb6b0 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.225+0000 7f10aeada640 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558423fdb6b0 0x558423fddb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.225+0000 7f10aeada640 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558423fdb6b0 0x558423fddb00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.226+0000 7f10aeada640 1 -- 172.21.15.165:0/902880311 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f10a0052940 con 0x558423fdb6b0 2024-04-27T03:30:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.226+0000 7f10b97dc800 1 Processor -- start 2024-04-27T03:30:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.226+0000 7f10b97dc800 1 -- start start 2024-04-27T03:30:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.226+0000 7f10b97dc800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423fec6c0 0x558423fecae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.226+0000 7f10b97dc800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558423dc46b0 con 0x558423fec6c0 2024-04-27T03:30:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.226+0000 7f10b72db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423fec6c0 0x558423fecae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.226+0000 7f10b72db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423fec6c0 0x558423fecae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58740/0 (socket says 172.21.15.165:58740) 2024-04-27T03:30:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.226+0000 7f10b72db640 1 -- 172.21.15.165:0/1162006595 learned_addr learned my addr 172.21.15.165:0/1162006595 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.227+0000 7f10b72db640 1 -- 172.21.15.165:0/1162006595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558423fed020 con 0x558423fec6c0 2024-04-27T03:30:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.227+0000 7f10b72db640 1 --2- 172.21.15.165:0/1162006595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423fec6c0 0x558423fecae0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f10a800b710 tx=0x7f10a8041d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.227+0000 7f0fa1602640 1 -- 172.21.15.165:0/1162006595 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10a80195a0 con 0x558423fec6c0 2024-04-27T03:30:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.227+0000 7f0fa1602640 1 -- 172.21.15.165:0/1162006595 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a8019740 con 0x558423fec6c0 2024-04-27T03:30:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.227+0000 7f10b97dc800 1 -- 172.21.15.165:0/1162006595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558423feda90 con 0x558423fec6c0 2024-04-27T03:30:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.227+0000 7f0fa1602640 1 -- 172.21.15.165:0/1162006595 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10a8044af0 con 0x558423fec6c0 2024-04-27T03:30:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.227+0000 7f10b97dc800 1 -- 172.21.15.165:0/1162006595 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558423fedf90 con 0x558423fec6c0 2024-04-27T03:30:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.228+0000 7f0fa1602640 1 -- 172.21.15.165:0/1162006595 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f10a8044c90 con 0x558423fec6c0 2024-04-27T03:30:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.228+0000 7f0fa1602640 1 --2- 172.21.15.165:0/1162006595 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f90036900 0x7f0f90038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.228+0000 7f0fa1602640 1 -- 172.21.15.165:0/1162006595 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f10a803bae0 con 0x558423fec6c0 2024-04-27T03:30:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.228+0000 7f10aeada640 1 --2- 172.21.15.165:0/1162006595 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f90036900 0x7f0f90038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.228+0000 7f10b97dc800 1 Processor -- start 2024-04-27T03:30:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.228+0000 7f10b97dc800 1 -- start start 2024-04-27T03:30:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.228+0000 7f10aeada640 1 --2- 172.21.15.165:0/1162006595 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f90036900 0x7f0f90038dc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x558423fd2410 tx=0x7f10a0005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.228+0000 7f10b97dc800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 0x558423ff9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.228+0000 7f10b97dc800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558423ed1bf0 con 0x558423ff9980 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.229+0000 7f10b72db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 0x558423ff9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.229+0000 7f10b72db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 0x558423ff9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58746/0 (socket says 172.21.15.165:58746) 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.229+0000 7f10b72db640 1 -- 172.21.15.165:0/2210306096 learned_addr learned my addr 172.21.15.165:0/2210306096 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.229+0000 7f10b72db640 1 -- 172.21.15.165:0/2210306096 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558423ffa370 con 0x558423ff9980 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.229+0000 7f10b72db640 1 --2- 172.21.15.165:0/2210306096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 0x558423ff9da0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f10a80187e0 tx=0x7f10a8030f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e217123b2b871c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.229+0000 7f0f9affd640 1 -- 172.21.15.165:0/2210306096 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10a805c070 con 0x558423ff9980 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.229+0000 7f0f9affd640 1 -- 172.21.15.165:0/2210306096 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a803f740 con 0x558423ff9980 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.229+0000 7f0f9affd640 1 -- 172.21.15.165:0/2210306096 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10a8008df0 con 0x558423ff9980 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b97dc800 1 -- 172.21.15.165:0/2210306096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 msgr2=0x558423ff9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b97dc800 1 --2- 172.21.15.165:0/2210306096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 0x558423ff9da0 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f10a80187e0 tx=0x7f10a8030f30 comp rx=0 tx=0).stop 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b97dc800 1 -- 172.21.15.165:0/2210306096 shutdown_connections 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b97dc800 1 --2- 172.21.15.165:0/2210306096 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 0x558423ff9da0 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b97dc800 1 -- 172.21.15.165:0/2210306096 >> 172.21.15.165:0/2210306096 conn(0x558423ff5ce0 msgr2=0x558423ff3d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b97dc800 1 -- 172.21.15.165:0/2210306096 shutdown_connections 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b97dc800 1 -- 172.21.15.165:0/2210306096 wait complete. 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b97dc800 1 Processor -- start 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b97dc800 1 -- start start 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b97dc800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 0x558424000c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b97dc800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558423ffa8a0 con 0x558423ff9980 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b72db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 0x558424000c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b72db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 0x558424000c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58750/0 (socket says 172.21.15.165:58750) 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.230+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 learned_addr learned my addr 172.21.15.165:0/1309295725 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.231+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558424001140 con 0x558423ff9980 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.231+0000 7f10b72db640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 0x558424000c00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f10a8008d70 tx=0x7f10a8003f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.231+0000 7f0f997fa640 1 -- 172.21.15.165:0/1309295725 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10a8008df0 con 0x558423ff9980 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.231+0000 7f0f997fa640 1 -- 172.21.15.165:0/1309295725 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f10a806c030 con 0x558423ff9980 2024-04-27T03:30:35.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.231+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558424001140 con 0x558423ff9980 2024-04-27T03:30:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.231+0000 7f0f997fa640 1 -- 172.21.15.165:0/1309295725 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f10a8012220 con 0x558423ff9980 2024-04-27T03:30:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.231+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558424001140 con 0x558423ff9980 2024-04-27T03:30:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.232+0000 7f0f997fa640 1 -- 172.21.15.165:0/1309295725 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f10a805f070 con 0x558423ff9980 2024-04-27T03:30:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.232+0000 7f0f997fa640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f80036900 0x7f0f80038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.232+0000 7f0f997fa640 1 -- 172.21.15.165:0/1309295725 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f10a8075170 con 0x558423ff9980 2024-04-27T03:30:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.232+0000 7f10aeada640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f80036900 0x7f0f80038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.232+0000 7f10aeada640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f80036900 0x7f0f80038dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f10a0052fd0 tx=0x7f10a004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.232+0000 7f10b97dc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.232+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55842400c170 0x55842400e630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.232+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55842400ed80 con 0x55842400c170 2024-04-27T03:30:35.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.232+0000 7f10b7adc640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55842400c170 0x55842400e630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b7adc640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55842400c170 0x55842400e630 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f10b000a4a0 con 0x55842400c170 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b97dc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b97dc800 20 realm 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b97dc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55842400f190 con 0x55842400c170 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f10b000a4a0 con 0x55842400c170 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b97dc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b97dc800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b97dc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55842400fa60 con 0x55842400c170 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f10b000a4a0 con 0x55842400c170 2024-04-27T03:30:35.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b97dc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b97dc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.233+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55842400fe00 con 0x55842400c170 2024-04-27T03:30:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.234+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f10b000a4a0 con 0x55842400c170 2024-04-27T03:30:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.234+0000 7f10b97dc800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.234+0000 7f10b97dc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.234+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558424010b60 0x558424012f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.234+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558424013560 con 0x558424010b60 2024-04-27T03:30:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.234+0000 7f10b72db640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558424010b60 0x558424012f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.234+0000 7f10b72db640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558424010b60 0x558424012f50 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.234+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.234+0000 7f10b97dc800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:35.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.234+0000 7f10b97dc800 20 searching for the correct realm 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.234+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558424014e90 con 0x558424010b60 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.235+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.235+0000 7f10b72db640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f10a8061070 0x7f10a803b050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.235+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558424014e90 con 0x7f10a8061070 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.235+0000 7f10aeada640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f10a8061070 0x7f10a803b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.235+0000 7f10aeada640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f10a8061070 0x7f10a803b050 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.236+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.236+0000 7f10aeada640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f10a0076040 0x7f10a0083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.236+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558424014e90 con 0x7f10a0076040 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.236+0000 7f10b7adc640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f10a0076040 0x7f10a0083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.236+0000 7f10b7adc640 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f10a0076040 0x7f10a0083410 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.236+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f10b000a4a0 con 0x7f10a0076040 2024-04-27T03:30:35.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.236+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55842400fe00 con 0x55842400c170 2024-04-27T03:30:35.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.237+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b000a4a0 con 0x55842400c170 2024-04-27T03:30:35.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.237+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55842400fe00 con 0x55842400c170 2024-04-27T03:30:35.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.237+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b000a4a0 con 0x55842400c170 2024-04-27T03:30:35.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.237+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55842400fe00 con 0x7f10a0076040 2024-04-27T03:30:35.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.237+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b000a4a0 con 0x7f10a0076040 2024-04-27T03:30:35.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.237+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55842400fe00 con 0x7f10a0076040 2024-04-27T03:30:35.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.237+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b000a4a0 con 0x7f10a0076040 2024-04-27T03:30:35.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.237+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55842400fe00 con 0x7f10a8061070 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.238+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.238+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55842400fe00 con 0x558424010b60 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.238+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.238+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55842400fe00 con 0x55842400c170 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.238+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b000a4a0 con 0x55842400c170 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.238+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55842400fe00 con 0x7f10a0076040 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.239+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b000a4a0 con 0x7f10a0076040 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.239+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55842400fe00 con 0x55842400c170 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.239+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b000a4a0 con 0x55842400c170 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.239+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55842400fe00 con 0x7f10a8061070 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.239+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.239+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x558424010b60 2024-04-27T03:30:35.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.239+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.239+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x558424010b60 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.240+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.240+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x7f10a0076040 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.240+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b000a4a0 con 0x7f10a0076040 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.240+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x558424010b60 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.240+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.240+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x558424010b60 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.241+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.241+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x55842400c170 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.241+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b000a4a0 con 0x55842400c170 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.241+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x55842400c170 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.241+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f10b000a4a0 con 0x55842400c170 2024-04-27T03:30:35.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.241+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x7f10a0076040 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.241+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b0033040 con 0x7f10a0076040 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.241+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x558424010b60 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.242+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.242+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x7f10a8061070 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.242+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.242+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x55842400c170 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.242+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.242+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x7f10a0076040 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.242+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.242+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x558424010b60 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.243+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.243+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x7f10a0076040 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.243+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.243+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x7f10a8061070 2024-04-27T03:30:35.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.243+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.243+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x558424010b60 2024-04-27T03:30:35.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.244+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.244+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x7f10a0076040 2024-04-27T03:30:35.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.244+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.244+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x55842400c170 2024-04-27T03:30:35.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.244+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.244+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f10a0090060 con 0x7f10a8061070 2024-04-27T03:30:35.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:35.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:35.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:35.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:35.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558424013e20 con 0x55842400c170 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558424014210 con 0x55842400c170 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.245+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558424014210 con 0x7f10a0076040 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:35.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 20 zone default found 2024-04-27T03:30:35.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 4 Realm: () 2024-04-27T03:30:35.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:35.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:35.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:35.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 20 zonegroup default 2024-04-27T03:30:35.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558424013a80 con 0x7f10a0076040 2024-04-27T03:30:35.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558424013a80 con 0x558424010b60 2024-04-27T03:30:35.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558424016de0 con 0x558424010b60 2024-04-27T03:30:35.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.246+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.247+0000 7f10b97dc800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.247+0000 7f10b97dc800 20 started sync module instance, tier type = 2024-04-27T03:30:35.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.247+0000 7f10b97dc800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:35.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.247+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424016190 con 0x7f10a8061070 2024-04-27T03:30:35.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.248+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.248+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94026028115376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424015df0 con 0x7f10a8061070 2024-04-27T03:30:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.248+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424017d10 con 0x55842400c170 2024-04-27T03:30:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.249+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94026028115376] v25'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.249+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.249+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94026028120160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424018130 con 0x55842400c170 2024-04-27T03:30:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.249+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424018af0 con 0x558424010b60 2024-04-27T03:30:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.250+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.250+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94026028123712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424019340 con 0x558424010b60 2024-04-27T03:30:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.250+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424019e20 con 0x7f10a8061070 2024-04-27T03:30:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.250+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94026028120160] v25'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.251+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94026028123712] v25'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.251+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.251+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94026028128560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842401a210 con 0x7f10a8061070 2024-04-27T03:30:35.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.251+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842401aaa0 con 0x55842400c170 2024-04-27T03:30:35.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.252+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94026028128560] v25'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.252+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.252+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94026028121216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842401ae90 con 0x55842400c170 2024-04-27T03:30:35.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.252+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842401b8f0 con 0x7f10a0076040 2024-04-27T03:30:35.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.253+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94026028121216] v25'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.254+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.254+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94026028131760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842401b8f0 con 0x7f10a0076040 2024-04-27T03:30:35.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.254+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842401c300 con 0x7f10a0076040 2024-04-27T03:30:35.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.255+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.255+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94026028138064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842401c720 con 0x7f10a0076040 2024-04-27T03:30:35.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.255+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842401d240 con 0x7f10a0076040 2024-04-27T03:30:35.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.255+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94026028131760] v25'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.255+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94026028138064] v25'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.257+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'117 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.257+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94026028141840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842401c270 con 0x7f10a0076040 2024-04-27T03:30:35.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.257+0000 7f10b97dc800 20 add_watcher() i=0 2024-04-27T03:30:35.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.257+0000 7f10b97dc800 20 add_watcher() i=1 2024-04-27T03:30:35.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.257+0000 7f10b97dc800 20 add_watcher() i=2 2024-04-27T03:30:35.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.257+0000 7f10b97dc800 20 add_watcher() i=3 2024-04-27T03:30:35.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.257+0000 7f10b97dc800 20 add_watcher() i=4 2024-04-27T03:30:35.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.257+0000 7f10b97dc800 20 add_watcher() i=5 2024-04-27T03:30:35.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.257+0000 7f10b97dc800 20 add_watcher() i=6 2024-04-27T03:30:35.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.258+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94026028141840] v25'118 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.258+0000 7f10b97dc800 20 add_watcher() i=7 2024-04-27T03:30:35.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.258+0000 7f10b97dc800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:35.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.258+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55842401c6f0 con 0x55842400c170 2024-04-27T03:30:35.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.258+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.258+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94026028135376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842401c6f0 con 0x55842400c170 2024-04-27T03:30:35.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.259+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94026028135376] v25'118 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.260+0000 7f0f6dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:35.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.260+0000 7f10b97dc800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:35.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.260+0000 7f10b97dc800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:35.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.260+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424028aa0 con 0x55842400c170 2024-04-27T03:30:35.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.261+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'44 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.261+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424029170 con 0x7f10a0076040 2024-04-27T03:30:35.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.262+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'20 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.262+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424029590 con 0x558424010b60 2024-04-27T03:30:35.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.263+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'21 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.263+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424029590 con 0x55842400c170 2024-04-27T03:30:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.264+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'119 uv15 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.264+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424029b00 con 0x7f10a8061070 2024-04-27T03:30:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.265+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'38 uv17 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.265+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402a070 con 0x558424010b60 2024-04-27T03:30:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.266+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'34 uv16 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.266+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402a070 con 0x55842400c170 2024-04-27T03:30:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.267+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'22 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.267+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402a100 con 0x7f10a0076040 2024-04-27T03:30:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.268+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'27 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.268+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402a190 con 0x7f10a8061070 2024-04-27T03:30:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.269+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'14 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.269+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402a700 con 0x558424010b60 2024-04-27T03:30:35.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.270+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'20 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.270+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402a700 con 0x7f10a8061070 2024-04-27T03:30:35.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.271+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'39 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.271+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402abb0 con 0x7f10a0076040 2024-04-27T03:30:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.272+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'16 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.273+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402abb0 con 0x7f10a8061070 2024-04-27T03:30:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.274+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'20 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.274+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x7f10a0076040 2024-04-27T03:30:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.275+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'39 uv22 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.275+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x55842400c170 2024-04-27T03:30:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.276+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'120 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.276+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x558424010b60 2024-04-27T03:30:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.277+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'33 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.277+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x7f10a8061070 2024-04-27T03:30:35.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.278+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'40 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.278+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x558424010b60 2024-04-27T03:30:35.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.279+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'26 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.279+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x55842400c170 2024-04-27T03:30:35.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.281+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'22 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.281+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x55842400c170 2024-04-27T03:30:35.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.282+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'121 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.282+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x558424010b60 2024-04-27T03:30:35.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.283+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'34 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.283+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x7f10a8061070 2024-04-27T03:30:35.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.284+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'30 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.284+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x558424010b60 2024-04-27T03:30:35.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.285+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'22 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.285+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x558424010b60 2024-04-27T03:30:35.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.286+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'33 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.286+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x558424010b60 2024-04-27T03:30:35.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.287+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'16 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.287+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac40 con 0x558424010b60 2024-04-27T03:30:35.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.288+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'35 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.288+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac60 con 0x7f10a0076040 2024-04-27T03:30:35.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.289+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'22 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.289+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac60 con 0x558424010b60 2024-04-27T03:30:35.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.290+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'26 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.290+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac60 con 0x7f10a0076040 2024-04-27T03:30:35.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.291+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'28 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.291+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac60 con 0x558424010b60 2024-04-27T03:30:35.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.292+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'35 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.292+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac60 con 0x7f10a0076040 2024-04-27T03:30:35.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.293+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'40 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.293+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55842402ac60 con 0x7f10a8061070 2024-04-27T03:30:35.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.294+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.294+0000 7f0f6cfe1640 20 reqs_thread_entry: start 2024-04-27T03:30:35.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.294+0000 7f10b97dc800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:35.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.294+0000 7f10b97dc800 20 Filter name: none 2024-04-27T03:30:35.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.294+0000 7f0ee67fc640 20 reqs_thread_entry: start 2024-04-27T03:30:35.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.294+0000 7f10b97dc800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T03:30:35.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.294+0000 7f10b97dc800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.294+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558424055cc0 con 0x7f10a0076040 2024-04-27T03:30:35.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.295+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(94 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.295+0000 7f10b97dc800 20 rados_obj.operate() r=0 bl.length=380 2024-04-27T03:30:35.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.295+0000 7f10b97dc800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T03:30:35.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.295+0000 7f10b97dc800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:30:35.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.295+0000 7f10b97dc800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:35.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.295+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5584240585f0 con 0x7f10a0076040 2024-04-27T03:30:35.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.295+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(95 fud.buckets [call out=183b] v0'0 uv1 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.302 INFO:teuthology.orchestra.run.smithi165.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-04-27T03:30:35.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.296+0000 7f0f6dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:35.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.296+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94026028135376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424009790 con 0x55842400c170 2024-04-27T03:30:35.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.297+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 94026028135376] v25'122 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.297+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94026028115376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5584240092e0 con 0x7f10a8061070 2024-04-27T03:30:35.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.298+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 94026028115376] v25'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.298+0000 7f10b97dc800 20 remove_watcher() i=0 2024-04-27T03:30:35.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.298+0000 7f10b97dc800 2 removed watcher, disabling cache 2024-04-27T03:30:35.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.298+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94026028120160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5584240165b0 con 0x55842400c170 2024-04-27T03:30:35.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.299+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 94026028120160] v25'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.299+0000 7f10b97dc800 20 remove_watcher() i=1 2024-04-27T03:30:35.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.299+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94026028123712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5584240165b0 con 0x558424010b60 2024-04-27T03:30:35.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.300+0000 7f10b72db640 1 -- 172.21.15.165:0/1309295725 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 94026028123712] v25'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10a805e030 con 0x558424010b60 2024-04-27T03:30:35.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.300+0000 7f10b97dc800 20 remove_watcher() i=2 2024-04-27T03:30:35.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.300+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94026028128560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424017860 con 0x7f10a8061070 2024-04-27T03:30:35.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.301+0000 7f10aeada640 1 -- 172.21.15.165:0/1309295725 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 94026028128560] v25'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10a0081070 con 0x7f10a8061070 2024-04-27T03:30:35.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.301+0000 7f10b97dc800 20 remove_watcher() i=3 2024-04-27T03:30:35.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.301+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94026028121216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424017860 con 0x55842400c170 2024-04-27T03:30:35.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.302+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 94026028121216] v25'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x55842400c170 2024-04-27T03:30:35.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.302+0000 7f10b97dc800 20 remove_watcher() i=4 2024-04-27T03:30:35.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.302+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94026028131760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424019930 con 0x7f10a0076040 2024-04-27T03:30:35.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.303+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 94026028131760] v25'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.303+0000 7f10b97dc800 20 remove_watcher() i=5 2024-04-27T03:30:35.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.303+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94026028138064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424019930 con 0x7f10a0076040 2024-04-27T03:30:35.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.304+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 94026028138064] v25'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.304+0000 7f10b97dc800 20 remove_watcher() i=6 2024-04-27T03:30:35.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.304+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94026028141840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558424019930 con 0x7f10a0076040 2024-04-27T03:30:35.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.305+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 94026028141840] v25'119 uv117 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f10b0031040 con 0x7f10a0076040 2024-04-27T03:30:35.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.305+0000 7f10b97dc800 20 remove_watcher() i=7 2024-04-27T03:30:35.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.305+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f10a0076040 msgr2=0x7f10a0083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.305+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f10a0076040 0x7f10a0083410 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.305+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55842400c170 msgr2=0x55842400e630 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.305+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55842400c170 0x55842400e630 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.305+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f10a8061070 msgr2=0x7f10a803b050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.305+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f10a8061070 0x7f10a803b050 crc :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.305+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558424010b60 msgr2=0x558424012f50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.305+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558424010b60 0x558424012f50 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f80036900 msgr2=0x7f0f80038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f80036900 0x7f0f80038dc0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f10a0052fd0 tx=0x7f10a004f000 comp rx=0 tx=0).stop 2024-04-27T03:30:35.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 msgr2=0x558424000c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 0x558424000c00 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f10a8008d70 tx=0x7f10a8003f00 comp rx=0 tx=0).stop 2024-04-27T03:30:35.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b7adc640 1 -- 172.21.15.165:0/1309295725 reap_dead start 2024-04-27T03:30:35.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 shutdown_connections 2024-04-27T03:30:35.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1309295725 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ff9980 0x558424000c00 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 >> 172.21.15.165:0/1309295725 conn(0x558423ff5ce0 msgr2=0x558423ff3340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 shutdown_connections 2024-04-27T03:30:35.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/1309295725 wait complete. 2024-04-27T03:30:35.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/1162006595 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f90036900 msgr2=0x7f0f90038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1162006595 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f90036900 0x7f0f90038dc0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x558423fd2410 tx=0x7f10a0005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/1162006595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423fec6c0 msgr2=0x558423fecae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1162006595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423fec6c0 0x558423fecae0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f10a800b710 tx=0x7f10a8041d50 comp rx=0 tx=0).stop 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/1162006595 shutdown_connections 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1162006595 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f90036900 0x7f0f90038dc0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 --2- 172.21.15.165:0/1162006595 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423fec6c0 0x558423fecae0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/1162006595 >> 172.21.15.165:0/1162006595 conn(0x558423fe0760 msgr2=0x558423fe0b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/1162006595 shutdown_connections 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/1162006595 wait complete. 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558423fdb6b0 msgr2=0x558423fddb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558423fdb6b0 0x558423fddb00 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558423ed7480 msgr2=0x558423fd4f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558423ed7480 0x558423fd4f20 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558423fd7770 msgr2=0x558423fd9b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.306+0000 7f10b97dc800 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558423fd7770 0x558423fd9b60 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.307+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f9c0368b0 msgr2=0x7f0f9c038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.307+0000 7f10b97dc800 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0f9c0368b0 0x7f0f9c038d70 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f10a0054000 tx=0x7f10a004e000 comp rx=0 tx=0).stop 2024-04-27T03:30:35.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.307+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 msgr2=0x558423fcdce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.307+0000 7f10b97dc800 1 --2- 172.21.15.165:0/902880311 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558423ed3970 0x558423fcdce0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f10a800b210 tx=0x7f10a8000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.307+0000 7f10b7adc640 1 -- 172.21.15.165:0/902880311 reap_dead start 2024-04-27T03:30:35.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.307+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 shutdown_connections 2024-04-27T03:30:35.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.307+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 >> 172.21.15.165:0/902880311 conn(0x558423ecd320 msgr2=0x558423ed2150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.307+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 shutdown_connections 2024-04-27T03:30:35.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.307+0000 7f10b97dc800 1 -- 172.21.15.165:0/902880311 wait complete. 2024-04-27T03:30:35.322 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-27T03:30:35.323 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-04-27T03:30:35.323 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'unlink', '--uid', 'fud', '--bucket', 'myfoo'] 2024-04-27T03:30:35.323 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket unlink --uid fud --bucket myfoo 2024-04-27T03:30:35.410 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:35.411 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:35.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.406+0000 7fdc54146800 1 Processor -- start 2024-04-27T03:30:35.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.406+0000 7fdc54146800 1 -- start start 2024-04-27T03:30:35.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.406+0000 7fdc54146800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x559f750144a0 0x559f75016890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.406+0000 7fdc54146800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559f74eced40 con 0x559f75010990 2024-04-27T03:30:35.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.406+0000 7fdc54146800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x559f74ea14d0 con 0x559f750144a0 2024-04-27T03:30:35.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.406+0000 7fdc5144b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x559f750144a0 0x559f75016890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.406+0000 7fdc5144b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x559f750144a0 0x559f75016890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58764/0 (socket says 172.21.15.165:58764) 2024-04-27T03:30:35.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.406+0000 7fdc51c4c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x559f75010990 0x559f75012d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35338/0 (socket says 172.21.15.165:35338) 2024-04-27T03:30:35.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.406+0000 7fdc5144b640 1 -- 172.21.15.165:0/2130109861 learned_addr learned my addr 172.21.15.165:0/2130109861 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.407+0000 7fdc5144b640 1 -- 172.21.15.165:0/2130109861 >> v1:172.21.15.89:6789/0 conn(0x559f75010990 legacy=0x559f75012d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.407+0000 7fdc50c4a640 1 -- 172.21.15.165:0/2130109861 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1617753903 0 0) 0x559f74eced40 con 0x559f75010990 2024-04-27T03:30:35.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.407+0000 7fdc5144b640 1 -- 172.21.15.165:0/2130109861 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f74ecd590 con 0x559f750144a0 2024-04-27T03:30:35.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.408+0000 7fdc5144b640 1 --2- 172.21.15.165:0/2130109861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f750144a0 0x559f75016890 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800a520 tx=0x7fdc4801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e2a703defb48c84 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.409+0000 7fdc50c4a640 1 -- 172.21.15.165:0/2130109861 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc48057070 con 0x559f750144a0 2024-04-27T03:30:35.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.409+0000 7fdc50c4a640 1 -- 172.21.15.165:0/2130109861 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc48052570 con 0x559f750144a0 2024-04-27T03:30:35.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.409+0000 7fdc50c4a640 1 -- 172.21.15.165:0/2130109861 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc4804ae60 con 0x559f750144a0 2024-04-27T03:30:35.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.409+0000 7fdc54146800 1 -- 172.21.15.165:0/2130109861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f750144a0 msgr2=0x559f75016890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.409+0000 7fdc54146800 1 --2- 172.21.15.165:0/2130109861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f750144a0 0x559f75016890 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fdc4800a520 tx=0x7fdc4801d010 comp rx=0 tx=0).stop 2024-04-27T03:30:35.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.409+0000 7fdc54146800 1 -- 172.21.15.165:0/2130109861 shutdown_connections 2024-04-27T03:30:35.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.409+0000 7fdc54146800 1 --2- 172.21.15.165:0/2130109861 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f750144a0 0x559f75016890 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.409+0000 7fdc54146800 1 -- 172.21.15.165:0/2130109861 >> 172.21.15.165:0/2130109861 conn(0x559f7500a340 msgr2=0x559f7500c760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.409+0000 7fdc54146800 1 -- 172.21.15.165:0/2130109861 shutdown_connections 2024-04-27T03:30:35.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.409+0000 7fdc54146800 1 -- 172.21.15.165:0/2130109861 wait complete. 2024-04-27T03:30:35.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.421+0000 7fdc54146800 1 Processor -- start 2024-04-27T03:30:35.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.421+0000 7fdc54146800 1 -- start start 2024-04-27T03:30:35.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.421+0000 7fdc54146800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 0x559f75100ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.421+0000 7fdc54146800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559f74f5b3b0 con 0x559f75010990 2024-04-27T03:30:35.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.422+0000 7fdc51c4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 0x559f75100ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.422+0000 7fdc51c4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 0x559f75100ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58774/0 (socket says 172.21.15.165:58774) 2024-04-27T03:30:35.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.422+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2835777507 learned_addr learned my addr 172.21.15.165:0/2835777507 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.422+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2835777507 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f75101470 con 0x559f75010990 2024-04-27T03:30:35.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.422+0000 7fdc51c4c640 1 --2- 172.21.15.165:0/2835777507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 0x559f75100ea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400b210 tx=0x7fdc4400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d42042be537e30a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.422+0000 7fdb3fe07640 1 -- 172.21.15.165:0/2835777507 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc44008070 con 0x559f75010990 2024-04-27T03:30:35.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.422+0000 7fdb3fe07640 1 -- 172.21.15.165:0/2835777507 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc4400d500 con 0x559f75010990 2024-04-27T03:30:35.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.422+0000 7fdb3fe07640 1 -- 172.21.15.165:0/2835777507 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc4400d840 con 0x559f75010990 2024-04-27T03:30:35.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.423+0000 7fdc54146800 1 -- 172.21.15.165:0/2835777507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 msgr2=0x559f75100ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.423+0000 7fdc54146800 1 --2- 172.21.15.165:0/2835777507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 0x559f75100ea0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400b210 tx=0x7fdc4400b6d0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.423+0000 7fdc54146800 1 -- 172.21.15.165:0/2835777507 shutdown_connections 2024-04-27T03:30:35.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.423+0000 7fdc54146800 1 --2- 172.21.15.165:0/2835777507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 0x559f75100ea0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.423+0000 7fdc54146800 1 -- 172.21.15.165:0/2835777507 >> 172.21.15.165:0/2835777507 conn(0x559f7500a340 msgr2=0x559f750fc7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.423+0000 7fdc54146800 1 -- 172.21.15.165:0/2835777507 shutdown_connections 2024-04-27T03:30:35.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.423+0000 7fdc54146800 1 -- 172.21.15.165:0/2835777507 wait complete. 2024-04-27T03:30:35.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.423+0000 7fdc54146800 1 Processor -- start 2024-04-27T03:30:35.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.423+0000 7fdc54146800 1 -- start start 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.424+0000 7fdc54146800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 0x559f75010260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.424+0000 7fdc54146800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559f750190e0 con 0x559f75010990 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.424+0000 7fdc51c4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 0x559f75010260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.424+0000 7fdc51c4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 0x559f75010260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58776/0 (socket says 172.21.15.165:58776) 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.424+0000 7fdc51c4c640 1 -- 172.21.15.165:0/1108280362 learned_addr learned my addr 172.21.15.165:0/1108280362 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.424+0000 7fdc51c4c640 1 -- 172.21.15.165:0/1108280362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f750107a0 con 0x559f75010990 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.424+0000 7fdc51c4c640 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 0x559f75010260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400b210 tx=0x7fdc44000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.424+0000 7fdb3e604640 1 -- 172.21.15.165:0/1108280362 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc4400f950 con 0x559f75010990 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.424+0000 7fdb3e604640 1 -- 172.21.15.165:0/1108280362 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc4400faf0 con 0x559f75010990 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.424+0000 7fdb3e604640 1 -- 172.21.15.165:0/1108280362 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc4400fdd0 con 0x559f75010990 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.424+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f7500e9a0 con 0x559f75010990 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.424+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7500eec0 con 0x559f75010990 2024-04-27T03:30:35.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.425+0000 7fdb3e604640 1 -- 172.21.15.165:0/1108280362 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fdc44017040 con 0x559f75010990 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.425+0000 7fdb3e604640 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb38036900 0x7fdb38038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.425+0000 7fdb3e604640 1 -- 172.21.15.165:0/1108280362 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fdc44015020 con 0x559f75010990 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.425+0000 7fdc5144b640 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb38036900 0x7fdb38038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.425+0000 7fdc54146800 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f750144a0 0x559f75111ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.425+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f75112420 con 0x559f750144a0 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.426+0000 7fdc5244d640 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f750144a0 0x559f75111ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.426+0000 7fdc5144b640 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb38036900 0x7fdb38038dc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdc48054000 tx=0x7fdc4804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.426+0000 7fdc5244d640 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f750144a0 0x559f75111ee0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.426+0000 7fdc5244d640 1 -- 172.21.15.165:0/1108280362 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f750144a0 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.426+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f7500f3a0 con 0x559f750144a0 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.426+0000 7fdc5244d640 1 -- 172.21.15.165:0/1108280362 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f750144a0 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.426+0000 7fdc54146800 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f75114710 0x559f75116b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.427+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f751171f0 con 0x559f75114710 2024-04-27T03:30:35.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.427+0000 7fdc51c4c640 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f75114710 0x559f75116b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.427+0000 7fdc51c4c640 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f75114710 0x559f75116b00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.427+0000 7fdc51c4c640 1 -- 172.21.15.165:0/1108280362 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fdc4402d070 con 0x559f75114710 2024-04-27T03:30:35.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.427+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f751136c0 con 0x559f750144a0 2024-04-27T03:30:35.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.427+0000 7fdc5244d640 1 -- 172.21.15.165:0/1108280362 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f750144a0 2024-04-27T03:30:35.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.427+0000 7fdc54146800 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559f751186a0 0x559f7511aaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.427+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f7511b250 con 0x559f751186a0 2024-04-27T03:30:35.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.428+0000 7fdc5144b640 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559f751186a0 0x559f7511aaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.428+0000 7fdc5144b640 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559f751186a0 0x559f7511aaf0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.428+0000 7fdc5144b640 1 -- 172.21.15.165:0/1108280362 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fdc4805a040 con 0x559f751186a0 2024-04-27T03:30:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.429+0000 7fdc54146800 1 Processor -- start 2024-04-27T03:30:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.429+0000 7fdc54146800 1 -- start start 2024-04-27T03:30:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.429+0000 7fdc54146800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751296b0 0x559f75129ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.429+0000 7fdc54146800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559f74f016b0 con 0x559f751296b0 2024-04-27T03:30:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.429+0000 7fdc51c4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751296b0 0x559f75129ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.429+0000 7fdc51c4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751296b0 0x559f75129ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58778/0 (socket says 172.21.15.165:58778) 2024-04-27T03:30:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.429+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2696880275 learned_addr learned my addr 172.21.15.165:0/2696880275 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.430+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2696880275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7512a010 con 0x559f751296b0 2024-04-27T03:30:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.430+0000 7fdc51c4c640 1 --2- 172.21.15.165:0/2696880275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751296b0 0x559f75129ad0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdc4404b4b0 tx=0x7fdc4401aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.430+0000 7fdb377fe640 1 -- 172.21.15.165:0/2696880275 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc4401ade0 con 0x559f751296b0 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.430+0000 7fdb377fe640 1 -- 172.21.15.165:0/2696880275 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc44019470 con 0x559f751296b0 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.430+0000 7fdb377fe640 1 -- 172.21.15.165:0/2696880275 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc44019750 con 0x559f751296b0 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.430+0000 7fdc54146800 1 -- 172.21.15.165:0/2696880275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f7512aa80 con 0x559f751296b0 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.430+0000 7fdc54146800 1 -- 172.21.15.165:0/2696880275 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7512af80 con 0x559f751296b0 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.431+0000 7fdb377fe640 1 -- 172.21.15.165:0/2696880275 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fdc4403f1b0 con 0x559f751296b0 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.431+0000 7fdb377fe640 1 --2- 172.21.15.165:0/2696880275 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb2c036950 0x7fdb2c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.431+0000 7fdb377fe640 1 -- 172.21.15.165:0/2696880275 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fdc4403caa0 con 0x559f751296b0 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.431+0000 7fdc5144b640 1 --2- 172.21.15.165:0/2696880275 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb2c036950 0x7fdb2c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.432+0000 7fdc54146800 1 Processor -- start 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.432+0000 7fdc54146800 1 -- start start 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.432+0000 7fdc5144b640 1 --2- 172.21.15.165:0/2696880275 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb2c036950 0x7fdb2c038e10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x559f75112cc0 tx=0x7fdc48005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.432+0000 7fdc54146800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 0x559f75136df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.432+0000 7fdc54146800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559f751017f0 con 0x559f751369d0 2024-04-27T03:30:35.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.432+0000 7fdc51c4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 0x559f75136df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.432+0000 7fdc51c4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 0x559f75136df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58792/0 (socket says 172.21.15.165:58792) 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.432+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2442624947 learned_addr learned my addr 172.21.15.165:0/2442624947 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.432+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2442624947 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f751373c0 con 0x559f751369d0 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.432+0000 7fdc51c4c640 1 --2- 172.21.15.165:0/2442624947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 0x559f75136df0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdc44037000 tx=0x7fdc44026b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3c8290ac4f647c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdb357fa640 1 -- 172.21.15.165:0/2442624947 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc44030cf0 con 0x559f751369d0 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdb357fa640 1 -- 172.21.15.165:0/2442624947 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc44040900 con 0x559f751369d0 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdb357fa640 1 -- 172.21.15.165:0/2442624947 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc4404e070 con 0x559f751369d0 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdc54146800 1 -- 172.21.15.165:0/2442624947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 msgr2=0x559f75136df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdc54146800 1 --2- 172.21.15.165:0/2442624947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 0x559f75136df0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fdc44037000 tx=0x7fdc44026b00 comp rx=0 tx=0).stop 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdc54146800 1 -- 172.21.15.165:0/2442624947 shutdown_connections 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdc54146800 1 --2- 172.21.15.165:0/2442624947 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 0x559f75136df0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdc54146800 1 -- 172.21.15.165:0/2442624947 >> 172.21.15.165:0/2442624947 conn(0x559f75132d30 msgr2=0x559f75130d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdc54146800 1 -- 172.21.15.165:0/2442624947 shutdown_connections 2024-04-27T03:30:35.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdc54146800 1 -- 172.21.15.165:0/2442624947 wait complete. 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdc54146800 1 Processor -- start 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdc54146800 1 -- start start 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.433+0000 7fdc54146800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 0x559f7513dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.434+0000 7fdc54146800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559f75137a70 con 0x559f751369d0 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.434+0000 7fdc51c4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 0x559f7513dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.434+0000 7fdc51c4c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 0x559f7513dd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58808/0 (socket says 172.21.15.165:58808) 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.434+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 learned_addr learned my addr 172.21.15.165:0/2548626397 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.434+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f7513e2d0 con 0x559f751369d0 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.434+0000 7fdc51c4c640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 0x559f7513dd90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdc44026960 tx=0x7fdc44036f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.434+0000 7fdb237fe640 1 -- 172.21.15.165:0/2548626397 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc44026e00 con 0x559f751369d0 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.434+0000 7fdb237fe640 1 -- 172.21.15.165:0/2548626397 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fdc4403f4a0 con 0x559f751369d0 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.434+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f7513e2d0 con 0x559f751369d0 2024-04-27T03:30:35.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.434+0000 7fdb237fe640 1 -- 172.21.15.165:0/2548626397 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdc44032a80 con 0x559f751369d0 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.435+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f7513e2d0 con 0x559f751369d0 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.435+0000 7fdb237fe640 1 -- 172.21.15.165:0/2548626397 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fdc44060070 con 0x559f751369d0 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.435+0000 7fdb237fe640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb1c036950 0x7fdb1c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.435+0000 7fdb237fe640 1 -- 172.21.15.165:0/2548626397 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fdc4405e020 con 0x559f751369d0 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.436+0000 7fdc5144b640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb1c036950 0x7fdb1c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.436+0000 7fdc5144b640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb1c036950 0x7fdb1c038e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdc48055040 tx=0x7fdc4804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.436+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.436+0000 7fdc54146800 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f75149580 0x559f7514ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.436+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f7514c190 con 0x559f75149580 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.436+0000 7fdc5244d640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f75149580 0x559f7514ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.436+0000 7fdc5244d640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f75149580 0x559f7514ba40 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f75149580 2024-04-27T03:30:35.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 20 realm 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f7514c5a0 con 0x559f75149580 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f75149580 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f7514ce70 con 0x559f75149580 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f75149580 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f75149580 2024-04-27T03:30:35.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f75149580 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f7514df70 0x559f75150360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.437+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f75150970 con 0x559f7514df70 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.438+0000 7fdc51c4c640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f7514df70 0x559f75150360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.438+0000 7fdc51c4c640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f7514df70 0x559f75150360 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.438+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.438+0000 7fdc54146800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.438+0000 7fdc54146800 20 searching for the correct realm 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.438+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f751522a0 con 0x559f7514df70 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.439+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.439+0000 7fdc51c4c640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fdc44032c20 0x7fdc440713f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.439+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f751522a0 con 0x7fdc44032c20 2024-04-27T03:30:35.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.439+0000 7fdc5144b640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fdc44032c20 0x7fdc440713f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.439+0000 7fdc5144b640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fdc44032c20 0x7fdc440713f0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.439+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.439+0000 7fdc5144b640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fdc48082040 0x7fdc48083430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.439+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f751522a0 con 0x7fdc48082040 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.440+0000 7fdc5244d640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fdc48082040 0x7fdc48083430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.440+0000 7fdc5244d640 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fdc48082040 0x7fdc48083430 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.440+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x7fdc48082040 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.440+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f75149580 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.440+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f75149580 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.440+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f75149580 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.441+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f75149580 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.441+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc48082040 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.441+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x7fdc48082040 2024-04-27T03:30:35.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.441+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc48082040 2024-04-27T03:30:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.441+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x7fdc48082040 2024-04-27T03:30:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.441+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc44032c20 2024-04-27T03:30:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.441+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.441+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f7514df70 2024-04-27T03:30:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.442+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.442+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f75149580 2024-04-27T03:30:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.442+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f75149580 2024-04-27T03:30:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.442+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc48082040 2024-04-27T03:30:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.442+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x7fdc48082040 2024-04-27T03:30:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.442+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f75149580 2024-04-27T03:30:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.442+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f75149580 2024-04-27T03:30:35.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.442+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc44032c20 2024-04-27T03:30:35.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.443+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.443+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f7514df70 2024-04-27T03:30:35.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.443+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.443+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f7514df70 2024-04-27T03:30:35.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.443+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.443+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc48082040 2024-04-27T03:30:35.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.444+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x7fdc48082040 2024-04-27T03:30:35.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.444+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f7514df70 2024-04-27T03:30:35.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.444+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.444+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f7514df70 2024-04-27T03:30:35.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.444+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.444+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f75149580 2024-04-27T03:30:35.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.444+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f75149580 2024-04-27T03:30:35.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.444+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f75149580 2024-04-27T03:30:35.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.444+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fdc4c00a4a0 con 0x559f75149580 2024-04-27T03:30:35.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.444+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc48082040 2024-04-27T03:30:35.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.445+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c033040 con 0x7fdc48082040 2024-04-27T03:30:35.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.445+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f7514df70 2024-04-27T03:30:35.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.445+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.445+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc44032c20 2024-04-27T03:30:35.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.445+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.445+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f75149580 2024-04-27T03:30:35.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.445+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.445+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc48082040 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.446+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.446+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f7514df70 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.446+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.446+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc48082040 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.446+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.446+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc44032c20 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.447+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.447+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f7514df70 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.447+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.447+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc48082040 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.447+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.447+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x559f75149580 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f7514d210 con 0x7fdc44032c20 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc54146800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc54146800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc54146800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc54146800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc54146800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc54146800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f75151230 con 0x559f75149580 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc54146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f75151620 con 0x559f75149580 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.448+0000 7fdc54146800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f75151620 con 0x7fdc48082040 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 20 zone default found 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 4 Realm: () 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 20 zonegroup default 2024-04-27T03:30:35.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f75152720 con 0x7fdc48082040 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f75154100 con 0x559f7514df70 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.449+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f751544a0 con 0x559f7514df70 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.450+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.450+0000 7fdc54146800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.450+0000 7fdc54146800 20 started sync module instance, tier type = 2024-04-27T03:30:35.476 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.450+0000 7fdc54146800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.450+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f751535a0 con 0x7fdc44032c20 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.451+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.451+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94143352484080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75154840 con 0x7fdc44032c20 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.451+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75155090 con 0x559f75149580 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.452+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94143352484080] v25'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.452+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.452+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94143352490976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75155000 con 0x559f75149580 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.452+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75155f50 con 0x559f7514df70 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.453+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94143352490976] v25'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.453+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.453+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94143352494752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f751562f0 con 0x559f7514df70 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.453+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75156cf0 con 0x7fdc44032c20 2024-04-27T03:30:35.477 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.454+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.454+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94143352500448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f751579b0 con 0x7fdc44032c20 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.454+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75158420 con 0x559f75149580 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.455+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94143352494752] v25'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.455+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.455+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94143352500448] v25'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.455+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94143352504048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75158810 con 0x559f75149580 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.456+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75157520 con 0x7fdc48082040 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.456+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.457+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94143352501536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f751591d0 con 0x7fdc48082040 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.457+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94143352504048] v25'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.457+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f751554b0 con 0x7fdc48082040 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.458+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94143352501536] v25'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.478 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.458+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.458+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94143352507680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75159140 con 0x7fdc48082040 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.458+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f7515a070 con 0x7fdc48082040 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.459+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94143352507680] v25'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.459+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'120 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.459+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94143352510224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75159fe0 con 0x7fdc48082040 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.459+0000 7fdc54146800 20 add_watcher() i=0 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.459+0000 7fdc54146800 20 add_watcher() i=1 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.459+0000 7fdc54146800 20 add_watcher() i=2 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.459+0000 7fdc54146800 20 add_watcher() i=3 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.459+0000 7fdc54146800 20 add_watcher() i=4 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.459+0000 7fdc54146800 20 add_watcher() i=5 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.459+0000 7fdc54146800 20 add_watcher() i=6 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.460+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94143352510224] v25'121 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.460+0000 7fdc54146800 20 add_watcher() i=7 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.460+0000 7fdc54146800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.460+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f7515d520 con 0x559f75149580 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.460+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.460+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94143352511744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f7515aa40 con 0x559f75149580 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.461+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94143352511744] v25'123 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.461+0000 7fdb037fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.461+0000 7fdc54146800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.461+0000 7fdc54146800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.461+0000 7fdc54146800 5 note: GC not initialized 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.461+0000 7fdb027fc640 20 reqs_thread_entry: start 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc54146800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc54146800 20 Filter name: none 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fda73fff640 20 reqs_thread_entry: start 2024-04-27T03:30:35.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc54146800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f75190800 con 0x7fdc48082040 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(62 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc54146800 20 rados_obj.operate() r=0 bl.length=380 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc54146800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc54146800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc54146800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc54146800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.462+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f751928e0 con 0x7fdc44032c20 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv20 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc54146800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc54146800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T03:30:35.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc54146800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc54146800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 : miss 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f75194710 con 0x559f75149580 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(64 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc54146800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc54146800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc54146800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc54146800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.463+0000 7fdc54146800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.464+0000 7fdc54146800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.464+0000 7fdc54146800 10 cache get: name=default.rgw.meta+users.uid+fud : hit (requested=0x13, cached=0x13) 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.464+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75195c00 con 0x7fdc48082040 2024-04-27T03:30:35.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.465+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(65 fud.buckets [call] v25'2 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.465+0000 7fdc54146800 10 cache get: name=default.rgw.meta+root+myfoo : type miss (requested=0x13, cached=0x11) 2024-04-27T03:30:35.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.465+0000 7fdc54146800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.465+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f75196000 con 0x7fdc44032c20 2024-04-27T03:30:35.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.465+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(66 myfoo [call out=48b,read 0~165 out=165b,getxattrs out=75b] v0'0 uv20 ondisk = 0) ==== 233+0+288 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.465+0000 7fdc54146800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:35.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.465+0000 7fdc54146800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x13 2024-04-27T03:30:35.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.465+0000 7fdc54146800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:35.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.465+0000 7fdc54146800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:35.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.465+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 8.9 8:91b71841:root::myfoo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f7519bbf0 con 0x7fdc44032c20 2024-04-27T03:30:35.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.467+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(67 myfoo [delete,create,call,call,writefull 0~165] v25'21 uv21 ondisk = 0) ==== 317+0+0 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.467+0000 7fdc54146800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.467+0000 7fdc54146800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.467+0000 7fdc54146800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.467+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:68 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94143352764160 in=350b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f75199100 con 0x7fdc48082040 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.467+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== watch-notify(notify (1) cookie 94143352507680 notify 107374182404 ret 0) ==== 380+0+0 (crc 0 0 0) 0x559f75194b00 con 0x7fdc48082040 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.467+0000 7fdb357fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182404 cookie 94143352507680 notifier 4504 bl.length()=338 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.467+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(68 notify.6 [notify cookie 94143352764160 out=8b] v0'0 uv120 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.467+0000 7fdb357fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x17 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.467+0000 7fdb357fa640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.467+0000 7fdb357fa640 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fdb14003ea0 con 0x7fdc48082040 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.468+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(69 notify.6 [notify-ack] v0'0 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.468+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== watch-notify(notify_complete (2) cookie 94143352764160 notify 107374182404 ret 0) ==== 42+0+48 (crc 0 0 0) 0x559f75193df0 con 0x7fdc48082040 2024-04-27T03:30:35.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.468+0000 7fdb037fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.468+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94143352511744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75191220 con 0x559f75149580 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.469+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94143352511744] v25'124 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.469+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94143352484080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75159cd0 con 0x7fdc44032c20 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.470+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94143352484080] v25'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.470+0000 7fdc54146800 20 remove_watcher() i=0 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.470+0000 7fdc54146800 2 removed watcher, disabling cache 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.470+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94143352490976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f751530f0 con 0x559f75149580 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.471+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94143352490976] v25'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.471+0000 7fdc54146800 20 remove_watcher() i=1 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.471+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94143352494752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75154be0 con 0x559f7514df70 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.473+0000 7fdc51c4c640 1 -- 172.21.15.165:0/2548626397 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94143352494752] v25'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc44077070 con 0x559f7514df70 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.473+0000 7fdc54146800 20 remove_watcher() i=2 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.473+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94143352500448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75155aa0 con 0x7fdc44032c20 2024-04-27T03:30:35.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.474+0000 7fdc5144b640 1 -- 172.21.15.165:0/2548626397 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94143352500448] v25'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4807f070 con 0x7fdc44032c20 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.474+0000 7fdc54146800 20 remove_watcher() i=3 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.474+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94143352504048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f751570e0 con 0x559f75149580 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.475+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94143352504048] v25'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x559f75149580 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.475+0000 7fdc54146800 20 remove_watcher() i=4 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.475+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94143352501536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f751570e0 con 0x7fdc48082040 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.476+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94143352501536] v25'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.476+0000 7fdc54146800 20 remove_watcher() i=5 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.476+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94143352507680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f751570e0 con 0x7fdc48082040 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.477+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94143352507680] v25'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.477+0000 7fdc54146800 20 remove_watcher() i=6 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.477+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94143352510224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f75158d20 con 0x7fdc48082040 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94143352510224] v25'122 uv120 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fdc4c031040 con 0x7fdc48082040 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 20 remove_watcher() i=7 2024-04-27T03:30:35.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fdc48082040 msgr2=0x7fdc48083430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fdc48082040 0x7fdc48083430 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f75149580 msgr2=0x559f7514ba40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f75149580 0x559f7514ba40 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fdc44032c20 msgr2=0x7fdc440713f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fdc44032c20 0x7fdc440713f0 crc :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f7514df70 msgr2=0x559f75150360 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f7514df70 0x559f75150360 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb1c036950 msgr2=0x7fdb1c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb1c036950 0x7fdb1c038e10 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdc48055040 tx=0x7fdc4804f000 comp rx=0 tx=0).stop 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 msgr2=0x559f7513dd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc5244d640 1 -- 172.21.15.165:0/2548626397 reap_dead start 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 0x559f7513dd90 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdc44026960 tx=0x7fdc44036f00 comp rx=0 tx=0).stop 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 shutdown_connections 2024-04-27T03:30:35.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 --2- 172.21.15.165:0/2548626397 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751369d0 0x559f7513dd90 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 >> 172.21.15.165:0/2548626397 conn(0x559f75132d30 msgr2=0x559f75130390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 shutdown_connections 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2548626397 wait complete. 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2696880275 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb2c036950 msgr2=0x7fdb2c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 --2- 172.21.15.165:0/2696880275 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb2c036950 0x7fdb2c038e10 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x559f75112cc0 tx=0x7fdc48005ac0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2696880275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751296b0 msgr2=0x559f75129ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 --2- 172.21.15.165:0/2696880275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751296b0 0x559f75129ad0 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fdc4404b4b0 tx=0x7fdc4401aaf0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2696880275 shutdown_connections 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 --2- 172.21.15.165:0/2696880275 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb2c036950 0x7fdb2c038e10 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 --2- 172.21.15.165:0/2696880275 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f751296b0 0x559f75129ad0 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2696880275 >> 172.21.15.165:0/2696880275 conn(0x559f7511d750 msgr2=0x559f7511db50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2696880275 shutdown_connections 2024-04-27T03:30:35.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.478+0000 7fdc54146800 1 -- 172.21.15.165:0/2696880275 wait complete. 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559f751186a0 msgr2=0x559f7511aaf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559f751186a0 0x559f7511aaf0 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f750144a0 msgr2=0x559f75111ee0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f750144a0 0x559f75111ee0 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f75114710 msgr2=0x559f75116b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f75114710 0x559f75116b00 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb38036900 msgr2=0x7fdb38038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fdb38036900 0x7fdb38038dc0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fdc48054000 tx=0x7fdc4804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 msgr2=0x559f75010260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 --2- 172.21.15.165:0/1108280362 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f75010990 0x559f75010260 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7fdc4400b210 tx=0x7fdc44000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc5244d640 1 -- 172.21.15.165:0/1108280362 reap_dead start 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 shutdown_connections 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 >> 172.21.15.165:0/1108280362 conn(0x559f7500a340 msgr2=0x559f750fafc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 shutdown_connections 2024-04-27T03:30:35.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.479+0000 7fdc54146800 1 -- 172.21.15.165:0/1108280362 wait complete. 2024-04-27T03:30:35.496 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1'] 2024-04-27T03:30:35.497 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--uid', 'foo', '--bucket', 'myfoo', '--bucket-id', '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1'] 2024-04-27T03:30:35.497 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --uid foo --bucket myfoo --bucket-id 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 2024-04-27T03:30:35.583 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:35.583 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:35.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.578+0000 7f2a7c90c800 1 Processor -- start 2024-04-27T03:30:35.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.578+0000 7f2a7c90c800 1 -- start start 2024-04-27T03:30:35.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.578+0000 7f2a7c90c800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x561cf6dbf3e0 0x561cf6dc17d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.578+0000 7f2a7c90c800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x561cf6c79d40 con 0x561cf6dbb8d0 2024-04-27T03:30:35.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.578+0000 7f2a7c90c800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x561cf6c4c4d0 con 0x561cf6dbf3e0 2024-04-27T03:30:35.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.578+0000 7f2a79c0a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x561cf6dbf3e0 0x561cf6dc17d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.578+0000 7f2a7a40b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x561cf6dbb8d0 0x561cf6dbdcc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35344/0 (socket says 172.21.15.165:35344) 2024-04-27T03:30:35.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.578+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2459555092 learned_addr learned my addr 172.21.15.165:0/2459555092 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:35.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.579+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2459555092 >> v1:172.21.15.89:6789/0 conn(0x561cf6dbb8d0 legacy=0x561cf6dbdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.579+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2459555092 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf6c78590 con 0x561cf6dbf3e0 2024-04-27T03:30:35.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.580+0000 7f2a79c0a640 1 --2- 172.21.15.165:0/2459555092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbf3e0 0x561cf6dc17d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000a470 tx=0x7f2a7001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6285ff9f8c8c2fe9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.581+0000 7f2a79409640 1 -- 172.21.15.165:0/2459555092 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a70057070 con 0x561cf6dbf3e0 2024-04-27T03:30:35.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.581+0000 7f2a79409640 1 -- 172.21.15.165:0/2459555092 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a70052570 con 0x561cf6dbf3e0 2024-04-27T03:30:35.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.581+0000 7f2a79409640 1 -- 172.21.15.165:0/2459555092 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a70052870 con 0x561cf6dbf3e0 2024-04-27T03:30:35.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.581+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2459555092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbf3e0 msgr2=0x561cf6dc17d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.581+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2459555092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbf3e0 0x561cf6dc17d0 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7f2a7000a470 tx=0x7f2a7001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:35.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.581+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2459555092 shutdown_connections 2024-04-27T03:30:35.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.581+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2459555092 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbf3e0 0x561cf6dc17d0 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.581+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2459555092 >> 172.21.15.165:0/2459555092 conn(0x561cf6db53a0 msgr2=0x561cf6db77c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.581+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2459555092 shutdown_connections 2024-04-27T03:30:35.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.581+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2459555092 wait complete. 2024-04-27T03:30:35.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.593+0000 7f2a7c90c800 1 Processor -- start 2024-04-27T03:30:35.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.593+0000 7f2a7c90c800 1 -- start start 2024-04-27T03:30:35.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.593+0000 7f2a7c90c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 0x561cf6eac1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.593+0000 7f2a7c90c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561cf6d063e0 con 0x561cf6dbb8d0 2024-04-27T03:30:35.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.593+0000 7f2a7a40b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 0x561cf6eac1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.594+0000 7f2a7a40b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 0x561cf6eac1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58816/0 (socket says 172.21.15.165:58816) 2024-04-27T03:30:35.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.594+0000 7f2a7a40b640 1 -- 172.21.15.165:0/410258217 learned_addr learned my addr 172.21.15.165:0/410258217 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.594+0000 7f2a7a40b640 1 -- 172.21.15.165:0/410258217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf6eac7a0 con 0x561cf6dbb8d0 2024-04-27T03:30:35.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.594+0000 7f2a7a40b640 1 --2- 172.21.15.165:0/410258217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 0x561cf6eac1d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c001de0 tx=0x7f2a6c00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d911cde7db86ac36 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.594+0000 7f2968608640 1 -- 172.21.15.165:0/410258217 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c012020 con 0x561cf6dbb8d0 2024-04-27T03:30:35.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.594+0000 7f2968608640 1 -- 172.21.15.165:0/410258217 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c00e5a0 con 0x561cf6dbb8d0 2024-04-27T03:30:35.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.594+0000 7f2968608640 1 -- 172.21.15.165:0/410258217 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c00e8c0 con 0x561cf6dbb8d0 2024-04-27T03:30:35.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.595+0000 7f2a7c90c800 1 -- 172.21.15.165:0/410258217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 msgr2=0x561cf6eac1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.595+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/410258217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 0x561cf6eac1d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c001de0 tx=0x7f2a6c00b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.595+0000 7f2a7c90c800 1 -- 172.21.15.165:0/410258217 shutdown_connections 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.595+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/410258217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 0x561cf6eac1d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.595+0000 7f2a7c90c800 1 -- 172.21.15.165:0/410258217 >> 172.21.15.165:0/410258217 conn(0x561cf6db53a0 msgr2=0x561cf6ea7af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.595+0000 7f2a7c90c800 1 -- 172.21.15.165:0/410258217 shutdown_connections 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.595+0000 7f2a7c90c800 1 -- 172.21.15.165:0/410258217 wait complete. 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.595+0000 7f2a7c90c800 1 Processor -- start 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.595+0000 7f2a7c90c800 1 -- start start 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.595+0000 7f2a7c90c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 0x561cf6eb3e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.595+0000 7f2a7c90c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561cf6dc2010 con 0x561cf6dbb8d0 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.596+0000 7f2a7a40b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 0x561cf6eb3e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.596+0000 7f2a7a40b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 0x561cf6eb3e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58826/0 (socket says 172.21.15.165:58826) 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.596+0000 7f2a7a40b640 1 -- 172.21.15.165:0/1039037506 learned_addr learned my addr 172.21.15.165:0/1039037506 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.596+0000 7f2a7a40b640 1 -- 172.21.15.165:0/1039037506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf6eb4340 con 0x561cf6dbb8d0 2024-04-27T03:30:35.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.596+0000 7f2a7a40b640 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 0x561cf6eb3e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00c010 tx=0x7f2a6c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.596+0000 7f2966e05640 1 -- 172.21.15.165:0/1039037506 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c012070 con 0x561cf6dbb8d0 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.596+0000 7f2966e05640 1 -- 172.21.15.165:0/1039037506 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c01a7a0 con 0x561cf6dbb8d0 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.596+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561cf6eb45d0 con 0x561cf6dbb8d0 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.596+0000 7f2966e05640 1 -- 172.21.15.165:0/1039037506 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c01aa80 con 0x561cf6dbb8d0 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.596+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf6eb71e0 con 0x561cf6dbb8d0 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.597+0000 7f2966e05640 1 -- 172.21.15.165:0/1039037506 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2a6c015020 con 0x561cf6dbb8d0 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.597+0000 7f2966e05640 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2960036950 0x7f2960038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.597+0000 7f2966e05640 1 -- 172.21.15.165:0/1039037506 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2a6c020070 con 0x561cf6dbb8d0 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.597+0000 7f2a79c0a640 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2960036950 0x7f2960038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.597+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561cf6dbf3e0 0x561cf6ebafa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.597+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ebb520 con 0x561cf6dbf3e0 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.597+0000 7f2a7ac0c640 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561cf6dbf3e0 0x561cf6ebafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.598+0000 7f2a79c0a640 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2960036950 0x7f2960038e10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2a70054000 tx=0x7f2a7004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.598+0000 7f2a7ac0c640 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561cf6dbf3e0 0x561cf6ebafa0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.598+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/1039037506 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6dbf3e0 2024-04-27T03:30:35.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.598+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6eb8570 con 0x561cf6dbf3e0 2024-04-27T03:30:35.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.598+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/1039037506 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6dbf3e0 2024-04-27T03:30:35.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.598+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561cf6ebd880 0x561cf6ebfc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.598+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ec0310 con 0x561cf6ebd880 2024-04-27T03:30:35.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.598+0000 7f2a7a40b640 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561cf6ebd880 0x561cf6ebfc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.599+0000 7f2a7a40b640 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561cf6ebd880 0x561cf6ebfc70 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.599+0000 7f2a7a40b640 1 -- 172.21.15.165:0/1039037506 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ebd880 2024-04-27T03:30:35.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.599+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ebc830 con 0x561cf6dbf3e0 2024-04-27T03:30:35.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.599+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/1039037506 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6dbf3e0 2024-04-27T03:30:35.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.599+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561cf6ec17c0 0x561cf6ec3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.599+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ec4370 con 0x561cf6ec17c0 2024-04-27T03:30:35.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.599+0000 7f2a79c0a640 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561cf6ec17c0 0x561cf6ec3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.600+0000 7f2a79c0a640 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561cf6ec17c0 0x561cf6ec3c10 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.600+0000 7f2a79c0a640 1 -- 172.21.15.165:0/1039037506 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2a70052a40 con 0x561cf6ec17c0 2024-04-27T03:30:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.601+0000 7f2a7c90c800 1 Processor -- start 2024-04-27T03:30:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.601+0000 7f2a7c90c800 1 -- start start 2024-04-27T03:30:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.601+0000 7f2a7c90c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6ed26c0 0x561cf6ed2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.601+0000 7f2a7c90c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561cf6cac6b0 con 0x561cf6ed26c0 2024-04-27T03:30:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.601+0000 7f2a7a40b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6ed26c0 0x561cf6ed2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.601+0000 7f2a7a40b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6ed26c0 0x561cf6ed2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58832/0 (socket says 172.21.15.165:58832) 2024-04-27T03:30:35.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.601+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2490595572 learned_addr learned my addr 172.21.15.165:0/2490595572 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.601+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2490595572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf6ed3020 con 0x561cf6ed26c0 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.601+0000 7f2a7a40b640 1 --2- 172.21.15.165:0/2490595572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6ed26c0 0x561cf6ed2ae0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c04aa30 tx=0x7f2a6c04ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.601+0000 7f295ffff640 1 -- 172.21.15.165:0/2490595572 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c0490e0 con 0x561cf6ed26c0 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.602+0000 7f295ffff640 1 -- 172.21.15.165:0/2490595572 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c049280 con 0x561cf6ed26c0 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.602+0000 7f295ffff640 1 -- 172.21.15.165:0/2490595572 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c049560 con 0x561cf6ed26c0 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.602+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2490595572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561cf6ed3a90 con 0x561cf6ed26c0 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.602+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2490595572 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf6ed3f90 con 0x561cf6ed26c0 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.602+0000 7f295ffff640 1 -- 172.21.15.165:0/2490595572 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2a6c044000 con 0x561cf6ed26c0 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.602+0000 7f295ffff640 1 --2- 172.21.15.165:0/2490595572 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2954036950 0x7f2954038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.602+0000 7f295ffff640 1 -- 172.21.15.165:0/2490595572 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2a6c04dc00 con 0x561cf6ed26c0 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.603+0000 7f2a79c0a640 1 --2- 172.21.15.165:0/2490595572 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2954036950 0x7f2954038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.603+0000 7f2a7c90c800 1 Processor -- start 2024-04-27T03:30:35.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.603+0000 7f2a7c90c800 1 -- start start 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.603+0000 7f2a79c0a640 1 --2- 172.21.15.165:0/2490595572 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2954036950 0x7f2954038e10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x561cf6eb8490 tx=0x7f2a70005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.603+0000 7f2a7c90c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 0x561cf6edfc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.603+0000 7f2a7c90c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561cf6eacb20 con 0x561cf6edf870 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.603+0000 7f2a7a40b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 0x561cf6edfc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.603+0000 7f2a7a40b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 0x561cf6edfc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58842/0 (socket says 172.21.15.165:58842) 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.603+0000 7f2a7a40b640 1 -- 172.21.15.165:0/3422531674 learned_addr learned my addr 172.21.15.165:0/3422531674 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.603+0000 7f2a7a40b640 1 -- 172.21.15.165:0/3422531674 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf6ee0260 con 0x561cf6edf870 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.604+0000 7f2a7a40b640 1 --2- 172.21.15.165:0/3422531674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 0x561cf6edfc90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0473f0 tx=0x7f2a6c046f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b8c96d947e9358 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.604+0000 7f295dffb640 1 -- 172.21.15.165:0/3422531674 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c05c070 con 0x561cf6edf870 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.604+0000 7f295dffb640 1 -- 172.21.15.165:0/3422531674 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c03f000 con 0x561cf6edf870 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.604+0000 7f295dffb640 1 -- 172.21.15.165:0/3422531674 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c03d070 con 0x561cf6edf870 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.604+0000 7f2a7c90c800 1 -- 172.21.15.165:0/3422531674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 msgr2=0x561cf6edfc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.604+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/3422531674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 0x561cf6edfc90 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0473f0 tx=0x7f2a6c046f50 comp rx=0 tx=0).stop 2024-04-27T03:30:35.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.604+0000 7f2a7c90c800 1 -- 172.21.15.165:0/3422531674 shutdown_connections 2024-04-27T03:30:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.604+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/3422531674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 0x561cf6edfc90 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.604+0000 7f2a7c90c800 1 -- 172.21.15.165:0/3422531674 >> 172.21.15.165:0/3422531674 conn(0x561cf6edbb40 msgr2=0x561cf6ed9ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.604+0000 7f2a7c90c800 1 -- 172.21.15.165:0/3422531674 shutdown_connections 2024-04-27T03:30:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.604+0000 7f2a7c90c800 1 -- 172.21.15.165:0/3422531674 wait complete. 2024-04-27T03:30:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.605+0000 7f2a7c90c800 1 Processor -- start 2024-04-27T03:30:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.605+0000 7f2a7c90c800 1 -- start start 2024-04-27T03:30:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.605+0000 7f2a7c90c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 0x561cf6dbb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.605+0000 7f2a7c90c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561cf6ee0930 con 0x561cf6edf870 2024-04-27T03:30:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.605+0000 7f2a7a40b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 0x561cf6dbb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.605+0000 7f2a7a40b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 0x561cf6dbb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58856/0 (socket says 172.21.15.165:58856) 2024-04-27T03:30:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.605+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 learned_addr learned my addr 172.21.15.165:0/2861605470 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.605+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561cf6dbb690 con 0x561cf6edf870 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.605+0000 7f2a7a40b640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 0x561cf6dbb150 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0473f0 tx=0x7f2a6c03f2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.605+0000 7f294bfff640 1 -- 172.21.15.165:0/2861605470 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c067020 con 0x561cf6edf870 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.606+0000 7f294bfff640 1 -- 172.21.15.165:0/2861605470 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2a6c047680 con 0x561cf6edf870 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.606+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561cf6dbb690 con 0x561cf6edf870 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.606+0000 7f294bfff640 1 -- 172.21.15.165:0/2861605470 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2a6c05c040 con 0x561cf6edf870 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.606+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561cf6dbb690 con 0x561cf6edf870 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.606+0000 7f294bfff640 1 -- 172.21.15.165:0/2861605470 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2a6c06a020 con 0x561cf6edf870 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.607+0000 7f294bfff640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2944036950 0x7f2944038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.607+0000 7f294bfff640 1 -- 172.21.15.165:0/2861605470 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2a6c052030 con 0x561cf6edf870 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.607+0000 7f2a79c0a640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2944036950 0x7f2944038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.607+0000 7f2a79c0a640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2944036950 0x7f2944038e10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2a70052fd0 tx=0x7f2a7004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.607+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.607+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561cf6ef4530 0x561cf6ef69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.607+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef7140 con 0x561cf6ef4530 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.607+0000 7f2a7ac0c640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561cf6ef4530 0x561cf6ef69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7ac0c640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561cf6ef4530 0x561cf6ef69f0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6ef4530 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 20 realm 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef7550 con 0x561cf6ef4530 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6ef4530 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef7e20 con 0x561cf6ef4530 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6ef4530 2024-04-27T03:30:35.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef81c0 con 0x561cf6ef4530 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6ef4530 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561cf6ef8f20 0x561cf6efb310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.608+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efb920 con 0x561cf6ef8f20 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.609+0000 7f2a7a40b640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561cf6ef8f20 0x561cf6efb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.609+0000 7f2a7a40b640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561cf6ef8f20 0x561cf6efb310 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.609+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.609+0000 7f2a7c90c800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.609+0000 7f2a7c90c800 20 searching for the correct realm 2024-04-27T03:30:35.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.609+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efd250 con 0x561cf6ef8f20 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.610+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.610+0000 7f2a7a40b640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2a6c077040 0x7f2a6c04c530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.610+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efd250 con 0x7f2a6c077040 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.610+0000 7f2a79c0a640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2a6c077040 0x7f2a6c04c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.610+0000 7f2a79c0a640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2a6c077040 0x7f2a6c04c530 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.610+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.610+0000 7f2a79c0a640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2a70076040 0x7f2a70083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.610+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efd250 con 0x7f2a70076040 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.610+0000 7f2a7ac0c640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2a70076040 0x7f2a70083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.611+0000 7f2a7ac0c640 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2a70076040 0x7f2a70083410 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.611+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f2a7400a4a0 con 0x7f2a70076040 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.611+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef81c0 con 0x561cf6ef4530 2024-04-27T03:30:35.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.611+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6ef4530 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.611+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef81c0 con 0x561cf6ef4530 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.611+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6ef4530 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.611+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef81c0 con 0x7f2a70076040 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.612+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a7400a4a0 con 0x7f2a70076040 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.612+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef81c0 con 0x7f2a70076040 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.612+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a7400a4a0 con 0x7f2a70076040 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.612+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef81c0 con 0x7f2a6c077040 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.612+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.612+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef81c0 con 0x561cf6ef8f20 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.612+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.612+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef81c0 con 0x561cf6ef4530 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.612+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6ef4530 2024-04-27T03:30:35.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.613+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef81c0 con 0x7f2a70076040 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.613+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a7400a4a0 con 0x7f2a70076040 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.613+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef81c0 con 0x561cf6ef4530 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.613+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6ef4530 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.613+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561cf6ef81c0 con 0x7f2a6c077040 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.613+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.613+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x561cf6ef8f20 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.613+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.613+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x561cf6ef8f20 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.614+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.614+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x7f2a70076040 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.614+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a7400a4a0 con 0x7f2a70076040 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.614+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x561cf6ef8f20 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.614+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.614+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x561cf6ef8f20 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.614+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.614+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x561cf6ef4530 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.615+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6ef4530 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.615+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x561cf6ef4530 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.615+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2a7400a4a0 con 0x561cf6ef4530 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.615+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x7f2a70076040 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.615+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a74033040 con 0x7f2a70076040 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.615+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x561cf6ef8f20 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.616+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.616+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x7f2a6c077040 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.616+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.616+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x561cf6ef4530 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.616+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.616+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x7f2a70076040 2024-04-27T03:30:35.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.616+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.616+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x561cf6ef8f20 2024-04-27T03:30:35.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.617+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.617+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x7f2a70076040 2024-04-27T03:30:35.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.617+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.617+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x7f2a6c077040 2024-04-27T03:30:35.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.617+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.617+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a70090060 con 0x561cf6ef8f20 2024-04-27T03:30:35.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.617+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.617+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a6c089040 con 0x7f2a70076040 2024-04-27T03:30:35.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a6c089040 con 0x561cf6ef4530 2024-04-27T03:30:35.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f2a6c089040 con 0x7f2a6c077040 2024-04-27T03:30:35.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7c90c800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:35.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7c90c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7c90c800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7c90c800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7c90c800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7c90c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efc1e0 con 0x561cf6ef4530 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7c90c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.618+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efc5d0 con 0x561cf6ef4530 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:35.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efc5d0 con 0x7f2a70076040 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 20 zone default found 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 4 Realm: () 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 20 zonegroup default 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efbdc0 con 0x7f2a70076040 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efbdc0 con 0x561cf6ef8f20 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.619+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6eff170 con 0x561cf6ef8f20 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.620+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.620+0000 7f2a7c90c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.620+0000 7f2a7c90c800 20 started sync module instance, tier type = 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.620+0000 7f2a7c90c800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.620+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efe550 con 0x7f2a6c077040 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.621+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.621+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94682401990976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efe1b0 con 0x7f2a6c077040 2024-04-27T03:30:35.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.621+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f000a0 con 0x561cf6ef4530 2024-04-27T03:30:35.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.622+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94682401990976] v25'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.622+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.622+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94682401995760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f004c0 con 0x561cf6ef4530 2024-04-27T03:30:35.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.622+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f00e80 con 0x561cf6ef8f20 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.623+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94682401995760] v25'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.623+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.623+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94682401996816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f00430 con 0x561cf6ef8f20 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.624+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f01d40 con 0x7f2a6c077040 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.625+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94682401996816] v25'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.625+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.625+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94682402003088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f02130 con 0x7f2a6c077040 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.625+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f02b30 con 0x561cf6ef4530 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.626+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.626+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94682402003088] v25'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.626+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94682402006592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f033d0 con 0x561cf6ef4530 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.626+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f012b0 con 0x7f2a70076040 2024-04-27T03:30:35.658 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.627+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94682402006592] v25'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.627+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.627+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94682402008784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f039c0 con 0x7f2a70076040 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.627+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f043c0 con 0x7f2a70076040 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.628+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.628+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94682402008784] v25'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.628+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94682402012880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f042f0 con 0x7f2a70076040 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.628+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f04d60 con 0x7f2a70076040 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.629+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94682402012880] v25'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.629+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'123 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.629+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94682402009840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f04d30 con 0x7f2a70076040 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.629+0000 7f2a7c90c800 20 add_watcher() i=0 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.629+0000 7f2a7c90c800 20 add_watcher() i=1 2024-04-27T03:30:35.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.629+0000 7f2a7c90c800 20 add_watcher() i=2 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.629+0000 7f2a7c90c800 20 add_watcher() i=3 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.629+0000 7f2a7c90c800 20 add_watcher() i=4 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.629+0000 7f2a7c90c800 20 add_watcher() i=5 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.629+0000 7f2a7c90c800 20 add_watcher() i=6 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.630+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94682402009840] v25'124 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.630+0000 7f2a7c90c800 20 add_watcher() i=7 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.630+0000 7f2a7c90c800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.630+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f081c0 con 0x561cf6ef4530 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.630+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.630+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94682402015824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f056e0 con 0x561cf6ef4530 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.631+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94682402015824] v25'125 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.631+0000 7f2930fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:35.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.631+0000 7f2a7c90c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.631+0000 7f2a7c90c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.631+0000 7f2a7c90c800 5 note: GC not initialized 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.631+0000 7f28ab7fe640 20 reqs_thread_entry: start 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7c90c800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7c90c800 20 Filter name: none 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f28a97fa640 20 reqs_thread_entry: start 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7c90c800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f3b520 con 0x561cf6ef4530 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7c90c800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7c90c800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:35.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7c90c800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7c90c800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7c90c800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f3d600 con 0x7f2a6c077040 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.632+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv21 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 : miss 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f3ee60 con 0x561cf6ef4530 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:35.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.633+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:65 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f43700 con 0x7f2a70076040 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.634+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(65 fud.buckets [call] v25'3 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.634+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f49600 con 0x561cf6ef4530 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [delete,create,call,call,writefull 0~379,setxattr (123)] v25'4 uv4 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f2a7c90c800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f2a7c90c800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f2a7c90c800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:35.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f2a7c90c800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1, ofs0, ns] 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [notify cookie 94682402256160 in=764b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f43b30 con 0x561cf6ef4530 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify (1) cookie 94682401995760 notify 107374182419 ret 0) ==== 794+0+0 (crc 0 0 0) 0x561cf6f3f250 con 0x561cf6ef4530 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f295dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182419 cookie 94682401995760 notifier 4511 bl.length()=752 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(67 notify.1 [notify cookie 94682402256160 out=8b] v0'0 uv123 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f295dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f295dffb640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f295dffb640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.635+0000 7f295dffb640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f2940003ea0 con 0x561cf6ef4530 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.636+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [notify-ack] v0'0 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.636+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== watch-notify(notify_complete (2) cookie 94682402256160 notify 107374182419 ret 0) ==== 42+0+48 (crc 0 0 0) 0x561cf6f3e3a0 con 0x561cf6ef4530 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.636+0000 7f2a7c90c800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): orig_sources= new_sources= 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.636+0000 7f2a7c90c800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): potential sources added=[] removed=[] 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.636+0000 7f2a7c90c800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): orig_dests= new_dests= 2024-04-27T03:30:35.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.636+0000 7f2a7c90c800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): potential dests added=[] removed=[] 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.636+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f3f500 con 0x7f2a6c077040 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.637+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(69 foo.buckets [call] v25'22 uv22 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.637+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f3f500 con 0x7f2a6c077040 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.638+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v25'23 uv23 ondisk = 0) ==== 275+0+0 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.638+0000 7f2a7c90c800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.638+0000 7f2a7c90c800 10 moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.638+0000 7f2a7c90c800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.638+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94682402256128 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f482d0 con 0x7f2a70076040 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.639+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== watch-notify(notify (1) cookie 94682402012880 notify 107374182405 ret 0) ==== 356+0+0 (crc 0 0 0) 0x7f2a7401e040 con 0x7f2a70076040 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.639+0000 7f295dffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182405 cookie 94682402012880 notifier 4511 bl.length()=314 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.639+0000 7f295dffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.639+0000 7f295dffb640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.639+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(71 notify.6 [notify cookie 94682402256128 out=8b] v0'0 uv123 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.639+0000 7f295dffb640 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f2940003ea0 con 0x7f2a70076040 2024-04-27T03:30:35.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.639+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== watch-notify(notify_complete (2) cookie 94682402256128 notify 107374182405 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f2a74027420 con 0x7f2a70076040 2024-04-27T03:30:35.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.639+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.640+0000 7f2930fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:35.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.640+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:73 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94682402015824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f400b0 con 0x561cf6ef4530 2024-04-27T03:30:35.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.641+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(73 data_loggenerations_metadata [watch unwatch cookie 94682402015824] v25'126 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.641+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94682401990976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f3f500 con 0x7f2a6c077040 2024-04-27T03:30:35.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.642+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 notify.0 [watch unwatch cookie 94682401990976] v25'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.642+0000 7f2a7c90c800 20 remove_watcher() i=0 2024-04-27T03:30:35.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.642+0000 7f2a7c90c800 2 removed watcher, disabling cache 2024-04-27T03:30:35.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.642+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94682401995760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efe940 con 0x561cf6ef4530 2024-04-27T03:30:35.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.643+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(75 notify.1 [watch unwatch cookie 94682401995760] v25'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.643+0000 7f2a7c90c800 20 remove_watcher() i=1 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.643+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:76 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94682401996816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6effbf0 con 0x561cf6ef8f20 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.644+0000 7f2a7a40b640 1 -- 172.21.15.165:0/2861605470 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(76 notify.2 [watch unwatch cookie 94682401996816] v25'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a6c02b070 con 0x561cf6ef8f20 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.644+0000 7f2a7c90c800 20 remove_watcher() i=2 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.644+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:77 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94682402003088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6efe940 con 0x7f2a6c077040 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.645+0000 7f2a79c0a640 1 -- 172.21.15.165:0/2861605470 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(77 notify.3 [watch unwatch cookie 94682402003088] v25'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a70081070 con 0x7f2a6c077040 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.645+0000 7f2a7c90c800 20 remove_watcher() i=3 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.645+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94682402006592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f01890 con 0x561cf6ef4530 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.646+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(78 notify.4 [watch unwatch cookie 94682402006592] v25'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x561cf6ef4530 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.646+0000 7f2a7c90c800 20 remove_watcher() i=4 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.646+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:79 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94682402008784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f01890 con 0x7f2a70076040 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.647+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(79 notify.5 [watch unwatch cookie 94682402008784] v25'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.647+0000 7f2a7c90c800 20 remove_watcher() i=5 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.647+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:80 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94682402012880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f02640 con 0x7f2a70076040 2024-04-27T03:30:35.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.648+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(80 notify.6 [watch unwatch cookie 94682402012880] v25'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.648+0000 7f2a7c90c800 20 remove_watcher() i=6 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.648+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:81 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94682402009840] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561cf6f03ed0 con 0x7f2a70076040 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.649+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(81 notify.7 [watch unwatch cookie 94682402009840] v25'125 uv123 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2a74031040 con 0x7f2a70076040 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.649+0000 7f2a7c90c800 20 remove_watcher() i=7 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.649+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2a70076040 msgr2=0x7f2a70083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.649+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2a70076040 0x7f2a70083410 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.649+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561cf6ef4530 msgr2=0x561cf6ef69f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.649+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561cf6ef4530 0x561cf6ef69f0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.649+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2a6c077040 msgr2=0x7f2a6c04c530 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.649+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2a6c077040 0x7f2a6c04c530 crc :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.649+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561cf6ef8f20 msgr2=0x561cf6efb310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.649+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561cf6ef8f20 0x561cf6efb310 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2944036950 msgr2=0x7f2944038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2944036950 0x7f2944038e10 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f2a70052fd0 tx=0x7f2a7004f000 comp rx=0 tx=0).stop 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 msgr2=0x561cf6dbb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2861605470 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6edf870 0x561cf6dbb150 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0473f0 tx=0x7f2a6c03f2e0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 reap_dead start 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/2861605470 reap_dead start 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 shutdown_connections 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 >> 172.21.15.165:0/2861605470 conn(0x561cf6edbb40 msgr2=0x561cf6ed91a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 shutdown_connections 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2861605470 wait complete. 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2490595572 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2954036950 msgr2=0x7f2954038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2490595572 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2954036950 0x7f2954038e10 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x561cf6eb8490 tx=0x7f2a70005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2490595572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6ed26c0 msgr2=0x561cf6ed2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2490595572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6ed26c0 0x561cf6ed2ae0 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c04aa30 tx=0x7f2a6c04ac30 comp rx=0 tx=0).stop 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2490595572 shutdown_connections 2024-04-27T03:30:35.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2490595572 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2954036950 0x7f2954038e10 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/2490595572 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6ed26c0 0x561cf6ed2ae0 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2490595572 >> 172.21.15.165:0/2490595572 conn(0x561cf6ec6760 msgr2=0x561cf6ec6b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2490595572 shutdown_connections 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/2490595572 wait complete. 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561cf6ec17c0 msgr2=0x561cf6ec3c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561cf6ec17c0 0x561cf6ec3c10 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561cf6dbf3e0 msgr2=0x561cf6ebafa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561cf6dbf3e0 0x561cf6ebafa0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561cf6ebd880 msgr2=0x561cf6ebfc70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561cf6ebd880 0x561cf6ebfc70 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2960036950 msgr2=0x7f2960038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2960036950 0x7f2960038e10 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f2a70054000 tx=0x7f2a7004e000 comp rx=0 tx=0).stop 2024-04-27T03:30:35.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 msgr2=0x561cf6eb3e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7c90c800 1 --2- 172.21.15.165:0/1039037506 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561cf6dbb8d0 0x561cf6eb3e00 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00c010 tx=0x7f2a6c000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.650+0000 7f2a7ac0c640 1 -- 172.21.15.165:0/1039037506 reap_dead start 2024-04-27T03:30:35.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.651+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 shutdown_connections 2024-04-27T03:30:35.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.651+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 >> 172.21.15.165:0/1039037506 conn(0x561cf6db53a0 msgr2=0x561cf6ea6350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.651+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 shutdown_connections 2024-04-27T03:30:35.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.651+0000 7f2a7c90c800 1 -- 172.21.15.165:0/1039037506 wait complete. 2024-04-27T03:30:35.672 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud'] 2024-04-27T03:30:35.672 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud'] 2024-04-27T03:30:35.672 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud 2024-04-27T03:30:35.715 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:35.716 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:35.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.710+0000 7fc426f2a800 1 Processor -- start 2024-04-27T03:30:35.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.710+0000 7fc426f2a800 1 -- start start 2024-04-27T03:30:35.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.711+0000 7fc426f2a800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562405feb7c0 0x562405febbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.711+0000 7fc426f2a800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562405eabd40 con 0x562405fec120 2024-04-27T03:30:35.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.711+0000 7fc426f2a800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x562405e7e4d0 con 0x562405feb7c0 2024-04-27T03:30:35.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.711+0000 7fc422c64640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562405feb7c0 0x562405febbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.711+0000 7fc422463640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x562405fec120 0x562405fec5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35352/0 (socket says 172.21.15.165:35352) 2024-04-27T03:30:35.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.711+0000 7fc422463640 1 -- 172.21.15.165:0/3533553906 learned_addr learned my addr 172.21.15.165:0/3533553906 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:35.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.711+0000 7fc422c64640 1 -- 172.21.15.165:0/3533553906 >> v1:172.21.15.89:6789/0 conn(0x562405fec120 legacy=0x562405fec5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.711+0000 7fc422c64640 1 -- 172.21.15.165:0/3533553906 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562405eaa590 con 0x562405feb7c0 2024-04-27T03:30:35.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.713+0000 7fc422c64640 1 --2- 172.21.15.165:0/3533553906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x562405febbe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc41400a470 tx=0x7fc41401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1ddbd4bc21542e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.713+0000 7fc421c62640 1 -- 172.21.15.165:0/3533553906 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc414057070 con 0x562405feb7c0 2024-04-27T03:30:35.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.713+0000 7fc421c62640 1 -- 172.21.15.165:0/3533553906 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41404adc0 con 0x562405feb7c0 2024-04-27T03:30:35.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.713+0000 7fc421c62640 1 -- 172.21.15.165:0/3533553906 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc414052590 con 0x562405feb7c0 2024-04-27T03:30:35.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.714+0000 7fc426f2a800 1 -- 172.21.15.165:0/3533553906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 msgr2=0x562405febbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.714+0000 7fc426f2a800 1 --2- 172.21.15.165:0/3533553906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x562405febbe0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7fc41400a470 tx=0x7fc41401ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:35.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.714+0000 7fc426f2a800 1 -- 172.21.15.165:0/3533553906 shutdown_connections 2024-04-27T03:30:35.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.714+0000 7fc426f2a800 1 --2- 172.21.15.165:0/3533553906 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x562405febbe0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.714+0000 7fc426f2a800 1 -- 172.21.15.165:0/3533553906 >> 172.21.15.165:0/3533553906 conn(0x562405fe7320 msgr2=0x562405fe9740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.714+0000 7fc426f2a800 1 -- 172.21.15.165:0/3533553906 shutdown_connections 2024-04-27T03:30:35.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.714+0000 7fc426f2a800 1 -- 172.21.15.165:0/3533553906 wait complete. 2024-04-27T03:30:35.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.725+0000 7fc426f2a800 1 Processor -- start 2024-04-27T03:30:35.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.726+0000 7fc426f2a800 1 -- start start 2024-04-27T03:30:35.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.726+0000 7fc426f2a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x5624060e0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.726+0000 7fc426f2a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562405f383c0 con 0x562405feb7c0 2024-04-27T03:30:35.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.726+0000 7fc422c64640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x5624060e0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.726+0000 7fc422c64640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x5624060e0400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58882/0 (socket says 172.21.15.165:58882) 2024-04-27T03:30:35.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.726+0000 7fc422c64640 1 -- 172.21.15.165:0/1319042163 learned_addr learned my addr 172.21.15.165:0/1319042163 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.726+0000 7fc422c64640 1 -- 172.21.15.165:0/1319042163 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624060e09d0 con 0x562405feb7c0 2024-04-27T03:30:35.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.727+0000 7fc422c64640 1 --2- 172.21.15.165:0/1319042163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x5624060e0400 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc414055040 tx=0x7fc41404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50bdee4787d6d096 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.727+0000 7fc310e09640 1 -- 172.21.15.165:0/1319042163 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc414057040 con 0x562405feb7c0 2024-04-27T03:30:35.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.727+0000 7fc310e09640 1 -- 172.21.15.165:0/1319042163 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc414062070 con 0x562405feb7c0 2024-04-27T03:30:35.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.727+0000 7fc310e09640 1 -- 172.21.15.165:0/1319042163 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc41405d470 con 0x562405feb7c0 2024-04-27T03:30:35.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.727+0000 7fc426f2a800 1 -- 172.21.15.165:0/1319042163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 msgr2=0x5624060e0400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.727+0000 7fc426f2a800 1 --2- 172.21.15.165:0/1319042163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x5624060e0400 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7fc414055040 tx=0x7fc41404c000 comp rx=0 tx=0).stop 2024-04-27T03:30:35.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.727+0000 7fc426f2a800 1 -- 172.21.15.165:0/1319042163 shutdown_connections 2024-04-27T03:30:35.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.727+0000 7fc426f2a800 1 --2- 172.21.15.165:0/1319042163 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x5624060e0400 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.727+0000 7fc426f2a800 1 -- 172.21.15.165:0/1319042163 >> 172.21.15.165:0/1319042163 conn(0x562405fe7320 msgr2=0x5624060dbd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.727+0000 7fc426f2a800 1 -- 172.21.15.165:0/1319042163 shutdown_connections 2024-04-27T03:30:35.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.727+0000 7fc426f2a800 1 -- 172.21.15.165:0/1319042163 wait complete. 2024-04-27T03:30:35.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.728+0000 7fc426f2a800 1 Processor -- start 2024-04-27T03:30:35.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.728+0000 7fc426f2a800 1 -- start start 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.728+0000 7fc426f2a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x5624060e7f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.728+0000 7fc426f2a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562405fecf20 con 0x562405feb7c0 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.728+0000 7fc422c64640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x5624060e7f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.728+0000 7fc422c64640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x5624060e7f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58892/0 (socket says 172.21.15.165:58892) 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.728+0000 7fc422c64640 1 -- 172.21.15.165:0/490418060 learned_addr learned my addr 172.21.15.165:0/490418060 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.728+0000 7fc422c64640 1 -- 172.21.15.165:0/490418060 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624060e8480 con 0x562405feb7c0 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.729+0000 7fc422c64640 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x5624060e7f40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc41401ce60 tx=0x7fc414002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.729+0000 7fc30f606640 1 -- 172.21.15.165:0/490418060 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc41406b070 con 0x562405feb7c0 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.729+0000 7fc30f606640 1 -- 172.21.15.165:0/490418060 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41405d970 con 0x562405feb7c0 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.729+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5624060e8710 con 0x562405feb7c0 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.729+0000 7fc30f606640 1 -- 172.21.15.165:0/490418060 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc414057070 con 0x562405feb7c0 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.729+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5624060eb320 con 0x562405feb7c0 2024-04-27T03:30:35.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.729+0000 7fc30f606640 1 -- 172.21.15.165:0/490418060 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc41405dc90 con 0x562405feb7c0 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.730+0000 7fc30f606640 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc308036950 0x7fc308038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.730+0000 7fc30f606640 1 -- 172.21.15.165:0/490418060 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fc41408dfa0 con 0x562405feb7c0 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.730+0000 7fc422463640 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc308036950 0x7fc308038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.730+0000 7fc426f2a800 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562405fec120 0x5624060ef0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.730+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5624060ef670 con 0x562405fec120 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.730+0000 7fc423465640 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562405fec120 0x5624060ef0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.730+0000 7fc422463640 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc308036950 0x7fc308038e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc418000f60 tx=0x7fc4180012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.730+0000 7fc423465640 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562405fec120 0x5624060ef0e0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.731+0000 7fc423465640 1 -- 172.21.15.165:0/490418060 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562405fec120 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.731+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5624060ec6b0 con 0x562405fec120 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.731+0000 7fc423465640 1 -- 172.21.15.165:0/490418060 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562405fec120 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.731+0000 7fc426f2a800 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5624060f1980 0x5624060f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.731+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5624060f4410 con 0x5624060f1980 2024-04-27T03:30:35.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.731+0000 7fc422c64640 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5624060f1980 0x5624060f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.731+0000 7fc422c64640 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5624060f1980 0x5624060f3d70 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.731+0000 7fc422c64640 1 -- 172.21.15.165:0/490418060 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc41405a040 con 0x5624060f1980 2024-04-27T03:30:35.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.732+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5624060f0900 con 0x562405fec120 2024-04-27T03:30:35.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.732+0000 7fc423465640 1 -- 172.21.15.165:0/490418060 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562405fec120 2024-04-27T03:30:35.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.732+0000 7fc426f2a800 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5624060f5870 0x5624060f7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.732+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5624060f8440 con 0x5624060f5870 2024-04-27T03:30:35.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.732+0000 7fc422463640 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5624060f5870 0x5624060f7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.732+0000 7fc422463640 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5624060f5870 0x5624060f7ce0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.733+0000 7fc422463640 1 -- 172.21.15.165:0/490418060 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fc418009050 con 0x5624060f5870 2024-04-27T03:30:35.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.733+0000 7fc426f2a800 1 Processor -- start 2024-04-27T03:30:35.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.733+0000 7fc426f2a800 1 -- start start 2024-04-27T03:30:35.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.733+0000 7fc426f2a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061068a0 0x562406106cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.733+0000 7fc426f2a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562405ede6b0 con 0x5624061068a0 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.733+0000 7fc422c64640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061068a0 0x562406106cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.733+0000 7fc422c64640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061068a0 0x562406106cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58904/0 (socket says 172.21.15.165:58904) 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.733+0000 7fc422c64640 1 -- 172.21.15.165:0/984699701 learned_addr learned my addr 172.21.15.165:0/984699701 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.734+0000 7fc422c64640 1 -- 172.21.15.165:0/984699701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562406107200 con 0x5624061068a0 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.734+0000 7fc422c64640 1 --2- 172.21.15.165:0/984699701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061068a0 0x562406106cc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc41408c520 tx=0x7fc4140098c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.734+0000 7fc30ce01640 1 -- 172.21.15.165:0/984699701 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc41405bcd0 con 0x5624061068a0 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.734+0000 7fc30ce01640 1 -- 172.21.15.165:0/984699701 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc41408c860 con 0x5624061068a0 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.734+0000 7fc426f2a800 1 -- 172.21.15.165:0/984699701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562406107c70 con 0x5624061068a0 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.734+0000 7fc30ce01640 1 -- 172.21.15.165:0/984699701 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc41408cb40 con 0x5624061068a0 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.734+0000 7fc426f2a800 1 -- 172.21.15.165:0/984699701 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562406107c70 con 0x5624061068a0 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.735+0000 7fc30ce01640 1 -- 172.21.15.165:0/984699701 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc41408cce0 con 0x5624061068a0 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.735+0000 7fc30ce01640 1 --2- 172.21.15.165:0/984699701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc2fc036950 0x7fc2fc038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.735+0000 7fc30ce01640 1 -- 172.21.15.165:0/984699701 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fc414091aa0 con 0x5624061068a0 2024-04-27T03:30:35.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.735+0000 7fc422463640 1 --2- 172.21.15.165:0/984699701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc2fc036950 0x7fc2fc038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.735+0000 7fc426f2a800 1 Processor -- start 2024-04-27T03:30:35.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.735+0000 7fc426f2a800 1 -- start start 2024-04-27T03:30:35.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.735+0000 7fc422463640 1 --2- 172.21.15.165:0/984699701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc2fc036950 0x7fc2fc038e10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc41800bf50 tx=0x7fc41800f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.735+0000 7fc426f2a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 0x562406113bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.735+0000 7fc426f2a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5624060e0c90 con 0x5624061137d0 2024-04-27T03:30:35.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.736+0000 7fc422c64640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 0x562406113bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.736+0000 7fc422c64640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 0x562406113bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58914/0 (socket says 172.21.15.165:58914) 2024-04-27T03:30:35.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.736+0000 7fc422c64640 1 -- 172.21.15.165:0/2756090536 learned_addr learned my addr 172.21.15.165:0/2756090536 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.736+0000 7fc422c64640 1 -- 172.21.15.165:0/2756090536 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5624061141c0 con 0x5624061137d0 2024-04-27T03:30:35.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.736+0000 7fc422c64640 1 --2- 172.21.15.165:0/2756090536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 0x562406113bf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc414072980 tx=0x7fc4140857d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b73a54e1d075f31 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.736+0000 7fc3067fc640 1 -- 172.21.15.165:0/2756090536 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc414095040 con 0x5624061137d0 2024-04-27T03:30:35.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.736+0000 7fc3067fc640 1 -- 172.21.15.165:0/2756090536 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc414070460 con 0x5624061137d0 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.736+0000 7fc3067fc640 1 -- 172.21.15.165:0/2756090536 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc414079b30 con 0x5624061137d0 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc426f2a800 1 -- 172.21.15.165:0/2756090536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 msgr2=0x562406113bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc426f2a800 1 --2- 172.21.15.165:0/2756090536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 0x562406113bf0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fc414072980 tx=0x7fc4140857d0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc426f2a800 1 -- 172.21.15.165:0/2756090536 shutdown_connections 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc426f2a800 1 --2- 172.21.15.165:0/2756090536 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 0x562406113bf0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc426f2a800 1 -- 172.21.15.165:0/2756090536 >> 172.21.15.165:0/2756090536 conn(0x56240610fb30 msgr2=0x56240610db90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc426f2a800 1 -- 172.21.15.165:0/2756090536 shutdown_connections 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc426f2a800 1 -- 172.21.15.165:0/2756090536 wait complete. 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc426f2a800 1 Processor -- start 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc426f2a800 1 -- start start 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc426f2a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 0x56240611ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc426f2a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562406114870 con 0x5624061137d0 2024-04-27T03:30:35.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc422c64640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 0x56240611ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc422c64640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 0x56240611ac20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58928/0 (socket says 172.21.15.165:58928) 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.737+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 learned_addr learned my addr 172.21.15.165:0/2402780793 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.738+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56240611b160 con 0x5624061137d0 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.738+0000 7fc422c64640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 0x56240611ac20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc414060e30 tx=0x7fc414060da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.738+0000 7fc304ff9640 1 -- 172.21.15.165:0/2402780793 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4140b2020 con 0x5624061137d0 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.738+0000 7fc304ff9640 1 -- 172.21.15.165:0/2402780793 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc4140513a0 con 0x5624061137d0 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.738+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56240611b160 con 0x5624061137d0 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.738+0000 7fc304ff9640 1 -- 172.21.15.165:0/2402780793 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc414095040 con 0x5624061137d0 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.738+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56240611b160 con 0x5624061137d0 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.739+0000 7fc304ff9640 1 -- 172.21.15.165:0/2402780793 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc4140b5020 con 0x5624061137d0 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.739+0000 7fc304ff9640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc2ec036900 0x7fc2ec038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.739+0000 7fc304ff9640 1 -- 172.21.15.165:0/2402780793 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fc4140a9070 con 0x5624061137d0 2024-04-27T03:30:35.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.739+0000 7fc422463640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc2ec036900 0x7fc2ec038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.739+0000 7fc422463640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc2ec036900 0x7fc2ec038dc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc418001b60 tx=0x7fc418010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.739+0000 7fc426f2a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.739+0000 7fc426f2a800 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562406126340 0x562406128800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.739+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562406128f50 con 0x562406126340 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.739+0000 7fc423465640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562406126340 0x562406128800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc423465640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562406126340 0x562406128800 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562406126340 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc426f2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc426f2a800 20 realm 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc426f2a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562406129360 con 0x562406126340 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562406126340 2024-04-27T03:30:35.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc426f2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc426f2a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc426f2a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562406129c30 con 0x562406126340 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562406126340 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc426f2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc426f2a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.740+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x562406126340 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.741+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562406126340 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.741+0000 7fc426f2a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.741+0000 7fc426f2a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.741+0000 7fc426f2a800 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56240612ad30 0x56240612d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.741+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56240612d730 con 0x56240612ad30 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.741+0000 7fc422c64640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56240612ad30 0x56240612d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.741+0000 7fc422c64640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56240612ad30 0x56240612d120 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.741+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.741+0000 7fc426f2a800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:35.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.741+0000 7fc426f2a800 20 searching for the correct realm 2024-04-27T03:30:35.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.741+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56240612f060 con 0x56240612ad30 2024-04-27T03:30:35.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.742+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.742+0000 7fc422c64640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc4140b7020 0x7fc414091050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.742+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56240612f060 con 0x7fc4140b7020 2024-04-27T03:30:35.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.742+0000 7fc422463640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc4140b7020 0x7fc414091050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.742+0000 7fc422463640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc4140b7020 0x7fc414091050 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.743+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc418009050 con 0x7fc4140b7020 2024-04-27T03:30:35.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.743+0000 7fc422463640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc418038040 0x7fc41801bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.743+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56240612f060 con 0x7fc418038040 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.743+0000 7fc423465640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc418038040 0x7fc41801bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.743+0000 7fc423465640 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc418038040 0x7fc41801bab0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.743+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fc41c00a4a0 con 0x7fc418038040 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.743+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x562406126340 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.744+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562406126340 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.744+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x562406126340 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.744+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562406126340 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.744+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc418038040 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.744+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c00a4a0 con 0x7fc418038040 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.744+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc418038040 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.744+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c00a4a0 con 0x7fc418038040 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.744+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc4140b7020 2024-04-27T03:30:35.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.744+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc418009050 con 0x7fc4140b7020 2024-04-27T03:30:35.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.744+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x56240612ad30 2024-04-27T03:30:35.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.745+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.745+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x562406126340 2024-04-27T03:30:35.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.745+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562406126340 2024-04-27T03:30:35.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.745+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc418038040 2024-04-27T03:30:35.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.745+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c00a4a0 con 0x7fc418038040 2024-04-27T03:30:35.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.745+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x562406126340 2024-04-27T03:30:35.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.746+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562406126340 2024-04-27T03:30:35.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.746+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc4140b7020 2024-04-27T03:30:35.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.746+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc418009050 con 0x7fc4140b7020 2024-04-27T03:30:35.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.746+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x56240612ad30 2024-04-27T03:30:35.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.746+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.746+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x56240612ad30 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.746+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.746+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc418038040 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.747+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c00a4a0 con 0x7fc418038040 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.747+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x56240612ad30 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.747+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.747+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x56240612ad30 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.747+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.747+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x562406126340 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.747+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562406126340 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.747+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x562406126340 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.747+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc41c00a4a0 con 0x562406126340 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.747+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc418038040 2024-04-27T03:30:35.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.748+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c033040 con 0x7fc418038040 2024-04-27T03:30:35.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.748+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x56240612ad30 2024-04-27T03:30:35.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.748+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.748+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc4140b7020 2024-04-27T03:30:35.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.748+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc418009050 con 0x7fc4140b7020 2024-04-27T03:30:35.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.748+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x562406126340 2024-04-27T03:30:35.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.748+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.748+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc418038040 2024-04-27T03:30:35.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.749+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.749+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x56240612ad30 2024-04-27T03:30:35.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.749+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.749+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc418038040 2024-04-27T03:30:35.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.749+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.749+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc4140b7020 2024-04-27T03:30:35.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.750+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc418009050 con 0x7fc4140b7020 2024-04-27T03:30:35.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.750+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x56240612ad30 2024-04-27T03:30:35.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.750+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.750+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc418038040 2024-04-27T03:30:35.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.750+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.750+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x562406126340 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.750+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562406129fd0 con 0x7fc4140b7020 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56240612dff0 con 0x562406126340 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56240612e3e0 con 0x562406126340 2024-04-27T03:30:35.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:35.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.751+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56240612e3e0 con 0x7fc418038040 2024-04-27T03:30:35.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:35.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 20 zone default found 2024-04-27T03:30:35.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 4 Realm: () 2024-04-27T03:30:35.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:35.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:35.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:35.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 20 zonegroup default 2024-04-27T03:30:35.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56240612dc50 con 0x7fc418038040 2024-04-27T03:30:35.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56240612dc50 con 0x56240612ad30 2024-04-27T03:30:35.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.752+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562406130f40 con 0x56240612ad30 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.753+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.753+0000 7fc426f2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.753+0000 7fc426f2a800 20 started sync module instance, tier type = 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.753+0000 7fc426f2a800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.753+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406130360 con 0x7fc4140b7020 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.754+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.754+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94712720721792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56240612ffc0 con 0x7fc4140b7020 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.754+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406131e70 con 0x562406126340 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.755+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94712720721792] v25'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.755+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.755+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94712720726464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406131de0 con 0x562406126340 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.755+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406132d60 con 0x56240612ad30 2024-04-27T03:30:35.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.756+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94712720726464] v25'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.756+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.756+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94712720730288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406133100 con 0x56240612ad30 2024-04-27T03:30:35.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.756+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406133b00 con 0x7fc4140b7020 2024-04-27T03:30:35.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.757+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94712720730288] v25'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.757+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.757+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94712720728720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5624061343a0 con 0x7fc4140b7020 2024-04-27T03:30:35.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.757+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406134d60 con 0x562406126340 2024-04-27T03:30:35.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.758+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.758+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94712720738960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406135520 con 0x562406126340 2024-04-27T03:30:35.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.758+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406136040 con 0x7fc418038040 2024-04-27T03:30:35.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.758+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94712720728720] v25'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.759+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94712720738960] v25'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.759+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.759+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94712720743184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406136000 con 0x7fc418038040 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.759+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5624061368e0 con 0x7fc418038040 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.760+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.760+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94712720745440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406136000 con 0x7fc418038040 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.760+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5624061373f0 con 0x7fc418038040 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.760+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94712720743184] v25'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.761+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94712720745440] v25'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.761+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'126 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.761+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94712720746496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406133ef0 con 0x7fc418038040 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.761+0000 7fc426f2a800 20 add_watcher() i=0 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.761+0000 7fc426f2a800 20 add_watcher() i=1 2024-04-27T03:30:35.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.761+0000 7fc426f2a800 20 add_watcher() i=2 2024-04-27T03:30:35.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.761+0000 7fc426f2a800 20 add_watcher() i=3 2024-04-27T03:30:35.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.761+0000 7fc426f2a800 20 add_watcher() i=4 2024-04-27T03:30:35.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.761+0000 7fc426f2a800 20 add_watcher() i=5 2024-04-27T03:30:35.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.761+0000 7fc426f2a800 20 add_watcher() i=6 2024-04-27T03:30:35.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.762+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94712720746496] v25'127 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.762+0000 7fc426f2a800 20 add_watcher() i=7 2024-04-27T03:30:35.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.762+0000 7fc426f2a800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:35.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.762+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56240613a420 con 0x562406126340 2024-04-27T03:30:35.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.763+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.763+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94712720748288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5624061378a0 con 0x562406126340 2024-04-27T03:30:35.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.763+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94712720748288] v25'127 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.764+0000 7fc2d97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.764+0000 7fc426f2a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.764+0000 7fc426f2a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.764+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406142c10 con 0x562406126340 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.765+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'45 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.765+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5624061432e0 con 0x7fc418038040 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.766+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'21 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.766+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406143700 con 0x56240612ad30 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.767+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'22 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.767+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5624061426b0 con 0x562406126340 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.768+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'128 uv15 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.768+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406143d30 con 0x7fc4140b7020 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.769+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'41 uv17 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.769+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406144750 con 0x56240612ad30 2024-04-27T03:30:35.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.770+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'36 uv16 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.770+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406144750 con 0x562406126340 2024-04-27T03:30:35.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.771+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'23 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.771+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406144750 con 0x7fc418038040 2024-04-27T03:30:35.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.772+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'29 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.772+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406144750 con 0x7fc4140b7020 2024-04-27T03:30:35.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.773+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'15 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.773+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406144cc0 con 0x56240612ad30 2024-04-27T03:30:35.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.774+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'21 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.774+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406144cc0 con 0x7fc4140b7020 2024-04-27T03:30:35.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.775+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'42 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.775+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145230 con 0x7fc418038040 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.776+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'17 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.776+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145230 con 0x7fc4140b7020 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.777+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'21 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.777+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145510 con 0x7fc418038040 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.778+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'41 uv22 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.778+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5624061454e0 con 0x562406126340 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.779+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'129 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.779+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x56240612ad30 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.780+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'36 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.780+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x7fc4140b7020 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.781+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'43 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.781+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x56240612ad30 2024-04-27T03:30:35.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.782+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'27 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.782+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x562406126340 2024-04-27T03:30:35.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.783+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'23 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.783+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x562406126340 2024-04-27T03:30:35.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.784+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'130 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.784+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x56240612ad30 2024-04-27T03:30:35.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.785+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'37 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.785+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x7fc4140b7020 2024-04-27T03:30:35.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.786+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'31 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.786+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x56240612ad30 2024-04-27T03:30:35.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.787+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'23 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.787+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x56240612ad30 2024-04-27T03:30:35.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.788+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'34 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.788+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x56240612ad30 2024-04-27T03:30:35.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.789+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'17 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.789+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x56240612ad30 2024-04-27T03:30:35.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.790+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'37 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.790+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x7fc418038040 2024-04-27T03:30:35.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.791+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'23 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.791+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x56240612ad30 2024-04-27T03:30:35.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.792+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'27 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.792+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x7fc418038040 2024-04-27T03:30:35.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.793+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'30 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.793+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x56240612ad30 2024-04-27T03:30:35.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.795+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'38 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.795+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x7fc418038040 2024-04-27T03:30:35.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.796+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'42 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.796+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406145450 con 0x7fc4140b7020 2024-04-27T03:30:35.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.797+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.797+0000 7fc253fff640 20 reqs_thread_entry: start 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.797+0000 7fc426f2a800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.797+0000 7fc426f2a800 20 Filter name: none 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.797+0000 7fc251ffb640 20 reqs_thread_entry: start 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.797+0000 7fc426f2a800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.797+0000 7fc426f2a800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.797+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5624061703b0 con 0x7fc418038040 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.798+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(94 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.798+0000 7fc426f2a800 20 rados_obj.operate() r=0 bl.length=380 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.798+0000 7fc426f2a800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.798+0000 7fc426f2a800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.798+0000 7fc426f2a800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:35.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.798+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562406172970 con 0x7fc418038040 2024-04-27T03:30:35.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.798+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(95 fud.buckets [call out=15b] v0'0 uv2 ondisk = 0) ==== 155+0+15 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.798+0000 7fc426f2a800 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-04-27T03:30:35.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.798+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:96 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406170970 con 0x7fc418038040 2024-04-27T03:30:35.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.799+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(96 p5YnriCv1nAtykxBrupQ [delete] v25'4 uv4 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.799+0000 7fc426f2a800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T03:30:35.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.799+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [notify cookie 94712720983136 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562406172270 con 0x56240612ad30 2024-04-27T03:30:35.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.799+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== watch-notify(notify (1) cookie 94712720730288 notify 107374182403 ret 0) ==== 212+0+0 (crc 0 0 0) 0x5624061442a0 con 0x56240612ad30 2024-04-27T03:30:35.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.799+0000 7fc3067fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182403 cookie 94712720730288 notifier 4518 bl.length()=170 2024-04-27T03:30:35.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.799+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(97 notify.2 [notify cookie 94712720983136 out=8b] v0'0 uv126 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.799+0000 7fc3067fc640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fc2e8003d60 con 0x56240612ad30 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.800+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 29 ==== watch-notify(notify_complete (2) cookie 94712720983136 notify 107374182403 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fc41407f070 con 0x56240612ad30 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.800+0000 7fc426f2a800 10 removing email index: 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.800+0000 7fc426f2a800 10 removing user buckets index 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.800+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 30 ==== osd_op_reply(98 notify.2 [notify-ack] v0'0 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.800+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:99 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406170460 con 0x7fc418038040 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.801+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(99 fud.buckets [delete] v25'4 uv4 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.801+0000 7fc426f2a800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.801+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:100 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94712720983136 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562406172270 con 0x7fc418038040 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.801+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== watch-notify(notify (1) cookie 94712720745440 notify 107374182406 ret 0) ==== 202+0+0 (crc 0 0 0) 0x562406144cc0 con 0x7fc418038040 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.801+0000 7fc3067fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182406 cookie 94712720745440 notifier 4518 bl.length()=160 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.801+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 30 ==== osd_op_reply(100 notify.6 [notify cookie 94712720983136 out=8b] v0'0 uv126 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.801+0000 7fc3067fc640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fc2e8003d60 con 0x7fc418038040 2024-04-27T03:30:35.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.801+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 31 ==== osd_op_reply(101 notify.6 [notify-ack] v0'0 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.801+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 32 ==== watch-notify(notify_complete (2) cookie 94712720983136 notify 107374182406 ret 0) ==== 42+0+48 (crc 0 0 0) 0x562406107200 con 0x7fc418038040 2024-04-27T03:30:35.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.801+0000 7fc426f2a800 10 removing user index: fud 2024-04-27T03:30:35.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.801+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406172090 con 0x7fc418038040 2024-04-27T03:30:35.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.802+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 33 ==== osd_op_reply(102 fud [call,call,delete] v25'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.802+0000 7fc426f2a800 10 removing default.rgw.meta+users.uid+fud from cache 2024-04-27T03:30:35.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.802+0000 7fc426f2a800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-27T03:30:35.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.802+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:103 7.3 7:c609908c:::notify.5:head [notify cookie 94712720985040 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562406172000 con 0x7fc418038040 2024-04-27T03:30:35.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.803+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 34 ==== watch-notify(notify (1) cookie 94712720743184 notify 107374182407 ret 0) ==== 194+0+0 (crc 0 0 0) 0x562406107200 con 0x7fc418038040 2024-04-27T03:30:35.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.803+0000 7fc305ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182407 cookie 94712720743184 notifier 4518 bl.length()=152 2024-04-27T03:30:35.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.803+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 35 ==== osd_op_reply(103 notify.5 [notify cookie 94712720985040 out=8b] v0'0 uv126 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.803+0000 7fc305ffb640 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:104 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fc2580032c0 con 0x7fc418038040 2024-04-27T03:30:35.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.803+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 36 ==== watch-notify(notify_complete (2) cookie 94712720985040 notify 107374182407 ret 0) ==== 42+0+48 (crc 0 0 0) 0x562406142da0 con 0x7fc418038040 2024-04-27T03:30:35.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.803+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 37 ==== osd_op_reply(104 notify.5 [notify-ack] v0'0 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.804+0000 7fc2d97e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:35.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.804+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:105 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94712720748288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5624061404e0 con 0x562406126340 2024-04-27T03:30:35.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.805+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(105 data_loggenerations_metadata [watch unwatch cookie 94712720748288] v25'131 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.805+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:106 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94712720721792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406136f00 con 0x7fc4140b7020 2024-04-27T03:30:35.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.806+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(106 notify.0 [watch unwatch cookie 94712720721792] v25'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.806+0000 7fc426f2a800 20 remove_watcher() i=0 2024-04-27T03:30:35.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.806+0000 7fc426f2a800 2 removed watcher, disabling cache 2024-04-27T03:30:35.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.806+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:107 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94712720726464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406130780 con 0x562406126340 2024-04-27T03:30:35.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.807+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(107 notify.1 [watch unwatch cookie 94712720726464] v25'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.807+0000 7fc426f2a800 20 remove_watcher() i=1 2024-04-27T03:30:35.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.807+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94712720730288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406130780 con 0x56240612ad30 2024-04-27T03:30:35.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.808+0000 7fc422c64640 1 -- 172.21.15.165:0/2402780793 <== osd.3 v2:172.21.15.165:6808/2380918637 31 ==== osd_op_reply(108 notify.2 [watch unwatch cookie 94712720730288] v25'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc414097070 con 0x56240612ad30 2024-04-27T03:30:35.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.808+0000 7fc426f2a800 20 remove_watcher() i=2 2024-04-27T03:30:35.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.808+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94712720728720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5624061328b0 con 0x7fc4140b7020 2024-04-27T03:30:35.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.809+0000 7fc422463640 1 -- 172.21.15.165:0/2402780793 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(109 notify.3 [watch unwatch cookie 94712720728720] v25'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc418045040 con 0x7fc4140b7020 2024-04-27T03:30:35.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.809+0000 7fc426f2a800 20 remove_watcher() i=3 2024-04-27T03:30:35.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.809+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:110 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94712720738960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406130780 con 0x562406126340 2024-04-27T03:30:35.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.811+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(110 notify.4 [watch unwatch cookie 94712720738960] v25'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x562406126340 2024-04-27T03:30:35.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.811+0000 7fc426f2a800 20 remove_watcher() i=4 2024-04-27T03:30:35.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.811+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94712720743184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406132290 con 0x7fc418038040 2024-04-27T03:30:35.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.812+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 38 ==== osd_op_reply(111 notify.5 [watch unwatch cookie 94712720743184] v25'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.812+0000 7fc426f2a800 20 remove_watcher() i=5 2024-04-27T03:30:35.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.812+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94712720745440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406134a90 con 0x7fc418038040 2024-04-27T03:30:35.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.813+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 39 ==== osd_op_reply(112 notify.6 [watch unwatch cookie 94712720745440] v25'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.813+0000 7fc426f2a800 20 remove_watcher() i=6 2024-04-27T03:30:35.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.813+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:113 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94712720746496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562406130780 con 0x7fc418038040 2024-04-27T03:30:35.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.813+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 <== osd.0 v2:172.21.15.89:6800/2353668432 40 ==== osd_op_reply(113 notify.7 [watch unwatch cookie 94712720746496] v25'128 uv126 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc41c031040 con 0x7fc418038040 2024-04-27T03:30:35.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 20 remove_watcher() i=7 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc418038040 msgr2=0x7fc41801bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc418038040 0x7fc41801bab0 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562406126340 msgr2=0x562406128800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562406126340 0x562406128800 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc4140b7020 msgr2=0x7fc414091050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc4140b7020 0x7fc414091050 crc :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56240612ad30 msgr2=0x56240612d120 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56240612ad30 0x56240612d120 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc2ec036900 msgr2=0x7fc2ec038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc2ec036900 0x7fc2ec038dc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fc418001b60 tx=0x7fc418010000 comp rx=0 tx=0).stop 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 msgr2=0x56240611ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc423465640 1 -- 172.21.15.165:0/2402780793 reap_dead start 2024-04-27T03:30:35.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 0x56240611ac20 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fc414060e30 tx=0x7fc414060da0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 shutdown_connections 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 --2- 172.21.15.165:0/2402780793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061137d0 0x56240611ac20 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 >> 172.21.15.165:0/2402780793 conn(0x56240610fb30 msgr2=0x56240610d190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 shutdown_connections 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/2402780793 wait complete. 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/984699701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc2fc036950 msgr2=0x7fc2fc038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 --2- 172.21.15.165:0/984699701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc2fc036950 0x7fc2fc038e10 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc41800bf50 tx=0x7fc41800f000 comp rx=0 tx=0).stop 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/984699701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061068a0 msgr2=0x562406106cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 --2- 172.21.15.165:0/984699701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061068a0 0x562406106cc0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7fc41408c520 tx=0x7fc4140098c0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/984699701 shutdown_connections 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 --2- 172.21.15.165:0/984699701 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc2fc036950 0x7fc2fc038e10 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 --2- 172.21.15.165:0/984699701 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5624061068a0 0x562406106cc0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/984699701 >> 172.21.15.165:0/984699701 conn(0x5624060fa940 msgr2=0x5624060fad40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/984699701 shutdown_connections 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.814+0000 7fc426f2a800 1 -- 172.21.15.165:0/984699701 wait complete. 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5624060f5870 msgr2=0x5624060f7ce0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5624060f5870 0x5624060f7ce0 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562405fec120 msgr2=0x5624060ef0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562405fec120 0x5624060ef0e0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5624060f1980 msgr2=0x5624060f3d70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5624060f1980 0x5624060f3d70 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc308036950 msgr2=0x7fc308038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc308036950 0x7fc308038e10 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fc418000f60 tx=0x7fc4180012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 msgr2=0x5624060e7f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 --2- 172.21.15.165:0/490418060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562405feb7c0 0x5624060e7f40 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fc41401ce60 tx=0x7fc414002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc423465640 1 -- 172.21.15.165:0/490418060 reap_dead start 2024-04-27T03:30:35.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 shutdown_connections 2024-04-27T03:30:35.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 >> 172.21.15.165:0/490418060 conn(0x562405fe7320 msgr2=0x5624060da520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 shutdown_connections 2024-04-27T03:30:35.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.815+0000 7fc426f2a800 1 -- 172.21.15.165:0/490418060 wait complete. 2024-04-27T03:30:35.828 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-27T03:30:35.829 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--tenant', 'testx', '--uid', 'tenanteduser', '--display-name', 'tenanted-user', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '1'] 2024-04-27T03:30:35.829 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --tenant testx --uid tenanteduser --display-name tenanted-user --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 1 2024-04-27T03:30:35.922 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:35.922 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:35.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.917+0000 7f5253612800 1 Processor -- start 2024-04-27T03:30:35.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.917+0000 7f5253612800 1 -- start start 2024-04-27T03:30:35.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.917+0000 7f5253612800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55806d5072d0 0x55806d5096c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.917+0000 7f5253612800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55806d3c1d40 con 0x55806d502f30 2024-04-27T03:30:35.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.917+0000 7f5253612800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55806d3944d0 con 0x55806d5072d0 2024-04-27T03:30:35.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.918+0000 7f524e3ed640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55806d5072d0 0x55806d5096c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.918+0000 7f524ebee640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55806d502f30 0x55806d503350 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35366/0 (socket says 172.21.15.165:35366) 2024-04-27T03:30:35.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.918+0000 7f524ebee640 1 -- 172.21.15.165:0/786725507 learned_addr learned my addr 172.21.15.165:0/786725507 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:35.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.918+0000 7f524e3ed640 1 -- 172.21.15.165:0/786725507 >> v1:172.21.15.89:6789/0 conn(0x55806d502f30 legacy=0x55806d503350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.918+0000 7f524e3ed640 1 -- 172.21.15.165:0/786725507 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806d3c0590 con 0x55806d5072d0 2024-04-27T03:30:35.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.920+0000 7f524e3ed640 1 --2- 172.21.15.165:0/786725507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d5072d0 0x55806d5096c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f524400a470 tx=0x7f524401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38755e79614638d5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.920+0000 7f524dbec640 1 -- 172.21.15.165:0/786725507 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5244057070 con 0x55806d5072d0 2024-04-27T03:30:35.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.920+0000 7f524dbec640 1 -- 172.21.15.165:0/786725507 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5244052570 con 0x55806d5072d0 2024-04-27T03:30:35.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.920+0000 7f524dbec640 1 -- 172.21.15.165:0/786725507 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5244052870 con 0x55806d5072d0 2024-04-27T03:30:35.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.920+0000 7f5253612800 1 -- 172.21.15.165:0/786725507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d5072d0 msgr2=0x55806d5096c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.920+0000 7f5253612800 1 --2- 172.21.15.165:0/786725507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d5072d0 0x55806d5096c0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f524400a470 tx=0x7f524401ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:35.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.920+0000 7f5253612800 1 -- 172.21.15.165:0/786725507 shutdown_connections 2024-04-27T03:30:35.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.920+0000 7f5253612800 1 --2- 172.21.15.165:0/786725507 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d5072d0 0x55806d5096c0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.920+0000 7f5253612800 1 -- 172.21.15.165:0/786725507 >> 172.21.15.165:0/786725507 conn(0x55806d4fe4d0 msgr2=0x55806d5008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.920+0000 7f5253612800 1 -- 172.21.15.165:0/786725507 shutdown_connections 2024-04-27T03:30:35.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.921+0000 7f5253612800 1 -- 172.21.15.165:0/786725507 wait complete. 2024-04-27T03:30:35.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.932+0000 7f5253612800 1 Processor -- start 2024-04-27T03:30:35.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.932+0000 7f5253612800 1 -- start start 2024-04-27T03:30:35.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.932+0000 7f5253612800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 0x55806d5f4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.933+0000 7f5253612800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55806d44e360 con 0x55806d502f30 2024-04-27T03:30:35.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.933+0000 7f524ebee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 0x55806d5f4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.933+0000 7f524ebee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 0x55806d5f4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58958/0 (socket says 172.21.15.165:58958) 2024-04-27T03:30:35.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.933+0000 7f524ebee640 1 -- 172.21.15.165:0/800832479 learned_addr learned my addr 172.21.15.165:0/800832479 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.933+0000 7f524ebee640 1 -- 172.21.15.165:0/800832479 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806d5f4720 con 0x55806d502f30 2024-04-27T03:30:35.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.933+0000 7f524ebee640 1 --2- 172.21.15.165:0/800832479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 0x55806d5f4150 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5240001e10 tx=0x7f524000b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=832998ef334ad42b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.933+0000 7f513ce09640 1 -- 172.21.15.165:0/800832479 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5240012020 con 0x55806d502f30 2024-04-27T03:30:35.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.933+0000 7f513ce09640 1 -- 172.21.15.165:0/800832479 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524000e5a0 con 0x55806d502f30 2024-04-27T03:30:35.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.933+0000 7f513ce09640 1 -- 172.21.15.165:0/800832479 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f524000e8e0 con 0x55806d502f30 2024-04-27T03:30:35.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.934+0000 7f5253612800 1 -- 172.21.15.165:0/800832479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 msgr2=0x55806d5f4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.934+0000 7f5253612800 1 --2- 172.21.15.165:0/800832479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 0x55806d5f4150 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f5240001e10 tx=0x7f524000b780 comp rx=0 tx=0).stop 2024-04-27T03:30:35.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.934+0000 7f5253612800 1 -- 172.21.15.165:0/800832479 shutdown_connections 2024-04-27T03:30:35.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.934+0000 7f5253612800 1 --2- 172.21.15.165:0/800832479 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 0x55806d5f4150 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.934+0000 7f5253612800 1 -- 172.21.15.165:0/800832479 >> 172.21.15.165:0/800832479 conn(0x55806d4fe4d0 msgr2=0x55806d5efa70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.934+0000 7f5253612800 1 -- 172.21.15.165:0/800832479 shutdown_connections 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.934+0000 7f5253612800 1 -- 172.21.15.165:0/800832479 wait complete. 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.934+0000 7f5253612800 1 Processor -- start 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.934+0000 7f5253612800 1 -- start start 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.935+0000 7f5253612800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 0x55806d5fbd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.935+0000 7f5253612800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55806d509f90 con 0x55806d502f30 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.935+0000 7f524ebee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 0x55806d5fbd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.935+0000 7f524ebee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 0x55806d5fbd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58968/0 (socket says 172.21.15.165:58968) 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.935+0000 7f524ebee640 1 -- 172.21.15.165:0/1583550930 learned_addr learned my addr 172.21.15.165:0/1583550930 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.935+0000 7f524ebee640 1 -- 172.21.15.165:0/1583550930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806d5fc240 con 0x55806d502f30 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.935+0000 7f524ebee640 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 0x55806d5fbd00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f524000b7c0 tx=0x7f5240000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.935+0000 7f513b606640 1 -- 172.21.15.165:0/1583550930 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5240012030 con 0x55806d502f30 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.935+0000 7f513b606640 1 -- 172.21.15.165:0/1583550930 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5240001470 con 0x55806d502f30 2024-04-27T03:30:35.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.935+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55806d5fc4d0 con 0x55806d502f30 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.935+0000 7f513b606640 1 -- 172.21.15.165:0/1583550930 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f524001a8d0 con 0x55806d502f30 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.936+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55806d5ff180 con 0x55806d502f30 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.936+0000 7f513b606640 1 -- 172.21.15.165:0/1583550930 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f5240015020 con 0x55806d502f30 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.936+0000 7f513b606640 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f51340369a0 0x7f5134038e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.936+0000 7f513b606640 1 -- 172.21.15.165:0/1583550930 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f5240020030 con 0x55806d502f30 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.937+0000 7f524e3ed640 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f51340369a0 0x7f5134038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.937+0000 7f5253612800 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55806d5072d0 0x55806d602f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.937+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d603480 con 0x55806d5072d0 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.937+0000 7f524f3ef640 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55806d5072d0 0x55806d602f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.937+0000 7f524e3ed640 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f51340369a0 0x7f5134038e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5244054000 tx=0x7f524404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.937+0000 7f524f3ef640 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55806d5072d0 0x55806d602f40 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.937+0000 7f524f3ef640 1 -- 172.21.15.165:0/1583550930 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d5072d0 2024-04-27T03:30:35.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.937+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d600510 con 0x55806d5072d0 2024-04-27T03:30:35.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.938+0000 7f524f3ef640 1 -- 172.21.15.165:0/1583550930 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d5072d0 2024-04-27T03:30:35.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.938+0000 7f5253612800 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55806d6057d0 0x55806d607bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.938+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d6082b0 con 0x55806d6057d0 2024-04-27T03:30:35.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.938+0000 7f524ebee640 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55806d6057d0 0x55806d607bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.938+0000 7f524ebee640 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55806d6057d0 0x55806d607bc0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.938+0000 7f524ebee640 1 -- 172.21.15.165:0/1583550930 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f5240018950 con 0x55806d6057d0 2024-04-27T03:30:35.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.938+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d604780 con 0x55806d5072d0 2024-04-27T03:30:35.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.939+0000 7f524f3ef640 1 -- 172.21.15.165:0/1583550930 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d5072d0 2024-04-27T03:30:35.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.939+0000 7f5253612800 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55806d609760 0x55806d60bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.939+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d60c310 con 0x55806d609760 2024-04-27T03:30:35.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.939+0000 7f524e3ed640 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55806d609760 0x55806d60bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.939+0000 7f524e3ed640 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55806d609760 0x55806d60bbb0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.940+0000 7f524e3ed640 1 -- 172.21.15.165:0/1583550930 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f5244052a40 con 0x55806d609760 2024-04-27T03:30:35.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.940+0000 7f5253612800 1 Processor -- start 2024-04-27T03:30:35.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.940+0000 7f5253612800 1 -- start start 2024-04-27T03:30:35.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.940+0000 7f5253612800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d478ba0 0x55806d477010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.940+0000 7f5253612800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55806d3f46b0 con 0x55806d478ba0 2024-04-27T03:30:35.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.940+0000 7f524ebee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d478ba0 0x55806d477010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.940+0000 7f524ebee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d478ba0 0x55806d477010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58972/0 (socket says 172.21.15.165:58972) 2024-04-27T03:30:35.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.940+0000 7f524ebee640 1 -- 172.21.15.165:0/2927526948 learned_addr learned my addr 172.21.15.165:0/2927526948 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.940+0000 7f524ebee640 1 -- 172.21.15.165:0/2927526948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806d477550 con 0x55806d478ba0 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.941+0000 7f524ebee640 1 --2- 172.21.15.165:0/2927526948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d478ba0 0x55806d477010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5240000fd0 tx=0x7f524000db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.941+0000 7f5138e01640 1 -- 172.21.15.165:0/2927526948 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f524000d510 con 0x55806d478ba0 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.941+0000 7f5138e01640 1 -- 172.21.15.165:0/2927526948 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524003b070 con 0x55806d478ba0 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.941+0000 7f5253612800 1 -- 172.21.15.165:0/2927526948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55806d4777e0 con 0x55806d478ba0 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.941+0000 7f5138e01640 1 -- 172.21.15.165:0/2927526948 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5240044a90 con 0x55806d478ba0 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.941+0000 7f5253612800 1 -- 172.21.15.165:0/2927526948 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55806d477ce0 con 0x55806d478ba0 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.942+0000 7f5138e01640 1 -- 172.21.15.165:0/2927526948 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f5240044c30 con 0x55806d478ba0 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.942+0000 7f5138e01640 1 --2- 172.21.15.165:0/2927526948 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5128036900 0x7f5128038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.942+0000 7f5138e01640 1 -- 172.21.15.165:0/2927526948 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f5240037030 con 0x55806d478ba0 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.942+0000 7f524e3ed640 1 --2- 172.21.15.165:0/2927526948 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5128036900 0x7f5128038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.942+0000 7f5253612800 1 Processor -- start 2024-04-27T03:30:35.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.942+0000 7f5253612800 1 -- start start 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.942+0000 7f524e3ed640 1 --2- 172.21.15.165:0/2927526948 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5128036900 0x7f5128038dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55806d603d80 tx=0x7f5244005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.942+0000 7f5253612800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 0x55806d629ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.942+0000 7f5253612800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55806d5f4aa0 con 0x55806d629ad0 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.943+0000 7f524ebee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 0x55806d629ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.943+0000 7f524ebee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 0x55806d629ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58986/0 (socket says 172.21.15.165:58986) 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.943+0000 7f524ebee640 1 -- 172.21.15.165:0/3722318679 learned_addr learned my addr 172.21.15.165:0/3722318679 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.943+0000 7f524ebee640 1 -- 172.21.15.165:0/3722318679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806d62a4c0 con 0x55806d629ad0 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.943+0000 7f524ebee640 1 --2- 172.21.15.165:0/3722318679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 0x55806d629ef0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f524004c010 tx=0x7f5240016d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=756752c2cf39760a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.943+0000 7f51327fc640 1 -- 172.21.15.165:0/3722318679 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5240036d60 con 0x55806d629ad0 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.943+0000 7f51327fc640 1 -- 172.21.15.165:0/3722318679 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f52400474a0 con 0x55806d629ad0 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.943+0000 7f51327fc640 1 -- 172.21.15.165:0/3722318679 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f524001c0e0 con 0x55806d629ad0 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.944+0000 7f5253612800 1 -- 172.21.15.165:0/3722318679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 msgr2=0x55806d629ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:35.970 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.944+0000 7f5253612800 1 --2- 172.21.15.165:0/3722318679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 0x55806d629ef0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f524004c010 tx=0x7f5240016d30 comp rx=0 tx=0).stop 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.944+0000 7f5253612800 1 -- 172.21.15.165:0/3722318679 shutdown_connections 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.944+0000 7f5253612800 1 --2- 172.21.15.165:0/3722318679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 0x55806d629ef0 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.944+0000 7f5253612800 1 -- 172.21.15.165:0/3722318679 >> 172.21.15.165:0/3722318679 conn(0x55806d625da0 msgr2=0x55806d623e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.944+0000 7f5253612800 1 -- 172.21.15.165:0/3722318679 shutdown_connections 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.944+0000 7f5253612800 1 -- 172.21.15.165:0/3722318679 wait complete. 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.944+0000 7f5253612800 1 Processor -- start 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.944+0000 7f5253612800 1 -- start start 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.944+0000 7f5253612800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 0x55806d630e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.944+0000 7f5253612800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55806d62ab90 con 0x55806d629ad0 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.944+0000 7f524ebee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 0x55806d630e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.945+0000 7f524ebee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 0x55806d630e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59002/0 (socket says 172.21.15.165:59002) 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.945+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 learned_addr learned my addr 172.21.15.165:0/1580471043 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:35.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.945+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55806d6313d0 con 0x55806d629ad0 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.945+0000 7f524ebee640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 0x55806d630e90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f524004c010 tx=0x7f5240032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.945+0000 7f5130ff9640 1 -- 172.21.15.165:0/1580471043 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f524002d730 con 0x55806d629ad0 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.945+0000 7f5130ff9640 1 -- 172.21.15.165:0/1580471043 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f524002d8d0 con 0x55806d629ad0 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.945+0000 7f5130ff9640 1 -- 172.21.15.165:0/1580471043 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5240024420 con 0x55806d629ad0 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.945+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55806d6313d0 con 0x55806d629ad0 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.945+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55806d6313d0 con 0x55806d629ad0 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.946+0000 7f5130ff9640 1 -- 172.21.15.165:0/1580471043 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f5240072030 con 0x55806d629ad0 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.946+0000 7f5130ff9640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5118036900 0x7f5118038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.946+0000 7f5130ff9640 1 -- 172.21.15.165:0/1580471043 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f5240070070 con 0x55806d629ad0 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.946+0000 7f524e3ed640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5118036900 0x7f5118038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f5253612800 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55806d63c5c0 0x55806d63ea80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d63f1d0 con 0x55806d63c5c0 2024-04-27T03:30:35.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f524e3ed640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5118036900 0x7f5118038dc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5244052fd0 tx=0x7f524404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f524f3ef640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55806d63c5c0 0x55806d63ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f524f3ef640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55806d63c5c0 0x55806d63ea80 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d63c5c0 2024-04-27T03:30:35.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f5253612800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f5253612800 20 realm 2024-04-27T03:30:35.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d63f5e0 con 0x55806d63c5c0 2024-04-27T03:30:35.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d63c5c0 2024-04-27T03:30:35.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f5253612800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f5253612800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:35.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.947+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.948+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d63feb0 con 0x55806d63c5c0 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.948+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d63c5c0 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.948+0000 7f5253612800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.948+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.948+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d640250 con 0x55806d63c5c0 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.948+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d63c5c0 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.948+0000 7f5253612800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.948+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.948+0000 7f5253612800 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55806d640fb0 0x55806d6433a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.948+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d6439b0 con 0x55806d640fb0 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.948+0000 7f524ebee640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55806d640fb0 0x55806d6433a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.948+0000 7f524ebee640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55806d640fb0 0x55806d6433a0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.974 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.949+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.949+0000 7f5253612800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.949+0000 7f5253612800 20 searching for the correct realm 2024-04-27T03:30:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.949+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d6452e0 con 0x55806d640fb0 2024-04-27T03:30:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.949+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.949+0000 7f524ebee640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f524007f3f0 0x7f5240057880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.949+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d6452e0 con 0x7f524007f3f0 2024-04-27T03:30:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.949+0000 7f524e3ed640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f524007f3f0 0x7f5240057880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.950+0000 7f524e3ed640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f524007f3f0 0x7f5240057880 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.950+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.950+0000 7f524e3ed640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f5244076040 0x7f5244083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.950+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d6452e0 con 0x7f5244076040 2024-04-27T03:30:35.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.950+0000 7f524f3ef640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f5244076040 0x7f5244083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:35.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.950+0000 7f524f3ef640 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f5244076040 0x7f5244083410 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:35.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.951+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f524800a4a0 con 0x7f5244076040 2024-04-27T03:30:35.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.951+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d640250 con 0x55806d63c5c0 2024-04-27T03:30:35.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.951+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d63c5c0 2024-04-27T03:30:35.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.951+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d640250 con 0x55806d63c5c0 2024-04-27T03:30:35.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.951+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d63c5c0 2024-04-27T03:30:35.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.951+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d640250 con 0x7f5244076040 2024-04-27T03:30:35.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.951+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f524800a4a0 con 0x7f5244076040 2024-04-27T03:30:35.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.951+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d640250 con 0x7f5244076040 2024-04-27T03:30:35.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.951+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f524800a4a0 con 0x7f5244076040 2024-04-27T03:30:35.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.951+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d640250 con 0x7f524007f3f0 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.952+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.952+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d640250 con 0x55806d640fb0 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.952+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.952+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d640250 con 0x55806d63c5c0 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.952+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d63c5c0 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.952+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d640250 con 0x7f5244076040 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.952+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f524800a4a0 con 0x7f5244076040 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.952+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d640250 con 0x55806d63c5c0 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.953+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d63c5c0 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.953+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55806d640250 con 0x7f524007f3f0 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.953+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.953+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x55806d640fb0 2024-04-27T03:30:35.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.953+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.953+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x55806d640fb0 2024-04-27T03:30:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.953+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.953+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x7f5244076040 2024-04-27T03:30:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.954+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f524800a4a0 con 0x7f5244076040 2024-04-27T03:30:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.954+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x55806d640fb0 2024-04-27T03:30:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.954+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.954+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x55806d640fb0 2024-04-27T03:30:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.954+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.954+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x55806d63c5c0 2024-04-27T03:30:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.954+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d63c5c0 2024-04-27T03:30:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.954+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x55806d63c5c0 2024-04-27T03:30:35.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.955+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f524800a4a0 con 0x55806d63c5c0 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.955+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x7f5244076040 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.955+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5248033040 con 0x7f5244076040 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.955+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x55806d640fb0 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.955+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.955+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x7f524007f3f0 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.956+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.956+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x55806d63c5c0 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.956+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.956+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x7f5244076040 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.956+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.956+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x55806d640fb0 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.957+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.979 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.957+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x7f5244076040 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.957+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.957+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x7f524007f3f0 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.957+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.957+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x55806d640fb0 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.957+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.957+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x7f5244076040 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x55806d63c5c0 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f5244090060 con 0x7f524007f3f0 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f5253612800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f5253612800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:35.980 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f5253612800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f5253612800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f5253612800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f5253612800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d644270 con 0x55806d63c5c0 2024-04-27T03:30:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f5253612800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.981 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.958+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d644660 con 0x55806d63c5c0 2024-04-27T03:30:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d644660 con 0x7f5244076040 2024-04-27T03:30:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 20 zone default found 2024-04-27T03:30:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 4 Realm: () 2024-04-27T03:30:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:35.982 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:35.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:35.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 20 zonegroup default 2024-04-27T03:30:35.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d643df0 con 0x7f5244076040 2024-04-27T03:30:35.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d643df0 con 0x55806d640fb0 2024-04-27T03:30:35.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.959+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.960+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d6471e0 con 0x55806d640fb0 2024-04-27T03:30:35.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.960+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.960+0000 7f5253612800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.960+0000 7f5253612800 20 started sync module instance, tier type = 2024-04-27T03:30:35.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.960+0000 7f5253612800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:35.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.960+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d6465e0 con 0x7f524007f3f0 2024-04-27T03:30:35.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.961+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:35.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.961+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94010079472080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d646240 con 0x7f524007f3f0 2024-04-27T03:30:35.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.961+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d648110 con 0x55806d63c5c0 2024-04-27T03:30:35.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.962+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94010079472080] v25'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:35.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.962+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:35.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.962+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94010079476832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d648530 con 0x55806d63c5c0 2024-04-27T03:30:35.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.962+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d648ef0 con 0x55806d640fb0 2024-04-27T03:30:35.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.963+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94010079476832] v25'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:35.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.963+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.963+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94010079477888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d6484a0 con 0x55806d640fb0 2024-04-27T03:30:35.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.963+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d649d60 con 0x7f524007f3f0 2024-04-27T03:30:35.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.964+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94010079477888] v25'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:35.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.965+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:35.995 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.965+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94010079482512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64a600 con 0x7f524007f3f0 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.965+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64afc0 con 0x55806d63c5c0 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.966+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94010079482512] v25'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.966+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.966+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94010079488784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64b3e0 con 0x55806d63c5c0 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.966+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64bda0 con 0x7f5244076040 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.967+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94010079488784] v25'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.967+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.967+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94010079490320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64bda0 con 0x7f5244076040 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.967+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64c760 con 0x7f5244076040 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.968+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.968+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94010079494832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64cb80 con 0x7f5244076040 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.968+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64d640 con 0x7f5244076040 2024-04-27T03:30:35.996 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.969+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94010079490320] v25'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.969+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94010079494832] v25'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.969+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'129 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.969+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94010079498608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64c6d0 con 0x7f5244076040 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.969+0000 7f5253612800 20 add_watcher() i=0 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.969+0000 7f5253612800 20 add_watcher() i=1 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.969+0000 7f5253612800 20 add_watcher() i=2 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.969+0000 7f5253612800 20 add_watcher() i=3 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.969+0000 7f5253612800 20 add_watcher() i=4 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.969+0000 7f5253612800 20 add_watcher() i=5 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.969+0000 7f5253612800 20 add_watcher() i=6 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.970+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94010079498608] v25'130 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.970+0000 7f5253612800 20 add_watcher() i=7 2024-04-27T03:30:35.997 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.970+0000 7f5253612800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.970+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d650670 con 0x55806d63c5c0 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.970+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.970+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94010079492336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64daf0 con 0x55806d63c5c0 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.971+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94010079492336] v25'132 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.971+0000 7f51057e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.972+0000 7f5253612800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.972+0000 7f5253612800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.972+0000 7f5253612800 5 note: GC not initialized 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.972+0000 7f507ffff640 20 reqs_thread_entry: start 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.972+0000 7f5253612800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.972+0000 7f5253612800 20 Filter name: none 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.972+0000 7f507dffb640 20 reqs_thread_entry: start 2024-04-27T03:30:35.998 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.972+0000 7f5253612800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.972+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.972+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d683ab0 con 0x7f5244076040 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(62 testx$tenanteduser [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 246+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f5253612800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f5253612800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x0 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f5253612800 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f5253612800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f5253612800 20 rados->read ofs=0 len=0 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d6821a0 con 0x7f5244076040 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f5253612800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f5253612800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f5253612800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T03:30:35.999 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f5253612800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f5253612800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.973+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:64 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [delete,create,call version.set in=58b,writefull 0~419 in=419b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d686810 con 0x7f5244076040 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.974+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(64 testx$tenanteduser [delete,create,call,writefull 0~419] v25'5 uv5 ondisk = 0) ==== 288+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.974+0000 7f5253612800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.974+0000 7f5253612800 10 moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.974+0000 7f5253612800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.974+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 7.e 7:7759931f:::notify.3:head [notify cookie 94010079732320 in=622b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d687380 con 0x7f524007f3f0 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.975+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== watch-notify(notify (1) cookie 94010079482512 notify 107374182412 ret 0) ==== 652+0+0 (crc 0 0 0) 0x7f524408a040 con 0x7f524007f3f0 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.975+0000 7f51327fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182412 cookie 94010079482512 notifier 4525 bl.length()=610 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.975+0000 7f51327fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x17 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.975+0000 7f51327fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.975+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(65 notify.3 [notify cookie 94010079732320 out=8b] v0'0 uv129 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:36.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.975+0000 7f51327fc640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f5114003ea0 con 0x7f524007f3f0 2024-04-27T03:30:36.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.975+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(66 notify.3 [notify-ack] v0'0 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:36.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.975+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== watch-notify(notify_complete (2) cookie 94010079732320 notify 107374182412 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55806d3c0590 con 0x7f524007f3f0 2024-04-27T03:30:36.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.975+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d686260 con 0x7f5244076040 2024-04-27T03:30:36.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.976+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~22] v25'6 uv6 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:36.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.976+0000 7f5253612800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T03:30:36.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.976+0000 7f5253612800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T03:30:36.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.976+0000 7f5253612800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T03:30:36.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.976+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94010079732320 in=204b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55806d685640 con 0x55806d640fb0 2024-04-27T03:30:36.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.977+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== watch-notify(notify (1) cookie 94010079477888 notify 107374182404 ret 0) ==== 234+0+0 (crc 0 0 0) 0x55806d4777e0 con 0x55806d640fb0 2024-04-27T03:30:36.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.977+0000 7f5131ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182404 cookie 94010079477888 notifier 4525 bl.length()=192 2024-04-27T03:30:36.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.977+0000 7f5131ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T03:30:36.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.977+0000 7f5131ffb640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T03:30:36.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.977+0000 7f5131ffb640 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f5084003400 con 0x55806d640fb0 2024-04-27T03:30:36.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.977+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(68 notify.2 [notify cookie 94010079732320 out=8b] v0'0 uv129 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:36.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.977+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== watch-notify(notify_complete (2) cookie 94010079732320 notify 107374182404 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55806d477550 con 0x55806d640fb0 2024-04-27T03:30:36.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.977+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:36.002 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"testx$tenanteduser","display_name":"tenanted-user","email":"","suspended":0,"max_buckets":1,"subusers":[],"keys":[{"user":"testx$tenanteduser","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-04-27T03:30:35.974388Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:35.974369Z","tags":[],"group_ids":[]} 2024-04-27T03:30:36.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.978+0000 7f51057e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.978+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94010079492336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d6825f0 con 0x55806d63c5c0 2024-04-27T03:30:36.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.979+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94010079492336] v25'133 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:36.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.979+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94010079472080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64b8f0 con 0x7f524007f3f0 2024-04-27T03:30:36.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.980+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94010079472080] v25'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:36.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.980+0000 7f5253612800 20 remove_watcher() i=0 2024-04-27T03:30:36.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.980+0000 7f5253612800 2 removed watcher, disabling cache 2024-04-27T03:30:36.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.980+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94010079476832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64b8f0 con 0x55806d63c5c0 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.981+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94010079476832] v25'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.981+0000 7f5253612800 20 remove_watcher() i=1 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.981+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94010079477888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d647c60 con 0x55806d640fb0 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.982+0000 7f524ebee640 1 -- 172.21.15.165:0/1580471043 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94010079477888] v25'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5240018950 con 0x55806d640fb0 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.982+0000 7f5253612800 20 remove_watcher() i=2 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.982+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94010079482512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64b8f0 con 0x7f524007f3f0 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.983+0000 7f524e3ed640 1 -- 172.21.15.165:0/1580471043 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94010079482512] v25'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5244081070 con 0x7f524007f3f0 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.983+0000 7f5253612800 20 remove_watcher() i=3 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.983+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94010079488784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d649290 con 0x55806d63c5c0 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.984+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94010079488784] v25'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x55806d63c5c0 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.984+0000 7f5253612800 20 remove_watcher() i=4 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.984+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94010079490320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64b8f0 con 0x7f5244076040 2024-04-27T03:30:36.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.985+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94010079490320] v25'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.985+0000 7f5253612800 20 remove_watcher() i=5 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.985+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94010079494832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64ab10 con 0x7f5244076040 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.986+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94010079494832] v25'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.986+0000 7f5253612800 20 remove_watcher() i=6 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.986+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94010079498608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55806d64b110 con 0x7f5244076040 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94010079498608] v25'131 uv129 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f5248031040 con 0x7f5244076040 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 20 remove_watcher() i=7 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f5244076040 msgr2=0x7f5244083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f5244076040 0x7f5244083410 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55806d63c5c0 msgr2=0x55806d63ea80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55806d63c5c0 0x55806d63ea80 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f524007f3f0 msgr2=0x7f5240057880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f524007f3f0 0x7f5240057880 crc :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55806d640fb0 msgr2=0x55806d6433a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55806d640fb0 0x55806d6433a0 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5118036900 msgr2=0x7f5118038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5118036900 0x7f5118038dc0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f5244052fd0 tx=0x7f524404f000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 msgr2=0x55806d630e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f524f3ef640 1 -- 172.21.15.165:0/1580471043 reap_dead start 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.987+0000 7f5253612800 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 0x55806d630e90 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f524004c010 tx=0x7f5240032eb0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 shutdown_connections 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 --2- 172.21.15.165:0/1580471043 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d629ad0 0x55806d630e90 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 >> 172.21.15.165:0/1580471043 conn(0x55806d625da0 msgr2=0x55806d623400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 shutdown_connections 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/1580471043 wait complete. 2024-04-27T03:30:36.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/2927526948 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5128036900 msgr2=0x7f5128038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 --2- 172.21.15.165:0/2927526948 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5128036900 0x7f5128038dc0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x55806d603d80 tx=0x7f5244005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:36.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/2927526948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d478ba0 msgr2=0x55806d477010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 --2- 172.21.15.165:0/2927526948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d478ba0 0x55806d477010 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f5240000fd0 tx=0x7f524000db20 comp rx=0 tx=0).stop 2024-04-27T03:30:36.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/2927526948 shutdown_connections 2024-04-27T03:30:36.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 --2- 172.21.15.165:0/2927526948 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5128036900 0x7f5128038dc0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 --2- 172.21.15.165:0/2927526948 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d478ba0 0x55806d477010 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/2927526948 >> 172.21.15.165:0/2927526948 conn(0x55806d60e7d0 msgr2=0x55806d60ebd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/2927526948 shutdown_connections 2024-04-27T03:30:36.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/2927526948 wait complete. 2024-04-27T03:30:36.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55806d609760 msgr2=0x55806d60bbb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55806d609760 0x55806d60bbb0 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55806d5072d0 msgr2=0x55806d602f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55806d5072d0 0x55806d602f40 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55806d6057d0 msgr2=0x55806d607bc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55806d6057d0 0x55806d607bc0 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f51340369a0 msgr2=0x7f5134038e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f51340369a0 0x7f5134038e60 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f5244054000 tx=0x7f524404e000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 msgr2=0x55806d5fbd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f5253612800 1 --2- 172.21.15.165:0/1583550930 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55806d502f30 0x55806d5fbd00 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f524000b7c0 tx=0x7f5240000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.988+0000 7f524f3ef640 1 -- 172.21.15.165:0/1583550930 reap_dead start 2024-04-27T03:30:36.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.989+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 shutdown_connections 2024-04-27T03:30:36.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.989+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 >> 172.21.15.165:0/1583550930 conn(0x55806d4fe4d0 msgr2=0x55806d5ee270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.989+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 shutdown_connections 2024-04-27T03:30:36.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:35.989+0000 7f5253612800 1 -- 172.21.15.165:0/1583550930 wait complete. 2024-04-27T03:30:36.014 DEBUG:tasks.util.rgw: json result: {'user_id': 'testx$tenanteduser', 'display_name': 'tenanted-user', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'testx$tenanteduser', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-04-27T03:30:35.974388Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:35.974369Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:36.014 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-27T03:30:36.014 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', '/myfoo', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-27T03:30:36.014 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket /myfoo --tenant testx --uid tenanteduser 2024-04-27T03:30:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:36.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.056+0000 7f903a1a2800 1 Processor -- start 2024-04-27T03:30:36.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.056+0000 7f903a1a2800 1 -- start start 2024-04-27T03:30:36.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.056+0000 7f903a1a2800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5654be4997a0 0x5654be499bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.056+0000 7f903a1a2800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5654be359d40 con 0x5654be49a100 2024-04-27T03:30:36.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.056+0000 7f903a1a2800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5654be32c4d0 con 0x5654be4997a0 2024-04-27T03:30:36.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.056+0000 7f9037ca8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5654be4997a0 0x5654be499bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.056+0000 7f90374a7640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5654be49a100 0x5654be49a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35368/0 (socket says 172.21.15.165:35368) 2024-04-27T03:30:36.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.056+0000 7f90374a7640 1 -- 172.21.15.165:0/4201840633 learned_addr learned my addr 172.21.15.165:0/4201840633 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:36.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.057+0000 7f9037ca8640 1 -- 172.21.15.165:0/4201840633 >> v1:172.21.15.89:6789/0 conn(0x5654be49a100 legacy=0x5654be49a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.057+0000 7f9037ca8640 1 -- 172.21.15.165:0/4201840633 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654be358590 con 0x5654be4997a0 2024-04-27T03:30:36.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.059+0000 7f9037ca8640 1 --2- 172.21.15.165:0/4201840633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be499bc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f903000a470 tx=0x7f903001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bcfd9eea69b0cbd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.059+0000 7f9036ca6640 1 -- 172.21.15.165:0/4201840633 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9030057070 con 0x5654be4997a0 2024-04-27T03:30:36.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.059+0000 7f9036ca6640 1 -- 172.21.15.165:0/4201840633 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903004adc0 con 0x5654be4997a0 2024-04-27T03:30:36.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.059+0000 7f9036ca6640 1 -- 172.21.15.165:0/4201840633 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9030052590 con 0x5654be4997a0 2024-04-27T03:30:36.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.059+0000 7f903a1a2800 1 -- 172.21.15.165:0/4201840633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 msgr2=0x5654be499bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.059+0000 7f903a1a2800 1 --2- 172.21.15.165:0/4201840633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be499bc0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f903000a470 tx=0x7f903001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:36.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.059+0000 7f903a1a2800 1 -- 172.21.15.165:0/4201840633 shutdown_connections 2024-04-27T03:30:36.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.059+0000 7f903a1a2800 1 --2- 172.21.15.165:0/4201840633 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be499bc0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.059+0000 7f903a1a2800 1 -- 172.21.15.165:0/4201840633 >> 172.21.15.165:0/4201840633 conn(0x5654be495340 msgr2=0x5654be497760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.059+0000 7f903a1a2800 1 -- 172.21.15.165:0/4201840633 shutdown_connections 2024-04-27T03:30:36.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.059+0000 7f903a1a2800 1 -- 172.21.15.165:0/4201840633 wait complete. 2024-04-27T03:30:36.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.071+0000 7f903a1a2800 1 Processor -- start 2024-04-27T03:30:36.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.071+0000 7f903a1a2800 1 -- start start 2024-04-27T03:30:36.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.071+0000 7f903a1a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be58e410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.071+0000 7f903a1a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5654be3e63e0 con 0x5654be4997a0 2024-04-27T03:30:36.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.072+0000 7f9037ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be58e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.072+0000 7f9037ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be58e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59020/0 (socket says 172.21.15.165:59020) 2024-04-27T03:30:36.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.072+0000 7f9037ca8640 1 -- 172.21.15.165:0/2358155747 learned_addr learned my addr 172.21.15.165:0/2358155747 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.072+0000 7f9037ca8640 1 -- 172.21.15.165:0/2358155747 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654be58e9e0 con 0x5654be4997a0 2024-04-27T03:30:36.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.072+0000 7f9037ca8640 1 --2- 172.21.15.165:0/2358155747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be58e410 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9030055040 tx=0x7f903004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b14f2542c17694 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.072+0000 7f8f25e0b640 1 -- 172.21.15.165:0/2358155747 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9030057040 con 0x5654be4997a0 2024-04-27T03:30:36.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.072+0000 7f8f25e0b640 1 -- 172.21.15.165:0/2358155747 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9030062070 con 0x5654be4997a0 2024-04-27T03:30:36.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.072+0000 7f8f25e0b640 1 -- 172.21.15.165:0/2358155747 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f903005d470 con 0x5654be4997a0 2024-04-27T03:30:36.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.073+0000 7f903a1a2800 1 -- 172.21.15.165:0/2358155747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 msgr2=0x5654be58e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.073+0000 7f903a1a2800 1 --2- 172.21.15.165:0/2358155747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be58e410 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f9030055040 tx=0x7f903004c000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.073+0000 7f903a1a2800 1 -- 172.21.15.165:0/2358155747 shutdown_connections 2024-04-27T03:30:36.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.073+0000 7f903a1a2800 1 --2- 172.21.15.165:0/2358155747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be58e410 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.073+0000 7f903a1a2800 1 -- 172.21.15.165:0/2358155747 >> 172.21.15.165:0/2358155747 conn(0x5654be495340 msgr2=0x5654be589d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.073+0000 7f903a1a2800 1 -- 172.21.15.165:0/2358155747 shutdown_connections 2024-04-27T03:30:36.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.073+0000 7f903a1a2800 1 -- 172.21.15.165:0/2358155747 wait complete. 2024-04-27T03:30:36.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.073+0000 7f903a1a2800 1 Processor -- start 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.073+0000 7f903a1a2800 1 -- start start 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.073+0000 7f903a1a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be595e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.073+0000 7f903a1a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5654be49af60 con 0x5654be4997a0 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.074+0000 7f9037ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be595e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.074+0000 7f9037ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be595e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59028/0 (socket says 172.21.15.165:59028) 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.074+0000 7f9037ca8640 1 -- 172.21.15.165:0/3144715627 learned_addr learned my addr 172.21.15.165:0/3144715627 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.074+0000 7f9037ca8640 1 -- 172.21.15.165:0/3144715627 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654be5963c0 con 0x5654be4997a0 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.074+0000 7f9037ca8640 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be595e80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f903005f040 tx=0x7f9030002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.074+0000 7f8f1ffff640 1 -- 172.21.15.165:0/3144715627 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f903006b070 con 0x5654be4997a0 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.074+0000 7f8f1ffff640 1 -- 172.21.15.165:0/3144715627 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903005d9a0 con 0x5654be4997a0 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.074+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5654be596650 con 0x5654be4997a0 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.074+0000 7f8f1ffff640 1 -- 172.21.15.165:0/3144715627 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9030057070 con 0x5654be4997a0 2024-04-27T03:30:36.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.074+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654be599300 con 0x5654be4997a0 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.075+0000 7f8f1ffff640 1 -- 172.21.15.165:0/3144715627 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f903005dca0 con 0x5654be4997a0 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.075+0000 7f8f1ffff640 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f20036900 0x7f8f20038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.075+0000 7f8f1ffff640 1 -- 172.21.15.165:0/3144715627 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f903008e090 con 0x5654be4997a0 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.075+0000 7f90374a7640 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f20036900 0x7f8f20038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.075+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5654be49a100 0x5654be59d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.075+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be59d600 con 0x5654be49a100 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.075+0000 7f90384a9640 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5654be49a100 0x5654be59d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.076+0000 7f90374a7640 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f20036900 0x7f8f20038dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f902c000f60 tx=0x7f902c0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.076+0000 7f90384a9640 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5654be49a100 0x5654be59d0c0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.076+0000 7f90384a9640 1 -- 172.21.15.165:0/3144715627 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be49a100 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.076+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be59a690 con 0x5654be49a100 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.076+0000 7f90384a9640 1 -- 172.21.15.165:0/3144715627 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be49a100 2024-04-27T03:30:36.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.076+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5654be59f9a0 0x5654be5a1d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.076+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5a2430 con 0x5654be59f9a0 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.076+0000 7f9037ca8640 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5654be59f9a0 0x5654be5a1d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.077+0000 7f9037ca8640 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5654be59f9a0 0x5654be5a1d90 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.077+0000 7f9037ca8640 1 -- 172.21.15.165:0/3144715627 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f903005a040 con 0x5654be59f9a0 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.077+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be59e950 con 0x5654be49a100 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.077+0000 7f90384a9640 1 -- 172.21.15.165:0/3144715627 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be49a100 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.077+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5654be5a38e0 0x5654be5a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.077+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5a6490 con 0x5654be5a38e0 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.078+0000 7f90374a7640 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5654be5a38e0 0x5654be5a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.078+0000 7f90374a7640 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5654be5a38e0 0x5654be5a5d30 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.078+0000 7f90374a7640 1 -- 172.21.15.165:0/3144715627 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f902c009050 con 0x5654be5a38e0 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.078+0000 7f903a1a2800 1 Processor -- start 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.078+0000 7f903a1a2800 1 -- start start 2024-04-27T03:30:36.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.079+0000 7f903a1a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5b4880 0x5654be5b4ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.079+0000 7f903a1a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5654be38c6b0 con 0x5654be5b4880 2024-04-27T03:30:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.079+0000 7f9037ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5b4880 0x5654be5b4ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.079+0000 7f9037ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5b4880 0x5654be5b4ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59040/0 (socket says 172.21.15.165:59040) 2024-04-27T03:30:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.079+0000 7f9037ca8640 1 -- 172.21.15.165:0/3510659578 learned_addr learned my addr 172.21.15.165:0/3510659578 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.079+0000 7f9037ca8640 1 -- 172.21.15.165:0/3510659578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654be5b51e0 con 0x5654be5b4880 2024-04-27T03:30:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.079+0000 7f9037ca8640 1 --2- 172.21.15.165:0/3510659578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5b4880 0x5654be5b4ca0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f903008c690 tx=0x7f90300785d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.079+0000 7f8f1d7fa640 1 -- 172.21.15.165:0/3510659578 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f903006e420 con 0x5654be5b4880 2024-04-27T03:30:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.079+0000 7f8f1d7fa640 1 -- 172.21.15.165:0/3510659578 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9030002a50 con 0x5654be5b4880 2024-04-27T03:30:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.079+0000 7f903a1a2800 1 -- 172.21.15.165:0/3510659578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5654be5b5c50 con 0x5654be5b4880 2024-04-27T03:30:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.079+0000 7f8f1d7fa640 1 -- 172.21.15.165:0/3510659578 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9030002c20 con 0x5654be5b4880 2024-04-27T03:30:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.079+0000 7f903a1a2800 1 -- 172.21.15.165:0/3510659578 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654be5b6150 con 0x5654be5b4880 2024-04-27T03:30:36.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.080+0000 7f8f1d7fa640 1 -- 172.21.15.165:0/3510659578 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9030078730 con 0x5654be5b4880 2024-04-27T03:30:36.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.080+0000 7f8f1d7fa640 1 --2- 172.21.15.165:0/3510659578 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f10036950 0x7f8f10038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.080+0000 7f8f1d7fa640 1 -- 172.21.15.165:0/3510659578 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f9030088030 con 0x5654be5b4880 2024-04-27T03:30:36.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.080+0000 7f90374a7640 1 --2- 172.21.15.165:0/3510659578 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f10036950 0x7f8f10038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.081+0000 7f903a1a2800 1 Processor -- start 2024-04-27T03:30:36.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.081+0000 7f903a1a2800 1 -- start start 2024-04-27T03:30:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.081+0000 7f90374a7640 1 --2- 172.21.15.165:0/3510659578 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f10036950 0x7f8f10038e10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f902c00bf50 tx=0x7f902c00f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.081+0000 7f903a1a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 0x5654be5c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.081+0000 7f903a1a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5654be58eca0 con 0x5654be5c19a0 2024-04-27T03:30:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.081+0000 7f9037ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 0x5654be5c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.081+0000 7f9037ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 0x5654be5c1dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59056/0 (socket says 172.21.15.165:59056) 2024-04-27T03:30:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.081+0000 7f9037ca8640 1 -- 172.21.15.165:0/3236100679 learned_addr learned my addr 172.21.15.165:0/3236100679 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.081+0000 7f9037ca8640 1 -- 172.21.15.165:0/3236100679 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654be5c2390 con 0x5654be5c19a0 2024-04-27T03:30:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.082+0000 7f9037ca8640 1 --2- 172.21.15.165:0/3236100679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 0x5654be5c1dc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9030059010 tx=0x7f9030080c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d9488bf4eee62c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.082+0000 7f8f16ffd640 1 -- 172.21.15.165:0/3236100679 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9030009540 con 0x5654be5c19a0 2024-04-27T03:30:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.082+0000 7f8f16ffd640 1 -- 172.21.15.165:0/3236100679 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9030009ab0 con 0x5654be5c19a0 2024-04-27T03:30:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.082+0000 7f8f16ffd640 1 -- 172.21.15.165:0/3236100679 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9030002170 con 0x5654be5c19a0 2024-04-27T03:30:36.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.082+0000 7f903a1a2800 1 -- 172.21.15.165:0/3236100679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 msgr2=0x5654be5c1dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.082+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3236100679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 0x5654be5c1dc0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f9030059010 tx=0x7f9030080c30 comp rx=0 tx=0).stop 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.082+0000 7f903a1a2800 1 -- 172.21.15.165:0/3236100679 shutdown_connections 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.082+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3236100679 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 0x5654be5c1dc0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.082+0000 7f903a1a2800 1 -- 172.21.15.165:0/3236100679 >> 172.21.15.165:0/3236100679 conn(0x5654be5bdd00 msgr2=0x5654be5bbd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.082+0000 7f903a1a2800 1 -- 172.21.15.165:0/3236100679 shutdown_connections 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.082+0000 7f903a1a2800 1 -- 172.21.15.165:0/3236100679 wait complete. 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.082+0000 7f903a1a2800 1 Processor -- start 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.083+0000 7f903a1a2800 1 -- start start 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.083+0000 7f903a1a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 0x5654be5c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.083+0000 7f903a1a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5654be5c2a40 con 0x5654be5c19a0 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.083+0000 7f9037ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 0x5654be5c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.083+0000 7f9037ca8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 0x5654be5c8dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59066/0 (socket says 172.21.15.165:59066) 2024-04-27T03:30:36.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.083+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 learned_addr learned my addr 172.21.15.165:0/2167035104 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.083+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654be5c9300 con 0x5654be5c19a0 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.083+0000 7f9037ca8640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 0x5654be5c8dc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f903007c010 tx=0x7f9030059e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.084+0000 7f8f157fa640 1 -- 172.21.15.165:0/2167035104 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90300aa040 con 0x5654be5c19a0 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.084+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5654be5c9300 con 0x5654be5c19a0 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.084+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654be5c9300 con 0x5654be5c19a0 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.084+0000 7f8f157fa640 1 -- 172.21.15.165:0/2167035104 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f903006eae0 con 0x5654be5c19a0 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.084+0000 7f8f157fa640 1 -- 172.21.15.165:0/2167035104 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f903006ee40 con 0x5654be5c19a0 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.084+0000 7f8f157fa640 1 -- 172.21.15.165:0/2167035104 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f90300b6020 con 0x5654be5c19a0 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.085+0000 7f8f157fa640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f000368b0 0x7f8f00038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.085+0000 7f8f157fa640 1 -- 172.21.15.165:0/2167035104 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f90300b4020 con 0x5654be5c19a0 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.085+0000 7f90374a7640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f000368b0 0x7f8f00038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.085+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.085+0000 7f903a1a2800 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5654be5d4530 0x5654be5d69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.085+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d7140 con 0x5654be5d4530 2024-04-27T03:30:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.085+0000 7f90374a7640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f000368b0 0x7f8f00038d70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f902c001b60 tx=0x7f902c010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.085+0000 7f90384a9640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5654be5d4530 0x5654be5d69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f90384a9640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5654be5d4530 0x5654be5d69f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be5d4530 2024-04-27T03:30:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f903a1a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f903a1a2800 20 realm 2024-04-27T03:30:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d7550 con 0x5654be5d4530 2024-04-27T03:30:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be5d4530 2024-04-27T03:30:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f903a1a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f903a1a2800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d7e20 con 0x5654be5d4530 2024-04-27T03:30:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be5d4530 2024-04-27T03:30:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f903a1a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.086+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d4530 2024-04-27T03:30:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.087+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be5d4530 2024-04-27T03:30:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.087+0000 7f903a1a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.087+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.087+0000 7f903a1a2800 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5654be5d8f20 0x5654be5db310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.087+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5db920 con 0x5654be5d8f20 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.087+0000 7f9037ca8640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5654be5d8f20 0x5654be5db310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.087+0000 7f9037ca8640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5654be5d8f20 0x5654be5db310 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.087+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.087+0000 7f903a1a2800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.087+0000 7f903a1a2800 20 searching for the correct realm 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.087+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5dd250 con 0x5654be5d8f20 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.088+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.088+0000 7f9037ca8640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9030097b60 0x7f903008e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.088+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5dd250 con 0x7f9030097b60 2024-04-27T03:30:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.088+0000 7f90374a7640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9030097b60 0x7f903008e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.088+0000 7f90374a7640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9030097b60 0x7f903008e8f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.088+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.088+0000 7f90374a7640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f902c037040 0x7f902c036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.088+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5dd250 con 0x7f902c037040 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.089+0000 7f90384a9640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f902c037040 0x7f902c036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.089+0000 7f90384a9640 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f902c037040 0x7f902c036410 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.089+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f902800a4a0 con 0x7f902c037040 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.089+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d4530 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.089+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be5d4530 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.089+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d4530 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.090+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be5d4530 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.090+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f902c037040 2024-04-27T03:30:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.090+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902800a4a0 con 0x7f902c037040 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.090+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f902c037040 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.090+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902800a4a0 con 0x7f902c037040 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.090+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f9030097b60 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.090+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.090+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d8f20 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.091+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.091+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d4530 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.091+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be5d4530 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.091+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f902c037040 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.091+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902800a4a0 con 0x7f902c037040 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.091+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d4530 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.091+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be5d4530 2024-04-27T03:30:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.091+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f9030097b60 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.092+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.092+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d8f20 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.092+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.092+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d8f20 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.092+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.092+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f902c037040 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.093+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902800a4a0 con 0x7f902c037040 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.093+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d8f20 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.093+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.093+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d8f20 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.093+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.093+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d4530 2024-04-27T03:30:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.093+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be5d4530 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.093+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d4530 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.094+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f902800a4a0 con 0x5654be5d4530 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.094+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f902c037040 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.094+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9028033040 con 0x7f902c037040 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.094+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d8f20 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.094+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.094+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f9030097b60 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.094+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.094+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d4530 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.095+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.095+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f902c037040 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.095+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.095+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d8f20 2024-04-27T03:30:36.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.095+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.095+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f902c037040 2024-04-27T03:30:36.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.096+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.096+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f9030097b60 2024-04-27T03:30:36.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.096+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.096+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d8f20 2024-04-27T03:30:36.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.096+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.096+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f902c037040 2024-04-27T03:30:36.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x5654be5d4530 2024-04-27T03:30:36.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5654be5d81c0 con 0x7f9030097b60 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f903a1a2800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f903a1a2800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f903a1a2800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f903a1a2800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f903a1a2800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f903a1a2800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5dc1e0 con 0x5654be5d4530 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f903a1a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.097+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5dc5d0 con 0x5654be5d4530 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f903a1a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5dc5d0 con 0x7f902c037040 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f903a1a2800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f903a1a2800 20 zone default found 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f903a1a2800 4 Realm: () 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f903a1a2800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f903a1a2800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f903a1a2800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f903a1a2800 20 zonegroup default 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.098+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5dbd60 con 0x7f902c037040 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f903a1a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5df0b0 con 0x5654be5d8f20 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f903a1a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5df450 con 0x5654be5d8f20 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f903a1a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f903a1a2800 20 started sync module instance, tier type = 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f903a1a2800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.099+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5de550 con 0x7f9030097b60 2024-04-27T03:30:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.100+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.100+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94921971067248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5de1b0 con 0x7f9030097b60 2024-04-27T03:30:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.100+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e0150 con 0x5654be5d4530 2024-04-27T03:30:36.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.101+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94921971067248] v25'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.101+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.101+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94921971071440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e03c0 con 0x5654be5d4530 2024-04-27T03:30:36.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.101+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e0ee0 con 0x5654be5d8f20 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.102+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94921971071440] v25'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.102+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.103+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94921971075504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e12d0 con 0x5654be5d8f20 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.103+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e1d10 con 0x7f9030097b60 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.104+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.104+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94921971075504] v25'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.104+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94921971079136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e25b0 con 0x7f9030097b60 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.104+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e30b0 con 0x5654be5d4530 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.105+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.105+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94921971084192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e34a0 con 0x5654be5d4530 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.105+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e3ec0 con 0x7f902c037040 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.105+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94921971079136] v25'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.106+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.106+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94921971087792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e3e80 con 0x7f902c037040 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.106+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e48f0 con 0x7f902c037040 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.106+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94921971084192] v25'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.107+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94921971087792] v25'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.107+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.107+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94921971081472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5dfe30 con 0x7f902c037040 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.107+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e4da0 con 0x7f902c037040 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.108+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94921971081472] v25'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.108+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'132 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.108+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94921971090368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e4d60 con 0x7f902c037040 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.108+0000 7f903a1a2800 20 add_watcher() i=0 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.108+0000 7f903a1a2800 20 add_watcher() i=1 2024-04-27T03:30:36.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.108+0000 7f903a1a2800 20 add_watcher() i=2 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.108+0000 7f903a1a2800 20 add_watcher() i=3 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.108+0000 7f903a1a2800 20 add_watcher() i=4 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.108+0000 7f903a1a2800 20 add_watcher() i=5 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.108+0000 7f903a1a2800 20 add_watcher() i=6 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.109+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94921971090368] v25'133 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.109+0000 7f903a1a2800 20 add_watcher() i=7 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.109+0000 7f903a1a2800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.109+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e81c0 con 0x5654be5d4530 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.109+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.109+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94921971092096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be5e5690 con 0x5654be5d4530 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.110+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94921971092096] v25'134 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.110+0000 7f8eedfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.110+0000 7f903a1a2800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.110+0000 7f903a1a2800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.110+0000 7f903a1a2800 5 note: GC not initialized 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.110+0000 7f8eecfe1640 20 reqs_thread_entry: start 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f903a1a2800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f903a1a2800 20 Filter name: none 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f8e667fc640 20 reqs_thread_entry: start 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f903a1a2800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be61b660 con 0x7f902c037040 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(62 testx$tenanteduser [call out=48b,read 0~419 out=419b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 246+0+542 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f903a1a2800 20 rados_obj.operate() r=0 bl.length=419 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f903a1a2800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f903a1a2800 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T03:30:36.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f903a1a2800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f903a1a2800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.111+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be61b0b0 con 0x7f9030097b60 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(63 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv23 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f903a1a2800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f903a1a2800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f903a1a2800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f903a1a2800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 : miss 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f903a1a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be61f280 con 0x5654be5d4530 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(64 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv4 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f903a1a2800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f903a1a2800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:36.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f903a1a2800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:36.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f903a1a2800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:36.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f903a1a2800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:36.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.112+0000 7f903a1a2800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 2024-04-27T03:30:36.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.113+0000 7f903a1a2800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : hit (requested=0x13, cached=0x13) 2024-04-27T03:30:36.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.113+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be623920 con 0x7f9030097b60 2024-04-27T03:30:36.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.114+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(65 foo.buckets [call] v25'24 uv24 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.114+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 8.14 8:2e249026:root::.bucket.meta.testx%3amyfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [create,call version.inc in=38b,writefull 0~412 in=412b,setxattr user.rgw.acl (203) in=215b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be627230 con 0x5654be5d8f20 2024-04-27T03:30:36.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.115+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 .bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [create,call,writefull 0~412,setxattr (203)] v25'1 uv1 ondisk = 0) ==== 338+0+0 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.115+0000 7f903a1a2800 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x7 2024-04-27T03:30:36.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.115+0000 7f903a1a2800 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:36.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.115+0000 7f903a1a2800 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-27T03:30:36.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.115+0000 7f903a1a2800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1, ofs0, ns] 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94921971355312 in=859b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be626b90 con 0x7f9030097b60 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== watch-notify(notify (1) cookie 94921971067248 notify 107374182413 ret 0) ==== 889+0+0 (crc 0 0 0) 0x7f902c041060 con 0x7f9030097b60 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f8f16ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182413 cookie 94921971067248 notifier 4532 bl.length()=847 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f8f16ffd640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x7 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f8f16ffd640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f8f16ffd640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=203 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(67 notify.0 [notify cookie 94921971355312 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f8f16ffd640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f8efc003ea0 con 0x7f9030097b60 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== watch-notify(notify_complete (2) cookie 94921971355312 notify 107374182413 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5654be359d40 con 0x7f9030097b60 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(68 notify.0 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f903a1a2800 20 handle_bi_update(): bucket=testx:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): orig_sources= new_sources= 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f903a1a2800 20 handle_bi_update(): bucket=testx:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): potential sources added=[] removed=[] 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f903a1a2800 20 handle_bi_update(): bucket=testx:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): orig_dests= new_dests= 2024-04-27T03:30:36.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f903a1a2800 20 handle_bi_update(): bucket=testx:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): potential dests added=[] removed=[] 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.116+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x5654be5d4530 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.117+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(69 testx$tenanteduser.buckets [call] v25'1 uv1 ondisk = 0) ==== 170+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.117+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [delete,create,call version.inc in=38b,writefull 0~193 in=193b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x5654be5d4530 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.118+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(70 testx/myfoo [delete,create,call,writefull 0~193] v25'1 uv1 ondisk = 0) ==== 281+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.118+0000 7f903a1a2800 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.118+0000 7f903a1a2800 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.118+0000 7f903a1a2800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.118+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify cookie 94921971361472 in=360b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be627220 con 0x5654be5d4530 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.119+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify (1) cookie 94921971084192 notify 107374182420 ret 0) ==== 390+0+0 (crc 0 0 0) 0x5654be626fc0 con 0x5654be5d4530 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.119+0000 7f8f167fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182420 cookie 94921971084192 notifier 4532 bl.length()=348 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.119+0000 7f8f167fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x7 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.119+0000 7f8f167fc640 10 rgw watcher librados: moving default.rgw.meta+root+testx/myfoo to cache LRU end 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.119+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(71 notify.4 [notify cookie 94921971361472 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.119+0000 7f8f167fc640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f8e6c003430 con 0x5654be5d4530 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.119+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(72 notify.4 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.119+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== watch-notify(notify_complete (2) cookie 94921971361472 notify 107374182420 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5654be625ec0 con 0x5654be5d4530 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.119+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:73 8.9 8:91b71841:root::myfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x7f9030097b60 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.120+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(73 myfoo [call,delete] v25'25 uv25 ondisk = 0) ==== 191+0+0 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.120+0000 7f903a1a2800 10 removing default.rgw.meta+root+myfoo from cache 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.120+0000 7f903a1a2800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.120+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94921971364384 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be626b70 con 0x7f902c037040 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.121+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== watch-notify(notify (1) cookie 94921971081472 notify 107374182408 ret 0) ==== 191+0+0 (crc 0 0 0) 0x5654be625ec0 con 0x7f902c037040 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.121+0000 7f8f167fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182408 cookie 94921971081472 notifier 4532 bl.length()=149 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.121+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(74 notify.6 [notify cookie 94921971364384 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.121+0000 7f8f167fc640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f8e6c003430 con 0x7f902c037040 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.121+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== watch-notify(notify_complete (2) cookie 94921971364384 notify 107374182408 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5654be61f670 con 0x7f902c037040 2024-04-27T03:30:36.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.121+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(75 notify.6 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.121+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x5654be5d4530 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.122+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(76 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call,delete] v25'5 uv5 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.122+0000 7f903a1a2800 10 removing default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 from cache 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.122+0000 7f903a1a2800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1, ofs0, ns] 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.122+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:77 7.0 7:05bf5b68:::notify.1:head [notify cookie 94921971362480 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x5654be5d4530 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.122+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== watch-notify(notify (1) cookie 94921971071440 notify 107374182421 ret 0) ==== 248+0+0 (crc 0 0 0) 0x5654be61f670 con 0x5654be5d4530 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.122+0000 7f8f16ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182421 cookie 94921971071440 notifier 4532 bl.length()=206 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.122+0000 7f8f16ffd640 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:78 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f8efc003ea0 con 0x5654be5d4530 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.122+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(77 notify.1 [notify cookie 94921971362480 out=8b] v0'0 uv132 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.123+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== watch-notify(notify_complete (2) cookie 94921971362480 notify 107374182421 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5654be61e7c0 con 0x5654be5d4530 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.123+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(78 notify.1 [notify-ack] v0'0 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.123+0000 7f8eedfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.123+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94921971092096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x5654be5d4530 2024-04-27T03:30:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.124+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94921971092096] v25'135 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.124+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94921971067248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x7f9030097b60 2024-04-27T03:30:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.125+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94921971067248] v25'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.125+0000 7f903a1a2800 20 remove_watcher() i=0 2024-04-27T03:30:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.125+0000 7f903a1a2800 2 removed watcher, disabling cache 2024-04-27T03:30:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.125+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94921971071440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x5654be5d4530 2024-04-27T03:30:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.126+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94921971071440] v25'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.126+0000 7f903a1a2800 20 remove_watcher() i=1 2024-04-27T03:30:36.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.126+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94921971075504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x5654be5d8f20 2024-04-27T03:30:36.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.127+0000 7f9037ca8640 1 -- 172.21.15.165:0/2167035104 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94921971075504] v25'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9030094020 con 0x5654be5d8f20 2024-04-27T03:30:36.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.127+0000 7f903a1a2800 20 remove_watcher() i=2 2024-04-27T03:30:36.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.128+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94921971079136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x7f9030097b60 2024-04-27T03:30:36.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.128+0000 7f90374a7640 1 -- 172.21.15.165:0/2167035104 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94921971079136] v25'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f902c009050 con 0x7f9030097b60 2024-04-27T03:30:36.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.129+0000 7f903a1a2800 20 remove_watcher() i=3 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.129+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94921971084192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x5654be5d4530 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.130+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94921971084192] v25'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x5654be5d4530 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.130+0000 7f903a1a2800 20 remove_watcher() i=4 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.130+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94921971087792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x7f902c037040 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.131+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94921971087792] v25'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.131+0000 7f903a1a2800 20 remove_watcher() i=5 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.131+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94921971081472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x7f902c037040 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.132+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94921971081472] v25'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.132+0000 7f903a1a2800 20 remove_watcher() i=6 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.132+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94921971090368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5654be6266c0 con 0x7f902c037040 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94921971090368] v25'134 uv132 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9028031040 con 0x7f902c037040 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 20 remove_watcher() i=7 2024-04-27T03:30:36.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f902c037040 msgr2=0x7f902c036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f902c037040 0x7f902c036410 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5654be5d4530 msgr2=0x5654be5d69f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5654be5d4530 0x5654be5d69f0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9030097b60 msgr2=0x7f903008e8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9030097b60 0x7f903008e8f0 crc :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5654be5d8f20 msgr2=0x5654be5db310 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5654be5d8f20 0x5654be5db310 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f000368b0 msgr2=0x7f8f00038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f000368b0 0x7f8f00038d70 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f902c001b60 tx=0x7f902c010000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 msgr2=0x5654be5c8dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f90384a9640 1 -- 172.21.15.165:0/2167035104 reap_dead start 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 0x5654be5c8dc0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f903007c010 tx=0x7f9030059e70 comp rx=0 tx=0).stop 2024-04-27T03:30:36.150 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 shutdown_connections 2024-04-27T03:30:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 --2- 172.21.15.165:0/2167035104 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5c19a0 0x5654be5c8dc0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 >> 172.21.15.165:0/2167035104 conn(0x5654be5bdd00 msgr2=0x5654be5bb360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 shutdown_connections 2024-04-27T03:30:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.133+0000 7f903a1a2800 1 -- 172.21.15.165:0/2167035104 wait complete. 2024-04-27T03:30:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3510659578 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f10036950 msgr2=0x7f8f10038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3510659578 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f10036950 0x7f8f10038e10 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f902c00bf50 tx=0x7f902c00f000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3510659578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5b4880 msgr2=0x5654be5b4ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3510659578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5b4880 0x5654be5b4ca0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f903008c690 tx=0x7f90300785d0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3510659578 shutdown_connections 2024-04-27T03:30:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3510659578 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f10036950 0x7f8f10038e10 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3510659578 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be5b4880 0x5654be5b4ca0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3510659578 >> 172.21.15.165:0/3510659578 conn(0x5654be5a8920 msgr2=0x5654be5a8d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3510659578 shutdown_connections 2024-04-27T03:30:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3510659578 wait complete. 2024-04-27T03:30:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5654be5a38e0 msgr2=0x5654be5a5d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5654be5a38e0 0x5654be5a5d30 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5654be49a100 msgr2=0x5654be59d0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5654be49a100 0x5654be59d0c0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5654be59f9a0 msgr2=0x5654be5a1d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5654be59f9a0 0x5654be5a1d90 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f20036900 msgr2=0x7f8f20038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8f20036900 0x7f8f20038dc0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f902c000f60 tx=0x7f902c0012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 msgr2=0x5654be595e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 --2- 172.21.15.165:0/3144715627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5654be4997a0 0x5654be595e80 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f903005f040 tx=0x7f9030002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f90384a9640 1 -- 172.21.15.165:0/3144715627 reap_dead start 2024-04-27T03:30:36.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 shutdown_connections 2024-04-27T03:30:36.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 >> 172.21.15.165:0/3144715627 conn(0x5654be495340 msgr2=0x5654be588550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 shutdown_connections 2024-04-27T03:30:36.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.134+0000 7f903a1a2800 1 -- 172.21.15.165:0/3144715627 wait complete. 2024-04-27T03:30:36.153 INFO:tasks.util.rgw:rgwadmin: client.0 : ['metadata', 'get', 'bucket:testx/myfoo'] 2024-04-27T03:30:36.153 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'metadata', 'get', 'bucket:testx/myfoo'] 2024-04-27T03:30:36.153 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph metadata get bucket:testx/myfoo 2024-04-27T03:30:36.195 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:36.195 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:36.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.190+0000 7f9b0c9a2800 1 Processor -- start 2024-04-27T03:30:36.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.190+0000 7f9b0c9a2800 1 -- start start 2024-04-27T03:30:36.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.190+0000 7f9b0c9a2800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55aabeef1d20 0x55aabeef2140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.190+0000 7f9b0c9a2800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55aabedb1d40 con 0x55aabeef2f80 2024-04-27T03:30:36.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.190+0000 7f9b0c9a2800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55aabed844d0 con 0x55aabeef1d20 2024-04-27T03:30:36.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.191+0000 7f9b0a4a1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55aabeef1d20 0x55aabeef2140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.191+0000 7f9b09ca0640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55aabeef2f80 0x55aabeef3400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35376/0 (socket says 172.21.15.165:35376) 2024-04-27T03:30:36.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.191+0000 7f9b09ca0640 1 -- 172.21.15.165:0/1534688034 learned_addr learned my addr 172.21.15.165:0/1534688034 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:36.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.191+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/1534688034 >> v1:172.21.15.89:6789/0 conn(0x55aabeef2f80 legacy=0x55aabeef3400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.191+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/1534688034 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabedb0590 con 0x55aabeef1d20 2024-04-27T03:30:36.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.193+0000 7f9b0a4a1640 1 --2- 172.21.15.165:0/1534688034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabeef2140 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00a470 tx=0x7f9afc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79192d4f63788bf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.193+0000 7f9b0949f640 1 -- 172.21.15.165:0/1534688034 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9afc057070 con 0x55aabeef1d20 2024-04-27T03:30:36.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.193+0000 7f9b0949f640 1 -- 172.21.15.165:0/1534688034 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc052570 con 0x55aabeef1d20 2024-04-27T03:30:36.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.193+0000 7f9b0949f640 1 -- 172.21.15.165:0/1534688034 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9afc052870 con 0x55aabeef1d20 2024-04-27T03:30:36.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.193+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1534688034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 msgr2=0x55aabeef2140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.193+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/1534688034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabeef2140 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f9afc00a470 tx=0x7f9afc01ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.193+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1534688034 shutdown_connections 2024-04-27T03:30:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.193+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/1534688034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabeef2140 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.193+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1534688034 >> 172.21.15.165:0/1534688034 conn(0x55aabeeed380 msgr2=0x55aabeeef7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.193+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1534688034 shutdown_connections 2024-04-27T03:30:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.194+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1534688034 wait complete. 2024-04-27T03:30:36.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.205+0000 7f9b0c9a2800 1 Processor -- start 2024-04-27T03:30:36.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.205+0000 7f9b0c9a2800 1 -- start start 2024-04-27T03:30:36.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.205+0000 7f9b0c9a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabee65ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.205+0000 7f9b0c9a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aabee3e3c0 con 0x55aabeef1d20 2024-04-27T03:30:36.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.206+0000 7f9b0a4a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabee65ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.206+0000 7f9b0a4a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabee65ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59084/0 (socket says 172.21.15.165:59084) 2024-04-27T03:30:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.206+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/1953930110 learned_addr learned my addr 172.21.15.165:0/1953930110 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.206+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/1953930110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabefe50e0 con 0x55aabeef1d20 2024-04-27T03:30:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.206+0000 7f9b0a4a1640 1 --2- 172.21.15.165:0/1953930110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabee65ff0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9afc055040 tx=0x7f9afc04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b299c90b5257c437 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.206+0000 7f99f8608640 1 -- 172.21.15.165:0/1953930110 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9afc057070 con 0x55aabeef1d20 2024-04-27T03:30:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.206+0000 7f99f8608640 1 -- 172.21.15.165:0/1953930110 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc063070 con 0x55aabeef1d20 2024-04-27T03:30:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.206+0000 7f99f8608640 1 -- 172.21.15.165:0/1953930110 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9afc052720 con 0x55aabeef1d20 2024-04-27T03:30:36.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.207+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1953930110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 msgr2=0x55aabee65ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.207+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/1953930110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabee65ff0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f9afc055040 tx=0x7f9afc04c000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.207+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1953930110 shutdown_connections 2024-04-27T03:30:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.207+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/1953930110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabee65ff0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.207+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1953930110 >> 172.21.15.165:0/1953930110 conn(0x55aabeeed380 msgr2=0x55aabefdede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.207+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1953930110 shutdown_connections 2024-04-27T03:30:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.207+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1953930110 wait complete. 2024-04-27T03:30:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.207+0000 7f9b0c9a2800 1 Processor -- start 2024-04-27T03:30:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.207+0000 7f9b0c9a2800 1 -- start start 2024-04-27T03:30:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.208+0000 7f9b0c9a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabefee190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.208+0000 7f9b0c9a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aabeefb3f0 con 0x55aabeef1d20 2024-04-27T03:30:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.208+0000 7f9b0a4a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabefee190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.208+0000 7f9b0a4a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabefee190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59094/0 (socket says 172.21.15.165:59094) 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.208+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/1673952299 learned_addr learned my addr 172.21.15.165:0/1673952299 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.208+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/1673952299 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabefee6d0 con 0x55aabeef1d20 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.208+0000 7f9b0a4a1640 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabefee190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9afc055040 tx=0x7f9afc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.208+0000 7f99f6e05640 1 -- 172.21.15.165:0/1673952299 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9afc052cd0 con 0x55aabeef1d20 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.208+0000 7f99f6e05640 1 -- 172.21.15.165:0/1673952299 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc06c070 con 0x55aabeef1d20 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.208+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aabefee960 con 0x55aabeef1d20 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.208+0000 7f99f6e05640 1 -- 172.21.15.165:0/1673952299 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9afc05e480 con 0x55aabeef1d20 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.208+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aabeff15c0 con 0x55aabeef1d20 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.209+0000 7f99f6e05640 1 -- 172.21.15.165:0/1673952299 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9afc05e620 con 0x55aabeef1d20 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.209+0000 7f99f6e05640 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99f0036900 0x7f99f0038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.209+0000 7f99f6e05640 1 -- 172.21.15.165:0/1673952299 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f9afc08e650 con 0x55aabeef1d20 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.209+0000 7f9b09ca0640 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99f0036900 0x7f99f0038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.210+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aabeef2f80 0x55aabeff53e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.210+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabeff5960 con 0x55aabeef2f80 2024-04-27T03:30:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.210+0000 7f9b0aca2640 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aabeef2f80 0x55aabeff53e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.210+0000 7f9b09ca0640 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99f0036900 0x7f99f0038dc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b00000f60 tx=0x7f9b000012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.210+0000 7f9b0aca2640 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aabeef2f80 0x55aabeff53e0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.210+0000 7f9b0aca2640 1 -- 172.21.15.165:0/1673952299 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabeef2f80 2024-04-27T03:30:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.210+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabeff29b0 con 0x55aabeef2f80 2024-04-27T03:30:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.211+0000 7f9b0aca2640 1 -- 172.21.15.165:0/1673952299 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabeef2f80 2024-04-27T03:30:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.211+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aabeff7c90 0x55aabeffa080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.211+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabeffa720 con 0x55aabeff7c90 2024-04-27T03:30:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.211+0000 7f9b0a4a1640 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aabeff7c90 0x55aabeffa080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.211+0000 7f9b0a4a1640 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aabeff7c90 0x55aabeffa080 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.211+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/1673952299 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabeff7c90 2024-04-27T03:30:36.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.211+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabeff6c40 con 0x55aabeef2f80 2024-04-27T03:30:36.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.211+0000 7f9b0aca2640 1 -- 172.21.15.165:0/1673952299 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabeef2f80 2024-04-27T03:30:36.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.212+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55aabeffbbd0 0x55aabeffe040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.212+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabeffe7a0 con 0x55aabeffbbd0 2024-04-27T03:30:36.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.212+0000 7f9b09ca0640 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55aabeffbbd0 0x55aabeffe040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.212+0000 7f9b09ca0640 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55aabeffbbd0 0x55aabeffe040 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.212+0000 7f9b09ca0640 1 -- 172.21.15.165:0/1673952299 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9b00009050 con 0x55aabeffbbd0 2024-04-27T03:30:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.213+0000 7f9b0c9a2800 1 Processor -- start 2024-04-27T03:30:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.213+0000 7f9b0c9a2800 1 -- start start 2024-04-27T03:30:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.213+0000 7f9b0c9a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf00cbd0 0x55aabf00cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.213+0000 7f9b0c9a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aabede46b0 con 0x55aabf00cbd0 2024-04-27T03:30:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.213+0000 7f9b0a4a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf00cbd0 0x55aabf00cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.213+0000 7f9b0a4a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf00cbd0 0x55aabf00cff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59110/0 (socket says 172.21.15.165:59110) 2024-04-27T03:30:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.213+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3935145689 learned_addr learned my addr 172.21.15.165:0/3935145689 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.213+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3935145689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabf00d530 con 0x55aabf00cbd0 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.214+0000 7f9b0a4a1640 1 --2- 172.21.15.165:0/3935145689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf00cbd0 0x55aabf00cff0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9afc060040 tx=0x7f9afc068f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.214+0000 7f99effff640 1 -- 172.21.15.165:0/3935145689 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9afc092020 con 0x55aabf00cbd0 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.214+0000 7f99effff640 1 -- 172.21.15.165:0/3935145689 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc058040 con 0x55aabf00cbd0 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.214+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3935145689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aabf00dfa0 con 0x55aabf00cbd0 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.214+0000 7f99effff640 1 -- 172.21.15.165:0/3935145689 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9afc08eb60 con 0x55aabf00cbd0 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.214+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3935145689 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aabf00e4a0 con 0x55aabf00cbd0 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.214+0000 7f99effff640 1 -- 172.21.15.165:0/3935145689 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9afc095020 con 0x55aabf00cbd0 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.215+0000 7f99effff640 1 --2- 172.21.15.165:0/3935145689 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99e4036950 0x7f99e4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.215+0000 7f99effff640 1 -- 172.21.15.165:0/3935145689 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f9afc098ab0 con 0x55aabf00cbd0 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.215+0000 7f9b09ca0640 1 --2- 172.21.15.165:0/3935145689 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99e4036950 0x7f99e4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.215+0000 7f9b0c9a2800 1 Processor -- start 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.215+0000 7f9b0c9a2800 1 -- start start 2024-04-27T03:30:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.215+0000 7f9b09ca0640 1 --2- 172.21.15.165:0/3935145689 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99e4036950 0x7f99e4038e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000bf50 tx=0x7f9b0000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.215+0000 7f9b0c9a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 0x55aabf01a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.215+0000 7f9b0c9a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aabefe97b0 con 0x55aabf019dd0 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.215+0000 7f9b0a4a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 0x55aabf01a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.215+0000 7f9b0a4a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 0x55aabf01a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59120/0 (socket says 172.21.15.165:59120) 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.215+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/2478147072 learned_addr learned my addr 172.21.15.165:0/2478147072 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.216+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/2478147072 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabf01a7c0 con 0x55aabf019dd0 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.216+0000 7f9b0a4a1640 1 --2- 172.21.15.165:0/2478147072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 0x55aabf01a1f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9afc07add0 tx=0x7f9afc07ae00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977608f6aeb7634e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.216+0000 7f99edffb640 1 -- 172.21.15.165:0/2478147072 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9afc08ee30 con 0x55aabf019dd0 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.216+0000 7f99edffb640 1 -- 172.21.15.165:0/2478147072 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc0600d0 con 0x55aabf019dd0 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.216+0000 7f99edffb640 1 -- 172.21.15.165:0/2478147072 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9afc05d3e0 con 0x55aabf019dd0 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/2478147072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 msgr2=0x55aabf01a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/2478147072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 0x55aabf01a1f0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f9afc07add0 tx=0x7f9afc07ae00 comp rx=0 tx=0).stop 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/2478147072 shutdown_connections 2024-04-27T03:30:36.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/2478147072 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 0x55aabf01a1f0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/2478147072 >> 172.21.15.165:0/2478147072 conn(0x55aabf0160a0 msgr2=0x55aabf014100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/2478147072 shutdown_connections 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/2478147072 wait complete. 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0c9a2800 1 Processor -- start 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0c9a2800 1 -- start start 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0c9a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 0x55aabf021070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0c9a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aabf01ae20 con 0x55aabf019dd0 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0a4a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 0x55aabf021070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0a4a1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 0x55aabf021070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59132/0 (socket says 172.21.15.165:59132) 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.217+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 learned_addr learned my addr 172.21.15.165:0/3260786929 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.218+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aabf0215b0 con 0x55aabf019dd0 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.218+0000 7f9b0a4a1640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 0x55aabf021070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9afc08cb90 tx=0x7f9afc058ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.218+0000 7f99dbfff640 1 -- 172.21.15.165:0/3260786929 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9afc067d00 con 0x55aabf019dd0 2024-04-27T03:30:36.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.218+0000 7f99dbfff640 1 -- 172.21.15.165:0/3260786929 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9afc0bb070 con 0x55aabf019dd0 2024-04-27T03:30:36.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.218+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aabf0215b0 con 0x55aabf019dd0 2024-04-27T03:30:36.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.218+0000 7f99dbfff640 1 -- 172.21.15.165:0/3260786929 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9afc07bae0 con 0x55aabf019dd0 2024-04-27T03:30:36.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.218+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aabf0215b0 con 0x55aabf019dd0 2024-04-27T03:30:36.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.219+0000 7f99dbfff640 1 -- 172.21.15.165:0/3260786929 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9afc07bc80 con 0x55aabf019dd0 2024-04-27T03:30:36.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.219+0000 7f99dbfff640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99d4036950 0x7f99d4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.219+0000 7f9b0c9a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.219+0000 7f99dbfff640 1 -- 172.21.15.165:0/3260786929 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f9afc0b21b0 con 0x55aabf019dd0 2024-04-27T03:30:36.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.219+0000 7f9b09ca0640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99d4036950 0x7f99d4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.219+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aabf02c850 0x55aabf02ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.219+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf02f460 con 0x55aabf02c850 2024-04-27T03:30:36.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.219+0000 7f9b0aca2640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aabf02c850 0x55aabf02ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b09ca0640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99d4036950 0x7f99d4038e10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9b00001b60 tx=0x7f9b00010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0aca2640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aabf02c850 0x55aabf02ed10 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabf02c850 2024-04-27T03:30:36.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0c9a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0c9a2800 20 realm 2024-04-27T03:30:36.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0c9a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf02f870 con 0x55aabf02c850 2024-04-27T03:30:36.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabf02c850 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0c9a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0c9a2800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0c9a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf030110 con 0x55aabf02c850 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabf02c850 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0c9a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0c9a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.220+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf02c850 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.221+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabf02c850 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.221+0000 7f9b0c9a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.221+0000 7f9b0c9a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.221+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aabf030fe0 0x55aabf0333d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.221+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf033b10 con 0x55aabf030fe0 2024-04-27T03:30:36.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.221+0000 7f9b0a4a1640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aabf030fe0 0x55aabf0333d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.221+0000 7f9b0a4a1640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aabf030fe0 0x55aabf0333d0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.221+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.221+0000 7f9b0c9a2800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:36.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.221+0000 7f9b0c9a2800 20 searching for the correct realm 2024-04-27T03:30:36.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.221+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0354e0 con 0x55aabf030fe0 2024-04-27T03:30:36.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.222+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.222+0000 7f9b0a4a1640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9afc0a2040 0x7f9afc08b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.222+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0354e0 con 0x7f9afc0a2040 2024-04-27T03:30:36.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.222+0000 7f9b09ca0640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9afc0a2040 0x7f9afc08b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.222+0000 7f9b09ca0640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9afc0a2040 0x7f9afc08b3a0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.223+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b00009050 con 0x7f9afc0a2040 2024-04-27T03:30:36.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.223+0000 7f9b09ca0640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9b00038040 0x7f9b0001bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.223+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0354e0 con 0x7f9b00038040 2024-04-27T03:30:36.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.223+0000 7f9b0aca2640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9b00038040 0x7f9b0001bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.223+0000 7f9b0aca2640 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9b00038040 0x7f9b0001bab0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.223+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f9b0400a4a0 con 0x7f9b00038040 2024-04-27T03:30:36.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.223+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf02c850 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.224+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabf02c850 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.224+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf02c850 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.224+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabf02c850 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.224+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9b00038040 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.224+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b0400a4a0 con 0x7f9b00038040 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.224+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9b00038040 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.224+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b0400a4a0 con 0x7f9b00038040 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.224+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9afc0a2040 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.225+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b00009050 con 0x7f9afc0a2040 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.225+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf030fe0 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.225+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.225+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf02c850 2024-04-27T03:30:36.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.225+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabf02c850 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.225+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9b00038040 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.225+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b0400a4a0 con 0x7f9b00038040 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.225+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf02c850 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.226+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabf02c850 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.226+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9afc0a2040 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.226+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9b00009050 con 0x7f9afc0a2040 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.226+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf030fe0 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.226+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.226+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf030fe0 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.226+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.226+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9b00038040 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.227+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b0400a4a0 con 0x7f9b00038040 2024-04-27T03:30:36.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.227+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf030fe0 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.227+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.227+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf030fe0 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.227+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.227+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf02c850 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.227+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabf02c850 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.227+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf02c850 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.228+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9b0400a4a0 con 0x55aabf02c850 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.228+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9b00038040 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.228+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b04033040 con 0x7f9b00038040 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.228+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf030fe0 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.228+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.228+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9afc0a2040 2024-04-27T03:30:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.228+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b00009050 con 0x7f9afc0a2040 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.228+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf02c850 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.229+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.229+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9b00038040 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.229+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.229+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf030fe0 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.229+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.229+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9b00038040 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.229+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.229+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9afc0a2040 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.230+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b00009050 con 0x7f9afc0a2040 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.230+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf030fe0 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.230+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.230+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9b00038040 2024-04-27T03:30:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.230+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.230+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x55aabf02c850 2024-04-27T03:30:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.230+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.230+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0304b0 con 0x7f9afc0a2040 2024-04-27T03:30:36.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9b00045040 con 0x7f9afc0a2040 2024-04-27T03:30:36.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:36.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:36.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:36.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:36.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:36.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:36.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0344b0 con 0x55aabf02c850 2024-04-27T03:30:36.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0348a0 con 0x55aabf02c850 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.231+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf0348a0 con 0x7f9b00038040 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0c9a2800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0c9a2800 20 zone default found 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0c9a2800 4 Realm: () 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0c9a2800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0c9a2800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0c9a2800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0c9a2800 20 zonegroup default 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0c9a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf034cc0 con 0x7f9b00038040 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0c9a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0c9a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.232+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf034cc0 con 0x55aabf030fe0 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.233+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.233+0000 7f9b0c9a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.233+0000 7f9b0c9a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.233+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf037390 con 0x55aabf030fe0 2024-04-27T03:30:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.233+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.233+0000 7f9b0c9a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.233+0000 7f9b0c9a2800 20 started sync module instance, tier type = 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.233+0000 7f9b0c9a2800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.233+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf036830 con 0x7f9afc0a2040 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.234+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b00045040 con 0x7f9afc0a2040 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.234+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94191837473824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf036490 con 0x7f9afc0a2040 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.234+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf038110 con 0x55aabf02c850 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.235+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94191837473824] v25'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b00045040 con 0x7f9afc0a2040 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.235+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.235+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94191837476656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf038500 con 0x55aabf02c850 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.235+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf038f60 con 0x55aabf030fe0 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.236+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94191837476656] v25'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.236+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.236+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94191837481488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf039350 con 0x55aabf030fe0 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.236+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf039db0 con 0x7f9afc0a2040 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.237+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94191837481488] v25'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.237+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b00045040 con 0x7f9afc0a2040 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.237+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94191837477712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf03a650 con 0x7f9afc0a2040 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.237+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf03b010 con 0x55aabf02c850 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.238+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.238+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94191837490496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf03b810 con 0x55aabf02c850 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.238+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94191837477712] v25'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b00045040 con 0x7f9afc0a2040 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.238+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf03a1a0 con 0x7f9b00038040 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.239+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94191837490496] v25'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.239+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.239+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94191837491808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf03a1a0 con 0x7f9b00038040 2024-04-27T03:30:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.239+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf03c8a0 con 0x7f9b00038040 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.240+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.240+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94191837496304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf03ccc0 con 0x7f9b00038040 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.240+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf03d710 con 0x7f9b00038040 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.240+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94191837491808] v25'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.241+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'135 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.241+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94191837499904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf03d6d0 con 0x7f9b00038040 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.241+0000 7f9b0c9a2800 20 add_watcher() i=0 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.241+0000 7f9b0c9a2800 20 add_watcher() i=1 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.241+0000 7f9b0c9a2800 20 add_watcher() i=2 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.241+0000 7f9b0c9a2800 20 add_watcher() i=3 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.241+0000 7f9b0c9a2800 20 add_watcher() i=4 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.241+0000 7f9b0c9a2800 20 add_watcher() i=5 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.241+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94191837496304] v25'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.241+0000 7f9b0c9a2800 20 add_watcher() i=6 2024-04-27T03:30:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.242+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94191837499904] v25'136 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.242+0000 7f9b0c9a2800 20 add_watcher() i=7 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.242+0000 7f9b0c9a2800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.242+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf03c810 con 0x55aabf02c850 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.243+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.243+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94191837494736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf03c810 con 0x55aabf02c850 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.244+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94191837494736] v25'136 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.244+0000 7f99c0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.244+0000 7f9b0c9a2800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.244+0000 7f9b0c9a2800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.244+0000 7f9b0c9a2800 5 note: GC not initialized 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.244+0000 7f993b7fe640 20 reqs_thread_entry: start 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.244+0000 7f9b0c9a2800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:36.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.244+0000 7f9b0c9a2800 20 Filter name: none 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.244+0000 7f99397fa640 20 reqs_thread_entry: start 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.244+0000 7f9b0c9a2800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.244+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55aabf073320 con 0x55aabf02c850 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.245+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 testx/myfoo [call out=48b,stat out=16b,read 0~193 out=193b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 281+0+332 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.245+0000 7f9b0c9a2800 20 rados_obj.operate() r=0 bl.length=193 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.245+0000 7f99c0fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.246+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94191837494736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf049d50 con 0x55aabf02c850 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.246+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94191837494736] v25'137 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.246+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94191837473824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf03bdd0 con 0x7f9afc0a2040 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.247+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94191837473824] v25'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b00045040 con 0x7f9afc0a2040 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.247+0000 7f9b0c9a2800 20 remove_watcher() i=0 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.247+0000 7f9b0c9a2800 2 removed watcher, disabling cache 2024-04-27T03:30:36.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.247+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94191837476656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf036c20 con 0x55aabf02c850 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.248+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94191837476656] v25'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.248+0000 7f9b0c9a2800 20 remove_watcher() i=1 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.248+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94191837481488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf037730 con 0x55aabf030fe0 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.250+0000 7f9b0a4a1640 1 -- 172.21.15.165:0/3260786929 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94191837481488] v25'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9afc08cd70 con 0x55aabf030fe0 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.250+0000 7f9b0c9a2800 20 remove_watcher() i=2 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.250+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94191837477712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf037730 con 0x7f9afc0a2040 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.251+0000 7f9b09ca0640 1 -- 172.21.15.165:0/3260786929 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94191837477712] v25'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b00045040 con 0x7f9afc0a2040 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.251+0000 7f9b0c9a2800 20 remove_watcher() i=3 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.251+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94191837490496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf038a10 con 0x55aabf02c850 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.252+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94191837490496] v25'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x55aabf02c850 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.252+0000 7f9b0c9a2800 20 remove_watcher() i=4 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.252+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94191837491808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf038a10 con 0x7f9b00038040 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.253+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94191837491808] v25'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.253+0000 7f9b0c9a2800 20 remove_watcher() i=5 2024-04-27T03:30:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.253+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94191837496304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf037be0 con 0x7f9b00038040 2024-04-27T03:30:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.254+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94191837496304] v25'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.254+0000 7f9b0c9a2800 20 remove_watcher() i=6 2024-04-27T03:30:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.254+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94191837499904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55aabf037be0 con 0x7f9b00038040 2024-04-27T03:30:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.255+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94191837499904] v25'137 uv135 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9b04031040 con 0x7f9b00038040 2024-04-27T03:30:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.255+0000 7f9b0c9a2800 20 remove_watcher() i=7 2024-04-27T03:30:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.255+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9b00038040 msgr2=0x7f9b0001bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.255+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9b00038040 0x7f9b0001bab0 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.255+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aabf02c850 msgr2=0x55aabf02ed10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.255+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aabf02c850 0x55aabf02ed10 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.255+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9afc0a2040 msgr2=0x7f9afc08b3a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.255+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9afc0a2040 0x7f9afc08b3a0 crc :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.255+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aabf030fe0 msgr2=0x55aabf0333d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.255+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aabf030fe0 0x55aabf0333d0 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99d4036950 msgr2=0x7f99d4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99d4036950 0x7f99d4038e10 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f9b00001b60 tx=0x7f9b00010000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 msgr2=0x55aabf021070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/3260786929 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf019dd0 0x55aabf021070 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9afc08cb90 tx=0x7f9afc058ee0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 reap_dead start 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0aca2640 1 -- 172.21.15.165:0/3260786929 reap_dead start 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 shutdown_connections 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 >> 172.21.15.165:0/3260786929 conn(0x55aabf0160a0 msgr2=0x55aabf013700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 shutdown_connections 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3260786929 wait complete. 2024-04-27T03:30:36.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3935145689 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99e4036950 msgr2=0x7f99e4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/3935145689 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99e4036950 0x7f99e4038e10 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f9b0000bf50 tx=0x7f9b0000f000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3935145689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf00cbd0 msgr2=0x55aabf00cff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/3935145689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf00cbd0 0x55aabf00cff0 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f9afc060040 tx=0x7f9afc068f10 comp rx=0 tx=0).stop 2024-04-27T03:30:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3935145689 shutdown_connections 2024-04-27T03:30:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/3935145689 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99e4036950 0x7f99e4038e10 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/3935145689 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabf00cbd0 0x55aabf00cff0 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3935145689 >> 172.21.15.165:0/3935145689 conn(0x55aabf000c70 msgr2=0x55aabf001070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3935145689 shutdown_connections 2024-04-27T03:30:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/3935145689 wait complete. 2024-04-27T03:30:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55aabeffbbd0 msgr2=0x55aabeffe040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55aabeffbbd0 0x55aabeffe040 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aabeef2f80 msgr2=0x55aabeff53e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aabeef2f80 0x55aabeff53e0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aabeff7c90 msgr2=0x55aabeffa080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aabeff7c90 0x55aabeffa080 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99f0036900 msgr2=0x7f99f0038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.256+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f99f0036900 0x7f99f0038dc0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f9b00000f60 tx=0x7f9b000012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.257+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 msgr2=0x55aabefee190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.257+0000 7f9b0c9a2800 1 --2- 172.21.15.165:0/1673952299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aabeef1d20 0x55aabefee190 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f9afc055040 tx=0x7f9afc002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.257+0000 7f9b0aca2640 1 -- 172.21.15.165:0/1673952299 reap_dead start 2024-04-27T03:30:36.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.257+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 shutdown_connections 2024-04-27T03:30:36.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.257+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 >> 172.21.15.165:0/1673952299 conn(0x55aabeeed380 msgr2=0x55aabefdd640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.257+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 shutdown_connections 2024-04-27T03:30:36.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.257+0000 7f9b0c9a2800 1 -- 172.21.15.165:0/1673952299 wait complete. 2024-04-27T03:30:36.275 INFO:teuthology.orchestra.run.smithi165.stdout:{"key":"bucket:testx/myfoo","ver":{"tag":"xWAxA+M/4aF7gzeFaSq3Ot6S","ver":2},"mtime":"2024-04-27T03:30:36.118820Z","data":{"bucket":{"name":"myfoo","marker":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1","bucket_id":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1","tenant":"testx","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""}},"owner":"testx$tenanteduser","creation_time":"2024-04-27T03:30:33.422040Z","linked":true,"has_bucket_info":false}} 2024-04-27T03:30:36.276 DEBUG:tasks.util.rgw: json result: {'key': 'bucket:testx/myfoo', 'ver': {'tag': 'xWAxA+M/4aF7gzeFaSq3Ot6S', 'ver': 2}, 'mtime': '2024-04-27T03:30:36.118820Z', 'data': {'bucket': {'name': 'myfoo', 'marker': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1', 'bucket_id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1', 'tenant': 'testx', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}}, 'owner': 'testx$tenanteduser', 'creation_time': '2024-04-27T03:30:33.422040Z', 'linked': True, 'has_bucket_info': False}} 2024-04-27T03:30:36.276 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-04-27T03:30:36.276 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'link', '--bucket', 'testx/myfoo', '--uid', 'foo'] 2024-04-27T03:30:36.276 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket link --bucket testx/myfoo --uid foo 2024-04-27T03:30:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.355+0000 7fd4c1531800 1 Processor -- start 2024-04-27T03:30:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.355+0000 7fd4c1531800 1 -- start start 2024-04-27T03:30:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.356+0000 7fd4c1531800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x561726c00150 0x561726c02610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.356+0000 7fd4c1531800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x561726abad40 con 0x561726b6f1e0 2024-04-27T03:30:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.356+0000 7fd4c1531800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x561726a8d4d0 con 0x561726c00150 2024-04-27T03:30:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.356+0000 7fd4b7fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x561726c00150 0x561726c02610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.356+0000 7fd4bd22f640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x561726b6f1e0 0x561726bfea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35382/0 (socket says 172.21.15.165:35382) 2024-04-27T03:30:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.356+0000 7fd4bd22f640 1 -- 172.21.15.165:0/1576382306 learned_addr learned my addr 172.21.15.165:0/1576382306 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.356+0000 7fd4b7fff640 1 -- 172.21.15.165:0/1576382306 >> v1:172.21.15.89:6789/0 conn(0x561726b6f1e0 legacy=0x561726bfea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.356+0000 7fd4b7fff640 1 -- 172.21.15.165:0/1576382306 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561726ab9590 con 0x561726c00150 2024-04-27T03:30:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.356+0000 7fd4bca2e640 1 -- 172.21.15.165:0/1576382306 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 974485491 0 0) 0x561726abad40 con 0x561726b6f1e0 2024-04-27T03:30:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.358+0000 7fd4b7fff640 1 --2- 172.21.15.165:0/1576382306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726c00150 0x561726c02610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800a470 tx=0x7fd4a801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79aa2f9dc19162b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.358+0000 7fd4bca2e640 1 -- 172.21.15.165:0/1576382306 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a8057070 con 0x561726c00150 2024-04-27T03:30:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.358+0000 7fd4bca2e640 1 -- 172.21.15.165:0/1576382306 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4a8052570 con 0x561726c00150 2024-04-27T03:30:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.358+0000 7fd4bca2e640 1 -- 172.21.15.165:0/1576382306 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4a8052870 con 0x561726c00150 2024-04-27T03:30:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.359+0000 7fd4c1531800 1 -- 172.21.15.165:0/1576382306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726c00150 msgr2=0x561726c02610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.359+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1576382306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726c00150 0x561726c02610 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fd4a800a470 tx=0x7fd4a801ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.359+0000 7fd4c1531800 1 -- 172.21.15.165:0/1576382306 shutdown_connections 2024-04-27T03:30:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.359+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1576382306 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726c00150 0x561726c02610 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.359+0000 7fd4c1531800 1 -- 172.21.15.165:0/1576382306 >> 172.21.15.165:0/1576382306 conn(0x561726bf63a0 msgr2=0x561726bf87c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.359+0000 7fd4c1531800 1 -- 172.21.15.165:0/1576382306 shutdown_connections 2024-04-27T03:30:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.359+0000 7fd4c1531800 1 -- 172.21.15.165:0/1576382306 wait complete. 2024-04-27T03:30:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.371+0000 7fd4c1531800 1 Processor -- start 2024-04-27T03:30:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.371+0000 7fd4c1531800 1 -- start start 2024-04-27T03:30:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.371+0000 7fd4c1531800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 0x561726cecf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.371+0000 7fd4c1531800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561726b473b0 con 0x561726b6f1e0 2024-04-27T03:30:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.371+0000 7fd4bd22f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 0x561726cecf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.371+0000 7fd4bd22f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 0x561726cecf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59150/0 (socket says 172.21.15.165:59150) 2024-04-27T03:30:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.371+0000 7fd4bd22f640 1 -- 172.21.15.165:0/1967139945 learned_addr learned my addr 172.21.15.165:0/1967139945 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.371+0000 7fd4bd22f640 1 -- 172.21.15.165:0/1967139945 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561726ced4f0 con 0x561726b6f1e0 2024-04-27T03:30:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.372+0000 7fd4bd22f640 1 --2- 172.21.15.165:0/1967139945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 0x561726cecf20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000b210 tx=0x7fd4b000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af00575fc2f502b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.372+0000 7fd3ab606640 1 -- 172.21.15.165:0/1967139945 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4b0008070 con 0x561726b6f1e0 2024-04-27T03:30:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.372+0000 7fd3ab606640 1 -- 172.21.15.165:0/1967139945 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b000d500 con 0x561726b6f1e0 2024-04-27T03:30:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.372+0000 7fd3ab606640 1 -- 172.21.15.165:0/1967139945 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4b000d840 con 0x561726b6f1e0 2024-04-27T03:30:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.372+0000 7fd4c1531800 1 -- 172.21.15.165:0/1967139945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 msgr2=0x561726cecf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.372+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1967139945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 0x561726cecf20 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000b210 tx=0x7fd4b000b6d0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.372+0000 7fd4c1531800 1 -- 172.21.15.165:0/1967139945 shutdown_connections 2024-04-27T03:30:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.372+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1967139945 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 0x561726cecf20 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.372+0000 7fd4c1531800 1 -- 172.21.15.165:0/1967139945 >> 172.21.15.165:0/1967139945 conn(0x561726bf63a0 msgr2=0x561726ce8840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.372+0000 7fd4c1531800 1 -- 172.21.15.165:0/1967139945 shutdown_connections 2024-04-27T03:30:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.373+0000 7fd4c1531800 1 -- 172.21.15.165:0/1967139945 wait complete. 2024-04-27T03:30:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.373+0000 7fd4c1531800 1 Processor -- start 2024-04-27T03:30:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.373+0000 7fd4c1531800 1 -- start start 2024-04-27T03:30:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.373+0000 7fd4c1531800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 0x561726bfc330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.373+0000 7fd4c1531800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561726c02fc0 con 0x561726b6f1e0 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.373+0000 7fd4bd22f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 0x561726bfc330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.373+0000 7fd4bd22f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 0x561726bfc330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59158/0 (socket says 172.21.15.165:59158) 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.373+0000 7fd4bd22f640 1 -- 172.21.15.165:0/1407047807 learned_addr learned my addr 172.21.15.165:0/1407047807 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.373+0000 7fd4bd22f640 1 -- 172.21.15.165:0/1407047807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561726bfaa20 con 0x561726b6f1e0 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.374+0000 7fd4bd22f640 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 0x561726bfc330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000b210 tx=0x7fd4b0000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.374+0000 7fd3a9e03640 1 -- 172.21.15.165:0/1407047807 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4b000f950 con 0x561726b6f1e0 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.374+0000 7fd3a9e03640 1 -- 172.21.15.165:0/1407047807 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b000faf0 con 0x561726b6f1e0 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.374+0000 7fd3a9e03640 1 -- 172.21.15.165:0/1407047807 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4b000fdd0 con 0x561726b6f1e0 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.374+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561726bfacb0 con 0x561726b6f1e0 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.374+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561726bfb1d0 con 0x561726b6f1e0 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.375+0000 7fd3a9e03640 1 -- 172.21.15.165:0/1407047807 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fd4b0017040 con 0x561726b6f1e0 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.375+0000 7fd3a9e03640 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd3a4036900 0x7fd3a4038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.375+0000 7fd3a9e03640 1 -- 172.21.15.165:0/1407047807 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fd4b0015020 con 0x561726b6f1e0 2024-04-27T03:30:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.375+0000 7fd4b7fff640 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd3a4036900 0x7fd3a4038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.375+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561726c00150 0x561726cfde90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.375+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726cfe420 con 0x561726c00150 2024-04-27T03:30:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.375+0000 7fd4bda30640 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561726c00150 0x561726cfde90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.375+0000 7fd4b7fff640 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd3a4036900 0x7fd3a4038dc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8054000 tx=0x7fd4a804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.375+0000 7fd4bda30640 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561726c00150 0x561726cfde90 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.376+0000 7fd4bda30640 1 -- 172.21.15.165:0/1407047807 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726c00150 2024-04-27T03:30:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.376+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726cfed30 con 0x561726c00150 2024-04-27T03:30:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.376+0000 7fd4bda30640 1 -- 172.21.15.165:0/1407047807 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726c00150 2024-04-27T03:30:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.376+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561726d00760 0x561726d02c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.376+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d033b0 con 0x561726d00760 2024-04-27T03:30:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.376+0000 7fd4bd22f640 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561726d00760 0x561726d02c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.376+0000 7fd4bd22f640 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561726d00760 0x561726d02c20 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.377+0000 7fd4bd22f640 1 -- 172.21.15.165:0/1407047807 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd4b002d070 con 0x561726d00760 2024-04-27T03:30:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.377+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726cff5b0 con 0x561726c00150 2024-04-27T03:30:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.377+0000 7fd4bda30640 1 -- 172.21.15.165:0/1407047807 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726c00150 2024-04-27T03:30:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.377+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561726d048b0 0x561726d06d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.377+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d07480 con 0x561726d048b0 2024-04-27T03:30:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.377+0000 7fd4b7fff640 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561726d048b0 0x561726d06d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.378+0000 7fd4b7fff640 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561726d048b0 0x561726d06d20 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.378+0000 7fd4b7fff640 1 -- 172.21.15.165:0/1407047807 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd4a8052940 con 0x561726d048b0 2024-04-27T03:30:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.378+0000 7fd4c1531800 1 Processor -- start 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.378+0000 7fd4c1531800 1 -- start start 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.379+0000 7fd4c1531800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d15890 0x561726d15cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.379+0000 7fd4c1531800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561726bfc870 con 0x561726d15890 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.379+0000 7fd4bd22f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d15890 0x561726d15cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.379+0000 7fd4bd22f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d15890 0x561726d15cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59164/0 (socket says 172.21.15.165:59164) 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.379+0000 7fd4bd22f640 1 -- 172.21.15.165:0/468087218 learned_addr learned my addr 172.21.15.165:0/468087218 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.379+0000 7fd4bd22f640 1 -- 172.21.15.165:0/468087218 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561726d161f0 con 0x561726d15890 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.379+0000 7fd4bd22f640 1 --2- 172.21.15.165:0/468087218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d15890 0x561726d15cb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd4b004b4b0 tx=0x7fd4b001aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.379+0000 7fd3a2ffd640 1 -- 172.21.15.165:0/468087218 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4b001ade0 con 0x561726d15890 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.379+0000 7fd3a2ffd640 1 -- 172.21.15.165:0/468087218 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b0019470 con 0x561726d15890 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.379+0000 7fd3a2ffd640 1 -- 172.21.15.165:0/468087218 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4b0019750 con 0x561726d15890 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.379+0000 7fd4c1531800 1 -- 172.21.15.165:0/468087218 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561726d16c60 con 0x561726d15890 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.379+0000 7fd4c1531800 1 -- 172.21.15.165:0/468087218 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561726d17160 con 0x561726d15890 2024-04-27T03:30:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.380+0000 7fd3a2ffd640 1 -- 172.21.15.165:0/468087218 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fd4b003f1b0 con 0x561726d15890 2024-04-27T03:30:36.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.380+0000 7fd3a2ffd640 1 --2- 172.21.15.165:0/468087218 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd394036970 0x7fd394038e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.380+0000 7fd3a2ffd640 1 -- 172.21.15.165:0/468087218 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fd4b003caa0 con 0x561726d15890 2024-04-27T03:30:36.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.380+0000 7fd4b7fff640 1 --2- 172.21.15.165:0/468087218 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd394036970 0x7fd394038e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.380+0000 7fd4c1531800 1 Processor -- start 2024-04-27T03:30:36.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.381+0000 7fd4c1531800 1 -- start start 2024-04-27T03:30:36.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.381+0000 7fd4b7fff640 1 --2- 172.21.15.165:0/468087218 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd394036970 0x7fd394038e30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x561726bfb5f0 tx=0x7fd4a8005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.381+0000 7fd4c1531800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 0x561726d22ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.381+0000 7fd4c1531800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561726aed6b0 con 0x561726d22bd0 2024-04-27T03:30:36.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.381+0000 7fd4bd22f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 0x561726d22ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.381+0000 7fd4bd22f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 0x561726d22ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59170/0 (socket says 172.21.15.165:59170) 2024-04-27T03:30:36.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.381+0000 7fd4bd22f640 1 -- 172.21.15.165:0/1419607205 learned_addr learned my addr 172.21.15.165:0/1419607205 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.381+0000 7fd4bd22f640 1 -- 172.21.15.165:0/1419607205 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561726d235e0 con 0x561726d22bd0 2024-04-27T03:30:36.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.381+0000 7fd4bd22f640 1 --2- 172.21.15.165:0/1419607205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 0x561726d22ff0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0026aa0 tx=0x7fd4b00267f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a36f19ed1b340c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.381+0000 7fd3a0ff9640 1 -- 172.21.15.165:0/1419607205 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4b005c020 con 0x561726d22bd0 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.381+0000 7fd3a0ff9640 1 -- 172.21.15.165:0/1419607205 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b0041db0 con 0x561726d22bd0 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.381+0000 7fd3a0ff9640 1 -- 172.21.15.165:0/1419607205 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4b0030e20 con 0x561726d22bd0 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.382+0000 7fd4c1531800 1 -- 172.21.15.165:0/1419607205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 msgr2=0x561726d22ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.382+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1419607205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 0x561726d22ff0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fd4b0026aa0 tx=0x7fd4b00267f0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.382+0000 7fd4c1531800 1 -- 172.21.15.165:0/1419607205 shutdown_connections 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.382+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1419607205 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 0x561726d22ff0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.382+0000 7fd4c1531800 1 -- 172.21.15.165:0/1419607205 >> 172.21.15.165:0/1419607205 conn(0x561726d1ef10 msgr2=0x561726d1cf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.382+0000 7fd4c1531800 1 -- 172.21.15.165:0/1419607205 shutdown_connections 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.382+0000 7fd4c1531800 1 -- 172.21.15.165:0/1419607205 wait complete. 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.382+0000 7fd4c1531800 1 Processor -- start 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.382+0000 7fd4c1531800 1 -- start start 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.382+0000 7fd4c1531800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 0x561726d29e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.382+0000 7fd4c1531800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561726d23b80 con 0x561726d22bd0 2024-04-27T03:30:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.383+0000 7fd4bd22f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 0x561726d29e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.383+0000 7fd4bd22f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 0x561726d29e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59178/0 (socket says 172.21.15.165:59178) 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.383+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 learned_addr learned my addr 172.21.15.165:0/4178876637 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.383+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561726d2a380 con 0x561726d22bd0 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.383+0000 7fd4bd22f640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 0x561726d29e40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd4b003cd90 tx=0x7fd4b002b0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.383+0000 7fd392ffd640 1 -- 172.21.15.165:0/4178876637 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4b003f4a0 con 0x561726d22bd0 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.383+0000 7fd392ffd640 1 -- 172.21.15.165:0/4178876637 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd4b003f640 con 0x561726d22bd0 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.383+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561726d2a380 con 0x561726d22bd0 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.383+0000 7fd392ffd640 1 -- 172.21.15.165:0/4178876637 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd4b0026d20 con 0x561726d22bd0 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.384+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561726d2a380 con 0x561726d22bd0 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.384+0000 7fd392ffd640 1 -- 172.21.15.165:0/4178876637 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fd4b005f020 con 0x561726d22bd0 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.384+0000 7fd392ffd640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd388036950 0x7fd388038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.384+0000 7fd392ffd640 1 -- 172.21.15.165:0/4178876637 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fd4b006a070 con 0x561726d22bd0 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.385+0000 7fd4b7fff640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd388036950 0x7fd388038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.385+0000 7fd4b7fff640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd388036950 0x7fd388038e10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8052fd0 tx=0x7fd4a804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.385+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.385+0000 7fd4c1531800 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561726d355f0 0x561726d37ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.385+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d38200 con 0x561726d355f0 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.385+0000 7fd4bda30640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561726d355f0 0x561726d37ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.385+0000 7fd4bda30640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561726d355f0 0x561726d37ab0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726d355f0 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 20 realm 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d38610 con 0x561726d355f0 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726d355f0 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d38ee0 con 0x561726d355f0 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726d355f0 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d39280 con 0x561726d355f0 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726d355f0 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561726d39fe0 0x561726d3c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.386+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d3c9e0 con 0x561726d39fe0 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.387+0000 7fd4bd22f640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561726d39fe0 0x561726d3c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.387+0000 7fd4bd22f640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561726d39fe0 0x561726d3c3d0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.387+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.387+0000 7fd4c1531800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:36.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.387+0000 7fd4c1531800 20 searching for the correct realm 2024-04-27T03:30:36.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.387+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d3e310 con 0x561726d39fe0 2024-04-27T03:30:36.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.388+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.388+0000 7fd4bd22f640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fd4b0061050 0x7fd4b004c7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.388+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d3e310 con 0x7fd4b0061050 2024-04-27T03:30:36.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.388+0000 7fd4b7fff640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fd4b0061050 0x7fd4b004c7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.388+0000 7fd4b7fff640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fd4b0061050 0x7fd4b004c7b0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.388+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.388+0000 7fd4b7fff640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd4a8076040 0x7fd4a8083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.388+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d3e310 con 0x7fd4a8076040 2024-04-27T03:30:36.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.388+0000 7fd4bda30640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd4a8076040 0x7fd4a8083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.389+0000 7fd4bda30640 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd4a8076040 0x7fd4a8083410 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.389+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fd4b800a4a0 con 0x7fd4a8076040 2024-04-27T03:30:36.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.389+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d39280 con 0x561726d355f0 2024-04-27T03:30:36.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.389+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726d355f0 2024-04-27T03:30:36.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.389+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d39280 con 0x561726d355f0 2024-04-27T03:30:36.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.390+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726d355f0 2024-04-27T03:30:36.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.390+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d39280 con 0x7fd4a8076040 2024-04-27T03:30:36.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.390+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b800a4a0 con 0x7fd4a8076040 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.390+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d39280 con 0x7fd4a8076040 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.390+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b800a4a0 con 0x7fd4a8076040 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.390+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d39280 con 0x7fd4b0061050 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.390+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.390+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d39280 con 0x561726d39fe0 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.390+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.390+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d39280 con 0x561726d355f0 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.391+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726d355f0 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.391+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d39280 con 0x7fd4a8076040 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.391+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b800a4a0 con 0x7fd4a8076040 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.391+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d39280 con 0x561726d355f0 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.391+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726d355f0 2024-04-27T03:30:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.391+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561726d39280 con 0x7fd4b0061050 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.392+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.392+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x561726d39fe0 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.392+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.392+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x561726d39fe0 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.392+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.392+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x7fd4a8076040 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.392+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b800a4a0 con 0x7fd4a8076040 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.392+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x561726d39fe0 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.393+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.393+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x561726d39fe0 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.393+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.393+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x561726d355f0 2024-04-27T03:30:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.393+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726d355f0 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.393+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x561726d355f0 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.393+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd4b800a4a0 con 0x561726d355f0 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.393+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x7fd4a8076040 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.394+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b8033040 con 0x7fd4a8076040 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.394+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x561726d39fe0 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.394+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.394+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x7fd4b0061050 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.394+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.394+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x561726d355f0 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.394+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.394+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x7fd4a8076040 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.395+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b8031040 con 0x7fd4a8076040 2024-04-27T03:30:36.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.395+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x561726d39fe0 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.395+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.395+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x7fd4a8076040 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.395+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b8031040 con 0x7fd4a8076040 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.395+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x7fd4b0061050 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.395+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.395+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x561726d39fe0 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.396+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.396+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x7fd4a8076040 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.396+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4b8031040 con 0x7fd4a8076040 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.396+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x561726d355f0 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.396+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.396+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd4a8090060 con 0x7fd4b0061050 2024-04-27T03:30:36.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d3d2a0 con 0x561726d355f0 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d3d690 con 0x561726d355f0 2024-04-27T03:30:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.397+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d3d690 con 0x7fd4a8076040 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd4b8031040 con 0x7fd4a8076040 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 20 zone default found 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 4 Realm: () 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 20 zonegroup default 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d3cf00 con 0x7fd4a8076040 2024-04-27T03:30:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x7fd4a8076040 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d3cf00 con 0x561726d39fe0 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d40270 con 0x561726d39fe0 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 20 started sync module instance, tier type = 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.398+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d3f610 con 0x7fd4b0061050 2024-04-27T03:30:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.399+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.399+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94657435662848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d3f270 con 0x7fd4b0061050 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.399+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d411a0 con 0x561726d355f0 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.400+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94657435662848] v25'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.401+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.401+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94657435667696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d415c0 con 0x561726d355f0 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.401+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d41fc0 con 0x561726d39fe0 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.402+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94657435667696] v25'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.402+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.402+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94657435668752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d41530 con 0x561726d39fe0 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.402+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d42e80 con 0x7fd4b0061050 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.403+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.403+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94657435668752] v25'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.403+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94657435677216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d43af0 con 0x7fd4b0061050 2024-04-27T03:30:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.403+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d423b0 con 0x561726d355f0 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.404+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94657435677216] v25'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.404+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.404+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94657435678272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d444b0 con 0x561726d355f0 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.404+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d44e70 con 0x7fd4a8076040 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.405+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94657435678272] v25'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.405+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x7fd4a8076040 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.405+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94657435681040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d44e70 con 0x7fd4a8076040 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.405+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d45830 con 0x7fd4a8076040 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.406+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x7fd4a8076040 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.406+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94657435685760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d45c50 con 0x7fd4a8076040 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.406+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d466a0 con 0x7fd4a8076040 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.406+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94657435681040] v25'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x7fd4a8076040 2024-04-27T03:30:36.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.407+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'138 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x7fd4a8076040 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.407+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94657435689360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d46660 con 0x7fd4a8076040 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.407+0000 7fd4c1531800 20 add_watcher() i=0 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.407+0000 7fd4c1531800 20 add_watcher() i=1 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.407+0000 7fd4c1531800 20 add_watcher() i=2 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.407+0000 7fd4c1531800 20 add_watcher() i=3 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.407+0000 7fd4c1531800 20 add_watcher() i=4 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.407+0000 7fd4c1531800 20 add_watcher() i=5 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.407+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94657435685760] v25'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x7fd4a8076040 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.407+0000 7fd4c1531800 20 add_watcher() i=6 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.408+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94657435689360] v25'139 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x7fd4a8076040 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.408+0000 7fd4c1531800 20 add_watcher() i=7 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.408+0000 7fd4c1531800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:36.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.408+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d457a0 con 0x561726d355f0 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.408+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.408+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94657435683264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d457a0 con 0x561726d355f0 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.409+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94657435683264] v25'138 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.409+0000 7fd3737e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.409+0000 7fd4c1531800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.409+0000 7fd4c1531800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.409+0000 7fd4c1531800 5 note: GC not initialized 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd3727e4640 20 reqs_thread_entry: start 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4c1531800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4c1531800 20 Filter name: none 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd2e3fff640 20 reqs_thread_entry: start 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4c1531800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:36.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d7c8d0 con 0x561726d355f0 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4c1531800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4c1531800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4c1531800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4c1531800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4c1531800 10 cache get: name=default.rgw.meta+root+testx/myfoo : miss 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.410+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d7c420 con 0x561726d355f0 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 testx/myfoo [call out=48b,read 0~193 out=193b] v0'0 uv1 ondisk = 0) ==== 197+0+241 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 20 rados_obj.operate() r=0 bl.length=193 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 10 cache put: name=default.rgw.meta+root+testx/myfoo info.flags=0x11 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 10 adding default.rgw.meta+root+testx/myfoo to cache LRU end 2024-04-27T03:30:36.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 10 cache get: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 : miss 2024-04-27T03:30:36.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 8.14 8:2e249026:root::.bucket.meta.testx%3amyfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d80590 con 0x561726d39fe0 2024-04-27T03:30:36.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 .bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~412 out=412b,getxattrs out=298b] v0'0 uv1 ondisk = 0) ==== 338+0+774 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 20 rados_obj.operate() r=0 bl.length=412 2024-04-27T03:30:36.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 10 cache put: name=default.rgw.meta+root+.bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:36.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 10 adding default.rgw.meta+root+.bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:36.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:36.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 10 updating xattr: name=user.rgw.acl bl.length()=203 2024-04-27T03:30:36.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 2024-04-27T03:30:36.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 10 cache get: name=default.rgw.meta+users.uid+foo : hit (requested=0x13, cached=0x13) 2024-04-27T03:30:36.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.411+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d84830 con 0x561726d355f0 2024-04-27T03:30:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.412+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 testx$tenanteduser.buckets [call] v25'2 uv2 ondisk = 0) ==== 170+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.412+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [create,call version.inc in=38b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d88140 con 0x561726d355f0 2024-04-27T03:30:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.413+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(66 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [create,call,writefull 0~379,setxattr (123)] v25'6 uv6 ondisk = 0) ==== 332+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.413+0000 7fd4c1531800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x7 2024-04-27T03:30:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.413+0000 7fd4c1531800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.413+0000 7fd4c1531800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.413+0000 7fd4c1531800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1, ofs0, ns] 2024-04-27T03:30:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.413+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [notify cookie 94657435950816 in=740b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d87ba0 con 0x561726d355f0 2024-04-27T03:30:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== watch-notify(notify (1) cookie 94657435667696 notify 107374182422 ret 0) ==== 770+0+0 (crc 0 0 0) 0x561726d884e0 con 0x561726d355f0 2024-04-27T03:30:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd3a0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182422 cookie 94657435667696 notifier 4546 bl.length()=728 2024-04-27T03:30:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd3a0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x7 2024-04-27T03:30:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(67 notify.1 [notify cookie 94657435950816 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd3a0ff9640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd3a0ff9640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd3a0ff9640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fd380003ea0 con 0x561726d355f0 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(68 notify.1 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b8031040 con 0x561726d355f0 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== watch-notify(notify_complete (2) cookie 94657435950816 notify 107374182422 ret 0) ==== 42+0+48 (crc 0 0 0) 0x561726d84260 con 0x561726d355f0 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd4c1531800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): orig_sources= new_sources= 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd4c1531800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): potential sources added=[] removed=[] 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd4c1531800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): orig_dests= new_dests= 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd4c1531800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): potential dests added=[] removed=[] 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.414+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d87c60 con 0x7fd4b0061050 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.415+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(69 foo.buckets [call] v25'26 uv26 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.415+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 8.9 8:91b71841:root::myfoo:head [delete,create,call version.inc in=38b,writefull 0~165 in=165b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d87c60 con 0x7fd4b0061050 2024-04-27T03:30:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.416+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 myfoo [delete,create,call,writefull 0~165] v25'27 uv27 ondisk = 0) ==== 275+0+0 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.416+0000 7fd4c1531800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.416+0000 7fd4c1531800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.416+0000 7fd4c1531800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 0, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.416+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94657435957168 in=326b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d87bd0 con 0x7fd4a8076040 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.417+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== watch-notify(notify (1) cookie 94657435685760 notify 107374182409 ret 0) ==== 356+0+0 (crc 0 0 0) 0x7fd4b801e040 con 0x7fd4a8076040 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.417+0000 7fd3a0ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182409 cookie 94657435685760 notifier 4546 bl.length()=314 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.417+0000 7fd3a0ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo info.flags=0x7 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.417+0000 7fd3a0ff9640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.417+0000 7fd3a0ff9640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fd380003ea0 con 0x7fd4a8076040 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.417+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(71 notify.6 [notify cookie 94657435957168 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd4b803f040 con 0x7fd4a8076040 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.417+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(72 notify.6 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b803f040 con 0x7fd4a8076040 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.417+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== watch-notify(notify_complete (2) cookie 94657435957168 notify 107374182409 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fd4b8027420 con 0x7fd4a8076040 2024-04-27T03:30:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.417+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:73 8.5 8:a7a6c9d8:root::testx%2fmyfoo:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d87c60 con 0x561726d355f0 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.418+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(73 testx/myfoo [call,delete] v25'2 uv2 ondisk = 0) ==== 197+0+0 (crc 0 0 0) 0x7fd4b803f040 con 0x561726d355f0 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.418+0000 7fd4c1531800 10 removing default.rgw.meta+root+testx/myfoo from cache 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.418+0000 7fd4c1531800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:testx/myfoo, ofs0, ns] 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.418+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:74 7.d 7:b4812045:::notify.4:head [notify cookie 94657435957168 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d87bd0 con 0x561726d355f0 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.418+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== watch-notify(notify (1) cookie 94657435678272 notify 107374182423 ret 0) ==== 197+0+0 (crc 0 0 0) 0x7fd4b8027420 con 0x561726d355f0 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.418+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(74 notify.4 [notify cookie 94657435957168 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd4b803f040 con 0x561726d355f0 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.418+0000 7fd393fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182423 cookie 94657435678272 notifier 4546 bl.length()=155 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.418+0000 7fd393fff640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fd2f00032c0 con 0x561726d355f0 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.419+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== watch-notify(notify_complete (2) cookie 94657435957168 notify 107374182423 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fd4b8027790 con 0x561726d355f0 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.419+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(75 notify.4 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b803f040 con 0x561726d355f0 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.419+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:76 8.14 8:2e249026:root::.bucket.meta.testx%3amyfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d87c60 con 0x561726d39fe0 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.420+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(76 .bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call,delete] v25'2 uv2 ondisk = 0) ==== 254+0+0 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.420+0000 7fd4c1531800 10 removing default.rgw.meta+root+.bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 from cache 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.420+0000 7fd4c1531800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.testx:myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1, ofs0, ns] 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.420+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:77 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94657435927808 in=224b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561726d87c60 con 0x7fd4b0061050 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.420+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== watch-notify(notify (1) cookie 94657435662848 notify 107374182414 ret 0) ==== 254+0+0 (crc 0 0 0) 0x561726d88000 con 0x7fd4b0061050 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.420+0000 7fd393fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182414 cookie 94657435662848 notifier 4546 bl.length()=212 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.420+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(77 notify.0 [notify cookie 94657435927808 out=8b] v0'0 uv138 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.420+0000 7fd393fff640 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fd2f0003330 con 0x7fd4b0061050 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.421+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== watch-notify(notify_complete (2) cookie 94657435927808 notify 107374182414 ret 0) ==== 42+0+48 (crc 0 0 0) 0x561726d86ef0 con 0x7fd4b0061050 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.421+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(78 notify.0 [notify-ack] v0'0 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.421+0000 7fd3737e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.421+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:79 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94657435683264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d877b0 con 0x561726d355f0 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.422+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(79 data_loggenerations_metadata [watch unwatch cookie 94657435683264] v25'139 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd4b803f040 con 0x561726d355f0 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.422+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:80 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94657435662848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d449c0 con 0x7fd4b0061050 2024-04-27T03:30:36.444 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.423+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(80 notify.0 [watch unwatch cookie 94657435662848] v25'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.423+0000 7fd4c1531800 20 remove_watcher() i=0 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.423+0000 7fd4c1531800 2 removed watcher, disabling cache 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.423+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94657435667696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d449c0 con 0x561726d355f0 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.425+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(81 notify.1 [watch unwatch cookie 94657435667696] v25'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b803f040 con 0x561726d355f0 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.425+0000 7fd4c1531800 20 remove_watcher() i=1 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.425+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94657435668752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d40cf0 con 0x561726d39fe0 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.426+0000 7fd4bd22f640 1 -- 172.21.15.165:0/4178876637 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(82 notify.2 [watch unwatch cookie 94657435668752] v25'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b0074020 con 0x561726d39fe0 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.426+0000 7fd4c1531800 20 remove_watcher() i=2 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.426+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94657435677216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d449c0 con 0x7fd4b0061050 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.427+0000 7fd4b7fff640 1 -- 172.21.15.165:0/4178876637 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(83 notify.3 [watch unwatch cookie 94657435677216] v25'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4a8081070 con 0x7fd4b0061050 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.427+0000 7fd4c1531800 20 remove_watcher() i=3 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.427+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:84 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94657435678272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d43220 con 0x561726d355f0 2024-04-27T03:30:36.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.428+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.1 v2:172.21.15.165:6800/79879041 36 ==== osd_op_reply(84 notify.4 [watch unwatch cookie 94657435678272] v25'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b803f040 con 0x561726d355f0 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.428+0000 7fd4c1531800 20 remove_watcher() i=4 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.428+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:85 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94657435681040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d449c0 con 0x7fd4a8076040 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.428+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(85 notify.5 [watch unwatch cookie 94657435681040] v25'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b803f040 con 0x7fd4a8076040 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.428+0000 7fd4c1531800 20 remove_watcher() i=5 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.428+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:86 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94657435685760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d44110 con 0x7fd4a8076040 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.430+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(86 notify.6 [watch unwatch cookie 94657435685760] v25'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b803f040 con 0x7fd4a8076040 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.430+0000 7fd4c1531800 20 remove_watcher() i=6 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.430+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:87 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94657435689360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561726d449c0 con 0x7fd4a8076040 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.430+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(87 notify.7 [watch unwatch cookie 94657435689360] v25'140 uv138 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd4b803f040 con 0x7fd4a8076040 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 20 remove_watcher() i=7 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd4a8076040 msgr2=0x7fd4a8083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd4a8076040 0x7fd4a8083410 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561726d355f0 msgr2=0x561726d37ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561726d355f0 0x561726d37ab0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fd4b0061050 msgr2=0x7fd4b004c7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fd4b0061050 0x7fd4b004c7b0 crc :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561726d39fe0 msgr2=0x561726d3c3d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561726d39fe0 0x561726d3c3d0 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd388036950 msgr2=0x7fd388038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd388036950 0x7fd388038e10 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8052fd0 tx=0x7fd4a804f000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 msgr2=0x561726d29e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4bda30640 1 -- 172.21.15.165:0/4178876637 reap_dead start 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 0x561726d29e40 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fd4b003cd90 tx=0x7fd4b002b0c0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 shutdown_connections 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 --2- 172.21.15.165:0/4178876637 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d22bd0 0x561726d29e40 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 >> 172.21.15.165:0/4178876637 conn(0x561726d1ef10 msgr2=0x561726d1c570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 shutdown_connections 2024-04-27T03:30:36.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/4178876637 wait complete. 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/468087218 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd394036970 msgr2=0x7fd394038e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 --2- 172.21.15.165:0/468087218 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd394036970 0x7fd394038e30 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x561726bfb5f0 tx=0x7fd4a8005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/468087218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d15890 msgr2=0x561726d15cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 --2- 172.21.15.165:0/468087218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d15890 0x561726d15cb0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fd4b004b4b0 tx=0x7fd4b001aaf0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/468087218 shutdown_connections 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 --2- 172.21.15.165:0/468087218 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd394036970 0x7fd394038e30 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 --2- 172.21.15.165:0/468087218 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726d15890 0x561726d15cb0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/468087218 >> 172.21.15.165:0/468087218 conn(0x561726d09930 msgr2=0x561726d09d30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/468087218 shutdown_connections 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.431+0000 7fd4c1531800 1 -- 172.21.15.165:0/468087218 wait complete. 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561726d048b0 msgr2=0x561726d06d20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561726d048b0 0x561726d06d20 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561726c00150 msgr2=0x561726cfde90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561726c00150 0x561726cfde90 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561726d00760 msgr2=0x561726d02c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561726d00760 0x561726d02c20 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd3a4036900 msgr2=0x7fd3a4038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd3a4036900 0x7fd3a4038dc0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fd4a8054000 tx=0x7fd4a804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 msgr2=0x561726bfc330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 --2- 172.21.15.165:0/1407047807 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561726b6f1e0 0x561726bfc330 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fd4b000b210 tx=0x7fd4b0000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4bda30640 1 -- 172.21.15.165:0/1407047807 reap_dead start 2024-04-27T03:30:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 shutdown_connections 2024-04-27T03:30:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 >> 172.21.15.165:0/1407047807 conn(0x561726bf63a0 msgr2=0x561726ce7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 shutdown_connections 2024-04-27T03:30:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.432+0000 7fd4c1531800 1 -- 172.21.15.165:0/1407047807 wait complete. 2024-04-27T03:30:36.450 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-27T03:30:36.450 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--tenant', 'testx', '--uid', 'tenanteduser'] 2024-04-27T03:30:36.451 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --tenant testx --uid tenanteduser 2024-04-27T03:30:36.495 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:36.495 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.490+0000 7f49f3936800 1 Processor -- start 2024-04-27T03:30:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.490+0000 7f49f3936800 1 -- start start 2024-04-27T03:30:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.490+0000 7f49f3936800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5574c77ddd40 0x5574c77de160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.490+0000 7f49f3936800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5574c769dd40 con 0x5574c77defa0 2024-04-27T03:30:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.490+0000 7f49f3936800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5574c76704d0 con 0x5574c77ddd40 2024-04-27T03:30:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.491+0000 7f49ef6cc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5574c77ddd40 0x5574c77de160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.491+0000 7f49eeecb640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5574c77defa0 0x5574c77df420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35396/0 (socket says 172.21.15.165:35396) 2024-04-27T03:30:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.491+0000 7f49eeecb640 1 -- 172.21.15.165:0/1666477552 learned_addr learned my addr 172.21.15.165:0/1666477552 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.491+0000 7f49ef6cc640 1 -- 172.21.15.165:0/1666477552 >> v1:172.21.15.89:6789/0 conn(0x5574c77defa0 legacy=0x5574c77df420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.491+0000 7f49ee6ca640 1 -- 172.21.15.165:0/1666477552 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2824511910 0 0) 0x5574c769dd40 con 0x5574c77defa0 2024-04-27T03:30:36.497 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.491+0000 7f49ef6cc640 1 -- 172.21.15.165:0/1666477552 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c769c590 con 0x5574c77ddd40 2024-04-27T03:30:36.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.493+0000 7f49ef6cc640 1 --2- 172.21.15.165:0/1666477552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c77de160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f49e000a470 tx=0x7f49e001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f092ddfab23516d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.493+0000 7f49ee6ca640 1 -- 172.21.15.165:0/1666477552 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0057070 con 0x5574c77ddd40 2024-04-27T03:30:36.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.493+0000 7f49ee6ca640 1 -- 172.21.15.165:0/1666477552 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e0052570 con 0x5574c77ddd40 2024-04-27T03:30:36.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.493+0000 7f49ee6ca640 1 -- 172.21.15.165:0/1666477552 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0052870 con 0x5574c77ddd40 2024-04-27T03:30:36.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.494+0000 7f49f3936800 1 -- 172.21.15.165:0/1666477552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 msgr2=0x5574c77de160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.494+0000 7f49f3936800 1 --2- 172.21.15.165:0/1666477552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c77de160 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f49e000a470 tx=0x7f49e001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:36.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.494+0000 7f49f3936800 1 -- 172.21.15.165:0/1666477552 shutdown_connections 2024-04-27T03:30:36.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.494+0000 7f49f3936800 1 --2- 172.21.15.165:0/1666477552 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c77de160 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.494+0000 7f49f3936800 1 -- 172.21.15.165:0/1666477552 >> 172.21.15.165:0/1666477552 conn(0x5574c77d93a0 msgr2=0x5574c77db7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.494+0000 7f49f3936800 1 -- 172.21.15.165:0/1666477552 shutdown_connections 2024-04-27T03:30:36.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.494+0000 7f49f3936800 1 -- 172.21.15.165:0/1666477552 wait complete. 2024-04-27T03:30:36.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.505+0000 7f49f3936800 1 Processor -- start 2024-04-27T03:30:36.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.505+0000 7f49f3936800 1 -- start start 2024-04-27T03:30:36.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.506+0000 7f49f3936800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c78cf2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.506+0000 7f49f3936800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5574c772a3b0 con 0x5574c77ddd40 2024-04-27T03:30:36.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.506+0000 7f49ef6cc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c78cf2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.506+0000 7f49ef6cc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c78cf2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59202/0 (socket says 172.21.15.165:59202) 2024-04-27T03:30:36.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.506+0000 7f49ef6cc640 1 -- 172.21.15.165:0/2310946475 learned_addr learned my addr 172.21.15.165:0/2310946475 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.506+0000 7f49ef6cc640 1 -- 172.21.15.165:0/2310946475 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c78cf880 con 0x5574c77ddd40 2024-04-27T03:30:36.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.506+0000 7f49ef6cc640 1 --2- 172.21.15.165:0/2310946475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c78cf2b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f49e0055040 tx=0x7f49e004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=330c4e4e0da1c79 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.507+0000 7f48dd60a640 1 -- 172.21.15.165:0/2310946475 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0057070 con 0x5574c77ddd40 2024-04-27T03:30:36.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.507+0000 7f48dd60a640 1 -- 172.21.15.165:0/2310946475 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e0063070 con 0x5574c77ddd40 2024-04-27T03:30:36.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.507+0000 7f48dd60a640 1 -- 172.21.15.165:0/2310946475 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0052720 con 0x5574c77ddd40 2024-04-27T03:30:36.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.507+0000 7f49f3936800 1 -- 172.21.15.165:0/2310946475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 msgr2=0x5574c78cf2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.507+0000 7f49f3936800 1 --2- 172.21.15.165:0/2310946475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c78cf2b0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f49e0055040 tx=0x7f49e004c000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.507+0000 7f49f3936800 1 -- 172.21.15.165:0/2310946475 shutdown_connections 2024-04-27T03:30:36.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.507+0000 7f49f3936800 1 --2- 172.21.15.165:0/2310946475 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c78cf2b0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.507+0000 7f49f3936800 1 -- 172.21.15.165:0/2310946475 >> 172.21.15.165:0/2310946475 conn(0x5574c77d93a0 msgr2=0x5574c78cabd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.507+0000 7f49f3936800 1 -- 172.21.15.165:0/2310946475 shutdown_connections 2024-04-27T03:30:36.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.507+0000 7f49f3936800 1 -- 172.21.15.165:0/2310946475 wait complete. 2024-04-27T03:30:36.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.508+0000 7f49f3936800 1 Processor -- start 2024-04-27T03:30:36.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.508+0000 7f49f3936800 1 -- start start 2024-04-27T03:30:36.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.508+0000 7f49f3936800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c78d5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.508+0000 7f49f3936800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5574c77e7440 con 0x5574c77ddd40 2024-04-27T03:30:36.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.508+0000 7f49ef6cc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c78d5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.508+0000 7f49ef6cc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c78d5bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59212/0 (socket says 172.21.15.165:59212) 2024-04-27T03:30:36.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.508+0000 7f49ef6cc640 1 -- 172.21.15.165:0/1087040668 learned_addr learned my addr 172.21.15.165:0/1087040668 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.508+0000 7f49ef6cc640 1 -- 172.21.15.165:0/1087040668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c78d6110 con 0x5574c77ddd40 2024-04-27T03:30:36.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.508+0000 7f49ef6cc640 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c78d5bd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f49e0060040 tx=0x7f49e0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.509+0000 7f48dbe07640 1 -- 172.21.15.165:0/1087040668 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0052c70 con 0x5574c77ddd40 2024-04-27T03:30:36.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.509+0000 7f48dbe07640 1 -- 172.21.15.165:0/1087040668 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e006c070 con 0x5574c77ddd40 2024-04-27T03:30:36.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.509+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5574c78d63a0 con 0x5574c77ddd40 2024-04-27T03:30:36.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.509+0000 7f48dbe07640 1 -- 172.21.15.165:0/1087040668 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e005e480 con 0x5574c77ddd40 2024-04-27T03:30:36.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.509+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c78d9050 con 0x5574c77ddd40 2024-04-27T03:30:36.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.509+0000 7f48dbe07640 1 -- 172.21.15.165:0/1087040668 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f49e005e620 con 0x5574c77ddd40 2024-04-27T03:30:36.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.510+0000 7f48dbe07640 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48d4038b60 0x7f48d403b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.510+0000 7f48dbe07640 1 -- 172.21.15.165:0/1087040668 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f49e008e4b0 con 0x5574c77ddd40 2024-04-27T03:30:36.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.510+0000 7f49eeecb640 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48d4038b60 0x7f48d403b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.510+0000 7f49f3936800 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574c77defa0 0x5574c78dce70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.510+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c78dd440 con 0x5574c77defa0 2024-04-27T03:30:36.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.510+0000 7f49efecd640 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574c77defa0 0x5574c78dce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.510+0000 7f49eeecb640 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48d4038b60 0x7f48d403b020 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f49e4000c00 tx=0x7f49e40013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.510+0000 7f49efecd640 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574c77defa0 0x5574c78dce70 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.510+0000 7f49efecd640 1 -- 172.21.15.165:0/1087040668 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c77defa0 2024-04-27T03:30:36.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.511+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c78da400 con 0x5574c77defa0 2024-04-27T03:30:36.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.511+0000 7f49efecd640 1 -- 172.21.15.165:0/1087040668 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c77defa0 2024-04-27T03:30:36.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.511+0000 7f49f3936800 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574c78df730 0x5574c78e1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.511+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c78e2180 con 0x5574c78df730 2024-04-27T03:30:36.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.511+0000 7f49ef6cc640 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574c78df730 0x5574c78e1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.511+0000 7f49ef6cc640 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574c78df730 0x5574c78e1b20 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.511+0000 7f49ef6cc640 1 -- 172.21.15.165:0/1087040668 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f49e0009d10 con 0x5574c78df730 2024-04-27T03:30:36.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.512+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c78de6e0 con 0x5574c77defa0 2024-04-27T03:30:36.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.512+0000 7f49efecd640 1 -- 172.21.15.165:0/1087040668 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c77defa0 2024-04-27T03:30:36.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.512+0000 7f49f3936800 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5574c78e3630 0x5574c78e5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.512+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c78e61e0 con 0x5574c78e3630 2024-04-27T03:30:36.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.512+0000 7f49eeecb640 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5574c78e3630 0x5574c78e5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.512+0000 7f49eeecb640 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5574c78e3630 0x5574c78e5a80 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.513+0000 7f49eeecb640 1 -- 172.21.15.165:0/1087040668 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f49e400f670 con 0x5574c78e3630 2024-04-27T03:30:36.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.513+0000 7f49f3936800 1 Processor -- start 2024-04-27T03:30:36.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.513+0000 7f49f3936800 1 -- start start 2024-04-27T03:30:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.513+0000 7f49f3936800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c78f8a50 0x5574c78f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.513+0000 7f49f3936800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5574c76d06b0 con 0x5574c78f8a50 2024-04-27T03:30:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.513+0000 7f49ef6cc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c78f8a50 0x5574c78f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.513+0000 7f49ef6cc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c78f8a50 0x5574c78f8e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59228/0 (socket says 172.21.15.165:59228) 2024-04-27T03:30:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.513+0000 7f49ef6cc640 1 -- 172.21.15.165:0/3017681682 learned_addr learned my addr 172.21.15.165:0/3017681682 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.514+0000 7f49ef6cc640 1 -- 172.21.15.165:0/3017681682 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c78f93b0 con 0x5574c78f8a50 2024-04-27T03:30:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.514+0000 7f49ef6cc640 1 --2- 172.21.15.165:0/3017681682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c78f8a50 0x5574c78f8e70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f49e008cab0 tx=0x7f49e008e3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.514+0000 7f48d9602640 1 -- 172.21.15.165:0/3017681682 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0081040 con 0x5574c78f8a50 2024-04-27T03:30:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.514+0000 7f48d9602640 1 -- 172.21.15.165:0/3017681682 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e0058040 con 0x5574c78f8a50 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.514+0000 7f48d9602640 1 -- 172.21.15.165:0/3017681682 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0079890 con 0x5574c78f8a50 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.514+0000 7f49f3936800 1 -- 172.21.15.165:0/3017681682 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5574c78f9e20 con 0x5574c78f8a50 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.514+0000 7f49f3936800 1 -- 172.21.15.165:0/3017681682 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c78fa320 con 0x5574c78f8a50 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.515+0000 7f48d9602640 1 -- 172.21.15.165:0/3017681682 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f49e0095060 con 0x5574c78f8a50 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.515+0000 7f48d9602640 1 --2- 172.21.15.165:0/3017681682 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48c8036950 0x7f48c8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.515+0000 7f48d9602640 1 -- 172.21.15.165:0/3017681682 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f49e004c0e0 con 0x5574c78f8a50 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.515+0000 7f49eeecb640 1 --2- 172.21.15.165:0/3017681682 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48c8036950 0x7f48c8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.515+0000 7f49f3936800 1 Processor -- start 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.515+0000 7f49f3936800 1 -- start start 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.515+0000 7f49eeecb640 1 --2- 172.21.15.165:0/3017681682 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48c8036950 0x7f48c8038e10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5574c78da360 tx=0x7f49e4006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.515+0000 7f49f3936800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 0x5574c7906020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.515+0000 7f49f3936800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5574c78cfb10 con 0x5574c7905c00 2024-04-27T03:30:36.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.515+0000 7f49ef6cc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 0x5574c7906020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.515+0000 7f49ef6cc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 0x5574c7906020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59234/0 (socket says 172.21.15.165:59234) 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.516+0000 7f49ef6cc640 1 -- 172.21.15.165:0/2310752264 learned_addr learned my addr 172.21.15.165:0/2310752264 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.516+0000 7f49ef6cc640 1 -- 172.21.15.165:0/2310752264 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c79065f0 con 0x5574c7905c00 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.516+0000 7f49ef6cc640 1 --2- 172.21.15.165:0/2310752264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 0x5574c7906020 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f49e0066bb0 tx=0x7f49e0097af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78e5730b5c33e814 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.516+0000 7f48d2ffd640 1 -- 172.21.15.165:0/2310752264 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e005ee10 con 0x5574c7905c00 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.516+0000 7f48d2ffd640 1 -- 172.21.15.165:0/2310752264 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e00a0070 con 0x5574c7905c00 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.516+0000 7f48d2ffd640 1 -- 172.21.15.165:0/2310752264 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0097d00 con 0x5574c7905c00 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.516+0000 7f49f3936800 1 -- 172.21.15.165:0/2310752264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 msgr2=0x5574c7906020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.516+0000 7f49f3936800 1 --2- 172.21.15.165:0/2310752264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 0x5574c7906020 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f49e0066bb0 tx=0x7f49e0097af0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.516+0000 7f49f3936800 1 -- 172.21.15.165:0/2310752264 shutdown_connections 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.516+0000 7f49f3936800 1 --2- 172.21.15.165:0/2310752264 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 0x5574c7906020 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.516+0000 7f49f3936800 1 -- 172.21.15.165:0/2310752264 >> 172.21.15.165:0/2310752264 conn(0x5574c7901f10 msgr2=0x5574c78fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.517+0000 7f49f3936800 1 -- 172.21.15.165:0/2310752264 shutdown_connections 2024-04-27T03:30:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.517+0000 7f49f3936800 1 -- 172.21.15.165:0/2310752264 wait complete. 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.517+0000 7f49f3936800 1 Processor -- start 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.517+0000 7f49f3936800 1 -- start start 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.517+0000 7f49f3936800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 0x5574c790d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.517+0000 7f49f3936800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5574c7906c80 con 0x5574c7905c00 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.517+0000 7f49ef6cc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 0x5574c790d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.517+0000 7f49ef6cc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 0x5574c790d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59240/0 (socket says 172.21.15.165:59240) 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.517+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 learned_addr learned my addr 172.21.15.165:0/803476657 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.517+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c790d550 con 0x5574c7905c00 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.518+0000 7f49ef6cc640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 0x5574c790d010 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f49e0074000 tx=0x7f49e0062f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.518+0000 7f48d17fa640 1 -- 172.21.15.165:0/803476657 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e00b5070 con 0x5574c7905c00 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.518+0000 7f48d17fa640 1 -- 172.21.15.165:0/803476657 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49e0088880 con 0x5574c7905c00 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.518+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5574c790d550 con 0x5574c7905c00 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.518+0000 7f48d17fa640 1 -- 172.21.15.165:0/803476657 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49e0088b80 con 0x5574c7905c00 2024-04-27T03:30:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.518+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c790d550 con 0x5574c7905c00 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.518+0000 7f48d17fa640 1 -- 172.21.15.165:0/803476657 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f49e00b8070 con 0x5574c7905c00 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.519+0000 7f48d17fa640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48b8036900 0x7f48b8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.519+0000 7f48d17fa640 1 -- 172.21.15.165:0/803476657 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f49e00b1ae0 con 0x5574c7905c00 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.519+0000 7f49eeecb640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48b8036900 0x7f48b8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.519+0000 7f49f3936800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.519+0000 7f49f3936800 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574c79186e0 0x5574c791aba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.519+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c791b2f0 con 0x5574c79186e0 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.519+0000 7f49eeecb640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48b8036900 0x7f48b8038dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f49e400ffd0 tx=0x7f49e4023040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.519+0000 7f49efecd640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574c79186e0 0x5574c791aba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49efecd640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574c79186e0 0x5574c791aba0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 20 realm 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c791b700 con 0x5574c79186e0 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c791bfd0 con 0x5574c79186e0 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c79186e0 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.520+0000 7f49f3936800 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574c791d0d0 0x5574c791f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.521+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c791fad0 con 0x5574c791d0d0 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.521+0000 7f49ef6cc640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574c791d0d0 0x5574c791f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.521+0000 7f49ef6cc640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574c791d0d0 0x5574c791f4c0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.521+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.521+0000 7f49f3936800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.521+0000 7f49f3936800 20 searching for the correct realm 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.521+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c7921400 con 0x5574c791d0d0 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.522+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.522+0000 7f49ef6cc640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f49e00ba060 0x7f49e00a5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.522+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c7921400 con 0x7f49e00ba060 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.522+0000 7f49eeecb640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f49e00ba060 0x7f49e00a5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.522+0000 7f49eeecb640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f49e00ba060 0x7f49e00a5470 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.522+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.522+0000 7f49eeecb640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f49e402e910 0x7f49e403a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.522+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c7921400 con 0x7f49e402e910 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.523+0000 7f49efecd640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f49e402e910 0x7f49e403a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.523+0000 7f49efecd640 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f49e402e910 0x7f49e403a3c0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.523+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.523+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c79186e0 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.523+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.523+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c79186e0 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.524+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.524+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x7f49e402e910 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.524+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.524+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x7f49e402e910 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.524+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.524+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x7f49e00ba060 2024-04-27T03:30:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.524+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.524+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c791d0d0 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.525+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.525+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c79186e0 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.525+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.525+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x7f49e402e910 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.525+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.525+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c79186e0 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.525+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.525+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x7f49e00ba060 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.525+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.525+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c791d0d0 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.526+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.526+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c791d0d0 2024-04-27T03:30:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.526+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.526+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x7f49e402e910 2024-04-27T03:30:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.526+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.526+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c791d0d0 2024-04-27T03:30:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.526+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.526+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c791d0d0 2024-04-27T03:30:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.527+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.527+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c79186e0 2024-04-27T03:30:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.527+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.527+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c79186e0 2024-04-27T03:30:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.527+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.527+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x7f49e402e910 2024-04-27T03:30:36.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.528+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.528+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c791d0d0 2024-04-27T03:30:36.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.528+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.528+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x7f49e00ba060 2024-04-27T03:30:36.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.528+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.528+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c79186e0 2024-04-27T03:30:36.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.528+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.528+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x7f49e402e910 2024-04-27T03:30:36.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.529+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.529+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c791d0d0 2024-04-27T03:30:36.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.529+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.529+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x7f49e402e910 2024-04-27T03:30:36.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.529+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.529+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x7f49e00ba060 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.529+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.529+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c791d0d0 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x7f49e402e910 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c79186e0 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f49e8038040 con 0x7f49e00ba060 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49f3936800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49f3936800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49f3936800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:36.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49f3936800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49f3936800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49f3936800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49f3936800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c791c370 con 0x5574c79186e0 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.530+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c7920390 con 0x5574c79186e0 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c7920390 con 0x7f49e402e910 2024-04-27T03:30:36.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:36.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 20 zone default found 2024-04-27T03:30:36.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 4 Realm: () 2024-04-27T03:30:36.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:36.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:36.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:36.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 20 zonegroup default 2024-04-27T03:30:36.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c7920cf0 con 0x7f49e402e910 2024-04-27T03:30:36.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.531+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c7920cf0 con 0x5574c791d0d0 2024-04-27T03:30:36.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.532+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.532+0000 7f49f3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.532+0000 7f49f3936800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.532+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c7923260 con 0x5574c791d0d0 2024-04-27T03:30:36.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.532+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.532+0000 7f49f3936800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.532+0000 7f49f3936800 20 started sync module instance, tier type = 2024-04-27T03:30:36.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.532+0000 7f49f3936800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:36.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.532+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7922700 con 0x7f49e00ba060 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.533+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.533+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93960052810320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7923600 con 0x7f49e00ba060 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.533+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7923fc0 con 0x5574c79186e0 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.534+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93960052810320] v25'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.534+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.534+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93960052816656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c79243e0 con 0x5574c79186e0 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.534+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7924da0 con 0x5574c791d0d0 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.535+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93960052816656] v25'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.535+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.535+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93960052817712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7924350 con 0x5574c791d0d0 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.535+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7925c60 con 0x7f49e00ba060 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.536+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.536+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93960052823984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c79264b0 con 0x7f49e00ba060 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.536+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7926f80 con 0x5574c79186e0 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.537+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93960052817712] v25'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.537+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93960052823984] v25'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.538+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.538+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93960052826112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7925190 con 0x5574c79186e0 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.538+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7927990 con 0x7f49e402e910 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.538+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.538+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93960052831456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7927db0 con 0x7f49e402e910 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.539+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c79287d0 con 0x7f49e402e910 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.539+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93960052826112] v25'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.539+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93960052831456] v25'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.540+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.540+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93960052835008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7927900 con 0x7f49e402e910 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.540+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7929070 con 0x7f49e402e910 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.541+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93960052835008] v25'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.541+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'141 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.541+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93960052828784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7927900 con 0x7f49e402e910 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.541+0000 7f49f3936800 20 add_watcher() i=0 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.541+0000 7f49f3936800 20 add_watcher() i=1 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.541+0000 7f49f3936800 20 add_watcher() i=2 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.541+0000 7f49f3936800 20 add_watcher() i=3 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.541+0000 7f49f3936800 20 add_watcher() i=4 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.541+0000 7f49f3936800 20 add_watcher() i=5 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.541+0000 7f49f3936800 20 add_watcher() i=6 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.542+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93960052828784] v25'142 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.542+0000 7f49f3936800 20 add_watcher() i=7 2024-04-27T03:30:36.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.542+0000 7f49f3936800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.542+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c792c060 con 0x5574c79186e0 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.542+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.542+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93960052837104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7929580 con 0x5574c79186e0 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.543+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93960052837104] v25'140 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.543+0000 7f48a5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.543+0000 7f49f3936800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.543+0000 7f49f3936800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.543+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c792bd30 con 0x5574c79186e0 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.545+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'46 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.545+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c79350b0 con 0x7f49e402e910 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.546+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'22 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.546+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c79350b0 con 0x5574c791d0d0 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.547+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'23 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.547+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c79350b0 con 0x5574c79186e0 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.548+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'141 uv15 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.548+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c79350b0 con 0x7f49e00ba060 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.549+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'44 uv17 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.549+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7935fc0 con 0x5574c791d0d0 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.550+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'38 uv16 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.550+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7936380 con 0x5574c79186e0 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.551+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'24 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.551+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7936380 con 0x7f49e402e910 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.552+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'31 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.552+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7936380 con 0x7f49e00ba060 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.553+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'16 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.553+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7936830 con 0x5574c791d0d0 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.554+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'22 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.554+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7936830 con 0x7f49e00ba060 2024-04-27T03:30:36.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.555+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'45 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.555+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7936da0 con 0x7f49e402e910 2024-04-27T03:30:36.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.556+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'18 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.556+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7936da0 con 0x7f49e00ba060 2024-04-27T03:30:36.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.557+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'22 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.557+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x7f49e402e910 2024-04-27T03:30:36.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.558+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'43 uv22 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.558+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x5574c79186e0 2024-04-27T03:30:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.559+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'142 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.559+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x5574c791d0d0 2024-04-27T03:30:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.560+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'39 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.560+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x7f49e00ba060 2024-04-27T03:30:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.561+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'46 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.561+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x5574c791d0d0 2024-04-27T03:30:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.562+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'28 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.562+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x5574c79186e0 2024-04-27T03:30:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.563+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'24 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.563+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x5574c79186e0 2024-04-27T03:30:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.564+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'143 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.564+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x5574c791d0d0 2024-04-27T03:30:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.565+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'40 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.565+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x7f49e00ba060 2024-04-27T03:30:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.566+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'32 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.566+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x5574c791d0d0 2024-04-27T03:30:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.567+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'24 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.567+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x5574c791d0d0 2024-04-27T03:30:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.568+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'35 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.568+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x5574c791d0d0 2024-04-27T03:30:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.569+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'18 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.569+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x5574c791d0d0 2024-04-27T03:30:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.570+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'39 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.570+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x7f49e402e910 2024-04-27T03:30:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.571+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'24 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.571+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x5574c791d0d0 2024-04-27T03:30:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.572+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'28 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.572+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x7f49e402e910 2024-04-27T03:30:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.573+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'32 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.573+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x5574c791d0d0 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.574+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'41 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.574+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x7f49e402e910 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.576+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'44 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.576+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7937250 con 0x7f49e00ba060 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.577+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'22 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.577+0000 7f48a4fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.577+0000 7f49f3936800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.577+0000 7f49f3936800 20 Filter name: none 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.577+0000 7f481e7fc640 20 reqs_thread_entry: start 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.577+0000 7f49f3936800 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser : miss 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.577+0000 7f49f3936800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.577+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:94 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c7962280 con 0x7f49e402e910 2024-04-27T03:30:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.578+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(94 testx$tenanteduser [call out=48b,read 0~419 out=419b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 246+0+542 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.578+0000 7f49f3936800 20 rados_obj.operate() r=0 bl.length=419 2024-04-27T03:30:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.578+0000 7f49f3936800 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser info.flags=0x13 2024-04-27T03:30:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.578+0000 7f49f3936800 10 adding default.rgw.meta+users.uid+testx$tenanteduser to cache LRU end 2024-04-27T03:30:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.578+0000 7f49f3936800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.578+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:95 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c7961d60 con 0x5574c79186e0 2024-04-27T03:30:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.579+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(95 testx$tenanteduser.buckets [call out=15b] v0'0 uv2 ondisk = 0) ==== 170+0+15 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.579+0000 7f49f3936800 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-04-27T03:30:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.579+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:96 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7961d60 con 0x7f49e402e910 2024-04-27T03:30:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.580+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(96 p5YnriCv1nAtykxBrupQ [delete] v25'7 uv7 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.580+0000 7f49f3936800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T03:30:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.580+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [notify cookie 93960053078016 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c79650e0 con 0x5574c791d0d0 2024-04-27T03:30:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.580+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== watch-notify(notify (1) cookie 93960052817712 notify 107374182405 ret 0) ==== 212+0+0 (crc 0 0 0) 0x5574c7935ba0 con 0x5574c791d0d0 2024-04-27T03:30:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.580+0000 7f48d2ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182405 cookie 93960052817712 notifier 4553 bl.length()=170 2024-04-27T03:30:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.580+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(97 notify.2 [notify cookie 93960053078016 out=8b] v0'0 uv141 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.580+0000 7f48d2ffd640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:98 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f48b4003d60 con 0x5574c791d0d0 2024-04-27T03:30:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.580+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 29 ==== osd_op_reply(98 notify.2 [notify-ack] v0'0 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.580+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 30 ==== watch-notify(notify_complete (2) cookie 93960053078016 notify 107374182405 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5574c78f93b0 con 0x5574c791d0d0 2024-04-27T03:30:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.580+0000 7f49f3936800 10 removing email index: 2024-04-27T03:30:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.580+0000 7f49f3936800 10 removing user buckets index 2024-04-27T03:30:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.580+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:99 8.c 8:37c7a57e:users.uid::testx$tenanteduser.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7962c80 con 0x5574c79186e0 2024-04-27T03:30:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.581+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(99 testx$tenanteduser.buckets [delete] v25'3 uv3 ondisk = 0) ==== 170+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.581+0000 7f49f3936800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser.buckets, ofs0, ns] 2024-04-27T03:30:36.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.581+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:100 7.d 7:b4812045:::notify.4:head [notify cookie 93960053077984 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c79650c0 con 0x5574c79186e0 2024-04-27T03:30:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.582+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== watch-notify(notify (1) cookie 93960052826112 notify 107374182424 ret 0) ==== 217+0+0 (crc 0 0 0) 0x5574c7936380 con 0x5574c79186e0 2024-04-27T03:30:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.582+0000 7f48d27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182424 cookie 93960052826112 notifier 4553 bl.length()=175 2024-04-27T03:30:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.582+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(100 notify.4 [notify cookie 93960053077984 out=8b] v0'0 uv141 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.582+0000 7f48d27fc640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f48240032c0 con 0x5574c79186e0 2024-04-27T03:30:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.582+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(101 notify.4 [notify-ack] v0'0 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.582+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== watch-notify(notify_complete (2) cookie 93960053077984 notify 107374182424 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5574c792ba40 con 0x5574c79186e0 2024-04-27T03:30:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.582+0000 7f49f3936800 10 removing user index: testx$tenanteduser 2024-04-27T03:30:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.582+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 8.13 8:cdfb9947:users.uid::testx$tenanteduser:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7966d30 con 0x7f49e402e910 2024-04-27T03:30:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.584+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(102 testx$tenanteduser [call,call,delete] v25'8 uv8 ondisk = 0) ==== 246+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.584+0000 7f49f3936800 10 removing default.rgw.meta+users.uid+testx$tenanteduser from cache 2024-04-27T03:30:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.584+0000 7f49f3936800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.uid:testx$tenanteduser, ofs0, ns] 2024-04-27T03:30:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.584+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:103 7.e 7:7759931f:::notify.3:head [notify cookie 93960053073872 in=179b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574c79640b0 con 0x7f49e00ba060 2024-04-27T03:30:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.584+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== watch-notify(notify (1) cookie 93960052823984 notify 107374182415 ret 0) ==== 209+0+0 (crc 0 0 0) 0x5574c7936550 con 0x7f49e00ba060 2024-04-27T03:30:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.584+0000 7f48d27fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182415 cookie 93960052823984 notifier 4553 bl.length()=167 2024-04-27T03:30:36.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.584+0000 7f48d27fc640 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:104 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f48240032f0 con 0x7f49e00ba060 2024-04-27T03:30:36.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.584+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(103 notify.3 [notify cookie 93960053073872 out=8b] v0'0 uv141 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.584+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== watch-notify(notify_complete (2) cookie 93960053073872 notify 107374182415 ret 0) ==== 42+0+48 (crc 0 0 0) 0x5574c7936bd0 con 0x7f49e00ba060 2024-04-27T03:30:36.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.584+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== osd_op_reply(104 notify.3 [notify-ack] v0'0 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.585+0000 7f48a5fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.585+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:105 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93960052837104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7960c90 con 0x5574c79186e0 2024-04-27T03:30:36.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.586+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(105 data_loggenerations_metadata [watch unwatch cookie 93960052837104] v25'144 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.586+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:106 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93960052810320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7928af0 con 0x7f49e00ba060 2024-04-27T03:30:36.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.587+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(106 notify.0 [watch unwatch cookie 93960052810320] v25'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.587+0000 7f49f3936800 20 remove_watcher() i=0 2024-04-27T03:30:36.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.587+0000 7f49f3936800 2 removed watcher, disabling cache 2024-04-27T03:30:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.587+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:107 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93960052816656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7922250 con 0x5574c79186e0 2024-04-27T03:30:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.588+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(107 notify.1 [watch unwatch cookie 93960052816656] v25'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.588+0000 7f49f3936800 20 remove_watcher() i=1 2024-04-27T03:30:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.588+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:108 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93960052817712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7922250 con 0x5574c791d0d0 2024-04-27T03:30:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.589+0000 7f49ef6cc640 1 -- 172.21.15.165:0/803476657 <== osd.3 v2:172.21.15.165:6808/2380918637 31 ==== osd_op_reply(108 notify.2 [watch unwatch cookie 93960052817712] v25'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e00b7030 con 0x5574c791d0d0 2024-04-27T03:30:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.589+0000 7f49f3936800 20 remove_watcher() i=2 2024-04-27T03:30:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.589+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:109 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93960052823984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7922250 con 0x7f49e00ba060 2024-04-27T03:30:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.590+0000 7f49eeecb640 1 -- 172.21.15.165:0/803476657 <== osd.2 v2:172.21.15.165:6816/975457786 23 ==== osd_op_reply(109 notify.3 [watch unwatch cookie 93960052823984] v25'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e4037040 con 0x7f49e00ba060 2024-04-27T03:30:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.590+0000 7f49f3936800 20 remove_watcher() i=3 2024-04-27T03:30:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.590+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:110 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93960052826112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7922250 con 0x5574c79186e0 2024-04-27T03:30:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.591+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(110 notify.4 [watch unwatch cookie 93960052826112] v25'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x5574c79186e0 2024-04-27T03:30:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.591+0000 7f49f3936800 20 remove_watcher() i=4 2024-04-27T03:30:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.591+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:111 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93960052831456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7922250 con 0x7f49e402e910 2024-04-27T03:30:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.592+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(111 notify.5 [watch unwatch cookie 93960052831456] v25'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.592+0000 7f49f3936800 20 remove_watcher() i=5 2024-04-27T03:30:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.592+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:112 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93960052835008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c79274e0 con 0x7f49e402e910 2024-04-27T03:30:36.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.593+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== osd_op_reply(112 notify.6 [watch unwatch cookie 93960052835008] v25'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.593+0000 7f49f3936800 20 remove_watcher() i=6 2024-04-27T03:30:36.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.593+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:113 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93960052828784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574c7922250 con 0x7f49e402e910 2024-04-27T03:30:36.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 <== osd.0 v2:172.21.15.89:6800/2353668432 30 ==== osd_op_reply(113 notify.7 [watch unwatch cookie 93960052828784] v25'143 uv141 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f49e800a4a0 con 0x7f49e402e910 2024-04-27T03:30:36.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 20 remove_watcher() i=7 2024-04-27T03:30:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f49e402e910 msgr2=0x7f49e403a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f49e402e910 0x7f49e403a3c0 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574c79186e0 msgr2=0x5574c791aba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574c79186e0 0x5574c791aba0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f49e00ba060 msgr2=0x7f49e00a5470 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f49e00ba060 0x7f49e00a5470 crc :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574c791d0d0 msgr2=0x5574c791f4c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574c791d0d0 0x5574c791f4c0 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48b8036900 msgr2=0x7f48b8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48b8036900 0x7f48b8038dc0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f49e400ffd0 tx=0x7f49e4023040 comp rx=0 tx=0).stop 2024-04-27T03:30:36.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 msgr2=0x5574c790d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 --2- 172.21.15.165:0/803476657 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c7905c00 0x5574c790d010 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f49e0074000 tx=0x7f49e0062f00 comp rx=0 tx=0).stop 2024-04-27T03:30:36.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 reap_dead start 2024-04-27T03:30:36.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49efecd640 1 -- 172.21.15.165:0/803476657 reap_dead start 2024-04-27T03:30:36.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 shutdown_connections 2024-04-27T03:30:36.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 >> 172.21.15.165:0/803476657 conn(0x5574c7901f10 msgr2=0x5574c78ff570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 shutdown_connections 2024-04-27T03:30:36.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/803476657 wait complete. 2024-04-27T03:30:36.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/3017681682 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48c8036950 msgr2=0x7f48c8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 --2- 172.21.15.165:0/3017681682 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48c8036950 0x7f48c8038e10 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x5574c78da360 tx=0x7f49e4006000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/3017681682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c78f8a50 msgr2=0x5574c78f8e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 --2- 172.21.15.165:0/3017681682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c78f8a50 0x5574c78f8e70 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f49e008cab0 tx=0x7f49e008e3a0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/3017681682 shutdown_connections 2024-04-27T03:30:36.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 --2- 172.21.15.165:0/3017681682 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48c8036950 0x7f48c8038e10 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 --2- 172.21.15.165:0/3017681682 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c78f8a50 0x5574c78f8e70 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.595+0000 7f49f3936800 1 -- 172.21.15.165:0/3017681682 >> 172.21.15.165:0/3017681682 conn(0x5574c78e86e0 msgr2=0x5574c78e8ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 -- 172.21.15.165:0/3017681682 shutdown_connections 2024-04-27T03:30:36.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 -- 172.21.15.165:0/3017681682 wait complete. 2024-04-27T03:30:36.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5574c78e3630 msgr2=0x5574c78e5a80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5574c78e3630 0x5574c78e5a80 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574c77defa0 msgr2=0x5574c78dce70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574c77defa0 0x5574c78dce70 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574c78df730 msgr2=0x5574c78e1b20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574c78df730 0x5574c78e1b20 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48d4038b60 msgr2=0x7f48d403b020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f48d4038b60 0x7f48d403b020 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f49e4000c00 tx=0x7f49e40013b0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 msgr2=0x5574c78d5bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 --2- 172.21.15.165:0/1087040668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5574c77ddd40 0x5574c78d5bd0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f49e0060040 tx=0x7f49e0002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49efecd640 1 -- 172.21.15.165:0/1087040668 reap_dead start 2024-04-27T03:30:36.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 shutdown_connections 2024-04-27T03:30:36.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 >> 172.21.15.165:0/1087040668 conn(0x5574c77d93a0 msgr2=0x5574c78c9450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 shutdown_connections 2024-04-27T03:30:36.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.596+0000 7f49f3936800 1 -- 172.21.15.165:0/1087040668 wait complete. 2024-04-27T03:30:36.643 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-04-27T03:30:36.646 INFO:tasks.radosgw_admin:RL: get_obj myfoo foo bytes_out=0 bytes_in=4 failed=False 2024-04-27T03:30:36.649 INFO:tasks.radosgw_admin:RL: list_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:36.649 INFO:tasks.util.rgw:rgwadmin: client.0 : ['object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-04-27T03:30:36.649 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'object', 'rm', '--bucket', 'myfoo', '--object', 'four'] 2024-04-27T03:30:36.649 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph object rm --bucket myfoo --object four 2024-04-27T03:30:36.695 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:36.695 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:36.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.690+0000 7ffa99d19800 1 Processor -- start 2024-04-27T03:30:36.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.690+0000 7ffa99d19800 1 -- start start 2024-04-27T03:30:36.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.690+0000 7ffa99d19800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5595c1ca6120 0x5595c1ca6540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.690+0000 7ffa99d19800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5595c1b66d40 con 0x5595c1ca9160 2024-04-27T03:30:36.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.690+0000 7ffa99d19800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5595c1b394d0 con 0x5595c1ca6120 2024-04-27T03:30:36.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.690+0000 7ffa97818640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5595c1ca6120 0x5595c1ca6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.691+0000 7ffa97017640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5595c1ca9160 0x5595c1ca9610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35398/0 (socket says 172.21.15.165:35398) 2024-04-27T03:30:36.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.691+0000 7ffa97017640 1 -- 172.21.15.165:0/961151699 learned_addr learned my addr 172.21.15.165:0/961151699 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:36.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.691+0000 7ffa97818640 1 -- 172.21.15.165:0/961151699 >> v1:172.21.15.89:6789/0 conn(0x5595c1ca9160 legacy=0x5595c1ca9610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.691+0000 7ffa96816640 1 -- 172.21.15.165:0/961151699 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2943308252 0 0) 0x5595c1b66d40 con 0x5595c1ca9160 2024-04-27T03:30:36.697 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.691+0000 7ffa97818640 1 -- 172.21.15.165:0/961151699 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c1b65590 con 0x5595c1ca6120 2024-04-27T03:30:36.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.693+0000 7ffa97818640 1 --2- 172.21.15.165:0/961151699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1ca6540 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800a470 tx=0x7ffa8801ce50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf6fb8d4a1e39031 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.693+0000 7ffa96816640 1 -- 172.21.15.165:0/961151699 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa88057070 con 0x5595c1ca6120 2024-04-27T03:30:36.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.693+0000 7ffa96816640 1 -- 172.21.15.165:0/961151699 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa8804ad80 con 0x5595c1ca6120 2024-04-27T03:30:36.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.693+0000 7ffa96816640 1 -- 172.21.15.165:0/961151699 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa88052590 con 0x5595c1ca6120 2024-04-27T03:30:36.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.693+0000 7ffa99d19800 1 -- 172.21.15.165:0/961151699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 msgr2=0x5595c1ca6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.693+0000 7ffa99d19800 1 --2- 172.21.15.165:0/961151699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1ca6540 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7ffa8800a470 tx=0x7ffa8801ce50 comp rx=0 tx=0).stop 2024-04-27T03:30:36.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.693+0000 7ffa99d19800 1 -- 172.21.15.165:0/961151699 shutdown_connections 2024-04-27T03:30:36.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.693+0000 7ffa99d19800 1 --2- 172.21.15.165:0/961151699 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1ca6540 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.693+0000 7ffa99d19800 1 -- 172.21.15.165:0/961151699 >> 172.21.15.165:0/961151699 conn(0x5595c1ca1b20 msgr2=0x5595c1ca3f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.693+0000 7ffa99d19800 1 -- 172.21.15.165:0/961151699 shutdown_connections 2024-04-27T03:30:36.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.693+0000 7ffa99d19800 1 -- 172.21.15.165:0/961151699 wait complete. 2024-04-27T03:30:36.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.705+0000 7ffa99d19800 1 Processor -- start 2024-04-27T03:30:36.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.705+0000 7ffa99d19800 1 -- start start 2024-04-27T03:30:36.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.705+0000 7ffa99d19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1d99390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.705+0000 7ffa99d19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5595c1bf33b0 con 0x5595c1ca6120 2024-04-27T03:30:36.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.706+0000 7ffa97818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1d99390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.706+0000 7ffa97818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1d99390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59250/0 (socket says 172.21.15.165:59250) 2024-04-27T03:30:36.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.706+0000 7ffa97818640 1 -- 172.21.15.165:0/812456216 learned_addr learned my addr 172.21.15.165:0/812456216 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.706+0000 7ffa97818640 1 -- 172.21.15.165:0/812456216 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c1d99960 con 0x5595c1ca6120 2024-04-27T03:30:36.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.706+0000 7ffa97818640 1 --2- 172.21.15.165:0/812456216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1d99390 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ffa88055040 tx=0x7ffa8804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f362d257ec4017a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.706+0000 7ff985e0b640 1 -- 172.21.15.165:0/812456216 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa88062070 con 0x5595c1ca6120 2024-04-27T03:30:36.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.706+0000 7ff985e0b640 1 -- 172.21.15.165:0/812456216 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa8800a230 con 0x5595c1ca6120 2024-04-27T03:30:36.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.706+0000 7ff985e0b640 1 -- 172.21.15.165:0/812456216 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa88057040 con 0x5595c1ca6120 2024-04-27T03:30:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.707+0000 7ffa99d19800 1 -- 172.21.15.165:0/812456216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 msgr2=0x5595c1d99390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.707+0000 7ffa99d19800 1 --2- 172.21.15.165:0/812456216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1d99390 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7ffa88055040 tx=0x7ffa8804c000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.707+0000 7ffa99d19800 1 -- 172.21.15.165:0/812456216 shutdown_connections 2024-04-27T03:30:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.707+0000 7ffa99d19800 1 --2- 172.21.15.165:0/812456216 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1d99390 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.707+0000 7ffa99d19800 1 -- 172.21.15.165:0/812456216 >> 172.21.15.165:0/812456216 conn(0x5595c1ca1b20 msgr2=0x5595c1d94cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.707+0000 7ffa99d19800 1 -- 172.21.15.165:0/812456216 shutdown_connections 2024-04-27T03:30:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.707+0000 7ffa99d19800 1 -- 172.21.15.165:0/812456216 wait complete. 2024-04-27T03:30:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.707+0000 7ffa99d19800 1 Processor -- start 2024-04-27T03:30:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.707+0000 7ffa99d19800 1 -- start start 2024-04-27T03:30:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.708+0000 7ffa99d19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1ca8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.708+0000 7ffa99d19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5595c1cb15d0 con 0x5595c1ca6120 2024-04-27T03:30:36.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.708+0000 7ffa97818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1ca8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.708+0000 7ffa97818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1ca8aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59260/0 (socket says 172.21.15.165:59260) 2024-04-27T03:30:36.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.708+0000 7ffa97818640 1 -- 172.21.15.165:0/2371750070 learned_addr learned my addr 172.21.15.165:0/2371750070 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.708+0000 7ffa97818640 1 -- 172.21.15.165:0/2371750070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c1ca7190 con 0x5595c1ca6120 2024-04-27T03:30:36.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.708+0000 7ffa97818640 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1ca8aa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffa88005870 tx=0x7ffa88002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.708+0000 7ff97ffff640 1 -- 172.21.15.165:0/2371750070 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa8806b020 con 0x5595c1ca6120 2024-04-27T03:30:36.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.708+0000 7ff97ffff640 1 -- 172.21.15.165:0/2371750070 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa8805d910 con 0x5595c1ca6120 2024-04-27T03:30:36.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.708+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5595c1ca7420 con 0x5595c1ca6120 2024-04-27T03:30:36.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.708+0000 7ff97ffff640 1 -- 172.21.15.165:0/2371750070 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa88062040 con 0x5595c1ca6120 2024-04-27T03:30:36.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.709+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c1ca7940 con 0x5595c1ca6120 2024-04-27T03:30:36.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.709+0000 7ff97ffff640 1 -- 172.21.15.165:0/2371750070 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffa8806e020 con 0x5595c1ca6120 2024-04-27T03:30:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.709+0000 7ff97ffff640 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff980036950 0x7ff980038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.709+0000 7ff97ffff640 1 -- 172.21.15.165:0/2371750070 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ffa8808d4b0 con 0x5595c1ca6120 2024-04-27T03:30:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.710+0000 7ffa97017640 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff980036950 0x7ff980038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.710+0000 7ffa99d19800 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5595c1ca9160 0x5595c1daa240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.710+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1daa7d0 con 0x5595c1ca9160 2024-04-27T03:30:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.710+0000 7ffa98019640 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5595c1ca9160 0x5595c1daa240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.710+0000 7ffa97017640 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff980036950 0x7ff980038e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c000c00 tx=0x7ffa8c0013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.710+0000 7ffa98019640 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5595c1ca9160 0x5595c1daa240 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.710+0000 7ffa98019640 1 -- 172.21.15.165:0/2371750070 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1ca9160 2024-04-27T03:30:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.710+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dab070 con 0x5595c1ca9160 2024-04-27T03:30:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.711+0000 7ffa98019640 1 -- 172.21.15.165:0/2371750070 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1ca9160 2024-04-27T03:30:36.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.711+0000 7ffa99d19800 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5595c1dacaf0 0x5595c1daef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.711+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1daf710 con 0x5595c1dacaf0 2024-04-27T03:30:36.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.711+0000 7ffa97818640 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5595c1dacaf0 0x5595c1daef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.711+0000 7ffa97818640 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5595c1dacaf0 0x5595c1daef80 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.711+0000 7ffa97818640 1 -- 172.21.15.165:0/2371750070 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1dacaf0 2024-04-27T03:30:36.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.711+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dab8f0 con 0x5595c1ca9160 2024-04-27T03:30:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.712+0000 7ffa98019640 1 -- 172.21.15.165:0/2371750070 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1ca9160 2024-04-27T03:30:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.712+0000 7ffa99d19800 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5595c1db0bc0 0x5595c1db3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.712+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1db3790 con 0x5595c1db0bc0 2024-04-27T03:30:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.712+0000 7ffa97017640 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5595c1db0bc0 0x5595c1db3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.712+0000 7ffa97017640 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5595c1db0bc0 0x5595c1db3030 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.713+0000 7ffa97017640 1 -- 172.21.15.165:0/2371750070 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ffa8c00f670 con 0x5595c1db0bc0 2024-04-27T03:30:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.713+0000 7ffa99d19800 1 Processor -- start 2024-04-27T03:30:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.713+0000 7ffa99d19800 1 -- start start 2024-04-27T03:30:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.713+0000 7ffa99d19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dc1bf0 0x5595c1dc2010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.713+0000 7ffa99d19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5595c1ca8fe0 con 0x5595c1dc1bf0 2024-04-27T03:30:36.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.713+0000 7ffa97818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dc1bf0 0x5595c1dc2010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.713+0000 7ffa97818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dc1bf0 0x5595c1dc2010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59276/0 (socket says 172.21.15.165:59276) 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.713+0000 7ffa97818640 1 -- 172.21.15.165:0/1639818590 learned_addr learned my addr 172.21.15.165:0/1639818590 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.714+0000 7ffa97818640 1 -- 172.21.15.165:0/1639818590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c1dc2550 con 0x5595c1dc1bf0 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.714+0000 7ffa97818640 1 --2- 172.21.15.165:0/1639818590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dc1bf0 0x5595c1dc2010 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ffa8806ff50 tx=0x7ffa8808be80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.714+0000 7ff97d7fa640 1 -- 172.21.15.165:0/1639818590 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa88057040 con 0x5595c1dc1bf0 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.714+0000 7ff97d7fa640 1 -- 172.21.15.165:0/1639818590 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa88007fb0 con 0x5595c1dc1bf0 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.714+0000 7ffa99d19800 1 -- 172.21.15.165:0/1639818590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5595c1dc2fc0 con 0x5595c1dc1bf0 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.714+0000 7ff97d7fa640 1 -- 172.21.15.165:0/1639818590 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa88085100 con 0x5595c1dc1bf0 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.714+0000 7ffa99d19800 1 -- 172.21.15.165:0/1639818590 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c1dc34c0 con 0x5595c1dc1bf0 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.715+0000 7ff97d7fa640 1 -- 172.21.15.165:0/1639818590 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffa88065070 con 0x5595c1dc1bf0 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.715+0000 7ff97d7fa640 1 --2- 172.21.15.165:0/1639818590 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff970036950 0x7ff970038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.715+0000 7ff97d7fa640 1 -- 172.21.15.165:0/1639818590 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ffa88087020 con 0x5595c1dc1bf0 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.715+0000 7ffa97017640 1 --2- 172.21.15.165:0/1639818590 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff970036950 0x7ff970038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.715+0000 7ffa99d19800 1 Processor -- start 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.715+0000 7ffa99d19800 1 -- start start 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.715+0000 7ffa97017640 1 --2- 172.21.15.165:0/1639818590 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff970036950 0x7ff970038e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x5595c1ca7d90 tx=0x7ffa8c006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.715+0000 7ffa99d19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 0x5595c1dcf190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.715+0000 7ffa99d19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5595c1b996b0 con 0x5595c1dced70 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.715+0000 7ffa97818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 0x5595c1dcf190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.715+0000 7ffa97818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 0x5595c1dcf190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59290/0 (socket says 172.21.15.165:59290) 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.716+0000 7ffa97818640 1 -- 172.21.15.165:0/2839076288 learned_addr learned my addr 172.21.15.165:0/2839076288 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.716+0000 7ffa97818640 1 -- 172.21.15.165:0/2839076288 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c1dcf760 con 0x5595c1dced70 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.716+0000 7ffa97818640 1 --2- 172.21.15.165:0/2839076288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 0x5595c1dcf190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ffa88091030 tx=0x7ffa88066c70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cee13f2c10732b6a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.716+0000 7ff976ffd640 1 -- 172.21.15.165:0/2839076288 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa8805ee00 con 0x5595c1dced70 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.716+0000 7ff976ffd640 1 -- 172.21.15.165:0/2839076288 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa88007a80 con 0x5595c1dced70 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.716+0000 7ff976ffd640 1 -- 172.21.15.165:0/2839076288 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa8808a260 con 0x5595c1dced70 2024-04-27T03:30:36.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.716+0000 7ffa99d19800 1 -- 172.21.15.165:0/2839076288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 msgr2=0x5595c1dcf190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.716+0000 7ffa99d19800 1 --2- 172.21.15.165:0/2839076288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 0x5595c1dcf190 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7ffa88091030 tx=0x7ffa88066c70 comp rx=0 tx=0).stop 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.716+0000 7ffa99d19800 1 -- 172.21.15.165:0/2839076288 shutdown_connections 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.716+0000 7ffa99d19800 1 --2- 172.21.15.165:0/2839076288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 0x5595c1dcf190 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.716+0000 7ffa99d19800 1 -- 172.21.15.165:0/2839076288 >> 172.21.15.165:0/2839076288 conn(0x5595c1dcb040 msgr2=0x5595c1dc90a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.717+0000 7ffa99d19800 1 -- 172.21.15.165:0/2839076288 shutdown_connections 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.717+0000 7ffa99d19800 1 -- 172.21.15.165:0/2839076288 wait complete. 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.717+0000 7ffa99d19800 1 Processor -- start 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.717+0000 7ffa99d19800 1 -- start start 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.717+0000 7ffa99d19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 0x5595c1dd5f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.717+0000 7ffa99d19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5595c1dcfc40 con 0x5595c1dced70 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.717+0000 7ffa97818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 0x5595c1dd5f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.717+0000 7ffa97818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 0x5595c1dd5f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59296/0 (socket says 172.21.15.165:59296) 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.717+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 learned_addr learned my addr 172.21.15.165:0/136801114 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.717+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595c1dd6480 con 0x5595c1dced70 2024-04-27T03:30:36.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.718+0000 7ffa97818640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 0x5595c1dd5f40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ffa8808bb10 tx=0x7ffa88060f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.718+0000 7ff9757fa640 1 -- 172.21.15.165:0/136801114 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa88056ae0 con 0x5595c1dced70 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.718+0000 7ff9757fa640 1 -- 172.21.15.165:0/136801114 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa88056c80 con 0x5595c1dced70 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.718+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5595c1dd6480 con 0x5595c1dced70 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.718+0000 7ff9757fa640 1 -- 172.21.15.165:0/136801114 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa880073e0 con 0x5595c1dced70 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.718+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5595c1dd6480 con 0x5595c1dced70 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.719+0000 7ff9757fa640 1 -- 172.21.15.165:0/136801114 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffa880b2030 con 0x5595c1dced70 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.719+0000 7ff9757fa640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff960036950 0x7ff960038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.719+0000 7ff9757fa640 1 -- 172.21.15.165:0/136801114 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7ffa880b0070 con 0x5595c1dced70 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.719+0000 7ffa97017640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff960036950 0x7ff960038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.719+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.719+0000 7ffa99d19800 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5595c1de16b0 0x5595c1de3b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.719+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de42c0 con 0x5595c1de16b0 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.719+0000 7ffa97017640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff960036950 0x7ff960038e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00ffd0 tx=0x7ffa8c023040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.719+0000 7ffa98019640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5595c1de16b0 0x5595c1de3b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa98019640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5595c1de16b0 0x5595c1de3b70 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa99d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa99d19800 20 realm 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de46d0 con 0x5595c1de16b0 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa99d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa99d19800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de4fa0 con 0x5595c1de16b0 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa99d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.720+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de16b0 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.721+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.721+0000 7ffa99d19800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.721+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.721+0000 7ffa99d19800 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5595c1de60a0 0x5595c1de8490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.721+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de8aa0 con 0x5595c1de60a0 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.721+0000 7ffa97818640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5595c1de60a0 0x5595c1de8490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.721+0000 7ffa97818640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5595c1de60a0 0x5595c1de8490 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.721+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.721+0000 7ffa99d19800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.721+0000 7ffa99d19800 20 searching for the correct realm 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.721+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dea3d0 con 0x5595c1de60a0 2024-04-27T03:30:36.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.722+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.722+0000 7ffa97818640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffa88070020 0x7ffa88058280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.722+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dea3d0 con 0x7ffa88070020 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.722+0000 7ffa97017640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffa88070020 0x7ffa88058280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.722+0000 7ffa97017640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffa88070020 0x7ffa88058280 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.722+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.722+0000 7ffa97017640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffa8c02e910 0x7ffa8c03a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.722+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dea3d0 con 0x7ffa8c02e910 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.723+0000 7ffa98019640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffa8c02e910 0x7ffa8c03a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.723+0000 7ffa98019640 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffa8c02e910 0x7ffa8c03a3c0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.723+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.723+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de16b0 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.723+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.723+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de16b0 2024-04-27T03:30:36.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.724+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.724+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x7ffa8c02e910 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.724+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.724+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x7ffa8c02e910 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.724+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.724+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x7ffa88070020 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.724+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.724+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de60a0 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.725+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.725+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de16b0 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.725+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.725+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x7ffa8c02e910 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.725+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.725+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de16b0 2024-04-27T03:30:36.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.725+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.725+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x7ffa88070020 2024-04-27T03:30:36.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.726+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.726+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de60a0 2024-04-27T03:30:36.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.726+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.726+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de60a0 2024-04-27T03:30:36.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.726+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.726+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x7ffa8c02e910 2024-04-27T03:30:36.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.727+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.727+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de60a0 2024-04-27T03:30:36.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.727+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.727+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de60a0 2024-04-27T03:30:36.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.727+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.727+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de16b0 2024-04-27T03:30:36.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.727+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.727+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de16b0 2024-04-27T03:30:36.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.728+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.728+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x7ffa8c02e910 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.728+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.728+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de60a0 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.728+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.728+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x7ffa88070020 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.728+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.728+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de16b0 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.729+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.729+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x7ffa8c02e910 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.729+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.729+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de60a0 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.729+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.729+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x7ffa8c02e910 2024-04-27T03:30:36.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.729+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.729+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x7ffa88070020 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.730+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.730+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de60a0 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.730+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.730+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x7ffa8c02e910 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.730+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.730+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de16b0 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7ffa90038040 con 0x7ffa88070020 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:36.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de5340 con 0x5595c1de16b0 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de9360 con 0x5595c1de16b0 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.731+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de9360 con 0x7ffa8c02e910 2024-04-27T03:30:36.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 20 zone default found 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 4 Realm: () 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 20 zonegroup default 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de9c80 con 0x7ffa8c02e910 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1de9c80 con 0x5595c1de60a0 2024-04-27T03:30:36.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.732+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dec230 con 0x5595c1de60a0 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.733+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.733+0000 7ffa99d19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.733+0000 7ffa99d19800 20 started sync module instance, tier type = 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.733+0000 7ffa99d19800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.733+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1deb6d0 con 0x7ffa88070020 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.734+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.734+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94101691085344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dec5d0 con 0x7ffa88070020 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.734+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dece80 con 0x5595c1de16b0 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.735+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94101691085344] v25'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.735+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.735+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94101691091312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1ded270 con 0x5595c1de16b0 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.735+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dedcd0 con 0x5595c1de60a0 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.736+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94101691091312] v25'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.736+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.736+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94101691087552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dee0f0 con 0x5595c1de60a0 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.736+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1deeb30 con 0x7ffa88070020 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.737+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94101691087552] v25'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.737+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.737+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94101691095008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1def3d0 con 0x7ffa88070020 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.737+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1defd90 con 0x5595c1de16b0 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.738+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.738+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94101691103936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1df0590 con 0x5595c1de16b0 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.738+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1df1080 con 0x7ffa8c02e910 2024-04-27T03:30:36.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.738+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94101691095008] v25'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.739+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94101691103936] v25'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.739+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.739+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94101691108176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1df1040 con 0x7ffa8c02e910 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.739+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1df1920 con 0x7ffa8c02e910 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.740+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.740+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94101691110512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1df1040 con 0x7ffa8c02e910 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.740+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1df2440 con 0x7ffa8c02e910 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.740+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94101691108176] v25'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.741+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94101691110512] v25'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.741+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'144 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.741+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94101691113232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1df2400 con 0x7ffa8c02e910 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.741+0000 7ffa99d19800 20 add_watcher() i=0 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.741+0000 7ffa99d19800 20 add_watcher() i=1 2024-04-27T03:30:36.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.741+0000 7ffa99d19800 20 add_watcher() i=2 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.741+0000 7ffa99d19800 20 add_watcher() i=3 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.741+0000 7ffa99d19800 20 add_watcher() i=4 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.741+0000 7ffa99d19800 20 add_watcher() i=5 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.741+0000 7ffa99d19800 20 add_watcher() i=6 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.742+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94101691113232] v25'145 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.742+0000 7ffa99d19800 20 add_watcher() i=7 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.742+0000 7ffa99d19800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.743+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1df1890 con 0x5595c1de16b0 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.743+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.743+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94101691100960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1df2ba0 con 0x5595c1de16b0 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.744+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94101691100960] v25'145 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.744+0000 7ff94dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.744+0000 7ffa99d19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:36.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.744+0000 7ffa99d19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:36.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.744+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfdd10 con 0x5595c1de16b0 2024-04-27T03:30:36.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.745+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'47 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.745+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfe360 con 0x7ffa8c02e910 2024-04-27T03:30:36.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.746+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'23 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.746+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfe7a0 con 0x5595c1de60a0 2024-04-27T03:30:36.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.747+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'24 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.747+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfe6d0 con 0x5595c1de16b0 2024-04-27T03:30:36.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.748+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'146 uv15 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.748+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfed40 con 0x7ffa88070020 2024-04-27T03:30:36.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.749+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'47 uv17 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.749+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dff2b0 con 0x5595c1de60a0 2024-04-27T03:30:36.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.750+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'40 uv16 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.750+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dff760 con 0x5595c1de16b0 2024-04-27T03:30:36.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.751+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'25 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.751+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dff760 con 0x7ffa8c02e910 2024-04-27T03:30:36.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.752+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'33 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.752+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dff760 con 0x7ffa88070020 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.753+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'17 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.753+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbb90 con 0x5595c1de60a0 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.754+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'23 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.754+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbb90 con 0x7ffa88070020 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.755+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'48 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.755+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbb90 con 0x7ffa8c02e910 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.756+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'19 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.756+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbb90 con 0x7ffa88070020 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.757+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'23 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.757+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbd60 con 0x7ffa8c02e910 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.758+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'45 uv22 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.758+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbd60 con 0x5595c1de16b0 2024-04-27T03:30:36.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.759+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'147 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.759+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbd60 con 0x5595c1de60a0 2024-04-27T03:30:36.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.760+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'42 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.760+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbd60 con 0x7ffa88070020 2024-04-27T03:30:36.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.761+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'49 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.761+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbd60 con 0x5595c1de60a0 2024-04-27T03:30:36.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.762+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'29 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.762+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbcb0 con 0x5595c1de16b0 2024-04-27T03:30:36.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.763+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'25 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.763+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbcb0 con 0x5595c1de16b0 2024-04-27T03:30:36.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.764+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'148 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.764+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbcb0 con 0x5595c1de60a0 2024-04-27T03:30:36.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.765+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'43 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.765+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbcb0 con 0x7ffa88070020 2024-04-27T03:30:36.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.766+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'33 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.766+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e002c0 con 0x5595c1de60a0 2024-04-27T03:30:36.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.767+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'25 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.767+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e002c0 con 0x5595c1de60a0 2024-04-27T03:30:36.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.768+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'36 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.768+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e002c0 con 0x5595c1de60a0 2024-04-27T03:30:36.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.769+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'19 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.769+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbae0 con 0x5595c1de60a0 2024-04-27T03:30:36.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.770+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'41 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.770+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbcb0 con 0x7ffa8c02e910 2024-04-27T03:30:36.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.771+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'25 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.771+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbcb0 con 0x5595c1de60a0 2024-04-27T03:30:36.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.772+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'29 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.772+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbcb0 con 0x7ffa8c02e910 2024-04-27T03:30:36.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.774+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'34 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.774+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbcb0 con 0x5595c1de60a0 2024-04-27T03:30:36.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.775+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'44 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.775+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbcb0 con 0x7ffa8c02e910 2024-04-27T03:30:36.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.776+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'46 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.776+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1dfbcb0 con 0x7ffa88070020 2024-04-27T03:30:36.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.777+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'23 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.777+0000 7ff94cfe1640 20 reqs_thread_entry: start 2024-04-27T03:30:36.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.777+0000 7ffa99d19800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:36.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.777+0000 7ffa99d19800 20 Filter name: none 2024-04-27T03:30:36.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.777+0000 7ff8c67fc640 20 reqs_thread_entry: start 2024-04-27T03:30:36.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.777+0000 7ffa99d19800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T03:30:36.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.777+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.777+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:94 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e2b180 con 0x7ffa88070020 2024-04-27T03:30:36.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(94 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 : miss 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:95 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e2cbb0 con 0x5595c1de16b0 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(95 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv6 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:36.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 2024-04-27T03:30:36.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 20 get_obj_state: octx=0x7ffa9000aad0 obj=myfoo:four state=0x5595c1e2de98 s->prefetch_data=0 2024-04-27T03:30:36.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.778+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:96 3.24 3:24af57b8:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1_four:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e2ecc0 con 0x7ffa8c02e910 2024-04-27T03:30:36.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.779+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(96 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1_four [getxattrs out=849b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 234+0+865 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.779+0000 7ffa99d19800 10 manifest: total_size = 4 2024-04-27T03:30:36.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.779+0000 7ffa99d19800 20 get_obj_state: setting s->obj_tag to 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1231213195453622583 2024-04-27T03:30:36.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.779+0000 7ffa99d19800 20 get_obj_state: octx=0x7ffa9000aad0 obj=myfoo:four state=0x5595c1e2de98 s->prefetch_data=0 2024-04-27T03:30:36.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.779+0000 7ffa99d19800 20 prepare_atomic_modification: state is not atomic. state=0x5595c1e2de98 2024-04-27T03:30:36.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.779+0000 7ffa99d19800 20 bucket index object: :.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6 2024-04-27T03:30:36.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.779+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:97 4.1b 4:d9d9fd94:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=182b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e33590 con 0x7ffa88070020 2024-04-27T03:30:36.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.780+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(97 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6 [stat,call,call] v25'6 uv6 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.780+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:98 3.24 3:24af57b8:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1_four:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e33590 con 0x7ffa8c02e910 2024-04-27T03:30:36.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.781+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(98 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1_four [call] v25'2 uv2 ondisk = 0) ==== 192+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.781+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:99 4.1b 4:d9d9fd94:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=259b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e34020 con 0x7ffa88070020 2024-04-27T03:30:36.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.782+0000 7ff94dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.782+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:100 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94101691100960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e34020 con 0x5595c1de16b0 2024-04-27T03:30:36.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.782+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(99 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6 [stat,call,call] v25'7 uv7 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.782+0000 7ff976ffd640 20 handle_completion(): completion ok for obj=four 2024-04-27T03:30:36.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.783+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(100 data_loggenerations_metadata [watch unwatch cookie 94101691100960] v25'149 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.783+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:101 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94101691085344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e34020 con 0x7ffa88070020 2024-04-27T03:30:36.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.784+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== osd_op_reply(101 notify.0 [watch unwatch cookie 94101691085344] v25'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.784+0000 7ffa99d19800 20 remove_watcher() i=0 2024-04-27T03:30:36.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.784+0000 7ffa99d19800 2 removed watcher, disabling cache 2024-04-27T03:30:36.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.784+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:102 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94101691091312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e34020 con 0x5595c1de16b0 2024-04-27T03:30:36.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.785+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(102 notify.1 [watch unwatch cookie 94101691091312] v25'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.785+0000 7ffa99d19800 20 remove_watcher() i=1 2024-04-27T03:30:36.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.785+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:103 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94101691087552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e34020 con 0x5595c1de60a0 2024-04-27T03:30:36.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.786+0000 7ffa97818640 1 -- 172.21.15.165:0/136801114 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(103 notify.2 [watch unwatch cookie 94101691087552] v25'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa8805a040 con 0x5595c1de60a0 2024-04-27T03:30:36.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.786+0000 7ffa99d19800 20 remove_watcher() i=2 2024-04-27T03:30:36.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.786+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:104 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94101691095008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e34020 con 0x7ffa88070020 2024-04-27T03:30:36.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.787+0000 7ffa97017640 1 -- 172.21.15.165:0/136801114 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(104 notify.3 [watch unwatch cookie 94101691095008] v25'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa8c037040 con 0x7ffa88070020 2024-04-27T03:30:36.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.787+0000 7ffa99d19800 20 remove_watcher() i=3 2024-04-27T03:30:36.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.787+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:105 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94101691103936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e34020 con 0x5595c1de16b0 2024-04-27T03:30:36.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.788+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(105 notify.4 [watch unwatch cookie 94101691103936] v25'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x5595c1de16b0 2024-04-27T03:30:36.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.788+0000 7ffa99d19800 20 remove_watcher() i=4 2024-04-27T03:30:36.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.788+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:106 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94101691108176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e34020 con 0x7ffa8c02e910 2024-04-27T03:30:36.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.789+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(106 notify.5 [watch unwatch cookie 94101691108176] v25'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.789+0000 7ffa99d19800 20 remove_watcher() i=5 2024-04-27T03:30:36.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.789+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:107 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94101691110512] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e34020 con 0x7ffa8c02e910 2024-04-27T03:30:36.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.790+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(107 notify.6 [watch unwatch cookie 94101691110512] v25'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.790+0000 7ffa99d19800 20 remove_watcher() i=6 2024-04-27T03:30:36.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.790+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:108 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94101691113232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5595c1e34020 con 0x7ffa8c02e910 2024-04-27T03:30:36.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== osd_op_reply(108 notify.7 [watch unwatch cookie 94101691113232] v25'146 uv144 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa9000a4a0 con 0x7ffa8c02e910 2024-04-27T03:30:36.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 20 remove_watcher() i=7 2024-04-27T03:30:36.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffa8c02e910 msgr2=0x7ffa8c03a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffa8c02e910 0x7ffa8c03a3c0 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5595c1de16b0 msgr2=0x5595c1de3b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5595c1de16b0 0x5595c1de3b70 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffa88070020 msgr2=0x7ffa88058280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffa88070020 0x7ffa88058280 crc :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5595c1de60a0 msgr2=0x5595c1de8490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5595c1de60a0 0x5595c1de8490 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff960036950 msgr2=0x7ff960038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff960036950 0x7ff960038e10 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c00ffd0 tx=0x7ffa8c023040 comp rx=0 tx=0).stop 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 msgr2=0x5595c1dd5f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 --2- 172.21.15.165:0/136801114 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dced70 0x5595c1dd5f40 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7ffa8808bb10 tx=0x7ffa88060f30 comp rx=0 tx=0).stop 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 reap_dead start 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa98019640 1 -- 172.21.15.165:0/136801114 reap_dead start 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 shutdown_connections 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 >> 172.21.15.165:0/136801114 conn(0x5595c1dcb040 msgr2=0x5595c1dc86a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 shutdown_connections 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.791+0000 7ffa99d19800 1 -- 172.21.15.165:0/136801114 wait complete. 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/1639818590 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff970036950 msgr2=0x7ff970038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 --2- 172.21.15.165:0/1639818590 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff970036950 0x7ff970038e10 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x5595c1ca7d90 tx=0x7ffa8c006000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/1639818590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dc1bf0 msgr2=0x5595c1dc2010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 --2- 172.21.15.165:0/1639818590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dc1bf0 0x5595c1dc2010 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7ffa8806ff50 tx=0x7ffa8808be80 comp rx=0 tx=0).stop 2024-04-27T03:30:36.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/1639818590 shutdown_connections 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 --2- 172.21.15.165:0/1639818590 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff970036950 0x7ff970038e10 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 --2- 172.21.15.165:0/1639818590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1dc1bf0 0x5595c1dc2010 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/1639818590 >> 172.21.15.165:0/1639818590 conn(0x5595c1db5c90 msgr2=0x5595c1db6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/1639818590 shutdown_connections 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/1639818590 wait complete. 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5595c1db0bc0 msgr2=0x5595c1db3030 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5595c1db0bc0 0x5595c1db3030 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5595c1ca9160 msgr2=0x5595c1daa240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5595c1ca9160 0x5595c1daa240 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5595c1dacaf0 msgr2=0x5595c1daef80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5595c1dacaf0 0x5595c1daef80 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff980036950 msgr2=0x7ff980038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff980036950 0x7ff980038e10 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ffa8c000c00 tx=0x7ffa8c0013b0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 msgr2=0x5595c1ca8aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 --2- 172.21.15.165:0/2371750070 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5595c1ca6120 0x5595c1ca8aa0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7ffa88005870 tx=0x7ffa88002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa98019640 1 -- 172.21.15.165:0/2371750070 reap_dead start 2024-04-27T03:30:36.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 shutdown_connections 2024-04-27T03:30:36.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 >> 172.21.15.165:0/2371750070 conn(0x5595c1ca1b20 msgr2=0x5595c1d93530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 shutdown_connections 2024-04-27T03:30:36.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.792+0000 7ffa99d19800 1 -- 172.21.15.165:0/2371750070 wait complete. 2024-04-27T03:30:36.806 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T03:30:36.806 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'stats', '--bucket', 'myfoo'] 2024-04-27T03:30:36.806 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket stats --bucket myfoo 2024-04-27T03:30:36.850 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:36.850 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:36.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.845+0000 7fa3c6733800 1 Processor -- start 2024-04-27T03:30:36.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.845+0000 7fa3c6733800 1 -- start start 2024-04-27T03:30:36.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.845+0000 7fa3c6733800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5581879b1ac0 0x5581879b0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.845+0000 7fa3c6733800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5581878fad40 con 0x5581879b05b0 2024-04-27T03:30:36.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.845+0000 7fa3c6733800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5581878cd4d0 con 0x5581879b1ac0 2024-04-27T03:30:36.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.845+0000 7fa3c23f5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5581879b1ac0 0x5581879b0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.845+0000 7fa3c1bf4640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5581879b05b0 0x558187a43e20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35412/0 (socket says 172.21.15.165:35412) 2024-04-27T03:30:36.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.845+0000 7fa3c23f5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5581879b1ac0 0x5581879b0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59304/0 (socket says 172.21.15.165:59304) 2024-04-27T03:30:36.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.845+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/818004711 learned_addr learned my addr 172.21.15.165:0/818004711 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:36.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.846+0000 7fa3c23f5640 1 -- 172.21.15.165:0/818004711 >> v1:172.21.15.89:6789/0 conn(0x5581879b05b0 legacy=0x558187a43e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.846+0000 7fa3c23f5640 1 -- 172.21.15.165:0/818004711 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5581878f9590 con 0x5581879b1ac0 2024-04-27T03:30:36.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.848+0000 7fa3c23f5640 1 --2- 172.21.15.165:0/818004711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b1ac0 0x5581879b0070 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400a470 tx=0x7fa3b401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d0a1cc1b1a405ee server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.848+0000 7fa3c13f3640 1 -- 172.21.15.165:0/818004711 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b4057070 con 0x5581879b1ac0 2024-04-27T03:30:36.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.848+0000 7fa3c13f3640 1 -- 172.21.15.165:0/818004711 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b4052570 con 0x5581879b1ac0 2024-04-27T03:30:36.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.848+0000 7fa3c13f3640 1 -- 172.21.15.165:0/818004711 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b4052870 con 0x5581879b1ac0 2024-04-27T03:30:36.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.848+0000 7fa3c6733800 1 -- 172.21.15.165:0/818004711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b1ac0 msgr2=0x5581879b0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.848+0000 7fa3c6733800 1 --2- 172.21.15.165:0/818004711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b1ac0 0x5581879b0070 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400a470 tx=0x7fa3b401ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:36.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.848+0000 7fa3c6733800 1 -- 172.21.15.165:0/818004711 shutdown_connections 2024-04-27T03:30:36.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.848+0000 7fa3c6733800 1 --2- 172.21.15.165:0/818004711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b1ac0 0x5581879b0070 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.848+0000 7fa3c6733800 1 -- 172.21.15.165:0/818004711 >> 172.21.15.165:0/818004711 conn(0x558187a37530 msgr2=0x558187a39950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.848+0000 7fa3c6733800 1 -- 172.21.15.165:0/818004711 shutdown_connections 2024-04-27T03:30:36.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.848+0000 7fa3c6733800 1 -- 172.21.15.165:0/818004711 wait complete. 2024-04-27T03:30:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.860+0000 7fa3c6733800 1 Processor -- start 2024-04-27T03:30:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.860+0000 7fa3c6733800 1 -- start start 2024-04-27T03:30:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.860+0000 7fa3c6733800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 0x558187b2f810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.860+0000 7fa3c6733800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5581879873c0 con 0x5581879b05b0 2024-04-27T03:30:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.860+0000 7fa3c23f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 0x558187b2f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.861+0000 7fa3c23f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 0x558187b2f810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59310/0 (socket says 172.21.15.165:59310) 2024-04-27T03:30:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.861+0000 7fa3c23f5640 1 -- 172.21.15.165:0/3652868390 learned_addr learned my addr 172.21.15.165:0/3652868390 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.861+0000 7fa3c23f5640 1 -- 172.21.15.165:0/3652868390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558187b2fde0 con 0x5581879b05b0 2024-04-27T03:30:36.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.861+0000 7fa3c23f5640 1 --2- 172.21.15.165:0/3652868390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 0x558187b2f810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4000c00 tx=0x7fa3b404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cfb8af578f4776 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.861+0000 7fa2b0608640 1 -- 172.21.15.165:0/3652868390 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b4057070 con 0x5581879b05b0 2024-04-27T03:30:36.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.861+0000 7fa2b0608640 1 -- 172.21.15.165:0/3652868390 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b4063070 con 0x5581879b05b0 2024-04-27T03:30:36.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.861+0000 7fa2b0608640 1 -- 172.21.15.165:0/3652868390 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b4052720 con 0x5581879b05b0 2024-04-27T03:30:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.862+0000 7fa3c6733800 1 -- 172.21.15.165:0/3652868390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 msgr2=0x558187b2f810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.862+0000 7fa3c6733800 1 --2- 172.21.15.165:0/3652868390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 0x558187b2f810 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4000c00 tx=0x7fa3b404c000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.862+0000 7fa3c6733800 1 -- 172.21.15.165:0/3652868390 shutdown_connections 2024-04-27T03:30:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.862+0000 7fa3c6733800 1 --2- 172.21.15.165:0/3652868390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 0x558187b2f810 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.862+0000 7fa3c6733800 1 -- 172.21.15.165:0/3652868390 >> 172.21.15.165:0/3652868390 conn(0x558187a37530 msgr2=0x558187b2b130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.862+0000 7fa3c6733800 1 -- 172.21.15.165:0/3652868390 shutdown_connections 2024-04-27T03:30:36.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.862+0000 7fa3c6733800 1 -- 172.21.15.165:0/3652868390 wait complete. 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.862+0000 7fa3c6733800 1 Processor -- start 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.862+0000 7fa3c6733800 1 -- start start 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.862+0000 7fa3c6733800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 0x558187b372c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.862+0000 7fa3c6733800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558187a477f0 con 0x5581879b05b0 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.863+0000 7fa3c23f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 0x558187b372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.863+0000 7fa3c23f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 0x558187b372c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59316/0 (socket says 172.21.15.165:59316) 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.863+0000 7fa3c23f5640 1 -- 172.21.15.165:0/1063404534 learned_addr learned my addr 172.21.15.165:0/1063404534 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.863+0000 7fa3c23f5640 1 -- 172.21.15.165:0/1063404534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558187b37800 con 0x5581879b05b0 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.863+0000 7fa3c23f5640 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 0x558187b372c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4000c00 tx=0x7fa3b4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.863+0000 7fa2aee05640 1 -- 172.21.15.165:0/1063404534 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b4052c70 con 0x5581879b05b0 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.863+0000 7fa2aee05640 1 -- 172.21.15.165:0/1063404534 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b406c070 con 0x5581879b05b0 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.863+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558187b37a90 con 0x5581879b05b0 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.863+0000 7fa2aee05640 1 -- 172.21.15.165:0/1063404534 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b405e480 con 0x5581879b05b0 2024-04-27T03:30:36.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.863+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558187b3a740 con 0x5581879b05b0 2024-04-27T03:30:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.864+0000 7fa2aee05640 1 -- 172.21.15.165:0/1063404534 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa3b406e030 con 0x5581879b05b0 2024-04-27T03:30:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.864+0000 7fa2aee05640 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2a8036950 0x7fa2a8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.864+0000 7fa2aee05640 1 -- 172.21.15.165:0/1063404534 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fa3b408e100 con 0x5581879b05b0 2024-04-27T03:30:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.864+0000 7fa3c1bf4640 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2a8036950 0x7fa2a8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.864+0000 7fa3c6733800 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5581879b1ac0 0x558187b3e500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.864+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b3ea80 con 0x5581879b1ac0 2024-04-27T03:30:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.865+0000 7fa3c2bf6640 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5581879b1ac0 0x558187b3e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.865+0000 7fa3c1bf4640 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2a8036950 0x7fa2a8038e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8000f60 tx=0x7fa3b80012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.865+0000 7fa3c2bf6640 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5581879b1ac0 0x558187b3e500 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.865+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/1063404534 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x5581879b1ac0 2024-04-27T03:30:36.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.865+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b3bab0 con 0x5581879b1ac0 2024-04-27T03:30:36.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.865+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/1063404534 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x5581879b1ac0 2024-04-27T03:30:36.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.865+0000 7fa3c6733800 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558187b40d90 0x558187b43180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.865+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b43820 con 0x558187b40d90 2024-04-27T03:30:36.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.865+0000 7fa3c23f5640 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558187b40d90 0x558187b43180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.866+0000 7fa3c23f5640 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558187b40d90 0x558187b43180 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.866+0000 7fa3c23f5640 1 -- 172.21.15.165:0/1063404534 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa3b4070410 con 0x558187b40d90 2024-04-27T03:30:36.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.866+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b3fd40 con 0x5581879b1ac0 2024-04-27T03:30:36.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.866+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/1063404534 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x5581879b1ac0 2024-04-27T03:30:36.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.866+0000 7fa3c6733800 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558187b44cd0 0x558187b47140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.866+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b478a0 con 0x558187b44cd0 2024-04-27T03:30:36.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.867+0000 7fa3c1bf4640 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558187b44cd0 0x558187b47140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.867+0000 7fa3c1bf4640 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558187b44cd0 0x558187b47140 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.867+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/1063404534 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fa3b8009050 con 0x558187b44cd0 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.868+0000 7fa3c6733800 1 Processor -- start 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.868+0000 7fa3c6733800 1 -- start start 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.868+0000 7fa3c6733800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b55cd0 0x558187b560f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.868+0000 7fa3c6733800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55818792d6b0 con 0x558187b55cd0 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.868+0000 7fa3c23f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b55cd0 0x558187b560f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.868+0000 7fa3c23f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b55cd0 0x558187b560f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59330/0 (socket says 172.21.15.165:59330) 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.868+0000 7fa3c23f5640 1 -- 172.21.15.165:0/1677450688 learned_addr learned my addr 172.21.15.165:0/1677450688 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.868+0000 7fa3c23f5640 1 -- 172.21.15.165:0/1677450688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558187b56630 con 0x558187b55cd0 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.868+0000 7fa3c23f5640 1 --2- 172.21.15.165:0/1677450688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b55cd0 0x558187b560f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa3b406fb70 tx=0x7fa3b4067f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.869+0000 7fa2a7fff640 1 -- 172.21.15.165:0/1677450688 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b4094070 con 0x558187b55cd0 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.869+0000 7fa2a7fff640 1 -- 172.21.15.165:0/1677450688 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b4058040 con 0x558187b55cd0 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.869+0000 7fa3c6733800 1 -- 172.21.15.165:0/1677450688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558187b570a0 con 0x558187b55cd0 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.869+0000 7fa2a7fff640 1 -- 172.21.15.165:0/1677450688 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b4084040 con 0x558187b55cd0 2024-04-27T03:30:36.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.869+0000 7fa3c6733800 1 -- 172.21.15.165:0/1677450688 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558187b575a0 con 0x558187b55cd0 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.869+0000 7fa2a7fff640 1 -- 172.21.15.165:0/1677450688 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa3b4097070 con 0x558187b55cd0 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.869+0000 7fa2a7fff640 1 --2- 172.21.15.165:0/1677450688 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa29c036950 0x7fa29c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.870+0000 7fa2a7fff640 1 -- 172.21.15.165:0/1677450688 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fa3b409aaf0 con 0x558187b55cd0 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.870+0000 7fa3c1bf4640 1 --2- 172.21.15.165:0/1677450688 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa29c036950 0x7fa29c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.870+0000 7fa3c6733800 1 Processor -- start 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.870+0000 7fa3c6733800 1 -- start start 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.870+0000 7fa3c1bf4640 1 --2- 172.21.15.165:0/1677450688 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa29c036950 0x7fa29c038e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800bf50 tx=0x7fa3b800f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.870+0000 7fa3c6733800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 0x558187b63210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.870+0000 7fa3c6733800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558187b300a0 con 0x558187b62df0 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.870+0000 7fa3c23f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 0x558187b63210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.870+0000 7fa3c23f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 0x558187b63210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59342/0 (socket says 172.21.15.165:59342) 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.870+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2755829049 learned_addr learned my addr 172.21.15.165:0/2755829049 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.871+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2755829049 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558187b637e0 con 0x558187b62df0 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.871+0000 7fa3c23f5640 1 --2- 172.21.15.165:0/2755829049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 0x558187b63210 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa3b408ff70 tx=0x7fa3b407d170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e30a95640b8515ee server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.871+0000 7fa2a5ffb640 1 -- 172.21.15.165:0/2755829049 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b40ab070 con 0x558187b62df0 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.871+0000 7fa2a5ffb640 1 -- 172.21.15.165:0/2755829049 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b404fa30 con 0x558187b62df0 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.871+0000 7fa2a5ffb640 1 -- 172.21.15.165:0/2755829049 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b408ca30 con 0x558187b62df0 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.871+0000 7fa3c6733800 1 -- 172.21.15.165:0/2755829049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 msgr2=0x558187b63210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.871+0000 7fa3c6733800 1 --2- 172.21.15.165:0/2755829049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 0x558187b63210 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa3b408ff70 tx=0x7fa3b407d170 comp rx=0 tx=0).stop 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.871+0000 7fa3c6733800 1 -- 172.21.15.165:0/2755829049 shutdown_connections 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.871+0000 7fa3c6733800 1 --2- 172.21.15.165:0/2755829049 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 0x558187b63210 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.871+0000 7fa3c6733800 1 -- 172.21.15.165:0/2755829049 >> 172.21.15.165:0/2755829049 conn(0x558187b5f150 msgr2=0x558187b5d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.871+0000 7fa3c6733800 1 -- 172.21.15.165:0/2755829049 shutdown_connections 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.871+0000 7fa3c6733800 1 -- 172.21.15.165:0/2755829049 wait complete. 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.872+0000 7fa3c6733800 1 Processor -- start 2024-04-27T03:30:36.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.872+0000 7fa3c6733800 1 -- start start 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.872+0000 7fa3c6733800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 0x558187b6a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.872+0000 7fa3c6733800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558187b63e90 con 0x558187b62df0 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.872+0000 7fa3c23f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 0x558187b6a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.872+0000 7fa3c23f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 0x558187b6a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59346/0 (socket says 172.21.15.165:59346) 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.872+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 learned_addr learned my addr 172.21.15.165:0/2614048197 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.872+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558187b6a750 con 0x558187b62df0 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.872+0000 7fa3c23f5640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 0x558187b6a210 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4082ff0 tx=0x7fa3b408ae70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.873+0000 7fa293fff640 1 -- 172.21.15.165:0/2614048197 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b40b6020 con 0x558187b62df0 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.873+0000 7fa293fff640 1 -- 172.21.15.165:0/2614048197 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa3b408cb60 con 0x558187b62df0 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.873+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x558187b6a750 con 0x558187b62df0 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.873+0000 7fa293fff640 1 -- 172.21.15.165:0/2614048197 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa3b40ab030 con 0x558187b62df0 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.873+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558187b6a750 con 0x558187b62df0 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.873+0000 7fa293fff640 1 -- 172.21.15.165:0/2614048197 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa3b40b9020 con 0x558187b62df0 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa293fff640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa28c036900 0x7fa28c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa293fff640 1 -- 172.21.15.165:0/2614048197 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fa3b40ac070 con 0x558187b62df0 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa3c1bf4640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa28c036900 0x7fa28c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa3c6733800 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558187b75990 0x558187b77e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b785a0 con 0x558187b75990 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa3c2bf6640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558187b75990 0x558187b77e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa3c1bf4640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa28c036900 0x7fa28c038dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8001b60 tx=0x7fa3b8010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa3c2bf6640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558187b75990 0x558187b77e50 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x558187b75990 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa3c6733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa3c6733800 20 realm 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.874+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b789b0 con 0x558187b75990 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x558187b75990 2024-04-27T03:30:36.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c6733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c6733800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b79250 con 0x558187b75990 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x558187b75990 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c6733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b75990 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x558187b75990 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c6733800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c6733800 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558187b7a120 0x558187b7c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b7cc50 con 0x558187b7a120 2024-04-27T03:30:36.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c23f5640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558187b7a120 0x558187b7c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.875+0000 7fa3c23f5640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558187b7a120 0x558187b7c510 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.876+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.876+0000 7fa3c6733800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:36.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.876+0000 7fa3c6733800 20 searching for the correct realm 2024-04-27T03:30:36.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.876+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b7e620 con 0x558187b7a120 2024-04-27T03:30:36.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.876+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.876+0000 7fa3c23f5640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa3b40c93f0 0x7fa3b40a6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.876+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b7e620 con 0x7fa3b40c93f0 2024-04-27T03:30:36.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.876+0000 7fa3c1bf4640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa3b40c93f0 0x7fa3b40a6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.877+0000 7fa3c1bf4640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa3b40c93f0 0x7fa3b40a6440 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.877+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b8009050 con 0x7fa3b40c93f0 2024-04-27T03:30:36.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.877+0000 7fa3c1bf4640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa3b8038040 0x7fa3b801bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.877+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b7e620 con 0x7fa3b8038040 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.877+0000 7fa3c2bf6640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa3b8038040 0x7fa3b801bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.878+0000 7fa3c2bf6640 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa3b8038040 0x7fa3b801bab0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.878+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x7fa3b8038040 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.878+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b75990 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.878+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x558187b75990 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.878+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b75990 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.878+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x558187b75990 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.878+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b8038040 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.879+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x7fa3b8038040 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.879+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b8038040 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.879+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x7fa3b8038040 2024-04-27T03:30:36.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.879+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.879+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b8009050 con 0x7fa3b40c93f0 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.879+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b7a120 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.879+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.879+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b75990 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.880+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x558187b75990 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.880+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b8038040 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.880+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x7fa3b8038040 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.880+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b75990 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.880+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x558187b75990 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.880+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.880+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa3b8009050 con 0x7fa3b40c93f0 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.880+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b7a120 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.881+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.881+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b7a120 2024-04-27T03:30:36.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.881+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.881+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b8038040 2024-04-27T03:30:36.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.881+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x7fa3b8038040 2024-04-27T03:30:36.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.881+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b7a120 2024-04-27T03:30:36.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.881+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.881+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b7a120 2024-04-27T03:30:36.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.881+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.881+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b75990 2024-04-27T03:30:36.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.882+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x558187b75990 2024-04-27T03:30:36.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.882+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b75990 2024-04-27T03:30:36.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.882+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa3bc00a4a0 con 0x558187b75990 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.882+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b8038040 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.882+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc033040 con 0x7fa3b8038040 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.882+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b7a120 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.883+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.883+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.883+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b8009050 con 0x7fa3b40c93f0 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.883+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b75990 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.883+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.883+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b8038040 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.884+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.884+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b7a120 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.884+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.884+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b8038040 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.884+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.884+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.884+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b8009050 con 0x7fa3b40c93f0 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.884+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b7a120 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b8038040 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x558187b75990 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x558187b795f0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c6733800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c6733800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:36.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c6733800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c6733800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c6733800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c6733800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.885+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b7d5f0 con 0x558187b75990 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b7d9e0 con 0x558187b75990 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b7d9e0 con 0x7fa3b8038040 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 20 zone default found 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 4 Realm: () 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 20 zonegroup default 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b7de00 con 0x7fa3b8038040 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.886+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b7de00 con 0x558187b7a120 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.887+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.887+0000 7fa3c6733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.887+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.887+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b804d0 con 0x558187b7a120 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.887+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.887+0000 7fa3c6733800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.887+0000 7fa3c6733800 20 started sync module instance, tier type = 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.887+0000 7fa3c6733800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.887+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b7f970 con 0x7fa3b40c93f0 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.888+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.888+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94014816124256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b7f5d0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.888+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b81340 con 0x558187b75990 2024-04-27T03:30:36.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.889+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94014816124256] v25'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.889+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.889+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94014816128656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b81760 con 0x558187b75990 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.889+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b82120 con 0x558187b7a120 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.890+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94014816128656] v25'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.891+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.891+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94014816129712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b816d0 con 0x558187b7a120 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.891+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b82fe0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.891+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94014816129712] v25'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.892+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.892+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94014816134416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b83880 con 0x7fa3b40c93f0 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.892+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b84240 con 0x558187b75990 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.893+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.893+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94014816141168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b84a20 con 0x558187b75990 2024-04-27T03:30:36.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.893+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b85520 con 0x7fa3b8038040 2024-04-27T03:30:36.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.893+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94014816134416] v25'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.894+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94014816141168] v25'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.894+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.894+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94014816145424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b854e0 con 0x7fa3b8038040 2024-04-27T03:30:36.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.894+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b85df0 con 0x7fa3b8038040 2024-04-27T03:30:36.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.895+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94014816145424] v25'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.895+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.895+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94014816138192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b854e0 con 0x7fa3b8038040 2024-04-27T03:30:36.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.895+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b863c0 con 0x7fa3b8038040 2024-04-27T03:30:36.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.896+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94014816138192] v25'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.896+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'147 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.896+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94014816147696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b863c0 con 0x7fa3b8038040 2024-04-27T03:30:36.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.896+0000 7fa3c6733800 20 add_watcher() i=0 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.896+0000 7fa3c6733800 20 add_watcher() i=1 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.896+0000 7fa3c6733800 20 add_watcher() i=2 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.896+0000 7fa3c6733800 20 add_watcher() i=3 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.896+0000 7fa3c6733800 20 add_watcher() i=4 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.896+0000 7fa3c6733800 20 add_watcher() i=5 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.896+0000 7fa3c6733800 20 add_watcher() i=6 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.897+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94014816147696] v25'148 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.897+0000 7fa3c6733800 20 add_watcher() i=7 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.897+0000 7fa3c6733800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.897+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187b897c0 con 0x558187b75990 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.897+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.897+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94014816149728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b86c80 con 0x558187b75990 2024-04-27T03:30:36.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.898+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94014816149728] v25'150 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.898+0000 7fa278fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa3c6733800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:36.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa3c6733800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:36.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa3c6733800 5 note: GC not initialized 2024-04-27T03:30:36.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa1f37fe640 20 reqs_thread_entry: start 2024-04-27T03:30:36.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa3c6733800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:36.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa3c6733800 20 Filter name: none 2024-04-27T03:30:36.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa1f17fa640 20 reqs_thread_entry: start 2024-04-27T03:30:36.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bbcad0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa3c6733800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:36.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.899+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bbd6b0 con 0x558187b75990 2024-04-27T03:30:36.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(63 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv6 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c6733800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:36.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c6733800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T03:30:36.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bbd6f0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(64 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c6733800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:36.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c6733800 20 rados->read ofs=0 len=0 2024-04-27T03:30:36.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bbee40 con 0x558187b75990 2024-04-27T03:30:36.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv6 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c6733800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:36.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c6733800 20 couldn't put binfo cache entry, might have raced with data changes 2024-04-27T03:30:36.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 4.2f 4:f7b16f5a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bbe330 con 0x558187b75990 2024-04-27T03:30:36.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 4.16 4:6b5ddb47:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bc39b0 con 0x558187b7a120 2024-04-27T03:30:36.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.900+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:68 4.7 4:e2bbac01:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bbe330 con 0x7fa3b8038040 2024-04-27T03:30:36.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 4.3e 4:7e0d8a50:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bc8ad0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:70 4.19 4:9b87498d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bcb810 con 0x558187b7a120 2024-04-27T03:30:36.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 4.2 4:42daa0e0:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bce080 con 0x558187b75990 2024-04-27T03:30:36.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 4.1b 4:d9d9fd94:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bd0550 con 0x7fa3b40c93f0 2024-04-27T03:30:36.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:73 4.e 4:70dfcb91:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bd2eb0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:74 4.3d 4:bd27934b:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bd5920 con 0x558187b7a120 2024-04-27T03:30:36.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(67 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.1 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:75 4.5 4:a1c15390:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bc2bd0 con 0x558187b7a120 2024-04-27T03:30:36.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:76 4.1a 4:59a985c3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x558187bd90a0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(70 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(69 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(71 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.5 [call out=124b] v0'0 uv7 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(72 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.6 [call out=124b] v0'0 uv7 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(68 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(73 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.901+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(76 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.902+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(74 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.8 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.902+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(75 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.902+0000 7fa278fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:36.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.902+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:77 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94014816149728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187bd90a0 con 0x558187b75990 2024-04-27T03:30:36.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.903+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(77 data_loggenerations_metadata [watch unwatch cookie 94014816149728] v25'151 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.903+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94014816124256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b860b0 con 0x7fa3b40c93f0 2024-04-27T03:30:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.904+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(78 notify.0 [watch unwatch cookie 94014816124256] v25'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.904+0000 7fa3c6733800 20 remove_watcher() i=0 2024-04-27T03:30:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.904+0000 7fa3c6733800 2 removed watcher, disabling cache 2024-04-27T03:30:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.904+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:79 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94014816128656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b7fd60 con 0x558187b75990 2024-04-27T03:30:36.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.906+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(79 notify.1 [watch unwatch cookie 94014816128656] v25'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.906+0000 7fa3c6733800 20 remove_watcher() i=1 2024-04-27T03:30:36.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.906+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:80 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94014816129712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b7fd60 con 0x558187b7a120 2024-04-27T03:30:36.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.907+0000 7fa3c23f5640 1 -- 172.21.15.165:0/2614048197 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(80 notify.2 [watch unwatch cookie 94014816129712] v25'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3b40a4070 con 0x558187b7a120 2024-04-27T03:30:36.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.907+0000 7fa3c6733800 20 remove_watcher() i=2 2024-04-27T03:30:36.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.907+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:81 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94014816134416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b7fd60 con 0x7fa3b40c93f0 2024-04-27T03:30:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.908+0000 7fa3c1bf4640 1 -- 172.21.15.165:0/2614048197 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(81 notify.3 [watch unwatch cookie 94014816134416] v25'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3b8045040 con 0x7fa3b40c93f0 2024-04-27T03:30:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.908+0000 7fa3c6733800 20 remove_watcher() i=3 2024-04-27T03:30:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.908+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:82 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94014816141168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b7fd60 con 0x558187b75990 2024-04-27T03:30:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.909+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(82 notify.4 [watch unwatch cookie 94014816141168] v25'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x558187b75990 2024-04-27T03:30:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.909+0000 7fa3c6733800 20 remove_watcher() i=4 2024-04-27T03:30:36.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.909+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:83 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94014816145424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b82510 con 0x7fa3b8038040 2024-04-27T03:30:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.910+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(83 notify.5 [watch unwatch cookie 94014816145424] v25'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.910+0000 7fa3c6733800 20 remove_watcher() i=5 2024-04-27T03:30:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.910+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:84 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94014816138192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b85010 con 0x7fa3b8038040 2024-04-27T03:30:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.911+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(84 notify.6 [watch unwatch cookie 94014816138192] v25'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.911+0000 7fa3c6733800 20 remove_watcher() i=6 2024-04-27T03:30:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.911+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:85 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94014816147696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x558187b833d0 con 0x7fa3b8038040 2024-04-27T03:30:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(85 notify.7 [watch unwatch cookie 94014816147696] v25'149 uv147 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa3bc031040 con 0x7fa3b8038040 2024-04-27T03:30:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 20 remove_watcher() i=7 2024-04-27T03:30:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa3b8038040 msgr2=0x7fa3b801bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa3b8038040 0x7fa3b801bab0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558187b75990 msgr2=0x558187b77e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x558187b75990 0x558187b77e50 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa3b40c93f0 msgr2=0x7fa3b40a6440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa3b40c93f0 0x7fa3b40a6440 crc :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558187b7a120 msgr2=0x558187b7c510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558187b7a120 0x558187b7c510 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa28c036900 msgr2=0x7fa28c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa28c036900 0x7fa28c038dc0 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8001b60 tx=0x7fa3b8010000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/2614048197 reap_dead start 2024-04-27T03:30:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 msgr2=0x558187b6a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 0x558187b6a210 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4082ff0 tx=0x7fa3b408ae70 comp rx=0 tx=0).stop 2024-04-27T03:30:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 shutdown_connections 2024-04-27T03:30:36.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 --2- 172.21.15.165:0/2614048197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b62df0 0x558187b6a210 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 >> 172.21.15.165:0/2614048197 conn(0x558187b5f150 msgr2=0x558187b5c7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 shutdown_connections 2024-04-27T03:30:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/2614048197 wait complete. 2024-04-27T03:30:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/1677450688 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa29c036950 msgr2=0x7fa29c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 --2- 172.21.15.165:0/1677450688 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa29c036950 0x7fa29c038e10 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800bf50 tx=0x7fa3b800f000 comp rx=0 tx=0).stop 2024-04-27T03:30:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/1677450688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b55cd0 msgr2=0x558187b560f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 --2- 172.21.15.165:0/1677450688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b55cd0 0x558187b560f0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa3b406fb70 tx=0x7fa3b4067f20 comp rx=0 tx=0).stop 2024-04-27T03:30:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/1677450688 shutdown_connections 2024-04-27T03:30:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 --2- 172.21.15.165:0/1677450688 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa29c036950 0x7fa29c038e10 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 --2- 172.21.15.165:0/1677450688 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558187b55cd0 0x558187b560f0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.913+0000 7fa3c6733800 1 -- 172.21.15.165:0/1677450688 >> 172.21.15.165:0/1677450688 conn(0x558187b49d70 msgr2=0x558187b4a170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 -- 172.21.15.165:0/1677450688 shutdown_connections 2024-04-27T03:30:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 -- 172.21.15.165:0/1677450688 wait complete. 2024-04-27T03:30:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558187b44cd0 msgr2=0x558187b47140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x558187b44cd0 0x558187b47140 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5581879b1ac0 msgr2=0x558187b3e500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5581879b1ac0 0x558187b3e500 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558187b40d90 msgr2=0x558187b43180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x558187b40d90 0x558187b43180 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2a8036950 msgr2=0x7fa2a8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa2a8036950 0x7fa2a8038e10 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8000f60 tx=0x7fa3b80012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 msgr2=0x558187b372c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 --2- 172.21.15.165:0/1063404534 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5581879b05b0 0x558187b372c0 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4000c00 tx=0x7fa3b4002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c2bf6640 1 -- 172.21.15.165:0/1063404534 reap_dead start 2024-04-27T03:30:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 shutdown_connections 2024-04-27T03:30:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 >> 172.21.15.165:0/1063404534 conn(0x558187a37530 msgr2=0x558187b29930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 shutdown_connections 2024-04-27T03:30:36.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:36.914+0000 7fa3c6733800 1 -- 172.21.15.165:0/1063404534 wait complete. 2024-04-27T03:30:36.951 INFO:teuthology.orchestra.run.smithi165.stdout:{"bucket":"myfoo","tenant":"","versioning":"off","zonegroup":"fd642a41-253a-48d4-9473-6eca68174247","placement_rule":"default-placement","explicit_placement":{"data_pool":"","data_extra_pool":"","index_pool":""},"id":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1","marker":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1","index_type":"Normal","index_generation":0,"num_shards":11,"object_lock_enabled":false,"mfa_enabled":false,"owner":"foo","ver":"0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1","master_ver":"0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0","mtime":"2024-04-27T03:30:36.413727Z","creation_time":"2024-04-27T03:30:33.422040Z","max_marker":"0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#","usage":{"rgw.main":{"size":0,"size_actual":0,"size_utilized":0,"size_kb":0,"size_kb_actual":0,"size_kb_utilized":0,"num_objects":0}},"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1}} 2024-04-27T03:30:36.951 DEBUG:tasks.util.rgw: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'fd642a41-253a-48d4-9473-6eca68174247', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1', 'marker': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T03:30:36.413727Z', 'creation_time': '2024-04-27T03:30:33.422040Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T03:30:36.952 INFO:tasks.util.rgw:rgwadmin: client.0 : ['log', 'list'] 2024-04-27T03:30:36.952 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'log', 'list'] 2024-04-27T03:30:36.952 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph log list 2024-04-27T03:30:37.032 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:37.033 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:37.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.027+0000 7f910c21f800 1 Processor -- start 2024-04-27T03:30:37.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.027+0000 7f910c21f800 1 -- start start 2024-04-27T03:30:37.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.027+0000 7f910c21f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x559f40aa8790 0x559f40aa8bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.027+0000 7f910c21f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559f40968d40 con 0x559f40aa90f0 2024-04-27T03:30:37.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.027+0000 7f910c21f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x559f4093b4d0 con 0x559f40aa8790 2024-04-27T03:30:37.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.028+0000 7f9109d25640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x559f40aa8790 0x559f40aa8bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.028+0000 7f9109524640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x559f40aa90f0 0x559f40aa95a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35428/0 (socket says 172.21.15.165:35428) 2024-04-27T03:30:37.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.028+0000 7f9109d25640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x559f40aa8790 0x559f40aa8bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59354/0 (socket says 172.21.15.165:59354) 2024-04-27T03:30:37.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.028+0000 7f9109524640 1 -- 172.21.15.165:0/1657328299 learned_addr learned my addr 172.21.15.165:0/1657328299 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:37.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.028+0000 7f9109d25640 1 -- 172.21.15.165:0/1657328299 >> v1:172.21.15.89:6789/0 conn(0x559f40aa90f0 legacy=0x559f40aa95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.028+0000 7f9109d25640 1 -- 172.21.15.165:0/1657328299 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f40967590 con 0x559f40aa8790 2024-04-27T03:30:37.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.030+0000 7f9109d25640 1 --2- 172.21.15.165:0/1657328299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40aa8bb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00a470 tx=0x7f90fc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbbfddf3740d9741 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.030+0000 7f9108d23640 1 -- 172.21.15.165:0/1657328299 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc057070 con 0x559f40aa8790 2024-04-27T03:30:37.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.030+0000 7f9108d23640 1 -- 172.21.15.165:0/1657328299 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc04adc0 con 0x559f40aa8790 2024-04-27T03:30:37.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.030+0000 7f9108d23640 1 -- 172.21.15.165:0/1657328299 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc052590 con 0x559f40aa8790 2024-04-27T03:30:37.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.030+0000 7f910c21f800 1 -- 172.21.15.165:0/1657328299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 msgr2=0x559f40aa8bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.030+0000 7f910c21f800 1 --2- 172.21.15.165:0/1657328299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40aa8bb0 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f90fc00a470 tx=0x7f90fc01ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:37.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.031+0000 7f910c21f800 1 -- 172.21.15.165:0/1657328299 shutdown_connections 2024-04-27T03:30:37.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.031+0000 7f910c21f800 1 --2- 172.21.15.165:0/1657328299 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40aa8bb0 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.031+0000 7f910c21f800 1 -- 172.21.15.165:0/1657328299 >> 172.21.15.165:0/1657328299 conn(0x559f40aa4310 msgr2=0x559f40aa6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.031+0000 7f910c21f800 1 -- 172.21.15.165:0/1657328299 shutdown_connections 2024-04-27T03:30:37.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.031+0000 7f910c21f800 1 -- 172.21.15.165:0/1657328299 wait complete. 2024-04-27T03:30:37.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.042+0000 7f910c21f800 1 Processor -- start 2024-04-27T03:30:37.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.042+0000 7f910c21f800 1 -- start start 2024-04-27T03:30:37.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.043+0000 7f910c21f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40b9d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.043+0000 7f910c21f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559f409f53b0 con 0x559f40aa8790 2024-04-27T03:30:37.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.043+0000 7f9109d25640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40b9d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.043+0000 7f9109d25640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40b9d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59364/0 (socket says 172.21.15.165:59364) 2024-04-27T03:30:37.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.043+0000 7f9109d25640 1 -- 172.21.15.165:0/1702499587 learned_addr learned my addr 172.21.15.165:0/1702499587 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.043+0000 7f9109d25640 1 -- 172.21.15.165:0/1702499587 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f40b9d9c0 con 0x559f40aa8790 2024-04-27T03:30:37.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.043+0000 7f9109d25640 1 --2- 172.21.15.165:0/1702499587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40b9d3f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f90fc055040 tx=0x7f90fc04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d06019a36a869a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.044+0000 7f8ff7e07640 1 -- 172.21.15.165:0/1702499587 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc057040 con 0x559f40aa8790 2024-04-27T03:30:37.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.044+0000 7f8ff7e07640 1 -- 172.21.15.165:0/1702499587 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc062070 con 0x559f40aa8790 2024-04-27T03:30:37.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.044+0000 7f8ff7e07640 1 -- 172.21.15.165:0/1702499587 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc05d470 con 0x559f40aa8790 2024-04-27T03:30:37.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.044+0000 7f910c21f800 1 -- 172.21.15.165:0/1702499587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 msgr2=0x559f40b9d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.044+0000 7f910c21f800 1 --2- 172.21.15.165:0/1702499587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40b9d3f0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f90fc055040 tx=0x7f90fc04c000 comp rx=0 tx=0).stop 2024-04-27T03:30:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.044+0000 7f910c21f800 1 -- 172.21.15.165:0/1702499587 shutdown_connections 2024-04-27T03:30:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.044+0000 7f910c21f800 1 --2- 172.21.15.165:0/1702499587 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40b9d3f0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.044+0000 7f910c21f800 1 -- 172.21.15.165:0/1702499587 >> 172.21.15.165:0/1702499587 conn(0x559f40aa4310 msgr2=0x559f40b98d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.044+0000 7f910c21f800 1 -- 172.21.15.165:0/1702499587 shutdown_connections 2024-04-27T03:30:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.044+0000 7f910c21f800 1 -- 172.21.15.165:0/1702499587 wait complete. 2024-04-27T03:30:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.045+0000 7f910c21f800 1 Processor -- start 2024-04-27T03:30:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.045+0000 7f910c21f800 1 -- start start 2024-04-27T03:30:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.045+0000 7f910c21f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40ba4f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.045+0000 7f910c21f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559f40aa9f50 con 0x559f40aa8790 2024-04-27T03:30:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.045+0000 7f9109d25640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40ba4f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.045+0000 7f9109d25640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40ba4f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59368/0 (socket says 172.21.15.165:59368) 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.045+0000 7f9109d25640 1 -- 172.21.15.165:0/1370021926 learned_addr learned my addr 172.21.15.165:0/1370021926 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.045+0000 7f9109d25640 1 -- 172.21.15.165:0/1370021926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f40ba5470 con 0x559f40aa8790 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.045+0000 7f9109d25640 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40ba4f30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f90fc01ce60 tx=0x7f90fc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.046+0000 7f8ff6604640 1 -- 172.21.15.165:0/1370021926 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc06b070 con 0x559f40aa8790 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.046+0000 7f8ff6604640 1 -- 172.21.15.165:0/1370021926 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc05d970 con 0x559f40aa8790 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.046+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f40ba5700 con 0x559f40aa8790 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.046+0000 7f8ff6604640 1 -- 172.21.15.165:0/1370021926 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc057070 con 0x559f40aa8790 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.046+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f40ba8310 con 0x559f40aa8790 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.046+0000 7f8ff6604640 1 -- 172.21.15.165:0/1370021926 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f90fc05dc90 con 0x559f40aa8790 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.047+0000 7f8ff6604640 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8ff0036950 0x7f8ff0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.047+0000 7f8ff6604640 1 -- 172.21.15.165:0/1370021926 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f90fc08dfe0 con 0x559f40aa8790 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.047+0000 7f9109524640 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8ff0036950 0x7f8ff0038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.047+0000 7f910c21f800 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f40aa90f0 0x559f40bac0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.047+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40bac610 con 0x559f40aa90f0 2024-04-27T03:30:37.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.047+0000 7f910a526640 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f40aa90f0 0x559f40bac0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.047+0000 7f9109524640 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8ff0036950 0x7f8ff0038e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9100000f60 tx=0x7f91000012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.047+0000 7f910a526640 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f40aa90f0 0x559f40bac0d0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.047+0000 7f910a526640 1 -- 172.21.15.165:0/1370021926 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40aa90f0 2024-04-27T03:30:37.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.048+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40ba96a0 con 0x559f40aa90f0 2024-04-27T03:30:37.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.048+0000 7f910a526640 1 -- 172.21.15.165:0/1370021926 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40aa90f0 2024-04-27T03:30:37.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.048+0000 7f910c21f800 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f40bae920 0x559f40bb0d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.048+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40bb1400 con 0x559f40bae920 2024-04-27T03:30:37.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.048+0000 7f9109d25640 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f40bae920 0x559f40bb0d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.048+0000 7f9109d25640 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f40bae920 0x559f40bb0d10 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.048+0000 7f9109d25640 1 -- 172.21.15.165:0/1370021926 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f90fc05a040 con 0x559f40bae920 2024-04-27T03:30:37.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.048+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40bad8d0 con 0x559f40aa90f0 2024-04-27T03:30:37.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.049+0000 7f910a526640 1 -- 172.21.15.165:0/1370021926 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40aa90f0 2024-04-27T03:30:37.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.049+0000 7f910c21f800 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559f40bb28b0 0x559f40bb4d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.049+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40bb5460 con 0x559f40bb28b0 2024-04-27T03:30:37.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.049+0000 7f9109524640 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559f40bb28b0 0x559f40bb4d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.049+0000 7f9109524640 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559f40bb28b0 0x559f40bb4d00 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.050+0000 7f9109524640 1 -- 172.21.15.165:0/1370021926 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9100009050 con 0x559f40bb28b0 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.050+0000 7f910c21f800 1 Processor -- start 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.050+0000 7f910c21f800 1 -- start start 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.050+0000 7f910c21f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bc38c0 0x559f40bc3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.050+0000 7f910c21f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559f4099b6b0 con 0x559f40bc38c0 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.050+0000 7f9109d25640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bc38c0 0x559f40bc3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.050+0000 7f9109d25640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bc38c0 0x559f40bc3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59370/0 (socket says 172.21.15.165:59370) 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.050+0000 7f9109d25640 1 -- 172.21.15.165:0/19417711 learned_addr learned my addr 172.21.15.165:0/19417711 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.051+0000 7f9109d25640 1 -- 172.21.15.165:0/19417711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f40bc4220 con 0x559f40bc38c0 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.051+0000 7f9109d25640 1 --2- 172.21.15.165:0/19417711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bc38c0 0x559f40bc3ce0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f90fc070f70 tx=0x7f90fc072da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.051+0000 7f8fef7fe640 1 -- 172.21.15.165:0/19417711 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc009750 con 0x559f40bc38c0 2024-04-27T03:30:37.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.051+0000 7f8fef7fe640 1 -- 172.21.15.165:0/19417711 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc0098f0 con 0x559f40bc38c0 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.051+0000 7f8fef7fe640 1 -- 172.21.15.165:0/19417711 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc08c9f0 con 0x559f40bc38c0 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.051+0000 7f910c21f800 1 -- 172.21.15.165:0/19417711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f40bc4c90 con 0x559f40bc38c0 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.051+0000 7f910c21f800 1 -- 172.21.15.165:0/19417711 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f40bc5190 con 0x559f40bc38c0 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.052+0000 7f8fef7fe640 1 -- 172.21.15.165:0/19417711 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f90fc08cb90 con 0x559f40bc38c0 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.052+0000 7f8fef7fe640 1 --2- 172.21.15.165:0/19417711 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fe40369a0 0x7f8fe4038e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.052+0000 7f8fef7fe640 1 -- 172.21.15.165:0/19417711 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f90fc092af0 con 0x559f40bc38c0 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.052+0000 7f9109524640 1 --2- 172.21.15.165:0/19417711 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fe40369a0 0x7f8fe4038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.052+0000 7f910c21f800 1 Processor -- start 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.052+0000 7f910c21f800 1 -- start start 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.052+0000 7f9109524640 1 --2- 172.21.15.165:0/19417711 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fe40369a0 0x7f8fe4038e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f910000bf50 tx=0x7f910000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.052+0000 7f910c21f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 0x559f40bd0ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.053+0000 7f910c21f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559f40b9dc80 con 0x559f40bd0aa0 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.053+0000 7f9109d25640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 0x559f40bd0ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.053+0000 7f9109d25640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 0x559f40bd0ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59382/0 (socket says 172.21.15.165:59382) 2024-04-27T03:30:37.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.053+0000 7f9109d25640 1 -- 172.21.15.165:0/1691350753 learned_addr learned my addr 172.21.15.165:0/1691350753 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.053+0000 7f9109d25640 1 -- 172.21.15.165:0/1691350753 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f40bd1490 con 0x559f40bd0aa0 2024-04-27T03:30:37.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.053+0000 7f9109d25640 1 --2- 172.21.15.165:0/1691350753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 0x559f40bd0ec0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f90fc05bfd0 tx=0x7f90fc06ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10da2ea184ee721f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.053+0000 7f8fed7fa640 1 -- 172.21.15.165:0/1691350753 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc070460 con 0x559f40bd0aa0 2024-04-27T03:30:37.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.053+0000 7f8fed7fa640 1 -- 172.21.15.165:0/1691350753 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc079b80 con 0x559f40bd0aa0 2024-04-27T03:30:37.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.053+0000 7f8fed7fa640 1 -- 172.21.15.165:0/1691350753 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc070600 con 0x559f40bd0aa0 2024-04-27T03:30:37.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f910c21f800 1 -- 172.21.15.165:0/1691350753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 msgr2=0x559f40bd0ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f910c21f800 1 --2- 172.21.15.165:0/1691350753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 0x559f40bd0ec0 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f90fc05bfd0 tx=0x7f90fc06ed50 comp rx=0 tx=0).stop 2024-04-27T03:30:37.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f910c21f800 1 -- 172.21.15.165:0/1691350753 shutdown_connections 2024-04-27T03:30:37.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f910c21f800 1 --2- 172.21.15.165:0/1691350753 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 0x559f40bd0ec0 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f910c21f800 1 -- 172.21.15.165:0/1691350753 >> 172.21.15.165:0/1691350753 conn(0x559f40bccd70 msgr2=0x559f40bcadd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f910c21f800 1 -- 172.21.15.165:0/1691350753 shutdown_connections 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f910c21f800 1 -- 172.21.15.165:0/1691350753 wait complete. 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f910c21f800 1 Processor -- start 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f910c21f800 1 -- start start 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f910c21f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 0x559f40bd7de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f910c21f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559f40bd1af0 con 0x559f40bd0aa0 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f9109d25640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 0x559f40bd7de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f9109d25640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 0x559f40bd7de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59396/0 (socket says 172.21.15.165:59396) 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.054+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 learned_addr learned my addr 172.21.15.165:0/807754085 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.055+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f40bd8320 con 0x559f40bd0aa0 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.055+0000 7f9109d25640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 0x559f40bd7de0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f90fc06ed90 tx=0x7f90fc067e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.055+0000 7f8fdb7fe640 1 -- 172.21.15.165:0/807754085 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc0b1070 con 0x559f40bd0aa0 2024-04-27T03:30:37.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.055+0000 7f8fdb7fe640 1 -- 172.21.15.165:0/807754085 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f90fc0513a0 con 0x559f40bd0aa0 2024-04-27T03:30:37.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.055+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559f40bd8320 con 0x559f40bd0aa0 2024-04-27T03:30:37.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.055+0000 7f8fdb7fe640 1 -- 172.21.15.165:0/807754085 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f90fc0516c0 con 0x559f40bd0aa0 2024-04-27T03:30:37.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.055+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f40bd8320 con 0x559f40bd0aa0 2024-04-27T03:30:37.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.056+0000 7f8fdb7fe640 1 -- 172.21.15.165:0/807754085 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f90fc0b4070 con 0x559f40bd0aa0 2024-04-27T03:30:37.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.056+0000 7f8fdb7fe640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fd4036950 0x7f8fd4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.056+0000 7f8fdb7fe640 1 -- 172.21.15.165:0/807754085 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f90fc0b7b00 con 0x559f40bd0aa0 2024-04-27T03:30:37.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.056+0000 7f910c21f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.056+0000 7f9109524640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fd4036950 0x7f8fd4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.056+0000 7f910c21f800 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f40be33d0 0x559f40be5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.056+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40be5fe0 con 0x559f40be33d0 2024-04-27T03:30:37.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.056+0000 7f910a526640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f40be33d0 0x559f40be5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.056+0000 7f9109524640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fd4036950 0x7f8fd4038e10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9100001b60 tx=0x7f9100010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.056+0000 7f910a526640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f40be33d0 0x559f40be5890 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40be33d0 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 20 realm 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40be63f0 con 0x559f40be33d0 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40be33d0 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40be6c90 con 0x559f40be33d0 2024-04-27T03:30:37.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40be33d0 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be33d0 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40be33d0 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.057+0000 7f910c21f800 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f40be7d90 0x559f40bea180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.058+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40bea700 con 0x559f40be7d90 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.058+0000 7f9109d25640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f40be7d90 0x559f40bea180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.058+0000 7f9109d25640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f40be7d90 0x559f40bea180 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.058+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.058+0000 7f910c21f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.058+0000 7f910c21f800 20 searching for the correct realm 2024-04-27T03:30:37.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.058+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bec080 con 0x559f40be7d90 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.059+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.059+0000 7f9109d25640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f90fc0b6070 0x7f90fc0b7040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.059+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bec080 con 0x7f90fc0b6070 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.059+0000 7f9109524640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f90fc0b6070 0x7f90fc0b7040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.059+0000 7f9109524640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f90fc0b6070 0x7f90fc0b7040 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.059+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9100009050 con 0x7f90fc0b6070 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.059+0000 7f9109524640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9100038040 0x7f910001bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.059+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bec080 con 0x7f9100038040 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.060+0000 7f910a526640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9100038040 0x7f910001bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.060+0000 7f910a526640 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9100038040 0x7f910001bab0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.060+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f910400a4a0 con 0x7f9100038040 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.060+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be33d0 2024-04-27T03:30:37.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.060+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40be33d0 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.060+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be33d0 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.061+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40be33d0 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.061+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f9100038040 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.061+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f910400a4a0 con 0x7f9100038040 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.061+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f9100038040 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.061+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f910400a4a0 con 0x7f9100038040 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.061+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f90fc0b6070 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.061+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9100009050 con 0x7f90fc0b6070 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.061+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be7d90 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.062+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.062+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be33d0 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.062+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40be33d0 2024-04-27T03:30:37.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.062+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f9100038040 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.062+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f910400a4a0 con 0x7f9100038040 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.062+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be33d0 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.062+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40be33d0 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.062+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f90fc0b6070 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.062+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9100009050 con 0x7f90fc0b6070 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.063+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be7d90 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.063+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.063+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be7d90 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.063+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.063+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f9100038040 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.063+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f910400a4a0 con 0x7f9100038040 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.063+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be7d90 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.063+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.063+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be7d90 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.064+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.064+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be33d0 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.064+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40be33d0 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.064+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be33d0 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.064+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f910400a4a0 con 0x559f40be33d0 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.064+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f9100038040 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.064+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9104033040 con 0x7f9100038040 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.064+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be7d90 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.065+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.065+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f90fc0b6070 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.065+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9100009050 con 0x7f90fc0b6070 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.065+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be33d0 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.065+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.065+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f9100038040 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.066+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.066+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be7d90 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.066+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.066+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f9100038040 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.066+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.066+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f90fc0b6070 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.066+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9100009050 con 0x7f90fc0b6070 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.066+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be7d90 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.066+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f9100038040 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x559f40be33d0 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40be7030 con 0x7f90fc0b6070 2024-04-27T03:30:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f910c21f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f910c21f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f910c21f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f910c21f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f910c21f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f910c21f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f910c21f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.067+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40beb050 con 0x559f40be33d0 2024-04-27T03:30:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.068+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.068+0000 7f910c21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.068+0000 7f910c21f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.068+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40beb440 con 0x559f40be33d0 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.068+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.068+0000 7f910c21f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.068+0000 7f910c21f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.068+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40beb440 con 0x7f9100038040 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.068+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 20 zone default found 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 4 Realm: () 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:37.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 20 zonegroup default 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40beb860 con 0x7f9100038040 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40beb860 con 0x559f40be7d90 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40bedf30 con 0x559f40be7d90 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 20 started sync module instance, tier type = 2024-04-27T03:30:37.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.069+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bed3d0 con 0x7f90fc0b6070 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.070+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.070+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94142474409920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bee780 con 0x7f90fc0b6070 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.070+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40beefd0 con 0x559f40be33d0 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.071+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.071+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94142474415152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef640 con 0x559f40be33d0 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.071+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf0160 con 0x559f40be7d90 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.072+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94142474409920] v25'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.073+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94142474415152] v25'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.073+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.073+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94142474419248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.073+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf0b70 con 0x7f90fc0b6070 2024-04-27T03:30:37.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.074+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94142474419248] v25'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.074+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.074+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94142474412752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf13c0 con 0x7f90fc0b6070 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.074+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf1c10 con 0x559f40be33d0 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.075+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.075+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94142474426720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf24b0 con 0x559f40be33d0 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.075+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf2f60 con 0x7f9100038040 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.075+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94142474412752] v25'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.076+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.076+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94142474424080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf2f60 con 0x7f9100038040 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.076+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf39b0 con 0x7f9100038040 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.076+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94142474426720] v25'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.077+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94142474424080] v25'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.077+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.077+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94142474431040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf34c0 con 0x7f9100038040 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.077+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf3e80 con 0x7f9100038040 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.078+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94142474431040] v25'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.078+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'150 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.078+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94142474428352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf3e80 con 0x7f9100038040 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.078+0000 7f910c21f800 20 add_watcher() i=0 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.078+0000 7f910c21f800 20 add_watcher() i=1 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.078+0000 7f910c21f800 20 add_watcher() i=2 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.078+0000 7f910c21f800 20 add_watcher() i=3 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.078+0000 7f910c21f800 20 add_watcher() i=4 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.078+0000 7f910c21f800 20 add_watcher() i=5 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.078+0000 7f910c21f800 20 add_watcher() i=6 2024-04-27T03:30:37.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.079+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94142474428352] v25'151 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.079+0000 7f910c21f800 20 add_watcher() i=7 2024-04-27T03:30:37.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.079+0000 7f910c21f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:37.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.079+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf71c0 con 0x559f40be33d0 2024-04-27T03:30:37.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.079+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.079+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94142474435392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf4410 con 0x559f40be33d0 2024-04-27T03:30:37.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.080+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94142474435392] v25'152 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.081+0000 7f8fbb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:37.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.081+0000 7f910c21f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:37.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.081+0000 7f910c21f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:37.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.081+0000 7f910c21f800 5 note: GC not initialized 2024-04-27T03:30:37.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.081+0000 7f8fba7fc640 20 reqs_thread_entry: start 2024-04-27T03:30:37.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.081+0000 7f910c21f800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.081+0000 7f910c21f800 20 Filter name: none 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.081+0000 7f8f2bfff640 20 reqs_thread_entry: start 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.081+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 6.0 6:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40c29af0 con 0x7f9100038040 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.082+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(62 [pgnls start_epoch 25 out=254b] v25'46 uv30 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.082+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.10 6:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40beb050 con 0x7f9100038040 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.083+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(63 [pgnls start_epoch 25 out=131b] v25'4 uv4 ondisk = 1) ==== 144+0+131 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.083+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.8 6:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40beb050 con 0x559f40be7d90 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.083+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 [pgnls start_epoch 25 out=295b] v25'36 uv28 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.083+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:65 6.18 6:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x7f9100038040 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.083+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(65 [pgnls start_epoch 25 out=172b] v25'23 uv15 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.083+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be33d0 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.084+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(66 [pgnls start_epoch 25 out=335b] v25'152 uv29 ondisk = 1) ==== 144+0+335 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.084+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 6.14 6:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x7f90fc0b6070 2024-04-27T03:30:37.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.084+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(67 [pgnls start_epoch 25 out=213b] v25'22 uv22 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.084+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 6.c 6:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.084+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(68 [pgnls start_epoch 25 out=254b] v25'10 uv10 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.084+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 6.1c 6:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.084+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(69 [pgnls start_epoch 25 out=295b] v25'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.084+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:70 6.2 6:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.085+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(70 [pgnls start_epoch 25 out=213b] v25'18 uv18 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.085+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 6.12 6:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x7f90fc0b6070 2024-04-27T03:30:37.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.085+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(71 [pgnls start_epoch 25 out=172b] v25'17 uv9 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.085+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.a 6:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x7f90fc0b6070 2024-04-27T03:30:37.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.085+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 [pgnls start_epoch 25 out=336b] v25'33 uv25 ondisk = 1) ==== 144+0+336 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.085+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 6.1a 6:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.086+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(73 [pgnls start_epoch 25 out=131b] v25'29 uv21 ondisk = 1) ==== 144+0+131 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.086+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:74 6.6 6:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.086+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(74 [pgnls start_epoch 25 out=254b] v25'12 uv12 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.086+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:75 6.16 6:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.086+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(75 [pgnls start_epoch 25 out=90b] v25'29 uv21 ondisk = 1) ==== 144+0+90 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.086+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.e 6:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be33d0 2024-04-27T03:30:37.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.086+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(76 [pgnls start_epoch 25 out=213b] v25'25 uv17 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.086+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.1e 6:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.087+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(77 [pgnls start_epoch 25 out=254b] v25'16 uv16 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.087+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:78 6.1 6:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be33d0 2024-04-27T03:30:37.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.087+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(78 [pgnls start_epoch 25 out=254b] v25'20 uv20 ondisk = 1) ==== 144+0+254 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.087+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.11 6:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.087+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(79 [pgnls start_epoch 25 out=213b] v25'25 uv17 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.087+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:80 6.9 6:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.088+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(80 [pgnls start_epoch 25 out=172b] v25'41 uv24 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.088+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.19 6:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be33d0 2024-04-27T03:30:37.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.088+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(81 [pgnls start_epoch 25 out=295b] v25'47 uv39 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.088+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:82 6.5 6:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x7f90fc0b6070 2024-04-27T03:30:37.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.088+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(82 [pgnls start_epoch 25 out=276b] v25'15 uv15 ondisk = 1) ==== 144+0+276 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.088+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.15 6:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x7f90fc0b6070 2024-04-27T03:30:37.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.088+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(83 [pgnls start_epoch 25 out=172b] v25'23 uv15 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.088+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:84 6.d 6:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be33d0 2024-04-27T03:30:37.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.089+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(84 [pgnls start_epoch 25 out=295b] v25'25 uv17 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.089+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:85 6.1d 6:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x7f9100038040 2024-04-27T03:30:37.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.089+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(85 [pgnls start_epoch 25 out=172b] v25'34 uv18 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.089+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:86 6.3 6:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x7f90fc0b6070 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.090+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(86 [pgnls start_epoch 25 out=213b] v25'49 uv25 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.090+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:87 6.13 6:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x7f9100038040 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.090+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(87 [pgnls start_epoch 25 out=172b] v25'19 uv11 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.090+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:88 6.b 6:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.090+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(88 [pgnls start_epoch 25 out=172b] v25'24 uv15 ondisk = 1) ==== 144+0+172 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.090+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1b 6:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.091+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(89 [pgnls start_epoch 25 out=90b] v25'19 uv11 ondisk = 1) ==== 144+0+90 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.091+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:90 6.7 6:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be7d90 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.091+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(90 [pgnls start_epoch 25 out=90b] v25'44 uv19 ondisk = 1) ==== 144+0+90 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.091+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:91 6.17 6:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x559f40be33d0 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.091+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(91 [pgnls start_epoch 25 out=131b] v25'12 uv12 ondisk = 1) ==== 144+0+131 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.091+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:92 6.f 6:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x7f90fc0b6070 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.091+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(92 [pgnls start_epoch 25 out=295b] v25'23 uv15 ondisk = 1) ==== 144+0+295 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.091+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:93 6.1f 6:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559f40bef090 con 0x7f9100038040 2024-04-27T03:30:37.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.092+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(93 [pgnls start_epoch 25 out=213b] v25'25 uv17 ondisk = 1) ==== 144+0+213 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.121 INFO:teuthology.orchestra.run.smithi165.stdout:["obj_delete_at_hint.0000000078","obj_delete_at_hint.0000000070","obj_delete_at_hint.0000000104","obj_delete_at_hint.0000000026","obj_delete_at_hint.0000000028","obj_delete_at_hint.0000000040","obj_delete_at_hint.0000000015","obj_delete_at_hint.0000000069","obj_delete_at_hint.0000000095","obj_delete_at_hint.0000000003","obj_delete_at_hint.0000000047","obj_delete_at_hint.0000000035","obj_delete_at_hint.0000000037","obj_delete_at_hint.0000000024","obj_delete_at_hint.0000000000","obj_delete_at_hint.0000000031","data_loggenerations_metadata","obj_delete_at_hint.0000000076","obj_delete_at_hint.0000000113","obj_delete_at_hint.0000000005","obj_delete_at_hint.0000000011","obj_delete_at_hint.0000000058","obj_delete_at_hint.0000000052","obj_delete_at_hint.0000000088","obj_delete_at_hint.0000000080","obj_delete_at_hint.0000000090","obj_delete_at_hint.0000000110","obj_delete_at_hint.0000000096","obj_delete_at_hint.0000000087","obj_delete_at_hint.0000000008","obj_delete_at_hint.0000000006","obj_delete_at_hint.0000000029","obj_delete_at_hint.0000000089","obj_delete_at_hint.0000000022","obj_delete_at_hint.0000000016","obj_delete_at_hint.0000000041","obj_delete_at_hint.0000000018","obj_delete_at_hint.0000000092","obj_delete_at_hint.0000000014","obj_delete_at_hint.0000000112","obj_delete_at_hint.0000000007","obj_delete_at_hint.0000000021","obj_delete_at_hint.0000000064","obj_delete_at_hint.0000000071","obj_delete_at_hint.0000000074","obj_delete_at_hint.0000000081","obj_delete_at_hint.0000000009","obj_delete_at_hint.0000000121","obj_delete_at_hint.0000000125","obj_delete_at_hint.0000000082","obj_delete_at_hint.0000000105","obj_delete_at_hint.0000000059","obj_delete_at_hint.0000000077","obj_delete_at_hint.0000000032","obj_delete_at_hint.0000000053","obj_delete_at_hint.0000000091","obj_delete_at_hint.0000000065","obj_delete_at_hint.0000000083","obj_delete_at_hint.0000000010","obj_delete_at_hint.0000000045","obj_delete_at_hint.0000000002","obj_delete_at_hint.0000000116","obj_delete_at_hint.0000000034","obj_delete_at_hint.0000000101","obj_delete_at_hint.0000000079","obj_delete_at_hint.0000000049","obj_delete_at_hint.0000000117","obj_delete_at_hint.0000000044","obj_delete_at_hint.0000000066","obj_delete_at_hint.0000000068","obj_delete_at_hint.0000000085","obj_delete_at_hint.0000000073","obj_delete_at_hint.0000000038","obj_delete_at_hint.0000000118","obj_delete_at_hint.0000000036","obj_delete_at_hint.0000000103","obj_delete_at_hint.0000000119","obj_delete_at_hint.0000000098","obj_delete_at_hint.0000000027","obj_delete_at_hint.0000000019","obj_delete_at_hint.0000000039","obj_delete_at_hint.0000000100","obj_delete_at_hint.0000000093","obj_delete_at_hint.0000000004","obj_delete_at_hint.0000000063","obj_delete_at_hint.0000000122","obj_delete_at_hint.0000000057","obj_delete_at_hint.0000000054","obj_delete_at_hint.0000000114","data_log.0","obj_delete_at_hint.0000000012","obj_delete_at_hint.0000000084","obj_delete_at_hint.0000000043","obj_delete_at_hint.0000000111","obj_delete_at_hint.0000000048","obj_delete_at_hint.0000000020","obj_delete_at_hint.0000000099","obj_delete_at_hint.0000000056","obj_delete_at_hint.0000000072","obj_delete_at_hint.0000000062","obj_delete_at_hint.0000000115","obj_delete_at_hint.0000000033","obj_delete_at_hint.0000000025","obj_delete_at_hint.0000000123","obj_delete_at_hint.0000000108","obj_delete_at_hint.0000000094","obj_delete_at_hint.0000000060","obj_delete_at_hint.0000000109","obj_delete_at_hint.0000000013","obj_delete_at_hint.0000000051","obj_delete_at_hint.0000000106","obj_delete_at_hint.0000000017","obj_delete_at_hint.0000000023","obj_delete_at_hint.0000000107","obj_delete_at_hint.0000000046","obj_delete_at_hint.0000000067","obj_delete_at_hint.0000000086","obj_delete_at_hint.0000000050","obj_delete_at_hint.0000000102","obj_delete_at_hint.0000000124","obj_delete_at_hint.0000000001","obj_delete_at_hint.0000000126","obj_delete_at_hint.0000000055","obj_delete_at_hint.0000000061","obj_delete_at_hint.0000000097","obj_delete_at_hint.0000000042","obj_delete_at_hint.0000000075","obj_delete_at_hint.0000000120","obj_delete_at_hint.0000000030"] 2024-04-27T03:30:37.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.094+0000 7f8fbb7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:37.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.094+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94142474435392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40beb050 con 0x559f40be33d0 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.095+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94142474435392] v25'153 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.095+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94142474409920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40c28000 con 0x7f90fc0b6070 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.096+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94142474409920] v25'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.096+0000 7f910c21f800 20 remove_watcher() i=0 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.096+0000 7f910c21f800 2 removed watcher, disabling cache 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.096+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94142474415152] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40beb050 con 0x559f40be33d0 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.097+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94142474415152] v25'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.097+0000 7f910c21f800 20 remove_watcher() i=1 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.097+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94142474419248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40beb050 con 0x559f40be7d90 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.098+0000 7f9109d25640 1 -- 172.21.15.165:0/807754085 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94142474419248] v25'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f90fc0bd070 con 0x559f40be7d90 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.098+0000 7f910c21f800 20 remove_watcher() i=2 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.098+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94142474412752] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40befc30 con 0x7f90fc0b6070 2024-04-27T03:30:37.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.099+0000 7f9109524640 1 -- 172.21.15.165:0/807754085 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94142474412752] v25'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9100045040 con 0x7f90fc0b6070 2024-04-27T03:30:37.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.099+0000 7f910c21f800 20 remove_watcher() i=3 2024-04-27T03:30:37.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.099+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94142474426720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bee2d0 con 0x559f40be33d0 2024-04-27T03:30:37.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.100+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94142474426720] v25'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x559f40be33d0 2024-04-27T03:30:37.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.100+0000 7f910c21f800 20 remove_watcher() i=4 2024-04-27T03:30:37.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.100+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94142474424080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40befc30 con 0x7f9100038040 2024-04-27T03:30:37.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.101+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94142474424080] v25'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.101+0000 7f910c21f800 20 remove_watcher() i=5 2024-04-27T03:30:37.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.101+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94142474431040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40befc30 con 0x7f9100038040 2024-04-27T03:30:37.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.102+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94142474431040] v25'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.102+0000 7f910c21f800 20 remove_watcher() i=6 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.102+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94142474428352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559f40bf2a40 con 0x7f9100038040 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94142474428352] v25'152 uv150 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9104031040 con 0x7f9100038040 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 20 remove_watcher() i=7 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9100038040 msgr2=0x7f910001bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9100038040 0x7f910001bab0 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f40be33d0 msgr2=0x559f40be5890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f40be33d0 0x559f40be5890 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f90fc0b6070 msgr2=0x7f90fc0b7040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f90fc0b6070 0x7f90fc0b7040 crc :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f40be7d90 msgr2=0x559f40bea180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f40be7d90 0x559f40bea180 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fd4036950 msgr2=0x7f8fd4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fd4036950 0x7f8fd4038e10 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f9100001b60 tx=0x7f9100010000 comp rx=0 tx=0).stop 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 msgr2=0x559f40bd7de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 --2- 172.21.15.165:0/807754085 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bd0aa0 0x559f40bd7de0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7f90fc06ed90 tx=0x7f90fc067e60 comp rx=0 tx=0).stop 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 reap_dead start 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910a526640 1 -- 172.21.15.165:0/807754085 reap_dead start 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 shutdown_connections 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 >> 172.21.15.165:0/807754085 conn(0x559f40bccd70 msgr2=0x559f40bca3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 shutdown_connections 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/807754085 wait complete. 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/19417711 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fe40369a0 msgr2=0x7f8fe4038e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 --2- 172.21.15.165:0/19417711 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fe40369a0 0x7f8fe4038e60 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f910000bf50 tx=0x7f910000f000 comp rx=0 tx=0).stop 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/19417711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bc38c0 msgr2=0x559f40bc3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 --2- 172.21.15.165:0/19417711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bc38c0 0x559f40bc3ce0 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f90fc070f70 tx=0x7f90fc072da0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/19417711 shutdown_connections 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 --2- 172.21.15.165:0/19417711 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8fe40369a0 0x7f8fe4038e60 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 --2- 172.21.15.165:0/19417711 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40bc38c0 0x559f40bc3ce0 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/19417711 >> 172.21.15.165:0/19417711 conn(0x559f40bb7960 msgr2=0x559f40bb7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/19417711 shutdown_connections 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.103+0000 7f910c21f800 1 -- 172.21.15.165:0/19417711 wait complete. 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559f40bb28b0 msgr2=0x559f40bb4d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559f40bb28b0 0x559f40bb4d00 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f40aa90f0 msgr2=0x559f40bac0d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559f40aa90f0 0x559f40bac0d0 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f40bae920 msgr2=0x559f40bb0d10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559f40bae920 0x559f40bb0d10 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8ff0036950 msgr2=0x7f8ff0038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8ff0036950 0x7f8ff0038e10 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f9100000f60 tx=0x7f91000012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 msgr2=0x559f40ba4f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 --2- 172.21.15.165:0/1370021926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559f40aa8790 0x559f40ba4f30 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f90fc01ce60 tx=0x7f90fc002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910a526640 1 -- 172.21.15.165:0/1370021926 reap_dead start 2024-04-27T03:30:37.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 shutdown_connections 2024-04-27T03:30:37.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 >> 172.21.15.165:0/1370021926 conn(0x559f40aa4310 msgr2=0x559f40b97510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 shutdown_connections 2024-04-27T03:30:37.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.104+0000 7f910c21f800 1 -- 172.21.15.165:0/1370021926 wait complete. 2024-04-27T03:30:37.132 DEBUG:tasks.util.rgw: json result: ['obj_delete_at_hint.0000000078', 'obj_delete_at_hint.0000000070', 'obj_delete_at_hint.0000000104', 'obj_delete_at_hint.0000000026', 'obj_delete_at_hint.0000000028', 'obj_delete_at_hint.0000000040', 'obj_delete_at_hint.0000000015', 'obj_delete_at_hint.0000000069', 'obj_delete_at_hint.0000000095', 'obj_delete_at_hint.0000000003', 'obj_delete_at_hint.0000000047', 'obj_delete_at_hint.0000000035', 'obj_delete_at_hint.0000000037', 'obj_delete_at_hint.0000000024', 'obj_delete_at_hint.0000000000', 'obj_delete_at_hint.0000000031', 'data_loggenerations_metadata', 'obj_delete_at_hint.0000000076', 'obj_delete_at_hint.0000000113', 'obj_delete_at_hint.0000000005', 'obj_delete_at_hint.0000000011', 'obj_delete_at_hint.0000000058', 'obj_delete_at_hint.0000000052', 'obj_delete_at_hint.0000000088', 'obj_delete_at_hint.0000000080', 'obj_delete_at_hint.0000000090', 'obj_delete_at_hint.0000000110', 'obj_delete_at_hint.0000000096', 'obj_delete_at_hint.0000000087', 'obj_delete_at_hint.0000000008', 'obj_delete_at_hint.0000000006', 'obj_delete_at_hint.0000000029', 'obj_delete_at_hint.0000000089', 'obj_delete_at_hint.0000000022', 'obj_delete_at_hint.0000000016', 'obj_delete_at_hint.0000000041', 'obj_delete_at_hint.0000000018', 'obj_delete_at_hint.0000000092', 'obj_delete_at_hint.0000000014', 'obj_delete_at_hint.0000000112', 'obj_delete_at_hint.0000000007', 'obj_delete_at_hint.0000000021', 'obj_delete_at_hint.0000000064', 'obj_delete_at_hint.0000000071', 'obj_delete_at_hint.0000000074', 'obj_delete_at_hint.0000000081', 'obj_delete_at_hint.0000000009', 'obj_delete_at_hint.0000000121', 'obj_delete_at_hint.0000000125', 'obj_delete_at_hint.0000000082', 'obj_delete_at_hint.0000000105', 'obj_delete_at_hint.0000000059', 'obj_delete_at_hint.0000000077', 'obj_delete_at_hint.0000000032', 'obj_delete_at_hint.0000000053', 'obj_delete_at_hint.0000000091', 'obj_delete_at_hint.0000000065', 'obj_delete_at_hint.0000000083', 'obj_delete_at_hint.0000000010', 'obj_delete_at_hint.0000000045', 'obj_delete_at_hint.0000000002', 'obj_delete_at_hint.0000000116', 'obj_delete_at_hint.0000000034', 'obj_delete_at_hint.0000000101', 'obj_delete_at_hint.0000000079', 'obj_delete_at_hint.0000000049', 'obj_delete_at_hint.0000000117', 'obj_delete_at_hint.0000000044', 'obj_delete_at_hint.0000000066', 'obj_delete_at_hint.0000000068', 'obj_delete_at_hint.0000000085', 'obj_delete_at_hint.0000000073', 'obj_delete_at_hint.0000000038', 'obj_delete_at_hint.0000000118', 'obj_delete_at_hint.0000000036', 'obj_delete_at_hint.0000000103', 'obj_delete_at_hint.0000000119', 'obj_delete_at_hint.0000000098', 'obj_delete_at_hint.0000000027', 'obj_delete_at_hint.0000000019', 'obj_delete_at_hint.0000000039', 'obj_delete_at_hint.0000000100', 'obj_delete_at_hint.0000000093', 'obj_delete_at_hint.0000000004', 'obj_delete_at_hint.0000000063', 'obj_delete_at_hint.0000000122', 'obj_delete_at_hint.0000000057', 'obj_delete_at_hint.0000000054', 'obj_delete_at_hint.0000000114', 'data_log.0', 'obj_delete_at_hint.0000000012', 'obj_delete_at_hint.0000000084', 'obj_delete_at_hint.0000000043', 'obj_delete_at_hint.0000000111', 'obj_delete_at_hint.0000000048', 'obj_delete_at_hint.0000000020', 'obj_delete_at_hint.0000000099', 'obj_delete_at_hint.0000000056', 'obj_delete_at_hint.0000000072', 'obj_delete_at_hint.0000000062', 'obj_delete_at_hint.0000000115', 'obj_delete_at_hint.0000000033', 'obj_delete_at_hint.0000000025', 'obj_delete_at_hint.0000000123', 'obj_delete_at_hint.0000000108', 'obj_delete_at_hint.0000000094', 'obj_delete_at_hint.0000000060', 'obj_delete_at_hint.0000000109', 'obj_delete_at_hint.0000000013', 'obj_delete_at_hint.0000000051', 'obj_delete_at_hint.0000000106', 'obj_delete_at_hint.0000000017', 'obj_delete_at_hint.0000000023', 'obj_delete_at_hint.0000000107', 'obj_delete_at_hint.0000000046', 'obj_delete_at_hint.0000000067', 'obj_delete_at_hint.0000000086', 'obj_delete_at_hint.0000000050', 'obj_delete_at_hint.0000000102', 'obj_delete_at_hint.0000000124', 'obj_delete_at_hint.0000000001', 'obj_delete_at_hint.0000000126', 'obj_delete_at_hint.0000000055', 'obj_delete_at_hint.0000000061', 'obj_delete_at_hint.0000000097', 'obj_delete_at_hint.0000000042', 'obj_delete_at_hint.0000000075', 'obj_delete_at_hint.0000000120', 'obj_delete_at_hint.0000000030'] 2024-04-27T03:30:37.132 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'suspend', '--uid', 'foo'] 2024-04-27T03:30:37.132 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'suspend', '--uid', 'foo'] 2024-04-27T03:30:37.133 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user suspend --uid foo 2024-04-27T03:30:37.209 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:37.209 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:37.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.204+0000 7f823a412800 1 Processor -- start 2024-04-27T03:30:37.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.204+0000 7f823a412800 1 -- start start 2024-04-27T03:30:37.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.204+0000 7f823a412800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55664d4a8480 0x55664d4aa870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.204+0000 7f823a412800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55664d362d40 con 0x55664d4a4970 2024-04-27T03:30:37.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.204+0000 7f823a412800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55664d3354d0 con 0x55664d4a8480 2024-04-27T03:30:37.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.204+0000 7f82351ed640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55664d4a8480 0x55664d4aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.204+0000 7f82359ee640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55664d4a4970 0x55664d4a6d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35430/0 (socket says 172.21.15.165:35430) 2024-04-27T03:30:37.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.204+0000 7f82351ed640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55664d4a8480 0x55664d4aa870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59398/0 (socket says 172.21.15.165:59398) 2024-04-27T03:30:37.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.204+0000 7f82359ee640 1 -- 172.21.15.165:0/1810904550 learned_addr learned my addr 172.21.15.165:0/1810904550 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:37.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.204+0000 7f82351ed640 1 -- 172.21.15.165:0/1810904550 >> v1:172.21.15.89:6789/0 conn(0x55664d4a4970 legacy=0x55664d4a6d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.204+0000 7f82349ec640 1 -- 172.21.15.165:0/1810904550 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2764005611 0 0) 0x55664d362d40 con 0x55664d4a4970 2024-04-27T03:30:37.211 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.205+0000 7f82351ed640 1 -- 172.21.15.165:0/1810904550 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664d361590 con 0x55664d4a8480 2024-04-27T03:30:37.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.206+0000 7f82351ed640 1 --2- 172.21.15.165:0/1810904550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a8480 0x55664d4aa870 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f822000a470 tx=0x7f822001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0badc6894d1ef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.207+0000 7f82349ec640 1 -- 172.21.15.165:0/1810904550 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8220057070 con 0x55664d4a8480 2024-04-27T03:30:37.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.207+0000 7f82349ec640 1 -- 172.21.15.165:0/1810904550 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8220052570 con 0x55664d4a8480 2024-04-27T03:30:37.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.207+0000 7f82349ec640 1 -- 172.21.15.165:0/1810904550 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8220052870 con 0x55664d4a8480 2024-04-27T03:30:37.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.207+0000 7f823a412800 1 -- 172.21.15.165:0/1810904550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a8480 msgr2=0x55664d4aa870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.207+0000 7f823a412800 1 --2- 172.21.15.165:0/1810904550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a8480 0x55664d4aa870 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f822000a470 tx=0x7f822001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:37.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.207+0000 7f823a412800 1 -- 172.21.15.165:0/1810904550 shutdown_connections 2024-04-27T03:30:37.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.207+0000 7f823a412800 1 --2- 172.21.15.165:0/1810904550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a8480 0x55664d4aa870 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.207+0000 7f823a412800 1 -- 172.21.15.165:0/1810904550 >> 172.21.15.165:0/1810904550 conn(0x55664d49e300 msgr2=0x55664d4a0720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.207+0000 7f823a412800 1 -- 172.21.15.165:0/1810904550 shutdown_connections 2024-04-27T03:30:37.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.207+0000 7f823a412800 1 -- 172.21.15.165:0/1810904550 wait complete. 2024-04-27T03:30:37.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.219+0000 7f823a412800 1 Processor -- start 2024-04-27T03:30:37.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.219+0000 7f823a412800 1 -- start start 2024-04-27T03:30:37.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.219+0000 7f823a412800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 0x55664d594e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.219+0000 7f823a412800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55664d3ef3c0 con 0x55664d4a4970 2024-04-27T03:30:37.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.219+0000 7f82359ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 0x55664d594e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.219+0000 7f82359ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 0x55664d594e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59408/0 (socket says 172.21.15.165:59408) 2024-04-27T03:30:37.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.219+0000 7f82359ee640 1 -- 172.21.15.165:0/3078567550 learned_addr learned my addr 172.21.15.165:0/3078567550 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.220+0000 7f82359ee640 1 -- 172.21.15.165:0/3078567550 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664d595450 con 0x55664d4a4970 2024-04-27T03:30:37.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.220+0000 7f82359ee640 1 --2- 172.21.15.165:0/3078567550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 0x55664d594e80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f823000b210 tx=0x7f823000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e85dadac1af6f834 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.220+0000 7f8123e07640 1 -- 172.21.15.165:0/3078567550 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8230008070 con 0x55664d4a4970 2024-04-27T03:30:37.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.220+0000 7f8123e07640 1 -- 172.21.15.165:0/3078567550 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823000d500 con 0x55664d4a4970 2024-04-27T03:30:37.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.220+0000 7f8123e07640 1 -- 172.21.15.165:0/3078567550 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823000d840 con 0x55664d4a4970 2024-04-27T03:30:37.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f823a412800 1 -- 172.21.15.165:0/3078567550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 msgr2=0x55664d594e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f823a412800 1 --2- 172.21.15.165:0/3078567550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 0x55664d594e80 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f823000b210 tx=0x7f823000b6d0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f823a412800 1 -- 172.21.15.165:0/3078567550 shutdown_connections 2024-04-27T03:30:37.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f823a412800 1 --2- 172.21.15.165:0/3078567550 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 0x55664d594e80 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f823a412800 1 -- 172.21.15.165:0/3078567550 >> 172.21.15.165:0/3078567550 conn(0x55664d49e300 msgr2=0x55664d5907a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f823a412800 1 -- 172.21.15.165:0/3078567550 shutdown_connections 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f823a412800 1 -- 172.21.15.165:0/3078567550 wait complete. 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f823a412800 1 Processor -- start 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f823a412800 1 -- start start 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f823a412800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 0x55664d59cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f823a412800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55664d4ad0c0 con 0x55664d4a4970 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f82359ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 0x55664d59cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f82359ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 0x55664d59cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59412/0 (socket says 172.21.15.165:59412) 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.221+0000 7f82359ee640 1 -- 172.21.15.165:0/1013787287 learned_addr learned my addr 172.21.15.165:0/1013787287 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.222+0000 7f82359ee640 1 -- 172.21.15.165:0/1013787287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664d59cff0 con 0x55664d4a4970 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.222+0000 7f82359ee640 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 0x55664d59cab0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f823000b210 tx=0x7f8230000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.222+0000 7f8122604640 1 -- 172.21.15.165:0/1013787287 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823000f950 con 0x55664d4a4970 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.222+0000 7f8122604640 1 -- 172.21.15.165:0/1013787287 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823000faf0 con 0x55664d4a4970 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.222+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55664d59d280 con 0x55664d4a4970 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.222+0000 7f8122604640 1 -- 172.21.15.165:0/1013787287 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823000fdd0 con 0x55664d4a4970 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.222+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664d59fee0 con 0x55664d4a4970 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.223+0000 7f8122604640 1 -- 172.21.15.165:0/1013787287 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f8230017040 con 0x55664d4a4970 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.223+0000 7f8122604640 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f811c036900 0x7f811c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.223+0000 7f8122604640 1 -- 172.21.15.165:0/1013787287 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f8230015020 con 0x55664d4a4970 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.223+0000 7f82351ed640 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f811c036900 0x7f811c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.223+0000 7f823a412800 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55664d4a8480 0x55664d5a3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.223+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5a41e0 con 0x55664d4a8480 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.223+0000 7f82361ef640 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55664d4a8480 0x55664d5a3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.223+0000 7f82351ed640 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f811c036900 0x7f811c038dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8220054000 tx=0x7f822004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.223+0000 7f82361ef640 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55664d4a8480 0x55664d5a3ca0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.224+0000 7f82361ef640 1 -- 172.21.15.165:0/1013787287 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d4a8480 2024-04-27T03:30:37.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.224+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5a1250 con 0x55664d4a8480 2024-04-27T03:30:37.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.224+0000 7f82361ef640 1 -- 172.21.15.165:0/1013787287 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d4a8480 2024-04-27T03:30:37.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.224+0000 7f823a412800 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55664d5a6560 0x55664d5a8950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.224+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5a8ff0 con 0x55664d5a6560 2024-04-27T03:30:37.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.224+0000 7f82359ee640 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55664d5a6560 0x55664d5a8950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.224+0000 7f82359ee640 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55664d5a6560 0x55664d5a8950 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.225+0000 7f82359ee640 1 -- 172.21.15.165:0/1013787287 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f823002d070 con 0x55664d5a6560 2024-04-27T03:30:37.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.225+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5a5510 con 0x55664d4a8480 2024-04-27T03:30:37.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.225+0000 7f82361ef640 1 -- 172.21.15.165:0/1013787287 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d4a8480 2024-04-27T03:30:37.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.225+0000 7f823a412800 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55664d5aa4a0 0x55664d5ac8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.225+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5ad050 con 0x55664d5aa4a0 2024-04-27T03:30:37.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.225+0000 7f82351ed640 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55664d5aa4a0 0x55664d5ac8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.226+0000 7f82351ed640 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55664d5aa4a0 0x55664d5ac8f0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.226+0000 7f82351ed640 1 -- 172.21.15.165:0/1013787287 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f82200528e0 con 0x55664d5aa4a0 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.226+0000 7f823a412800 1 Processor -- start 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.226+0000 7f823a412800 1 -- start start 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.226+0000 7f823a412800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5bb430 0x55664d5bb850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.226+0000 7f823a412800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55664d3956b0 con 0x55664d5bb430 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.227+0000 7f82359ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5bb430 0x55664d5bb850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.227+0000 7f82359ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5bb430 0x55664d5bb850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59426/0 (socket says 172.21.15.165:59426) 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.227+0000 7f82359ee640 1 -- 172.21.15.165:0/2259988418 learned_addr learned my addr 172.21.15.165:0/2259988418 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.227+0000 7f82359ee640 1 -- 172.21.15.165:0/2259988418 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664d5bbd90 con 0x55664d5bb430 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.227+0000 7f82359ee640 1 --2- 172.21.15.165:0/2259988418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5bb430 0x55664d5bb850 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f823004b4b0 tx=0x7f823001aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.227+0000 7f811b7fe640 1 -- 172.21.15.165:0/2259988418 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8230019470 con 0x55664d5bb430 2024-04-27T03:30:37.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.227+0000 7f811b7fe640 1 -- 172.21.15.165:0/2259988418 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8230019610 con 0x55664d5bb430 2024-04-27T03:30:37.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.227+0000 7f823a412800 1 -- 172.21.15.165:0/2259988418 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55664d5bc800 con 0x55664d5bb430 2024-04-27T03:30:37.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.227+0000 7f811b7fe640 1 -- 172.21.15.165:0/2259988418 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f82300198f0 con 0x55664d5bb430 2024-04-27T03:30:37.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.227+0000 7f823a412800 1 -- 172.21.15.165:0/2259988418 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664d5bcd00 con 0x55664d5bb430 2024-04-27T03:30:37.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.228+0000 7f811b7fe640 1 -- 172.21.15.165:0/2259988418 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f823003f1b0 con 0x55664d5bb430 2024-04-27T03:30:37.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.228+0000 7f811b7fe640 1 --2- 172.21.15.165:0/2259988418 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8110036900 0x7f8110038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.228+0000 7f811b7fe640 1 -- 172.21.15.165:0/2259988418 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f823003caa0 con 0x55664d5bb430 2024-04-27T03:30:37.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.228+0000 7f82351ed640 1 --2- 172.21.15.165:0/2259988418 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8110036900 0x7f8110038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.228+0000 7f823a412800 1 Processor -- start 2024-04-27T03:30:37.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.228+0000 7f823a412800 1 -- start start 2024-04-27T03:30:37.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.228+0000 7f82351ed640 1 --2- 172.21.15.165:0/2259988418 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8110036900 0x7f8110038dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55664d5a1190 tx=0x7f8220005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.229+0000 7f823a412800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 0x55664d5c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.229+0000 7f823a412800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55664d5957d0 con 0x55664d5c87e0 2024-04-27T03:30:37.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.229+0000 7f82359ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 0x55664d5c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.229+0000 7f82359ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 0x55664d5c8c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59432/0 (socket says 172.21.15.165:59432) 2024-04-27T03:30:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.229+0000 7f82359ee640 1 -- 172.21.15.165:0/2254536078 learned_addr learned my addr 172.21.15.165:0/2254536078 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.229+0000 7f82359ee640 1 -- 172.21.15.165:0/2254536078 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664d5c91d0 con 0x55664d5c87e0 2024-04-27T03:30:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.229+0000 7f82359ee640 1 --2- 172.21.15.165:0/2254536078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 0x55664d5c8c00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f823004a1d0 tx=0x7f8230026a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3d688ce44090372 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.229+0000 7f81197fa640 1 -- 172.21.15.165:0/2254536078 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823001ddc0 con 0x55664d5c87e0 2024-04-27T03:30:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.229+0000 7f81197fa640 1 -- 172.21.15.165:0/2254536078 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8230030d50 con 0x55664d5c87e0 2024-04-27T03:30:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.229+0000 7f81197fa640 1 -- 172.21.15.165:0/2254536078 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8230040900 con 0x55664d5c87e0 2024-04-27T03:30:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.230+0000 7f823a412800 1 -- 172.21.15.165:0/2254536078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 msgr2=0x55664d5c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.230+0000 7f823a412800 1 --2- 172.21.15.165:0/2254536078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 0x55664d5c8c00 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f823004a1d0 tx=0x7f8230026a90 comp rx=0 tx=0).stop 2024-04-27T03:30:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.230+0000 7f823a412800 1 -- 172.21.15.165:0/2254536078 shutdown_connections 2024-04-27T03:30:37.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.230+0000 7f823a412800 1 --2- 172.21.15.165:0/2254536078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 0x55664d5c8c00 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.230+0000 7f823a412800 1 -- 172.21.15.165:0/2254536078 >> 172.21.15.165:0/2254536078 conn(0x55664d5c4ab0 msgr2=0x55664d5c2b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.230+0000 7f823a412800 1 -- 172.21.15.165:0/2254536078 shutdown_connections 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.230+0000 7f823a412800 1 -- 172.21.15.165:0/2254536078 wait complete. 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.230+0000 7f823a412800 1 Processor -- start 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.230+0000 7f823a412800 1 -- start start 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.230+0000 7f823a412800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 0x55664d5cfb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.230+0000 7f823a412800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55664d5c98a0 con 0x55664d5c87e0 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.231+0000 7f82359ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 0x55664d5cfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.231+0000 7f82359ee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 0x55664d5cfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59436/0 (socket says 172.21.15.165:59436) 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.231+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 learned_addr learned my addr 172.21.15.165:0/2852458099 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.231+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55664d5d0090 con 0x55664d5c87e0 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.231+0000 7f82359ee640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 0x55664d5cfb50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8230018a20 tx=0x7f823002b060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.231+0000 7f81077fe640 1 -- 172.21.15.165:0/2852458099 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8230044830 con 0x55664d5c87e0 2024-04-27T03:30:37.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.231+0000 7f81077fe640 1 -- 172.21.15.165:0/2852458099 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f823004b990 con 0x55664d5c87e0 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.231+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55664d5d0090 con 0x55664d5c87e0 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.231+0000 7f81077fe640 1 -- 172.21.15.165:0/2852458099 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f823004bc70 con 0x55664d5c87e0 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.231+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55664d5d0090 con 0x55664d5c87e0 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.232+0000 7f81077fe640 1 -- 172.21.15.165:0/2852458099 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f8230060070 con 0x55664d5c87e0 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.232+0000 7f81077fe640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8100036900 0x7f8100038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.232+0000 7f81077fe640 1 -- 172.21.15.165:0/2852458099 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f823005e020 con 0x55664d5c87e0 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.232+0000 7f82351ed640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8100036900 0x7f8100038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.233+0000 7f82351ed640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8100036900 0x7f8100038dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8220052fd0 tx=0x7f822004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.233+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.233+0000 7f823a412800 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55664d5db2d0 0x55664d5dd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.233+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5ddee0 con 0x55664d5db2d0 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.233+0000 7f82361ef640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55664d5db2d0 0x55664d5dd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.233+0000 7f82361ef640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55664d5db2d0 0x55664d5dd790 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.233+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d5db2d0 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.233+0000 7f823a412800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.233+0000 7f823a412800 20 realm 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.233+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.233+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5de2f0 con 0x55664d5db2d0 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d5db2d0 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f823a412800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f823a412800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5debc0 con 0x55664d5db2d0 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d5db2d0 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f823a412800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5def60 con 0x55664d5db2d0 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d5db2d0 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f823a412800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f823a412800 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55664d5dfcc0 0x55664d5e20b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e26c0 con 0x55664d5dfcc0 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f82359ee640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55664d5dfcc0 0x55664d5e20b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.234+0000 7f82359ee640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55664d5dfcc0 0x55664d5e20b0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.235+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.235+0000 7f823a412800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.235+0000 7f823a412800 20 searching for the correct realm 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.235+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e4020 con 0x55664d5dfcc0 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.235+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.235+0000 7f82359ee640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8230065830 0x7f823002f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.235+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e4020 con 0x7f8230065830 2024-04-27T03:30:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.235+0000 7f82351ed640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8230065830 0x7f823002f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.236+0000 7f82351ed640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8230065830 0x7f823002f3e0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.236+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.236+0000 7f82351ed640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f8220076040 0x7f822005e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.236+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e4020 con 0x7f8220076040 2024-04-27T03:30:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.236+0000 7f82361ef640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f8220076040 0x7f822005e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.237+0000 7f82361ef640 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f8220076040 0x7f822005e480 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.237+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f822800a4a0 con 0x7f8220076040 2024-04-27T03:30:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.237+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5def60 con 0x55664d5db2d0 2024-04-27T03:30:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.237+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d5db2d0 2024-04-27T03:30:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.237+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5def60 con 0x55664d5db2d0 2024-04-27T03:30:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.237+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d5db2d0 2024-04-27T03:30:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.237+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5def60 con 0x7f8220076040 2024-04-27T03:30:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.237+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f822800a4a0 con 0x7f8220076040 2024-04-27T03:30:37.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.237+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5def60 con 0x7f8220076040 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.238+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f822800a4a0 con 0x7f8220076040 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.238+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5def60 con 0x7f8230065830 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.238+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.238+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5def60 con 0x55664d5dfcc0 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.238+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.238+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5def60 con 0x55664d5db2d0 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.238+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d5db2d0 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.238+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5def60 con 0x7f8220076040 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.239+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f822800a4a0 con 0x7f8220076040 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.239+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5def60 con 0x55664d5db2d0 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.239+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d5db2d0 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.239+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55664d5def60 con 0x7f8230065830 2024-04-27T03:30:37.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.239+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.239+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x55664d5dfcc0 2024-04-27T03:30:37.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.239+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.239+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x55664d5dfcc0 2024-04-27T03:30:37.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.240+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.240+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x7f8220076040 2024-04-27T03:30:37.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.240+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f822800a4a0 con 0x7f8220076040 2024-04-27T03:30:37.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.240+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x55664d5dfcc0 2024-04-27T03:30:37.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.241+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.241+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x55664d5dfcc0 2024-04-27T03:30:37.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.241+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.241+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x55664d5db2d0 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.241+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d5db2d0 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.241+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x55664d5db2d0 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.241+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f822800a4a0 con 0x55664d5db2d0 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.241+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x7f8220076040 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.242+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8228033040 con 0x7f8220076040 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.242+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x55664d5dfcc0 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.242+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.242+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x7f8230065830 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.242+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.242+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x55664d5db2d0 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.242+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.242+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x7f8220076040 2024-04-27T03:30:37.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.243+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.243+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x55664d5dfcc0 2024-04-27T03:30:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.243+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.243+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x7f8220076040 2024-04-27T03:30:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.243+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.243+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x7f8230065830 2024-04-27T03:30:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.243+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.243+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x55664d5dfcc0 2024-04-27T03:30:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x7f8220076040 2024-04-27T03:30:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x55664d5db2d0 2024-04-27T03:30:37.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f8220090060 con 0x7f8230065830 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f823a412800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f823a412800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f823a412800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f823a412800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f823a412800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f823a412800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.244+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e2ad0 con 0x55664d5db2d0 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e2ec0 con 0x55664d5db2d0 2024-04-27T03:30:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e2ec0 con 0x7f8220076040 2024-04-27T03:30:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 20 zone default found 2024-04-27T03:30:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 4 Realm: () 2024-04-27T03:30:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:37.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 20 zonegroup default 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.245+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e3b70 con 0x7f8220076040 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f823a412800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e3b90 con 0x55664d5dfcc0 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f823a412800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e5fc0 con 0x55664d5dfcc0 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f823a412800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f823a412800 20 started sync module instance, tier type = 2024-04-27T03:30:37.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f823a412800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:37.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.246+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e5320 con 0x7f8230065830 2024-04-27T03:30:37.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.247+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.247+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93897873053456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e4f80 con 0x7f8230065830 2024-04-27T03:30:37.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.247+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e6ef0 con 0x55664d5db2d0 2024-04-27T03:30:37.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.248+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.248+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93897873058848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e7790 con 0x55664d5db2d0 2024-04-27T03:30:37.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.248+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e8260 con 0x55664d5dfcc0 2024-04-27T03:30:37.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.248+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93897873053456] v25'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.272 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.249+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.249+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93897873058848] v25'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.249+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93897873063248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e8650 con 0x55664d5dfcc0 2024-04-27T03:30:37.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.249+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e7320 con 0x7f8230065830 2024-04-27T03:30:37.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.250+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.250+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93897873067376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e9780 con 0x7f8230065830 2024-04-27T03:30:37.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.250+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5ea2a0 con 0x55664d5db2d0 2024-04-27T03:30:37.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.250+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93897873063248] v25'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.251+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93897873067376] v25'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.251+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.251+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93897873071472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e91d0 con 0x55664d5db2d0 2024-04-27T03:30:37.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.251+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5eacb0 con 0x7f8220076040 2024-04-27T03:30:37.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.252+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.252+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93897873074176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5eb0d0 con 0x7f8220076040 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.252+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5eba90 con 0x7f8220076040 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.252+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93897873071472] v25'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.253+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93897873074176] v25'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.254+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.254+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93897873077728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5eb0d0 con 0x7f8220076040 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.254+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5eba00 con 0x7f8220076040 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.255+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93897873077728] v25'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.255+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'153 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.255+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93897873075232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5eb040 con 0x7f8220076040 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.255+0000 7f823a412800 20 add_watcher() i=0 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.255+0000 7f823a412800 20 add_watcher() i=1 2024-04-27T03:30:37.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.255+0000 7f823a412800 20 add_watcher() i=2 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.255+0000 7f823a412800 20 add_watcher() i=3 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.255+0000 7f823a412800 20 add_watcher() i=4 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.255+0000 7f823a412800 20 add_watcher() i=5 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.255+0000 7f823a412800 20 add_watcher() i=6 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.256+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93897873075232] v25'154 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.256+0000 7f823a412800 20 add_watcher() i=7 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.256+0000 7f823a412800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.256+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d5ef300 con 0x55664d5db2d0 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.256+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.256+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93897873079712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5ec7f0 con 0x55664d5db2d0 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.257+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93897873079712] v25'154 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.257+0000 7f80e77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:37.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.257+0000 7f823a412800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.257+0000 7f823a412800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.257+0000 7f823a412800 5 note: GC not initialized 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f80e67fc640 20 reqs_thread_entry: start 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f823a412800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f823a412800 20 Filter name: none 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f8057fff640 20 reqs_thread_entry: start 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f823a412800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d622630 con 0x55664d5db2d0 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f823a412800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f823a412800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f823a412800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:37.283 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f823a412800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.258+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d6250c0 con 0x7f8230065830 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 20 disabling bucket name=myfoo 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d6230a0 con 0x7f8230065830 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 149+0+165 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 : miss 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.284 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d625a30 con 0x55664d5db2d0 2024-04-27T03:30:37.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv6 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:37.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:37.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:37.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:37.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:37.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 2024-04-27T03:30:37.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.259+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d62d5d0 con 0x55664d5db2d0 2024-04-27T03:30:37.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [delete,create,call,call,writefull 0~379,setxattr (123)] v25'7 uv7 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f823a412800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f823a412800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f823a412800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f823a412800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1, ofs0, ns] 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [notify cookie 93897873318128 in=764b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d628c80 con 0x55664d5db2d0 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify (1) cookie 93897873058848 notify 107374182425 ret 0) ==== 794+0+0 (crc 0 0 0) 0x55664d625e20 con 0x55664d5db2d0 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f81197fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182425 cookie 93897873058848 notifier 4581 bl.length()=752 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f81197fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f81197fa640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f81197fa640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(67 notify.1 [notify cookie 93897873318128 out=8b] v0'0 uv153 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f81197fa640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f80f8003ea0 con 0x55664d5db2d0 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [notify-ack] v0'0 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.261+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== watch-notify(notify_complete (2) cookie 93897873318128 notify 107374182425 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55664d624f20 con 0x55664d5db2d0 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.262+0000 7f823a412800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): orig_sources= new_sources= 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.262+0000 7f823a412800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): potential sources added=[] removed=[] 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.262+0000 7f823a412800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): orig_dests= new_dests= 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.262+0000 7f823a412800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): potential dests added=[] removed=[] 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.262+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d628790 con 0x55664d5db2d0 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~391] v25'12 uv12 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f823a412800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f823a412800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f823a412800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 93897873323760 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55664d628790 con 0x55664d5db2d0 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== watch-notify(notify (1) cookie 93897873071472 notify 107374182426 ret 0) ==== 609+0+0 (crc 0 0 0) 0x55664d624f20 con 0x55664d5db2d0 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f8118ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182426 cookie 93897873071472 notifier 4581 bl.length()=567 2024-04-27T03:30:37.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f8118ff9640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:37.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f8118ff9640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:37.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(70 notify.4 [notify cookie 93897873323760 out=8b] v0'0 uv153 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f8118ff9640 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f8060003430 con 0x55664d5db2d0 2024-04-27T03:30:37.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.263+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== watch-notify(notify_complete (2) cookie 93897873323760 notify 107374182426 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f822801e040 con 0x55664d5db2d0 2024-04-27T03:30:37.288 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":1,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:37.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.264+0000 7f80e77fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:37.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.264+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93897873079712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d624630 con 0x55664d5db2d0 2024-04-27T03:30:37.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.265+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 93897873079712] v25'155 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.265+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93897873053456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5ebda0 con 0x7f8230065830 2024-04-27T03:30:37.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.266+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 93897873053456] v25'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.266+0000 7f823a412800 20 remove_watcher() i=0 2024-04-27T03:30:37.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.266+0000 7f823a412800 2 removed watcher, disabling cache 2024-04-27T03:30:37.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.266+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93897873058848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5ebda0 con 0x55664d5db2d0 2024-04-27T03:30:37.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.267+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 93897873058848] v25'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.267+0000 7f823a412800 20 remove_watcher() i=1 2024-04-27T03:30:37.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.267+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93897873063248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5ebda0 con 0x55664d5dfcc0 2024-04-27T03:30:37.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.268+0000 7f82359ee640 1 -- 172.21.15.165:0/2852458099 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 93897873063248] v25'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8230077070 con 0x55664d5dfcc0 2024-04-27T03:30:37.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.268+0000 7f823a412800 20 remove_watcher() i=2 2024-04-27T03:30:37.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.268+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93897873067376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e7d50 con 0x7f8230065830 2024-04-27T03:30:37.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.269+0000 7f82351ed640 1 -- 172.21.15.165:0/2852458099 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 93897873067376] v25'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8220081070 con 0x7f8230065830 2024-04-27T03:30:37.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.269+0000 7f823a412800 20 remove_watcher() i=3 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.269+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93897873071472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e7d50 con 0x55664d5db2d0 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.270+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 93897873071472] v25'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x55664d5db2d0 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.270+0000 7f823a412800 20 remove_watcher() i=4 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.270+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93897873074176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5e7d50 con 0x7f8220076040 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.271+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 93897873074176] v25'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.271+0000 7f823a412800 20 remove_watcher() i=5 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.271+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93897873077728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5ea800 con 0x7f8220076040 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.272+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 93897873077728] v25'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.272+0000 7f823a412800 20 remove_watcher() i=6 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.272+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93897873075232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55664d5eb5e0 con 0x7f8220076040 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.273+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 93897873075232] v25'155 uv153 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f8228031040 con 0x7f8220076040 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.273+0000 7f823a412800 20 remove_watcher() i=7 2024-04-27T03:30:37.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f8220076040 msgr2=0x7f822005e480 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f8220076040 0x7f822005e480 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55664d5db2d0 msgr2=0x55664d5dd790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55664d5db2d0 0x55664d5dd790 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8230065830 msgr2=0x7f823002f3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f8230065830 0x7f823002f3e0 crc :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55664d5dfcc0 msgr2=0x55664d5e20b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55664d5dfcc0 0x55664d5e20b0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8100036900 msgr2=0x7f8100038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8100036900 0x7f8100038dc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f8220052fd0 tx=0x7f822004f000 comp rx=0 tx=0).stop 2024-04-27T03:30:37.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 msgr2=0x55664d5cfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 --2- 172.21.15.165:0/2852458099 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5c87e0 0x55664d5cfb50 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f8230018a20 tx=0x7f823002b060 comp rx=0 tx=0).stop 2024-04-27T03:30:37.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 reap_dead start 2024-04-27T03:30:37.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f82361ef640 1 -- 172.21.15.165:0/2852458099 reap_dead start 2024-04-27T03:30:37.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 shutdown_connections 2024-04-27T03:30:37.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 >> 172.21.15.165:0/2852458099 conn(0x55664d5c4ab0 msgr2=0x55664d5c2110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 shutdown_connections 2024-04-27T03:30:37.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2852458099 wait complete. 2024-04-27T03:30:37.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2259988418 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8110036900 msgr2=0x7f8110038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 --2- 172.21.15.165:0/2259988418 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8110036900 0x7f8110038dc0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x55664d5a1190 tx=0x7f8220005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:37.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2259988418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5bb430 msgr2=0x55664d5bb850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 --2- 172.21.15.165:0/2259988418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5bb430 0x55664d5bb850 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f823004b4b0 tx=0x7f823001aaf0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2259988418 shutdown_connections 2024-04-27T03:30:37.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 --2- 172.21.15.165:0/2259988418 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8110036900 0x7f8110038dc0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 --2- 172.21.15.165:0/2259988418 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d5bb430 0x55664d5bb850 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2259988418 >> 172.21.15.165:0/2259988418 conn(0x55664d5af4d0 msgr2=0x55664d5af8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2259988418 shutdown_connections 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.274+0000 7f823a412800 1 -- 172.21.15.165:0/2259988418 wait complete. 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55664d5aa4a0 msgr2=0x55664d5ac8f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55664d5aa4a0 0x55664d5ac8f0 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55664d4a8480 msgr2=0x55664d5a3ca0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55664d4a8480 0x55664d5a3ca0 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55664d5a6560 msgr2=0x55664d5a8950 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55664d5a6560 0x55664d5a8950 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f811c036900 msgr2=0x7f811c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f811c036900 0x7f811c038dc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8220054000 tx=0x7f822004e000 comp rx=0 tx=0).stop 2024-04-27T03:30:37.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 msgr2=0x55664d59cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 --2- 172.21.15.165:0/1013787287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55664d4a4970 0x55664d59cab0 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f823000b210 tx=0x7f8230000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f82361ef640 1 -- 172.21.15.165:0/1013787287 reap_dead start 2024-04-27T03:30:37.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 shutdown_connections 2024-04-27T03:30:37.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 >> 172.21.15.165:0/1013787287 conn(0x55664d49e300 msgr2=0x55664d58efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 shutdown_connections 2024-04-27T03:30:37.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.275+0000 7f823a412800 1 -- 172.21.15.165:0/1013787287 wait complete. 2024-04-27T03:30:37.300 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:37.303 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'enable', '--uid', 'foo'] 2024-04-27T03:30:37.303 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'enable', '--uid', 'foo'] 2024-04-27T03:30:37.303 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user enable --uid foo 2024-04-27T03:30:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:37.377 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:37.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.372+0000 7fd85510c800 1 Processor -- start 2024-04-27T03:30:37.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.373+0000 7fd85510c800 1 -- start start 2024-04-27T03:30:37.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.373+0000 7fd85510c800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x559cb385c370 0x559cb385e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.373+0000 7fd85510c800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559cb3716d40 con 0x559cb3856100 2024-04-27T03:30:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.373+0000 7fd85510c800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x559cb36e94d0 con 0x559cb385c370 2024-04-27T03:30:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.373+0000 7fd85240a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x559cb385c370 0x559cb385e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.373+0000 7fd852c0b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x559cb3856100 0x559cb3856520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:35432/0 (socket says 172.21.15.165:35432) 2024-04-27T03:30:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.373+0000 7fd852c0b640 1 -- 172.21.15.165:0/307334575 learned_addr learned my addr 172.21.15.165:0/307334575 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.373+0000 7fd85240a640 1 -- 172.21.15.165:0/307334575 >> v1:172.21.15.89:6789/0 conn(0x559cb3856100 legacy=0x559cb3856520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.373+0000 7fd85240a640 1 -- 172.21.15.165:0/307334575 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb3715590 con 0x559cb385c370 2024-04-27T03:30:37.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.373+0000 7fd851c09640 1 -- 172.21.15.165:0/307334575 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2282708246 0 0) 0x559cb3716d40 con 0x559cb3856100 2024-04-27T03:30:37.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.375+0000 7fd85240a640 1 --2- 172.21.15.165:0/307334575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb385c370 0x559cb385e760 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd84800a470 tx=0x7fd84801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511ccc2eaed02abd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.375+0000 7fd851c09640 1 -- 172.21.15.165:0/307334575 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd848057070 con 0x559cb385c370 2024-04-27T03:30:37.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.375+0000 7fd851c09640 1 -- 172.21.15.165:0/307334575 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd848052570 con 0x559cb385c370 2024-04-27T03:30:37.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.375+0000 7fd851c09640 1 -- 172.21.15.165:0/307334575 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd848052870 con 0x559cb385c370 2024-04-27T03:30:37.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.376+0000 7fd85510c800 1 -- 172.21.15.165:0/307334575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb385c370 msgr2=0x559cb385e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.376+0000 7fd85510c800 1 --2- 172.21.15.165:0/307334575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb385c370 0x559cb385e760 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7fd84800a470 tx=0x7fd84801ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:37.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.376+0000 7fd85510c800 1 -- 172.21.15.165:0/307334575 shutdown_connections 2024-04-27T03:30:37.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.376+0000 7fd85510c800 1 --2- 172.21.15.165:0/307334575 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb385c370 0x559cb385e760 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.376+0000 7fd85510c800 1 -- 172.21.15.165:0/307334575 >> 172.21.15.165:0/307334575 conn(0x559cb3851b00 msgr2=0x559cb3853f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.376+0000 7fd85510c800 1 -- 172.21.15.165:0/307334575 shutdown_connections 2024-04-27T03:30:37.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.376+0000 7fd85510c800 1 -- 172.21.15.165:0/307334575 wait complete. 2024-04-27T03:30:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.389+0000 7fd85510c800 1 Processor -- start 2024-04-27T03:30:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.389+0000 7fd85510c800 1 -- start start 2024-04-27T03:30:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.389+0000 7fd85510c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 0x559cb3948ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.389+0000 7fd85510c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559cb37a33c0 con 0x559cb3856100 2024-04-27T03:30:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.390+0000 7fd852c0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 0x559cb3948ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.390+0000 7fd852c0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 0x559cb3948ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59458/0 (socket says 172.21.15.165:59458) 2024-04-27T03:30:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.390+0000 7fd852c0b640 1 -- 172.21.15.165:0/1689043121 learned_addr learned my addr 172.21.15.165:0/1689043121 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.390+0000 7fd852c0b640 1 -- 172.21.15.165:0/1689043121 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb39494a0 con 0x559cb3856100 2024-04-27T03:30:37.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.390+0000 7fd852c0b640 1 --2- 172.21.15.165:0/1689043121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 0x559cb3948ed0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd84400b240 tx=0x7fd84400b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6f587c4a77628df server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.390+0000 7fd740e09640 1 -- 172.21.15.165:0/1689043121 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd844008070 con 0x559cb3856100 2024-04-27T03:30:37.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.390+0000 7fd740e09640 1 -- 172.21.15.165:0/1689043121 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd84400d500 con 0x559cb3856100 2024-04-27T03:30:37.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.390+0000 7fd740e09640 1 -- 172.21.15.165:0/1689043121 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd84400d860 con 0x559cb3856100 2024-04-27T03:30:37.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.391+0000 7fd85510c800 1 -- 172.21.15.165:0/1689043121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 msgr2=0x559cb3948ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.391+0000 7fd85510c800 1 --2- 172.21.15.165:0/1689043121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 0x559cb3948ed0 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7fd84400b240 tx=0x7fd84400b700 comp rx=0 tx=0).stop 2024-04-27T03:30:37.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.391+0000 7fd85510c800 1 -- 172.21.15.165:0/1689043121 shutdown_connections 2024-04-27T03:30:37.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.391+0000 7fd85510c800 1 --2- 172.21.15.165:0/1689043121 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 0x559cb3948ed0 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.391+0000 7fd85510c800 1 -- 172.21.15.165:0/1689043121 >> 172.21.15.165:0/1689043121 conn(0x559cb3851b00 msgr2=0x559cb39447f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.391+0000 7fd85510c800 1 -- 172.21.15.165:0/1689043121 shutdown_connections 2024-04-27T03:30:37.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.391+0000 7fd85510c800 1 -- 172.21.15.165:0/1689043121 wait complete. 2024-04-27T03:30:37.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.391+0000 7fd85510c800 1 Processor -- start 2024-04-27T03:30:37.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.392+0000 7fd85510c800 1 -- start start 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.392+0000 7fd85510c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 0x559cb3858a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.392+0000 7fd85510c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559cb385efa0 con 0x559cb3856100 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.392+0000 7fd852c0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 0x559cb3858a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.392+0000 7fd852c0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 0x559cb3858a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59460/0 (socket says 172.21.15.165:59460) 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.392+0000 7fd852c0b640 1 -- 172.21.15.165:0/2958574320 learned_addr learned my addr 172.21.15.165:0/2958574320 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.392+0000 7fd852c0b640 1 -- 172.21.15.165:0/2958574320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb3857170 con 0x559cb3856100 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.392+0000 7fd852c0b640 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 0x559cb3858a80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd844000f60 tx=0x7fd844000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.393+0000 7fd73f606640 1 -- 172.21.15.165:0/2958574320 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd844019870 con 0x559cb3856100 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.393+0000 7fd73f606640 1 -- 172.21.15.165:0/2958574320 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd844019a10 con 0x559cb3856100 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.393+0000 7fd73f606640 1 -- 172.21.15.165:0/2958574320 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd844019d10 con 0x559cb3856100 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.393+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559cb3857400 con 0x559cb3856100 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.393+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb3857920 con 0x559cb3856100 2024-04-27T03:30:37.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.393+0000 7fd73f606640 1 -- 172.21.15.165:0/2958574320 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fd844017040 con 0x559cb3856100 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.394+0000 7fd73f606640 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd738036900 0x7fd738038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.394+0000 7fd73f606640 1 -- 172.21.15.165:0/2958574320 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fd844015020 con 0x559cb3856100 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.394+0000 7fd85240a640 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd738036900 0x7fd738038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.394+0000 7fd85510c800 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559cb385c370 0x559cb3959e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.394+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb395a3e0 con 0x559cb385c370 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.394+0000 7fd85340c640 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559cb385c370 0x559cb3959e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.394+0000 7fd85240a640 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd738036900 0x7fd738038dc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd848054000 tx=0x7fd84804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.394+0000 7fd85340c640 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559cb385c370 0x559cb3959e60 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.394+0000 7fd85340c640 1 -- 172.21.15.165:0/2958574320 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb385c370 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.395+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb395ac80 con 0x559cb385c370 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.395+0000 7fd85340c640 1 -- 172.21.15.165:0/2958574320 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb385c370 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.395+0000 7fd85510c800 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559cb395c6b0 0x559cb395eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.395+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb395f300 con 0x559cb395c6b0 2024-04-27T03:30:37.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.395+0000 7fd852c0b640 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559cb395c6b0 0x559cb395eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.395+0000 7fd852c0b640 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559cb395c6b0 0x559cb395eb70 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.395+0000 7fd852c0b640 1 -- 172.21.15.165:0/2958574320 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fd84402d070 con 0x559cb395c6b0 2024-04-27T03:30:37.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.396+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb395b500 con 0x559cb385c370 2024-04-27T03:30:37.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.396+0000 7fd85340c640 1 -- 172.21.15.165:0/2958574320 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb385c370 2024-04-27T03:30:37.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.396+0000 7fd85510c800 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559cb3960800 0x559cb3962c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.396+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb39633d0 con 0x559cb3960800 2024-04-27T03:30:37.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.396+0000 7fd85240a640 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559cb3960800 0x559cb3962c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.397+0000 7fd85240a640 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559cb3960800 0x559cb3962c70 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.397+0000 7fd85240a640 1 -- 172.21.15.165:0/2958574320 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fd848052940 con 0x559cb3960800 2024-04-27T03:30:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.397+0000 7fd85510c800 1 Processor -- start 2024-04-27T03:30:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.397+0000 7fd85510c800 1 -- start start 2024-04-27T03:30:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.397+0000 7fd85510c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb39717e0 0x559cb3971c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.397+0000 7fd85510c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559cb3858fc0 con 0x559cb39717e0 2024-04-27T03:30:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.398+0000 7fd852c0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb39717e0 0x559cb3971c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.398+0000 7fd852c0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb39717e0 0x559cb3971c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59470/0 (socket says 172.21.15.165:59470) 2024-04-27T03:30:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.398+0000 7fd852c0b640 1 -- 172.21.15.165:0/3844727373 learned_addr learned my addr 172.21.15.165:0/3844727373 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.398+0000 7fd852c0b640 1 -- 172.21.15.165:0/3844727373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb3972140 con 0x559cb39717e0 2024-04-27T03:30:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.398+0000 7fd852c0b640 1 --2- 172.21.15.165:0/3844727373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb39717e0 0x559cb3971c00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd84400eb40 tx=0x7fd84401aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.398+0000 7fd73ce01640 1 -- 172.21.15.165:0/3844727373 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd844040950 con 0x559cb39717e0 2024-04-27T03:30:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.398+0000 7fd73ce01640 1 -- 172.21.15.165:0/3844727373 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd844040af0 con 0x559cb39717e0 2024-04-27T03:30:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.398+0000 7fd85510c800 1 -- 172.21.15.165:0/3844727373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559cb3972bb0 con 0x559cb39717e0 2024-04-27T03:30:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.398+0000 7fd73ce01640 1 -- 172.21.15.165:0/3844727373 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd844040dd0 con 0x559cb39717e0 2024-04-27T03:30:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.398+0000 7fd85510c800 1 -- 172.21.15.165:0/3844727373 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb39730b0 con 0x559cb39717e0 2024-04-27T03:30:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.399+0000 7fd73ce01640 1 -- 172.21.15.165:0/3844727373 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fd844049040 con 0x559cb39717e0 2024-04-27T03:30:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.399+0000 7fd73ce01640 1 --2- 172.21.15.165:0/3844727373 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd72c036970 0x7fd72c038e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.399+0000 7fd73ce01640 1 -- 172.21.15.165:0/3844727373 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fd844051060 con 0x559cb39717e0 2024-04-27T03:30:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.399+0000 7fd85240a640 1 --2- 172.21.15.165:0/3844727373 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd72c036970 0x7fd72c038e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.399+0000 7fd85510c800 1 Processor -- start 2024-04-27T03:30:37.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.400+0000 7fd85240a640 1 --2- 172.21.15.165:0/3844727373 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd72c036970 0x7fd72c038e30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x559cb3857d40 tx=0x7fd848005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.400+0000 7fd85510c800 1 -- start start 2024-04-27T03:30:37.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.400+0000 7fd85510c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 0x559cb397ef90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.400+0000 7fd85510c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559cb37496b0 con 0x559cb397eb70 2024-04-27T03:30:37.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.400+0000 7fd852c0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 0x559cb397ef90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.400+0000 7fd852c0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 0x559cb397ef90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59480/0 (socket says 172.21.15.165:59480) 2024-04-27T03:30:37.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.400+0000 7fd852c0b640 1 -- 172.21.15.165:0/3700006602 learned_addr learned my addr 172.21.15.165:0/3700006602 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.400+0000 7fd852c0b640 1 -- 172.21.15.165:0/3700006602 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb397f560 con 0x559cb397eb70 2024-04-27T03:30:37.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.400+0000 7fd852c0b640 1 --2- 172.21.15.165:0/3700006602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 0x559cb397ef90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd84401ac40 tx=0x7fd84400f530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39aa46664d3f8b94 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.400+0000 7fd7367fc640 1 -- 172.21.15.165:0/3700006602 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd844024340 con 0x559cb397eb70 2024-04-27T03:30:37.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.401+0000 7fd7367fc640 1 -- 172.21.15.165:0/3700006602 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd844050020 con 0x559cb397eb70 2024-04-27T03:30:37.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.401+0000 7fd7367fc640 1 -- 172.21.15.165:0/3700006602 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd84401c5e0 con 0x559cb397eb70 2024-04-27T03:30:37.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.401+0000 7fd85510c800 1 -- 172.21.15.165:0/3700006602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 msgr2=0x559cb397ef90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.401+0000 7fd85510c800 1 --2- 172.21.15.165:0/3700006602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 0x559cb397ef90 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fd84401ac40 tx=0x7fd84400f530 comp rx=0 tx=0).stop 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.401+0000 7fd85510c800 1 -- 172.21.15.165:0/3700006602 shutdown_connections 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.401+0000 7fd85510c800 1 --2- 172.21.15.165:0/3700006602 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 0x559cb397ef90 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.401+0000 7fd85510c800 1 -- 172.21.15.165:0/3700006602 >> 172.21.15.165:0/3700006602 conn(0x559cb397ae40 msgr2=0x559cb3978ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.401+0000 7fd85510c800 1 -- 172.21.15.165:0/3700006602 shutdown_connections 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.401+0000 7fd85510c800 1 -- 172.21.15.165:0/3700006602 wait complete. 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.401+0000 7fd85510c800 1 Processor -- start 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.401+0000 7fd85510c800 1 -- start start 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.402+0000 7fd85510c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 0x559cb3985cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.402+0000 7fd85510c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559cb397fa40 con 0x559cb397eb70 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.402+0000 7fd852c0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 0x559cb3985cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.402+0000 7fd852c0b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 0x559cb3985cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59494/0 (socket says 172.21.15.165:59494) 2024-04-27T03:30:37.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.402+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 learned_addr learned my addr 172.21.15.165:0/775551075 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.402+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559cb3986230 con 0x559cb397eb70 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.402+0000 7fd852c0b640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 0x559cb3985cf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd84401ac40 tx=0x7fd844005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.402+0000 7fd734ff9640 1 -- 172.21.15.165:0/775551075 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd844030d40 con 0x559cb397eb70 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.402+0000 7fd734ff9640 1 -- 172.21.15.165:0/775551075 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd844044e50 con 0x559cb397eb70 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.402+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559cb3986230 con 0x559cb397eb70 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.402+0000 7fd734ff9640 1 -- 172.21.15.165:0/775551075 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd844030450 con 0x559cb397eb70 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.402+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559cb3986230 con 0x559cb397eb70 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.403+0000 7fd734ff9640 1 -- 172.21.15.165:0/775551075 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fd84406c070 con 0x559cb397eb70 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.403+0000 7fd734ff9640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd71c036900 0x7fd71c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.403+0000 7fd734ff9640 1 -- 172.21.15.165:0/775551075 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fd844070ad0 con 0x559cb397eb70 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.403+0000 7fd85240a640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd71c036900 0x7fd71c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.404+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.404+0000 7fd85510c800 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559cb3991430 0x559cb39938f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.404+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb3994040 con 0x559cb3991430 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.404+0000 7fd85240a640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd71c036900 0x7fd71c038dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd848052fd0 tx=0x7fd84804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.404+0000 7fd85340c640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559cb3991430 0x559cb39938f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.404+0000 7fd85340c640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559cb3991430 0x559cb39938f0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.404+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb3991430 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.404+0000 7fd85510c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.404+0000 7fd85510c800 20 realm 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb3994450 con 0x559cb3991430 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb3991430 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb3994d20 con 0x559cb3991430 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb3991430 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb39950c0 con 0x559cb3991430 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb3991430 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559cb3995e20 0x559cb3998210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb3998820 con 0x559cb3995e20 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.405+0000 7fd852c0b640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559cb3995e20 0x559cb3998210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.406+0000 7fd852c0b640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559cb3995e20 0x559cb3998210 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.406+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.406+0000 7fd85510c800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:37.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.406+0000 7fd85510c800 20 searching for the correct realm 2024-04-27T03:30:37.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.406+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb399a180 con 0x559cb3995e20 2024-04-27T03:30:37.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.407+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.407+0000 7fd852c0b640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fd844062430 0x7fd844013130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.407+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb399a180 con 0x7fd844062430 2024-04-27T03:30:37.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.407+0000 7fd85240a640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fd844062430 0x7fd844013130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.407+0000 7fd85240a640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fd844062430 0x7fd844013130 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.408+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.408+0000 7fd85240a640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd848076040 0x7fd848083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:37.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.408+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb399a180 con 0x7fd848076040 2024-04-27T03:30:37.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.408+0000 7fd85340c640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd848076040 0x7fd848083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:37.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.408+0000 7fd85340c640 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd848076040 0x7fd848083410 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:37.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.408+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fd84c00a4a0 con 0x7fd848076040 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.408+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb39950c0 con 0x559cb3991430 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.409+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb3991430 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.409+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb39950c0 con 0x559cb3991430 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.409+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb3991430 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.409+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb39950c0 con 0x7fd848076040 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.409+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c00a4a0 con 0x7fd848076040 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.409+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb39950c0 con 0x7fd848076040 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.409+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c00a4a0 con 0x7fd848076040 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.409+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb39950c0 con 0x7fd844062430 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.410+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.410+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb39950c0 con 0x559cb3995e20 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.410+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.410+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb39950c0 con 0x559cb3991430 2024-04-27T03:30:37.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.410+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb3991430 2024-04-27T03:30:37.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.410+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb39950c0 con 0x7fd848076040 2024-04-27T03:30:37.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.410+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c00a4a0 con 0x7fd848076040 2024-04-27T03:30:37.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.410+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb39950c0 con 0x559cb3991430 2024-04-27T03:30:37.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.411+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb3991430 2024-04-27T03:30:37.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.411+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x559cb39950c0 con 0x7fd844062430 2024-04-27T03:30:37.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.411+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.411+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x559cb3995e20 2024-04-27T03:30:37.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.411+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.411+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x559cb3995e20 2024-04-27T03:30:37.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.411+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.411+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x7fd848076040 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.412+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c00a4a0 con 0x7fd848076040 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.412+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x559cb3995e20 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.412+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.412+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x559cb3995e20 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.412+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.412+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x559cb3991430 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.413+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb3991430 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.413+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x559cb3991430 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.413+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fd84c00a4a0 con 0x559cb3991430 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.413+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x7fd848076040 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.413+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c033040 con 0x7fd848076040 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.413+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x559cb3995e20 2024-04-27T03:30:37.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.413+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.413+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x7fd844062430 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.414+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.414+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x559cb3991430 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.414+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.414+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x7fd848076040 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.414+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.414+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x559cb3995e20 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.415+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.415+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x7fd848076040 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.415+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.415+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x7fd844062430 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.415+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.415+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x559cb3995e20 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.415+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.415+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x7fd848076040 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x559cb3991430 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7fd848090060 con 0x7fd844062430 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85510c800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85510c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85510c800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85510c800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85510c800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85510c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:37.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb3998c30 con 0x559cb3991430 2024-04-27T03:30:37.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85510c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.416+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb3999020 con 0x559cb3991430 2024-04-27T03:30:37.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85510c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:37.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb3999020 con 0x7fd848076040 2024-04-27T03:30:37.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85510c800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:37.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85510c800 20 zone default found 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85510c800 4 Realm: () 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85510c800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85510c800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85510c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85510c800 20 zonegroup default 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.417+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb399a600 con 0x7fd848076040 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd85510c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb399bfe0 con 0x559cb3995e20 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd85510c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb399c380 con 0x559cb3995e20 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd85510c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd85510c800 20 started sync module instance, tier type = 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd85510c800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.418+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb399b480 con 0x7fd844062430 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.419+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.419+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94131516452816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb399c720 con 0x7fd844062430 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.419+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb399cfb0 con 0x559cb3991430 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.420+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.420+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94131516459712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb399d850 con 0x559cb3991430 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.420+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb399e330 con 0x559cb3995e20 2024-04-27T03:30:37.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.420+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94131516452816] v25'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.421+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.421+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94131516464704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb399e720 con 0x559cb3995e20 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.421+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb399f180 con 0x7fd844062430 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.421+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94131516459712] v25'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.422+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94131516464704] v25'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.422+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.422+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94131516455024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb399ec60 con 0x7fd844062430 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.422+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb399fb90 con 0x559cb3991430 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.423+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.423+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94131516471488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a02d0 con 0x559cb3991430 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.423+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94131516455024] v25'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.423+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a0df0 con 0x7fd848076040 2024-04-27T03:30:37.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.424+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.424+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94131516471488] v25'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.424+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94131516462064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a0db0 con 0x7fd848076040 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.424+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a12f0 con 0x7fd848076040 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.425+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.425+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94131516472608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a12f0 con 0x7fd848076040 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.425+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a21f0 con 0x7fd848076040 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.426+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94131516462064] v25'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.426+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94131516472608] v25'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.427+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'156 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.427+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94131516475584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a1d00 con 0x7fd848076040 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.427+0000 7fd85510c800 20 add_watcher() i=0 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.427+0000 7fd85510c800 20 add_watcher() i=1 2024-04-27T03:30:37.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.427+0000 7fd85510c800 20 add_watcher() i=2 2024-04-27T03:30:37.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.427+0000 7fd85510c800 20 add_watcher() i=3 2024-04-27T03:30:37.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.427+0000 7fd85510c800 20 add_watcher() i=4 2024-04-27T03:30:37.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.427+0000 7fd85510c800 20 add_watcher() i=5 2024-04-27T03:30:37.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.427+0000 7fd85510c800 20 add_watcher() i=6 2024-04-27T03:30:37.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.429+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94131516475584] v25'157 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.429+0000 7fd85510c800 20 add_watcher() i=7 2024-04-27T03:30:37.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.429+0000 7fd85510c800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:37.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.429+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a51e0 con 0x559cb3991430 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.429+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.429+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94131516480768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a25d0 con 0x559cb3991430 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.430+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94131516480768] v25'156 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.430+0000 7fd7097e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.431+0000 7fd85510c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.431+0000 7fd85510c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.431+0000 7fd85510c800 5 note: GC not initialized 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.431+0000 7fd683fff640 20 reqs_thread_entry: start 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.431+0000 7fd85510c800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.431+0000 7fd85510c800 20 Filter name: none 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.431+0000 7fd681ffb640 20 reqs_thread_entry: start 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.431+0000 7fd85510c800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:37.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.431+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.431+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb39d8390 con 0x559cb3991430 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv12 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb39dae20 con 0x7fd844062430 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(63 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 20 enabling bucket name=myfoo 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 8.9 8:91b71841:root::myfoo:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb39d8e00 con 0x7fd844062430 2024-04-27T03:30:37.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(64 myfoo [read 0~165 out=165b] v0'0 uv27 ondisk = 0) ==== 149+0+165 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x1 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 : miss 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 20 rados->read ofs=0 len=0 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.432+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb39db6c0 con 0x559cb3991430 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.433+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv7 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.433+0000 7fd85510c800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.433+0000 7fd85510c800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.433+0000 7fd85510c800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.433+0000 7fd85510c800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.433+0000 7fd85510c800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:37.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.433+0000 7fd85510c800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.433+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 8.3 8:c65d367a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~379 in=379b,setxattr user.rgw.acl (123) in=135b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39e3290 con 0x559cb3991430 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 [delete,create,call,call,writefull 0~379,setxattr (123)] v25'8 uv8 ondisk = 0) ==== 416+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd85510c800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd85510c800 10 moving default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd85510c800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd85510c800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1, ofs0, ns] 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [notify cookie 94131516718464 in=764b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb39dd800 con 0x559cb3991430 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify (1) cookie 94131516459712 notify 107374182427 ret 0) ==== 794+0+0 (crc 0 0 0) 0x559cb39dbab0 con 0x559cb3991430 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd7367fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182427 cookie 94131516459712 notifier 4588 bl.length()=752 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd7367fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 info.flags=0x17 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd7367fc640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1 to cache LRU end 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd7367fc640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:37.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd7367fc640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fd718003ea0 con 0x559cb3991430 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.434+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(67 notify.1 [notify cookie 94131516718464 out=8b] v0'0 uv156 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.435+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [notify-ack] v0'0 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.435+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== watch-notify(notify_complete (2) cookie 94131516718464 notify 107374182427 ret 0) ==== 42+0+48 (crc 0 0 0) 0x559cb39dabb0 con 0x559cb3991430 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.435+0000 7fd85510c800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): orig_sources= new_sources= 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.435+0000 7fd85510c800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): potential sources added=[] removed=[] 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.435+0000 7fd85510c800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): orig_dests= new_dests= 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.435+0000 7fd85510c800 20 handle_bi_update(): bucket=:myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1]): potential dests added=[] removed=[] 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.435+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39de410 con 0x559cb3991430 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.436+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(69 foo [delete,create,call,call,writefull 0~391] v25'13 uv13 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.436+0000 7fd85510c800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.436+0000 7fd85510c800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.436+0000 7fd85510c800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:37.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.436+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify cookie 94131516697328 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x559cb39de410 con 0x559cb3991430 2024-04-27T03:30:37.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.436+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== watch-notify(notify (1) cookie 94131516471488 notify 107374182428 ret 0) ==== 609+0+0 (crc 0 0 0) 0x559cb39dabb0 con 0x559cb3991430 2024-04-27T03:30:37.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.436+0000 7fd735ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182428 cookie 94131516471488 notifier 4588 bl.length()=567 2024-04-27T03:30:37.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.436+0000 7fd735ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:37.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.436+0000 7fd735ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:37.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.436+0000 7fd735ffb640 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fd688003430 con 0x559cb3991430 2024-04-27T03:30:37.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.436+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(70 notify.4 [notify cookie 94131516697328 out=8b] v0'0 uv156 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.437+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== watch-notify(notify_complete (2) cookie 94131516697328 notify 107374182428 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fd84c01e040 con 0x559cb3991430 2024-04-27T03:30:37.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.437+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(71 notify.4 [notify-ack] v0'0 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.454 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:37.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.437+0000 7fd7097e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:37.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.437+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94131516480768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39da2c0 con 0x559cb3991430 2024-04-27T03:30:37.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.438+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(72 data_loggenerations_metadata [watch unwatch cookie 94131516480768] v25'157 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.438+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:73 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94131516452816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a1d00 con 0x7fd844062430 2024-04-27T03:30:37.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.439+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(73 notify.0 [watch unwatch cookie 94131516452816] v25'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.439+0000 7fd85510c800 20 remove_watcher() i=0 2024-04-27T03:30:37.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.439+0000 7fd85510c800 2 removed watcher, disabling cache 2024-04-27T03:30:37.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.439+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:74 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94131516459712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a1d00 con 0x559cb3991430 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.441+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(74 notify.1 [watch unwatch cookie 94131516459712] v25'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.441+0000 7fd85510c800 20 remove_watcher() i=1 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.441+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:75 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94131516464704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a1d00 con 0x559cb3995e20 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.442+0000 7fd852c0b640 1 -- 172.21.15.165:0/775551075 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(75 notify.2 [watch unwatch cookie 94131516464704] v25'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd844075020 con 0x559cb3995e20 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.442+0000 7fd85510c800 20 remove_watcher() i=2 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.442+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:76 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94131516455024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a1d00 con 0x7fd844062430 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.443+0000 7fd85240a640 1 -- 172.21.15.165:0/775551075 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(76 notify.3 [watch unwatch cookie 94131516455024] v25'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd848081070 con 0x7fd844062430 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.443+0000 7fd85510c800 20 remove_watcher() i=3 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.443+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94131516471488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb399b870 con 0x559cb3991430 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.444+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(77 notify.4 [watch unwatch cookie 94131516471488] v25'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x559cb3991430 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.444+0000 7fd85510c800 20 remove_watcher() i=4 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.444+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94131516462064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a1d00 con 0x7fd848076040 2024-04-27T03:30:37.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.445+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(78 notify.5 [watch unwatch cookie 94131516462064] v25'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.445+0000 7fd85510c800 20 remove_watcher() i=5 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.445+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:79 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94131516472608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb399b870 con 0x7fd848076040 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.446+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(79 notify.6 [watch unwatch cookie 94131516472608] v25'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.446+0000 7fd85510c800 20 remove_watcher() i=6 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.446+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:80 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94131516475584] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x559cb39a1d00 con 0x7fd848076040 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(80 notify.7 [watch unwatch cookie 94131516475584] v25'158 uv156 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fd84c031040 con 0x7fd848076040 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 20 remove_watcher() i=7 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd848076040 msgr2=0x7fd848083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd848076040 0x7fd848083410 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559cb3991430 msgr2=0x559cb39938f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559cb3991430 0x559cb39938f0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fd844062430 msgr2=0x7fd844013130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fd844062430 0x7fd844013130 crc :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.462 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559cb3995e20 msgr2=0x559cb3998210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559cb3995e20 0x559cb3998210 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd71c036900 msgr2=0x7fd71c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd71c036900 0x7fd71c038dc0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fd848052fd0 tx=0x7fd84804f000 comp rx=0 tx=0).stop 2024-04-27T03:30:37.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85340c640 1 -- 172.21.15.165:0/775551075 reap_dead start 2024-04-27T03:30:37.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 msgr2=0x559cb3985cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 0x559cb3985cf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fd84401ac40 tx=0x7fd844005ef0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 shutdown_connections 2024-04-27T03:30:37.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 --2- 172.21.15.165:0/775551075 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb397eb70 0x559cb3985cf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 >> 172.21.15.165:0/775551075 conn(0x559cb397ae40 msgr2=0x559cb39784a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 shutdown_connections 2024-04-27T03:30:37.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 -- 172.21.15.165:0/775551075 wait complete. 2024-04-27T03:30:37.463 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 -- 172.21.15.165:0/3844727373 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd72c036970 msgr2=0x7fd72c038e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 --2- 172.21.15.165:0/3844727373 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd72c036970 0x7fd72c038e30 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x559cb3857d40 tx=0x7fd848005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 -- 172.21.15.165:0/3844727373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb39717e0 msgr2=0x559cb3971c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.447+0000 7fd85510c800 1 --2- 172.21.15.165:0/3844727373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb39717e0 0x559cb3971c00 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fd84400eb40 tx=0x7fd84401aaf0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/3844727373 shutdown_connections 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 --2- 172.21.15.165:0/3844727373 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd72c036970 0x7fd72c038e30 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 --2- 172.21.15.165:0/3844727373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb39717e0 0x559cb3971c00 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/3844727373 >> 172.21.15.165:0/3844727373 conn(0x559cb3965880 msgr2=0x559cb3965c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/3844727373 shutdown_connections 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/3844727373 wait complete. 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559cb3960800 msgr2=0x559cb3962c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x559cb3960800 0x559cb3962c70 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559cb385c370 msgr2=0x559cb3959e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559cb385c370 0x559cb3959e60 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.464 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559cb395c6b0 msgr2=0x559cb395eb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559cb395c6b0 0x559cb395eb70 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd738036900 msgr2=0x7fd738038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd738036900 0x7fd738038dc0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7fd848054000 tx=0x7fd84804e000 comp rx=0 tx=0).stop 2024-04-27T03:30:37.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 msgr2=0x559cb3858a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:37.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 --2- 172.21.15.165:0/2958574320 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559cb3856100 0x559cb3858a80 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fd844000f60 tx=0x7fd844000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:37.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85340c640 1 -- 172.21.15.165:0/2958574320 reap_dead start 2024-04-27T03:30:37.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 shutdown_connections 2024-04-27T03:30:37.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 >> 172.21.15.165:0/2958574320 conn(0x559cb3851b00 msgr2=0x559cb39430f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:37.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 shutdown_connections 2024-04-27T03:30:37.465 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:37.448+0000 7fd85510c800 1 -- 172.21.15.165:0/2958574320 wait complete. 2024-04-27T03:30:37.466 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:37.473 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-27T03:30:37.708 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=30000000 bytes_in=0 failed=False 2024-04-27T03:30:37.715 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:52.716 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-04-27T03:30:52.716 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-04-27T03:30:52.717 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-04-27T03:30:52.766 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:52.766 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:52.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.762+0000 7f0f6bd25800 1 Processor -- start 2024-04-27T03:30:52.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.762+0000 7f0f6bd25800 1 -- start start 2024-04-27T03:30:52.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.762+0000 7f0f6bd25800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ab6ba278b0 0x55ab6ba29ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.762+0000 7f0f6bd25800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ab6b8e5d40 con 0x55ab6ba2a1e0 2024-04-27T03:30:52.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.762+0000 7f0f6bd25800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55ab6b8b84d0 con 0x55ab6ba278b0 2024-04-27T03:30:52.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.762+0000 7f0f67ab7640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ab6ba278b0 0x55ab6ba29ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.762+0000 7f0f67ab7640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ab6ba278b0 0x55ab6ba29ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49298/0 (socket says 172.21.15.165:49298) 2024-04-27T03:30:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.762+0000 7f0f672b6640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55ab6ba2a1e0 0x55ab6ba2c5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:56542/0 (socket says 172.21.15.165:56542) 2024-04-27T03:30:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.762+0000 7f0f67ab7640 1 -- 172.21.15.165:0/3922170748 learned_addr learned my addr 172.21.15.165:0/3922170748 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.763+0000 7f0f67ab7640 1 -- 172.21.15.165:0/3922170748 >> v1:172.21.15.89:6789/0 conn(0x55ab6ba2a1e0 legacy=0x55ab6ba2c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.763+0000 7f0f67ab7640 1 -- 172.21.15.165:0/3922170748 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6b8e4590 con 0x55ab6ba278b0 2024-04-27T03:30:52.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.765+0000 7f0f67ab7640 1 --2- 172.21.15.165:0/3922170748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6ba29ca0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800a520 tx=0x7f0f5801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b261a3e5871ef11 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.765+0000 7f0f66ab5640 1 -- 172.21.15.165:0/3922170748 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f58057070 con 0x55ab6ba278b0 2024-04-27T03:30:52.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.765+0000 7f0f66ab5640 1 -- 172.21.15.165:0/3922170748 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f58052570 con 0x55ab6ba278b0 2024-04-27T03:30:52.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.765+0000 7f0f66ab5640 1 -- 172.21.15.165:0/3922170748 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f5804ae60 con 0x55ab6ba278b0 2024-04-27T03:30:52.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.765+0000 7f0f6bd25800 1 -- 172.21.15.165:0/3922170748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 msgr2=0x55ab6ba29ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.765+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/3922170748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6ba29ca0 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f0f5800a520 tx=0x7f0f5801d010 comp rx=0 tx=0).stop 2024-04-27T03:30:52.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.765+0000 7f0f6bd25800 1 -- 172.21.15.165:0/3922170748 shutdown_connections 2024-04-27T03:30:52.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.765+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/3922170748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6ba29ca0 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.765+0000 7f0f6bd25800 1 -- 172.21.15.165:0/3922170748 >> 172.21.15.165:0/3922170748 conn(0x55ab6ba21380 msgr2=0x55ab6ba237a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:52.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.765+0000 7f0f6bd25800 1 -- 172.21.15.165:0/3922170748 shutdown_connections 2024-04-27T03:30:52.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.765+0000 7f0f6bd25800 1 -- 172.21.15.165:0/3922170748 wait complete. 2024-04-27T03:30:52.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.779+0000 7f0f6bd25800 1 Processor -- start 2024-04-27T03:30:52.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.779+0000 7f0f6bd25800 1 -- start start 2024-04-27T03:30:52.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.779+0000 7f0f6bd25800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6bb180d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.779+0000 7f0f6bd25800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ab6b9723c0 con 0x55ab6ba278b0 2024-04-27T03:30:52.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.780+0000 7f0f67ab7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6bb180d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.780+0000 7f0f67ab7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6bb180d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49308/0 (socket says 172.21.15.165:49308) 2024-04-27T03:30:52.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.780+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1030058812 learned_addr learned my addr 172.21.15.165:0/1030058812 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:52.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.780+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1030058812 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6bb186a0 con 0x55ab6ba278b0 2024-04-27T03:30:52.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.780+0000 7f0f67ab7640 1 --2- 172.21.15.165:0/1030058812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6bb180d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0f58055040 tx=0x7f0f5804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f89f122d44657c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.780+0000 7f0e55e0b640 1 -- 172.21.15.165:0/1030058812 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f58057040 con 0x55ab6ba278b0 2024-04-27T03:30:52.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.781+0000 7f0e55e0b640 1 -- 172.21.15.165:0/1030058812 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f58052720 con 0x55ab6ba278b0 2024-04-27T03:30:52.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.781+0000 7f0e55e0b640 1 -- 172.21.15.165:0/1030058812 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f58052a40 con 0x55ab6ba278b0 2024-04-27T03:30:52.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.781+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1030058812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 msgr2=0x55ab6bb180d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.781+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/1030058812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6bb180d0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7f0f58055040 tx=0x7f0f5804c000 comp rx=0 tx=0).stop 2024-04-27T03:30:52.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.781+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1030058812 shutdown_connections 2024-04-27T03:30:52.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.781+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/1030058812 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6bb180d0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.781+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1030058812 >> 172.21.15.165:0/1030058812 conn(0x55ab6ba21380 msgr2=0x55ab6bb139f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:52.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.781+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1030058812 shutdown_connections 2024-04-27T03:30:52.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.781+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1030058812 wait complete. 2024-04-27T03:30:52.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.782+0000 7f0f6bd25800 1 Processor -- start 2024-04-27T03:30:52.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.782+0000 7f0f6bd25800 1 -- start start 2024-04-27T03:30:52.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.782+0000 7f0f6bd25800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6bb1fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.782+0000 7f0f6bd25800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ab6ba2df90 con 0x55ab6ba278b0 2024-04-27T03:30:52.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.782+0000 7f0f67ab7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6bb1fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.782+0000 7f0f67ab7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6bb1fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49320/0 (socket says 172.21.15.165:49320) 2024-04-27T03:30:52.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.782+0000 7f0f67ab7640 1 -- 172.21.15.165:0/673512316 learned_addr learned my addr 172.21.15.165:0/673512316 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:52.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.783+0000 7f0f67ab7640 1 -- 172.21.15.165:0/673512316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6bb20150 con 0x55ab6ba278b0 2024-04-27T03:30:52.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.783+0000 7f0f67ab7640 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6bb1fc10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0f58052540 tx=0x7f0f58003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.783+0000 7f0e4ffff640 1 -- 172.21.15.165:0/673512316 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f58073070 con 0x55ab6ba278b0 2024-04-27T03:30:52.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.783+0000 7f0e4ffff640 1 -- 172.21.15.165:0/673512316 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5805d440 con 0x55ab6ba278b0 2024-04-27T03:30:52.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.783+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ab6bb203e0 con 0x55ab6ba278b0 2024-04-27T03:30:52.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.783+0000 7f0e4ffff640 1 -- 172.21.15.165:0/673512316 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f58057070 con 0x55ab6ba278b0 2024-04-27T03:30:52.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.783+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6bb22ff0 con 0x55ab6ba278b0 2024-04-27T03:30:52.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.784+0000 7f0e4ffff640 1 -- 172.21.15.165:0/673512316 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0f58076070 con 0x55ab6ba278b0 2024-04-27T03:30:52.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.784+0000 7f0e4ffff640 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e50036950 0x7f0e50038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.784+0000 7f0e4ffff640 1 -- 172.21.15.165:0/673512316 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f0f5808da80 con 0x55ab6ba278b0 2024-04-27T03:30:52.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.785+0000 7f0f672b6640 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e50036950 0x7f0e50038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.785+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ab6ba2a1e0 0x55ab6bb26db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.785+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb272f0 con 0x55ab6ba2a1e0 2024-04-27T03:30:52.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.785+0000 7f0f682b8640 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ab6ba2a1e0 0x55ab6bb26db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.785+0000 7f0f672b6640 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e50036950 0x7f0e50038e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c000f60 tx=0x7f0f5c001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.785+0000 7f0f682b8640 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ab6ba2a1e0 0x55ab6bb26db0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.785+0000 7f0f682b8640 1 -- 172.21.15.165:0/673512316 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6ba2a1e0 2024-04-27T03:30:52.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.785+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb24380 con 0x55ab6ba2a1e0 2024-04-27T03:30:52.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.786+0000 7f0f682b8640 1 -- 172.21.15.165:0/673512316 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6ba2a1e0 2024-04-27T03:30:52.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.786+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ab6bb295e0 0x55ab6bb2b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.786+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb2c0c0 con 0x55ab6bb295e0 2024-04-27T03:30:52.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.786+0000 7f0f67ab7640 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ab6bb295e0 0x55ab6bb2b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.786+0000 7f0f67ab7640 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ab6bb295e0 0x55ab6bb2b9d0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.787+0000 7f0f67ab7640 1 -- 172.21.15.165:0/673512316 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0f58075030 con 0x55ab6bb295e0 2024-04-27T03:30:52.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.787+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb28590 con 0x55ab6ba2a1e0 2024-04-27T03:30:52.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.787+0000 7f0f682b8640 1 -- 172.21.15.165:0/673512316 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6ba2a1e0 2024-04-27T03:30:52.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.787+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ab6bb2d570 0x55ab6bb2f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.787+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb30120 con 0x55ab6bb2d570 2024-04-27T03:30:52.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.787+0000 7f0f672b6640 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ab6bb2d570 0x55ab6bb2f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.788+0000 7f0f672b6640 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ab6bb2d570 0x55ab6bb2f9c0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.788+0000 7f0f672b6640 1 -- 172.21.15.165:0/673512316 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0f5c00f690 con 0x55ab6bb2d570 2024-04-27T03:30:52.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.788+0000 7f0f6bd25800 1 Processor -- start 2024-04-27T03:30:52.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.788+0000 7f0f6bd25800 1 -- start start 2024-04-27T03:30:52.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.788+0000 7f0f6bd25800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba27430 0x55ab6ba258a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.788+0000 7f0f6bd25800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ab6b9186b0 con 0x55ab6ba27430 2024-04-27T03:30:52.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.789+0000 7f0f67ab7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba27430 0x55ab6ba258a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.789+0000 7f0f67ab7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba27430 0x55ab6ba258a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49334/0 (socket says 172.21.15.165:49334) 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.789+0000 7f0f67ab7640 1 -- 172.21.15.165:0/2099057495 learned_addr learned my addr 172.21.15.165:0/2099057495 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.789+0000 7f0f67ab7640 1 -- 172.21.15.165:0/2099057495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6ba25de0 con 0x55ab6ba27430 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.789+0000 7f0f67ab7640 1 --2- 172.21.15.165:0/2099057495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba27430 0x55ab6ba258a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0f58005280 tx=0x7f0f5806e6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.789+0000 7f0e4d7fa640 1 -- 172.21.15.165:0/2099057495 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f58083040 con 0x55ab6ba27430 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.790+0000 7f0e4d7fa640 1 -- 172.21.15.165:0/2099057495 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f58054ca0 con 0x55ab6ba27430 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.790+0000 7f0f6bd25800 1 -- 172.21.15.165:0/2099057495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ab6ba26010 con 0x55ab6ba27430 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.790+0000 7f0e4d7fa640 1 -- 172.21.15.165:0/2099057495 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f5805ca80 con 0x55ab6ba27430 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.790+0000 7f0f6bd25800 1 -- 172.21.15.165:0/2099057495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6ba26510 con 0x55ab6ba27430 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.790+0000 7f0e4d7fa640 1 -- 172.21.15.165:0/2099057495 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0f5805cc20 con 0x55ab6ba27430 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.791+0000 7f0e4d7fa640 1 --2- 172.21.15.165:0/2099057495 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e40036950 0x7f0e40038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.791+0000 7f0e4d7fa640 1 -- 172.21.15.165:0/2099057495 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f0f58098ab0 con 0x55ab6ba27430 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.791+0000 7f0f672b6640 1 --2- 172.21.15.165:0/2099057495 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e40036950 0x7f0e40038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.791+0000 7f0f6bd25800 1 Processor -- start 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.791+0000 7f0f6bd25800 1 -- start start 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.791+0000 7f0f672b6640 1 --2- 172.21.15.165:0/2099057495 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e40036950 0x7f0e40038e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55ab6bb27b90 tx=0x7f0f5c00f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.791+0000 7f0f6bd25800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 0x55ab6bb4dda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.791+0000 7f0f6bd25800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ab6bb18960 con 0x55ab6bb4d980 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.791+0000 7f0f67ab7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 0x55ab6bb4dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.791+0000 7f0f67ab7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 0x55ab6bb4dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49342/0 (socket says 172.21.15.165:49342) 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.791+0000 7f0f67ab7640 1 -- 172.21.15.165:0/3229205111 learned_addr learned my addr 172.21.15.165:0/3229205111 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.792+0000 7f0f67ab7640 1 -- 172.21.15.165:0/3229205111 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6bb4e370 con 0x55ab6bb4d980 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.792+0000 7f0f67ab7640 1 --2- 172.21.15.165:0/3229205111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 0x55ab6bb4dda0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0f5805c500 tx=0x7f0f5808d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1029594254a126f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.792+0000 7f0e46ffd640 1 -- 172.21.15.165:0/3229205111 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f580a5030 con 0x55ab6bb4d980 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.792+0000 7f0e46ffd640 1 -- 172.21.15.165:0/3229205111 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f5808d760 con 0x55ab6bb4d980 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.792+0000 7f0e46ffd640 1 -- 172.21.15.165:0/3229205111 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f5809c070 con 0x55ab6bb4d980 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f6bd25800 1 -- 172.21.15.165:0/3229205111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 msgr2=0x55ab6bb4dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/3229205111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 0x55ab6bb4dda0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f0f5805c500 tx=0x7f0f5808d4a0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f6bd25800 1 -- 172.21.15.165:0/3229205111 shutdown_connections 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/3229205111 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 0x55ab6bb4dda0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f6bd25800 1 -- 172.21.15.165:0/3229205111 >> 172.21.15.165:0/3229205111 conn(0x55ab6bb49c50 msgr2=0x55ab6bb47cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f6bd25800 1 -- 172.21.15.165:0/3229205111 shutdown_connections 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f6bd25800 1 -- 172.21.15.165:0/3229205111 wait complete. 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f6bd25800 1 Processor -- start 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f6bd25800 1 -- start start 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f6bd25800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 0x55ab6bb54c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f6bd25800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ab6bb4e9d0 con 0x55ab6bb4d980 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f67ab7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 0x55ab6bb54c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f67ab7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 0x55ab6bb54c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49358/0 (socket says 172.21.15.165:49358) 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.793+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 learned_addr learned my addr 172.21.15.165:0/1616042716 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:52.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.794+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ab6bb551a0 con 0x55ab6bb4d980 2024-04-27T03:30:52.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.794+0000 7f0f67ab7640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 0x55ab6bb54c60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0f58088ea0 tx=0x7f0f58087ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.810 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.794+0000 7f0e457fa640 1 -- 172.21.15.165:0/1616042716 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f58088b00 con 0x55ab6bb4d980 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.794+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ab6bb551a0 con 0x55ab6bb4d980 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.794+0000 7f0e457fa640 1 -- 172.21.15.165:0/1616042716 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0f580bc070 con 0x55ab6bb4d980 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.794+0000 7f0e457fa640 1 -- 172.21.15.165:0/1616042716 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0f5806ac30 con 0x55ab6bb4d980 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.794+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ab6bb551a0 con 0x55ab6bb4d980 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.795+0000 7f0e457fa640 1 -- 172.21.15.165:0/1616042716 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0f5805d720 con 0x55ab6bb4d980 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.795+0000 7f0f6bd25800 20 rados->read ofs=0 len=0 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.795+0000 7f0e457fa640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e30036900 0x7f0e30038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.795+0000 7f0e457fa640 1 -- 172.21.15.165:0/1616042716 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f0f580a9070 con 0x55ab6bb4d980 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.796+0000 7f0f672b6640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e30036900 0x7f0e30038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.796+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ab6bb60250 0x55ab6bb62710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.796+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb62e60 con 0x55ab6bb60250 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.796+0000 7f0f682b8640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ab6bb60250 0x55ab6bb62710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.811 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.796+0000 7f0f672b6640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e30036900 0x7f0e30038dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ab6bb30520 tx=0x7f0f5c010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.796+0000 7f0f682b8640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ab6bb60250 0x55ab6bb62710 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.796+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6bb60250 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.796+0000 7f0f6bd25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.796+0000 7f0f6bd25800 20 realm 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.796+0000 7f0f6bd25800 20 rados->read ofs=0 len=0 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.796+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63270 con 0x55ab6bb60250 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6bb60250 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f6bd25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f6bd25800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f6bd25800 20 rados->read ofs=0 len=0 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63b10 con 0x55ab6bb60250 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6bb60250 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f6bd25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:52.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f6bd25800 20 rados->read ofs=0 len=0 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb60250 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6bb60250 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f6bd25800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f6bd25800 20 rados->read ofs=0 len=0 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ab6bb64c10 0x55ab6bb67000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb67580 con 0x55ab6bb64c10 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.797+0000 7f0f67ab7640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ab6bb64c10 0x55ab6bb67000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.798+0000 7f0f67ab7640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ab6bb64c10 0x55ab6bb67000 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.798+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.798+0000 7f0f6bd25800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.798+0000 7f0f6bd25800 20 searching for the correct realm 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.798+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb68f00 con 0x55ab6bb64c10 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.799+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.799+0000 7f0f67ab7640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0f5808a3e0 0x7f0f58087370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.799+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb68f00 con 0x7f0f5808a3e0 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.799+0000 7f0f672b6640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0f5808a3e0 0x7f0f58087370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.799+0000 7f0f672b6640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0f5808a3e0 0x7f0f58087370 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.799+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.799+0000 7f0f672b6640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0f5c00f070 0x7f0f5c02e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.799+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb68f00 con 0x7f0f5c00f070 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.800+0000 7f0f682b8640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0f5c00f070 0x7f0f5c02e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.800+0000 7f0f682b8640 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0f5c00f070 0x7f0f5c02e910 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.800+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f0f6000a4a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.800+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb60250 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.801+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6bb60250 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.801+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb60250 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.801+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6bb60250 2024-04-27T03:30:52.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.801+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x7f0f5c00f070 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.801+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f6000a4a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.801+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x7f0f5c00f070 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.801+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f6000a4a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.801+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.802+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.802+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb64c10 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.802+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.802+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb60250 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.802+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6bb60250 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.802+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x7f0f5c00f070 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.803+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f6000a4a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.803+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb60250 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.803+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6bb60250 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.803+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.803+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.803+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb64c10 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.803+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.803+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb64c10 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.804+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.804+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x7f0f5c00f070 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.804+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f6000a4a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.804+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb64c10 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.804+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.804+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb64c10 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.804+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.804+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb60250 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.805+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6bb60250 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.805+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb60250 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.805+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0f6000a4a0 con 0x55ab6bb60250 2024-04-27T03:30:52.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.805+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x7f0f5c00f070 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.805+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f60033040 con 0x7f0f5c00f070 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.805+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb64c10 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.806+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.806+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.806+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.806+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb60250 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.806+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.806+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x7f0f5c00f070 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.807+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.807+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb64c10 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.807+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.807+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x7f0f5c00f070 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.807+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.807+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.807+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.808+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb63eb0 con 0x55ab6bb64c10 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.808+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.808+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0f580e0040 con 0x7f0f5c00f070 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.808+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.808+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0f580e0040 con 0x55ab6bb60250 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.808+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.808+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f0f580e0040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:52.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 20 rados->read ofs=0 len=0 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb67ed0 con 0x55ab6bb60250 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 20 rados->read ofs=0 len=0 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb682c0 con 0x55ab6bb60250 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 20 rados->read ofs=0 len=0 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.809+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb682c0 con 0x7f0f5c00f070 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 20 zone default found 2024-04-27T03:30:52.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 4 Realm: () 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 20 zonegroup default 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 20 rados->read ofs=0 len=0 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb686e0 con 0x7f0f5c00f070 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 20 rados->read ofs=0 len=0 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb686e0 con 0x55ab6bb64c10 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:52.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 20 rados->read ofs=0 len=0 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.810+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6adb0 con 0x55ab6bb64c10 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.811+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.811+0000 7f0f6bd25800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.811+0000 7f0f6bd25800 20 started sync module instance, tier type = 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.811+0000 7f0f6bd25800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.811+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6a250 con 0x7f0f5808a3e0 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.812+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.812+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94194734900848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb69eb0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.812+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6bb30 con 0x55ab6bb60250 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.813+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94194734900848] v25'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.813+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.813+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94194734903632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6bf20 con 0x55ab6bb60250 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.813+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6c980 con 0x55ab6bb64c10 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.814+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.814+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94194734908464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6c980 con 0x55ab6bb64c10 2024-04-27T03:30:52.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.814+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6d980 con 0x7f0f5808a3e0 2024-04-27T03:30:52.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.815+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94194734903632] v25'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.815+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94194734908464] v25'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.815+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.815+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94194734912624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6cd70 con 0x7f0f5808a3e0 2024-04-27T03:30:52.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.816+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6e390 con 0x55ab6bb60250 2024-04-27T03:30:52.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.817+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94194734912624] v25'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.817+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.817+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94194734915296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6e7b0 con 0x55ab6bb60250 2024-04-27T03:30:52.828 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.817+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6f170 con 0x7f0f5c00f070 2024-04-27T03:30:52.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.818+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.818+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94194734918848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6f590 con 0x7f0f5c00f070 2024-04-27T03:30:52.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.818+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6ffc0 con 0x7f0f5c00f070 2024-04-27T03:30:52.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.818+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94194734915296] v25'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.819+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94194734918848] v25'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.819+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.819+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94194734922448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6f0e0 con 0x7f0f5c00f070 2024-04-27T03:30:52.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.819+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb70470 con 0x7f0f5c00f070 2024-04-27T03:30:52.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.820+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'159 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.820+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94194734923712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb703e0 con 0x7f0f5c00f070 2024-04-27T03:30:52.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.820+0000 7f0f6bd25800 20 add_watcher() i=0 2024-04-27T03:30:52.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.820+0000 7f0f6bd25800 20 add_watcher() i=1 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.820+0000 7f0f6bd25800 20 add_watcher() i=2 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.820+0000 7f0f6bd25800 20 add_watcher() i=3 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.820+0000 7f0f6bd25800 20 add_watcher() i=4 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.820+0000 7f0f6bd25800 20 add_watcher() i=5 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.820+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94194734922448] v25'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.820+0000 7f0f6bd25800 20 add_watcher() i=6 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.821+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94194734923712] v25'160 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.821+0000 7f0f6bd25800 20 add_watcher() i=7 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.821+0000 7f0f6bd25800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.821+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6e300 con 0x55ab6bb60250 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.822+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.822+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94194734925888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb6e300 con 0x55ab6bb60250 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.823+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94194734925888] v25'158 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.823+0000 7f0e1dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:52.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.823+0000 7f0f6bd25800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:52.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.823+0000 7f0f6bd25800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:52.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.823+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7bf70 con 0x55ab6bb60250 2024-04-27T03:30:52.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.824+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'48 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.833 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.824+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7c520 con 0x7f0f5c00f070 2024-04-27T03:30:52.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.826+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'24 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.826+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7ca90 con 0x55ab6bb64c10 2024-04-27T03:30:52.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.827+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'25 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.827+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7ca90 con 0x55ab6bb60250 2024-04-27T03:30:52.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.828+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'159 uv15 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.828+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7ce50 con 0x7f0f5808a3e0 2024-04-27T03:30:52.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.829+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'50 uv17 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.829+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7d3c0 con 0x55ab6bb64c10 2024-04-27T03:30:52.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.830+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'42 uv16 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.836 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.830+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7d7a0 con 0x55ab6bb60250 2024-04-27T03:30:52.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.831+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'26 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.837 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.831+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7c520 con 0x7f0f5c00f070 2024-04-27T03:30:52.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.832+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'35 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.838 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.832+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7d990 con 0x7f0f5808a3e0 2024-04-27T03:30:52.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.834+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'18 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.839 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.834+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7d990 con 0x55ab6bb64c10 2024-04-27T03:30:52.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.835+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'24 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.840 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.835+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7bd20 con 0x7f0f5808a3e0 2024-04-27T03:30:52.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.836+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'51 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.841 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.836+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7bd20 con 0x7f0f5c00f070 2024-04-27T03:30:52.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.837+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'21 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.842 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.837+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x7f0f5808a3e0 2024-04-27T03:30:52.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.838+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'24 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.843 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.838+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x7f0f5c00f070 2024-04-27T03:30:52.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.839+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'47 uv22 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.839+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x55ab6bb60250 2024-04-27T03:30:52.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.840+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'160 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.840+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x55ab6bb64c10 2024-04-27T03:30:52.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.841+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'45 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.841+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x7f0f5808a3e0 2024-04-27T03:30:52.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.842+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'52 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.842+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x55ab6bb64c10 2024-04-27T03:30:52.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.843+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'30 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.843+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x55ab6bb60250 2024-04-27T03:30:52.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.844+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'26 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.850 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.844+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x55ab6bb60250 2024-04-27T03:30:52.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.845+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'161 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.851 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.845+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x55ab6bb64c10 2024-04-27T03:30:52.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.847+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'46 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.847+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x7f0f5808a3e0 2024-04-27T03:30:52.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.848+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'34 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.848+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e0b0 con 0x55ab6bb64c10 2024-04-27T03:30:52.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.849+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'26 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.849+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7d990 con 0x55ab6bb64c10 2024-04-27T03:30:52.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.850+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'37 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.855 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.850+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7d990 con 0x55ab6bb64c10 2024-04-27T03:30:52.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.850+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'20 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.856 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.851+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x55ab6bb64c10 2024-04-27T03:30:52.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.851+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'43 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.857 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.851+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x7f0f5c00f070 2024-04-27T03:30:52.858 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.853+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'27 uv26 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.853+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x55ab6bb64c10 2024-04-27T03:30:52.859 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.854+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'30 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.854+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x7f0f5c00f070 2024-04-27T03:30:52.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.856+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'36 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.856+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x55ab6bb64c10 2024-04-27T03:30:52.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.857+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'47 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.857+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x7f0f5c00f070 2024-04-27T03:30:52.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.858+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'48 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.858+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7e020 con 0x7f0f5808a3e0 2024-04-27T03:30:52.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.859+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'24 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.859+0000 7f0e1cfe1640 20 reqs_thread_entry: start 2024-04-27T03:30:52.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.860+0000 7f0f6bd25800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:52.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.860+0000 7f0f6bd25800 20 Filter name: none 2024-04-27T03:30:52.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.860+0000 7f0d967fc640 20 reqs_thread_entry: start 2024-04-27T03:30:52.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.860+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:94 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.860+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(94 gc.0 [call out=15b] v0'0 uv11 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.860+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:95 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.860+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(95 gc.0 [call out=24b] v0'0 uv11 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.860+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:96 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.861+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(96 gc.0 [call out=15b] v0'0 uv11 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.861+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:97 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.861+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(97 gc.1 [call out=15b] v0'0 uv12 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.861+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:98 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.861+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(98 gc.1 [call out=24b] v0'0 uv12 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.861+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:99 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.862+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(99 gc.1 [call out=15b] v0'0 uv12 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.862+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:100 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.862+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(100 gc.2 [call out=15b] v0'0 uv7 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.862+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:101 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.862+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(101 gc.2 [call out=24b] v0'0 uv7 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.862+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:102 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.863+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 29 ==== osd_op_reply(102 gc.2 [call out=15b] v0'0 uv7 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.863+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:103 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.863+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(103 gc.3 [call out=15b] v0'0 uv15 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.863+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:104 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.863+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(104 gc.3 [call out=24b] v0'0 uv15 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.863+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:105 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.863+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(105 gc.3 [call out=15b] v0'0 uv15 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.863+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:106 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.864+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(106 gc.4 [call out=15b] v0'0 uv17 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.864+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:107 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.864+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(107 gc.4 [call out=24b] v0'0 uv17 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.864+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:108 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.864+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(108 gc.4 [call out=15b] v0'0 uv17 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.864+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:109 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.864+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 30 ==== osd_op_reply(109 gc.5 [call out=15b] v0'0 uv16 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.864+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:110 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.865+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 31 ==== osd_op_reply(110 gc.5 [call out=24b] v0'0 uv16 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.865+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:111 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.865+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 32 ==== osd_op_reply(111 gc.5 [call out=15b] v0'0 uv16 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.865+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:112 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.865+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(112 gc.6 [call out=15b] v0'0 uv11 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.865+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:113 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.865+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(113 gc.6 [call out=24b] v0'0 uv11 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.865+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:114 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.866+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(114 gc.6 [call out=15b] v0'0 uv11 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.866+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:115 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.866+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(115 gc.7 [call out=15b] v0'0 uv12 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.866+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:116 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.867+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== osd_op_reply(116 gc.7 [call out=24b] v0'0 uv12 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.867+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:117 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.867+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 30 ==== osd_op_reply(117 gc.7 [call out=15b] v0'0 uv12 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.867+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:118 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.868+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== osd_op_reply(118 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.868+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:119 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.868+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(119 gc.8 [call out=24b] v0'0 uv3 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.868+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:120 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.868+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 23 ==== osd_op_reply(120 gc.8 [call out=15b] v0'0 uv3 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.868+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:121 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.868+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 33 ==== osd_op_reply(121 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.868+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:122 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.869+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 34 ==== osd_op_reply(122 gc.9 [call out=24b] v0'0 uv9 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.869+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:123 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.869+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 35 ==== osd_op_reply(123 gc.9 [call out=15b] v0'0 uv9 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.869+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:124 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.869+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 24 ==== osd_op_reply(124 gc.10 [call out=15b] v0'0 uv19 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.869+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:125 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.869+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 25 ==== osd_op_reply(125 gc.10 [call out=24b] v0'0 uv19 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.869+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:126 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.870+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 26 ==== osd_op_reply(126 gc.10 [call out=15b] v0'0 uv19 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.870+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:127 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.870+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 31 ==== osd_op_reply(127 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.870+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:128 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.870+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 32 ==== osd_op_reply(128 gc.11 [call out=24b] v0'0 uv7 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.870+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:129 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.871+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 33 ==== osd_op_reply(129 gc.11 [call out=15b] v0'0 uv7 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.871+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:130 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.871+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 27 ==== osd_op_reply(130 gc.12 [call out=15b] v0'0 uv9 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.871+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:131 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.871+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 28 ==== osd_op_reply(131 gc.12 [call out=24b] v0'0 uv9 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.871+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:132 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.871+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 29 ==== osd_op_reply(132 gc.12 [call out=15b] v0'0 uv9 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.871+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:133 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.872+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 34 ==== osd_op_reply(133 gc.13 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.872+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:134 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.872+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 35 ==== osd_op_reply(134 gc.13 [call out=24b] v0'0 uv22 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.872+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:135 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.873+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 36 ==== osd_op_reply(135 gc.13 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.873+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:136 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.873+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 36 ==== osd_op_reply(136 gc.14 [call out=15b] v0'0 uv19 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.873+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:137 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.873+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 37 ==== osd_op_reply(137 gc.14 [call out=24b] v0'0 uv19 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.873+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:138 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.873+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 38 ==== osd_op_reply(138 gc.14 [call out=15b] v0'0 uv19 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.873+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:139 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.874+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 36 ==== osd_op_reply(139 gc.15 [call out=15b] v0'0 uv15 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.874+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:140 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.874+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 37 ==== osd_op_reply(140 gc.15 [call out=24b] v0'0 uv15 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.874+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:141 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.874+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 38 ==== osd_op_reply(141 gc.15 [call out=15b] v0'0 uv15 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.874+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:142 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.874+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 30 ==== osd_op_reply(142 gc.16 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.874+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:143 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.875+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 31 ==== osd_op_reply(143 gc.16 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.875+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:144 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.875+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 32 ==== osd_op_reply(144 gc.16 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.875+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:145 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.875+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 39 ==== osd_op_reply(145 gc.17 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.875+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:146 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.875+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 40 ==== osd_op_reply(146 gc.17 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.875+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:147 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.876+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 41 ==== osd_op_reply(147 gc.17 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.876+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:148 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.876+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 39 ==== osd_op_reply(148 gc.18 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.876+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:149 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.876+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 40 ==== osd_op_reply(149 gc.18 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.876+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:150 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.876+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 41 ==== osd_op_reply(150 gc.18 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.876+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:151 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.877+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 42 ==== osd_op_reply(151 gc.19 [call out=15b] v0'0 uv8 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.877+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:152 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.877+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 43 ==== osd_op_reply(152 gc.19 [call out=24b] v0'0 uv8 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.877+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:153 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb60250 2024-04-27T03:30:52.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.877+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 44 ==== osd_op_reply(153 gc.19 [call out=15b] v0'0 uv8 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.877+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:154 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.877+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 42 ==== osd_op_reply(154 gc.20 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.877+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:155 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.878+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 43 ==== osd_op_reply(155 gc.20 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.878+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:156 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.878+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 44 ==== osd_op_reply(156 gc.20 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.878+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:157 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.878+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 33 ==== osd_op_reply(157 gc.21 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.878+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:158 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.878+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 34 ==== osd_op_reply(158 gc.21 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.878+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:159 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.879+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 35 ==== osd_op_reply(159 gc.21 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.879+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:160 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.879+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 45 ==== osd_op_reply(160 gc.22 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.879+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:161 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.879+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 46 ==== osd_op_reply(161 gc.22 [call out=24b] v0'0 uv3 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.879+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:162 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.879+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 47 ==== osd_op_reply(162 gc.22 [call out=15b] v0'0 uv3 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.879+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:163 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.880+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 48 ==== osd_op_reply(163 gc.23 [call out=15b] v0'0 uv12 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.880+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:164 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.880+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 49 ==== osd_op_reply(164 gc.23 [call out=24b] v0'0 uv12 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.880+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:165 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.880+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 50 ==== osd_op_reply(165 gc.23 [call out=15b] v0'0 uv12 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.880+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:166 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.880+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 51 ==== osd_op_reply(166 gc.24 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.880+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:167 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.881+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 52 ==== osd_op_reply(167 gc.24 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.881+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:168 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.881+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 53 ==== osd_op_reply(168 gc.24 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.881+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:169 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.881+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 54 ==== osd_op_reply(169 gc.25 [call out=15b] v0'0 uv8 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.881+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:170 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.881+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 55 ==== osd_op_reply(170 gc.25 [call out=24b] v0'0 uv8 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.881+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:171 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x55ab6bb64c10 2024-04-27T03:30:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.882+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 56 ==== osd_op_reply(171 gc.25 [call out=15b] v0'0 uv8 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.882+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.882+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 37 ==== osd_op_reply(172 gc.26 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.882+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.882+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 38 ==== osd_op_reply(173 gc.26 [call out=24b] v0'0 uv26 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.882+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:174 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a3a0 con 0x7f0f5c00f070 2024-04-27T03:30:52.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.883+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 39 ==== osd_op_reply(174 gc.26 [call out=1699b] v0'0 uv26 ondisk = 0) ==== 149+0+1699 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.883+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:175 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bbaa240 con 0x55ab6bb64c10 2024-04-27T03:30:52.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.883+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 57 ==== osd_op_reply(175 gc.27 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.883+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:176 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bba9230 con 0x55ab6bb64c10 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.883+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 58 ==== osd_op_reply(176 gc.27 [call out=24b] v0'0 uv11 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.883+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bbaa240 con 0x55ab6bb64c10 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.884+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 59 ==== osd_op_reply(177 gc.27 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.884+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:178 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bbaa240 con 0x7f0f5c00f070 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.884+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 40 ==== osd_op_reply(178 gc.28 [call out=15b] v0'0 uv8 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.884+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:179 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bba9230 con 0x7f0f5c00f070 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.884+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 41 ==== osd_op_reply(179 gc.28 [call out=24b] v0'0 uv8 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.884+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:180 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bbaa240 con 0x7f0f5c00f070 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.885+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 42 ==== osd_op_reply(180 gc.28 [call out=15b] v0'0 uv8 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.885+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:181 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bbaa240 con 0x55ab6bb64c10 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.885+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 60 ==== osd_op_reply(181 gc.29 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.885+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:182 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bba9230 con 0x55ab6bb64c10 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.885+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 61 ==== osd_op_reply(182 gc.29 [call out=24b] v0'0 uv11 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.885+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:183 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bbaa240 con 0x55ab6bb64c10 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.885+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 62 ==== osd_op_reply(183 gc.29 [call out=15b] v0'0 uv11 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.886+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:184 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bbaa240 con 0x7f0f5c00f070 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.886+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 43 ==== osd_op_reply(184 gc.30 [call out=15b] v0'0 uv10 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.886+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:185 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bba9230 con 0x7f0f5c00f070 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.886+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 44 ==== osd_op_reply(185 gc.30 [call out=24b] v0'0 uv10 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.886+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:186 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bbaa240 con 0x7f0f5c00f070 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.887+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 45 ==== osd_op_reply(186 gc.30 [call out=15b] v0'0 uv10 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.887+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:187 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bbaa240 con 0x7f0f5808a3e0 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.887+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 36 ==== osd_op_reply(187 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.887+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:188 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bba9230 con 0x7f0f5808a3e0 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.887+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 37 ==== osd_op_reply(188 gc.31 [call out=24b] v0'0 uv5 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.887+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:189 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bbaa240 con 0x7f0f5808a3e0 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.887+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 38 ==== osd_op_reply(189 gc.31 [call out=15b] v0'0 uv5 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.888+0000 7f0e1dfe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:52.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.888+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:190 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94194734925888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb79a70 con 0x55ab6bb60250 2024-04-27T03:30:52.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.889+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 45 ==== osd_op_reply(190 data_loggenerations_metadata [watch unwatch cookie 94194734925888] v25'162 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.889+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:191 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94194734900848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a4e0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.890+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 39 ==== osd_op_reply(191 notify.0 [watch unwatch cookie 94194734900848] v25'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.890+0000 7f0f6bd25800 20 remove_watcher() i=0 2024-04-27T03:30:52.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.890+0000 7f0f6bd25800 2 removed watcher, disabling cache 2024-04-27T03:30:52.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.890+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:192 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94194734903632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a4e0 con 0x55ab6bb60250 2024-04-27T03:30:52.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.892+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 46 ==== osd_op_reply(192 notify.1 [watch unwatch cookie 94194734903632] v25'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.892+0000 7f0f6bd25800 20 remove_watcher() i=1 2024-04-27T03:30:52.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.892+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:193 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94194734908464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a4e0 con 0x55ab6bb64c10 2024-04-27T03:30:52.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.893+0000 7f0f67ab7640 1 -- 172.21.15.165:0/1616042716 <== osd.3 v2:172.21.15.165:6808/2380918637 63 ==== osd_op_reply(193 notify.2 [watch unwatch cookie 94194734908464] v25'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f5809e070 con 0x55ab6bb64c10 2024-04-27T03:30:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.893+0000 7f0f6bd25800 20 remove_watcher() i=2 2024-04-27T03:30:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.893+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:194 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94194734912624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a4e0 con 0x7f0f5808a3e0 2024-04-27T03:30:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.894+0000 7f0f672b6640 1 -- 172.21.15.165:0/1616042716 <== osd.2 v2:172.21.15.165:6816/975457786 40 ==== osd_op_reply(194 notify.3 [watch unwatch cookie 94194734912624] v25'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f5c037040 con 0x7f0f5808a3e0 2024-04-27T03:30:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.894+0000 7f0f6bd25800 20 remove_watcher() i=3 2024-04-27T03:30:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.894+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:195 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94194734915296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a4e0 con 0x55ab6bb60250 2024-04-27T03:30:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.895+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.1 v2:172.21.15.165:6800/79879041 47 ==== osd_op_reply(195 notify.4 [watch unwatch cookie 94194734915296] v25'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x55ab6bb60250 2024-04-27T03:30:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.895+0000 7f0f6bd25800 20 remove_watcher() i=4 2024-04-27T03:30:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.895+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:196 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94194734918848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a4e0 con 0x7f0f5c00f070 2024-04-27T03:30:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.896+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 46 ==== osd_op_reply(196 notify.5 [watch unwatch cookie 94194734918848] v25'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.896+0000 7f0f6bd25800 20 remove_watcher() i=5 2024-04-27T03:30:52.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.896+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:197 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94194734922448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a4e0 con 0x7f0f5c00f070 2024-04-27T03:30:52.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.897+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 47 ==== osd_op_reply(197 notify.6 [watch unwatch cookie 94194734922448] v25'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.897+0000 7f0f6bd25800 20 remove_watcher() i=6 2024-04-27T03:30:52.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.897+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:198 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94194734923712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ab6bb7a4e0 con 0x7f0f5c00f070 2024-04-27T03:30:52.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.898+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 <== osd.0 v2:172.21.15.89:6800/2353668432 48 ==== osd_op_reply(198 notify.7 [watch unwatch cookie 94194734923712] v25'161 uv159 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0f60031040 con 0x7f0f5c00f070 2024-04-27T03:30:52.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.898+0000 7f0f6bd25800 20 remove_watcher() i=7 2024-04-27T03:30:52.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0f5c00f070 msgr2=0x7f0f5c02e910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0f5c00f070 0x7f0f5c02e910 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ab6bb60250 msgr2=0x55ab6bb62710 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ab6bb60250 0x55ab6bb62710 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0f5808a3e0 msgr2=0x7f0f58087370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0f5808a3e0 0x7f0f58087370 crc :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ab6bb64c10 msgr2=0x55ab6bb67000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ab6bb64c10 0x55ab6bb67000 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e30036900 msgr2=0x7f0e30038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e30036900 0x7f0e30038dc0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x55ab6bb30520 tx=0x7f0f5c010000 comp rx=0 tx=0).stop 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 msgr2=0x55ab6bb54c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/1616042716 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6bb4d980 0x55ab6bb54c60 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f0f58088ea0 tx=0x7f0f58087ef0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 reap_dead start 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f682b8640 1 -- 172.21.15.165:0/1616042716 reap_dead start 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 shutdown_connections 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 >> 172.21.15.165:0/1616042716 conn(0x55ab6bb49c50 msgr2=0x55ab6bb472b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 shutdown_connections 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/1616042716 wait complete. 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/2099057495 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e40036950 msgr2=0x7f0e40038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/2099057495 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e40036950 0x7f0e40038e10 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x55ab6bb27b90 tx=0x7f0f5c00f000 comp rx=0 tx=0).stop 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/2099057495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba27430 msgr2=0x55ab6ba258a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/2099057495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba27430 0x55ab6ba258a0 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f0f58005280 tx=0x7f0f5806e6d0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/2099057495 shutdown_connections 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/2099057495 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e40036950 0x7f0e40038e10 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/2099057495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba27430 0x55ab6ba258a0 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/2099057495 >> 172.21.15.165:0/2099057495 conn(0x55ab6bb32620 msgr2=0x55ab6bb32a20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/2099057495 shutdown_connections 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.899+0000 7f0f6bd25800 1 -- 172.21.15.165:0/2099057495 wait complete. 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ab6bb2d570 msgr2=0x55ab6bb2f9c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ab6bb2d570 0x55ab6bb2f9c0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ab6ba2a1e0 msgr2=0x55ab6bb26db0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ab6ba2a1e0 0x55ab6bb26db0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ab6bb295e0 msgr2=0x55ab6bb2b9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ab6bb295e0 0x55ab6bb2b9d0 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e50036950 msgr2=0x7f0e50038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e50036950 0x7f0e50038e10 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f0f5c000f60 tx=0x7f0f5c001370 comp rx=0 tx=0).stop 2024-04-27T03:30:52.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 msgr2=0x55ab6bb1fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 --2- 172.21.15.165:0/673512316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ab6ba278b0 0x55ab6bb1fc10 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7f0f58052540 tx=0x7f0f58003020 comp rx=0 tx=0).stop 2024-04-27T03:30:52.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f682b8640 1 -- 172.21.15.165:0/673512316 reap_dead start 2024-04-27T03:30:52.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 shutdown_connections 2024-04-27T03:30:52.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 >> 172.21.15.165:0/673512316 conn(0x55ab6ba21380 msgr2=0x55ab6bb12210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:52.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 shutdown_connections 2024-04-27T03:30:52.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.900+0000 7f0f6bd25800 1 -- 172.21.15.165:0/673512316 wait complete. 2024-04-27T03:30:52.913 INFO:teuthology.orchestra.run.smithi165.stdout:[{"tag":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.44429703447353615\u0000","time":"2024-04-27T03:30:52.710093+0000","objs":[{"pool":"default.rgw.buckets.data","oid":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_1","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_2","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_3","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_4","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_5","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_6","key":"","instance":""},{"pool":"default.rgw.buckets.data","oid":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_7","key":"","instance":""}]}] 2024-04-27T03:30:52.913 DEBUG:tasks.util.rgw: json result: [{'tag': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.44429703447353615\x00', 'time': '2024-04-27T03:30:52.710093+0000', 'objs': [{'pool': 'default.rgw.buckets.data', 'oid': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_1', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_2', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_3', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_4', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_5', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_6', 'key': '', 'instance': ''}, {'pool': 'default.rgw.buckets.data', 'oid': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_7', 'key': '', 'instance': ''}]}] 2024-04-27T03:30:52.913 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process'] 2024-04-27T03:30:52.913 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process'] 2024-04-27T03:30:52.914 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process 2024-04-27T03:30:52.954 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:52.954 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:52.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.949+0000 7f7b949af800 1 Processor -- start 2024-04-27T03:30:52.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.949+0000 7f7b949af800 1 -- start start 2024-04-27T03:30:52.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.949+0000 7f7b949af800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ed1e6b21d0 0x55ed1e7419b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.950+0000 7f7b949af800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ed1e5fdd40 con 0x55ed1e741f50 2024-04-27T03:30:52.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.950+0000 7f7b949af800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55ed1e5d04d0 con 0x55ed1e6b21d0 2024-04-27T03:30:52.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.950+0000 7f7b924b5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ed1e6b21d0 0x55ed1e7419b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.950+0000 7f7b924b5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ed1e6b21d0 0x55ed1e7419b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49362/0 (socket says 172.21.15.165:49362) 2024-04-27T03:30:52.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.950+0000 7f7b924b5640 1 -- 172.21.15.165:0/1884353285 learned_addr learned my addr 172.21.15.165:0/1884353285 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:52.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.950+0000 7f7b924b5640 1 -- 172.21.15.165:0/1884353285 >> v1:172.21.15.89:6789/0 conn(0x55ed1e741f50 legacy=0x55ed1e7443f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.950+0000 7f7b924b5640 1 -- 172.21.15.165:0/1884353285 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1e5fc590 con 0x55ed1e6b21d0 2024-04-27T03:30:52.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.952+0000 7f7b924b5640 1 --2- 172.21.15.165:0/1884353285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e7419b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400a520 tx=0x7f7b8401d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8822b9671f26e0c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.952+0000 7f7b914b3640 1 -- 172.21.15.165:0/1884353285 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84057070 con 0x55ed1e6b21d0 2024-04-27T03:30:52.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.952+0000 7f7b914b3640 1 -- 172.21.15.165:0/1884353285 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b84052570 con 0x55ed1e6b21d0 2024-04-27T03:30:52.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.952+0000 7f7b914b3640 1 -- 172.21.15.165:0/1884353285 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b8404ae60 con 0x55ed1e6b21d0 2024-04-27T03:30:52.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.953+0000 7f7b949af800 1 -- 172.21.15.165:0/1884353285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 msgr2=0x55ed1e7419b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.953+0000 7f7b949af800 1 --2- 172.21.15.165:0/1884353285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e7419b0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f7b8400a520 tx=0x7f7b8401d010 comp rx=0 tx=0).stop 2024-04-27T03:30:52.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.953+0000 7f7b949af800 1 -- 172.21.15.165:0/1884353285 shutdown_connections 2024-04-27T03:30:52.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.953+0000 7f7b949af800 1 --2- 172.21.15.165:0/1884353285 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e7419b0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.953+0000 7f7b949af800 1 -- 172.21.15.165:0/1884353285 >> 172.21.15.165:0/1884353285 conn(0x55ed1e739370 msgr2=0x55ed1e73b790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:52.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.953+0000 7f7b949af800 1 -- 172.21.15.165:0/1884353285 shutdown_connections 2024-04-27T03:30:52.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.953+0000 7f7b949af800 1 -- 172.21.15.165:0/1884353285 wait complete. 2024-04-27T03:30:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.965+0000 7f7b949af800 1 Processor -- start 2024-04-27T03:30:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.965+0000 7f7b949af800 1 -- start start 2024-04-27T03:30:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.965+0000 7f7b949af800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e73f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.965+0000 7f7b949af800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ed1e68a3b0 con 0x55ed1e6b21d0 2024-04-27T03:30:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.965+0000 7f7b924b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e73f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.965+0000 7f7b924b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e73f2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49378/0 (socket says 172.21.15.165:49378) 2024-04-27T03:30:52.971 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.965+0000 7f7b924b5640 1 -- 172.21.15.165:0/3142240525 learned_addr learned my addr 172.21.15.165:0/3142240525 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:52.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.965+0000 7f7b924b5640 1 -- 172.21.15.165:0/3142240525 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1e73d8a0 con 0x55ed1e6b21d0 2024-04-27T03:30:52.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.966+0000 7f7b924b5640 1 --2- 172.21.15.165:0/3142240525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e73f2e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7b8401d050 tx=0x7f7b8404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6b24633e9a711 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.966+0000 7f7a80608640 1 -- 172.21.15.165:0/3142240525 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84057040 con 0x55ed1e6b21d0 2024-04-27T03:30:52.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.966+0000 7f7a80608640 1 -- 172.21.15.165:0/3142240525 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b84052720 con 0x55ed1e6b21d0 2024-04-27T03:30:52.972 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.966+0000 7f7a80608640 1 -- 172.21.15.165:0/3142240525 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84052a40 con 0x55ed1e6b21d0 2024-04-27T03:30:52.975 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.966+0000 7f7b949af800 1 -- 172.21.15.165:0/3142240525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 msgr2=0x55ed1e73f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.966+0000 7f7b949af800 1 --2- 172.21.15.165:0/3142240525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e73f2e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f7b8401d050 tx=0x7f7b8404c000 comp rx=0 tx=0).stop 2024-04-27T03:30:52.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.966+0000 7f7b949af800 1 -- 172.21.15.165:0/3142240525 shutdown_connections 2024-04-27T03:30:52.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.966+0000 7f7b949af800 1 --2- 172.21.15.165:0/3142240525 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e73f2e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.967+0000 7f7b949af800 1 -- 172.21.15.165:0/3142240525 >> 172.21.15.165:0/3142240525 conn(0x55ed1e739370 msgr2=0x55ed1e82b970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:52.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.967+0000 7f7b949af800 1 -- 172.21.15.165:0/3142240525 shutdown_connections 2024-04-27T03:30:52.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.967+0000 7f7b949af800 1 -- 172.21.15.165:0/3142240525 wait complete. 2024-04-27T03:30:52.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.967+0000 7f7b949af800 1 Processor -- start 2024-04-27T03:30:52.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.967+0000 7f7b949af800 1 -- start start 2024-04-27T03:30:52.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.967+0000 7f7b949af800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e839c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.967+0000 7f7b949af800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ed1e745e60 con 0x55ed1e6b21d0 2024-04-27T03:30:52.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.967+0000 7f7b924b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e839c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.967+0000 7f7b924b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e839c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49380/0 (socket says 172.21.15.165:49380) 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.967+0000 7f7b924b5640 1 -- 172.21.15.165:0/4236632037 learned_addr learned my addr 172.21.15.165:0/4236632037 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.967+0000 7f7b924b5640 1 -- 172.21.15.165:0/4236632037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1e83a1b0 con 0x55ed1e6b21d0 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.968+0000 7f7b924b5640 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e839c70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7b8401cfe0 tx=0x7f7b84003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.968+0000 7f7a7ee05640 1 -- 172.21.15.165:0/4236632037 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84073070 con 0x55ed1e6b21d0 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.968+0000 7f7a7ee05640 1 -- 172.21.15.165:0/4236632037 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8405d440 con 0x55ed1e6b21d0 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.968+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ed1e83a440 con 0x55ed1e6b21d0 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.968+0000 7f7a7ee05640 1 -- 172.21.15.165:0/4236632037 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84057070 con 0x55ed1e6b21d0 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.968+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed1e83d0f0 con 0x55ed1e6b21d0 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.969+0000 7f7a7ee05640 1 -- 172.21.15.165:0/4236632037 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f7b84076070 con 0x55ed1e6b21d0 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.969+0000 7f7a7ee05640 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a78036950 0x7f7a78038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.969+0000 7f7a7ee05640 1 -- 172.21.15.165:0/4236632037 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f7b8408db40 con 0x55ed1e6b21d0 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.969+0000 7f7b91cb4640 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a78036950 0x7f7a78038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.969+0000 7f7b949af800 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ed1e741f50 0x55ed1e840eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.969+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8413f0 con 0x55ed1e741f50 2024-04-27T03:30:52.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.969+0000 7f7b92cb6640 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ed1e741f50 0x55ed1e840eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.969+0000 7f7b91cb4640 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a78036950 0x7f7a78038e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7b88001e10 tx=0x7f7b88001390 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.969+0000 7f7b92cb6640 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ed1e741f50 0x55ed1e840eb0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.970+0000 7f7b92cb6640 1 -- 172.21.15.165:0/4236632037 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e741f50 2024-04-27T03:30:52.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.970+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e841cd0 con 0x55ed1e741f50 2024-04-27T03:30:52.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.970+0000 7f7b92cb6640 1 -- 172.21.15.165:0/4236632037 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e741f50 2024-04-27T03:30:52.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.970+0000 7f7b949af800 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ed1e843620 0x55ed1e845ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.970+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e846240 con 0x55ed1e843620 2024-04-27T03:30:52.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.970+0000 7f7b924b5640 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ed1e843620 0x55ed1e845ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.970+0000 7f7b924b5640 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ed1e843620 0x55ed1e845ab0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.971+0000 7f7b924b5640 1 -- 172.21.15.165:0/4236632037 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7b84075030 con 0x55ed1e843620 2024-04-27T03:30:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.971+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e842550 con 0x55ed1e741f50 2024-04-27T03:30:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.971+0000 7f7b92cb6640 1 -- 172.21.15.165:0/4236632037 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e741f50 2024-04-27T03:30:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.971+0000 7f7b949af800 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ed1e8476f0 0x55ed1e849b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.971+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e84a2c0 con 0x55ed1e8476f0 2024-04-27T03:30:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.971+0000 7f7b91cb4640 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ed1e8476f0 0x55ed1e849b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.972+0000 7f7b91cb4640 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ed1e8476f0 0x55ed1e849b60 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.972+0000 7f7b91cb4640 1 -- 172.21.15.165:0/4236632037 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7b880106a0 con 0x55ed1e8476f0 2024-04-27T03:30:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.972+0000 7f7b949af800 1 Processor -- start 2024-04-27T03:30:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.972+0000 7f7b949af800 1 -- start start 2024-04-27T03:30:52.984 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.973+0000 7f7b949af800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e858720 0x55ed1e858b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.973+0000 7f7b949af800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ed1e6306b0 con 0x55ed1e858720 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.973+0000 7f7b924b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e858720 0x55ed1e858b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.973+0000 7f7b924b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e858720 0x55ed1e858b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49386/0 (socket says 172.21.15.165:49386) 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.973+0000 7f7b924b5640 1 -- 172.21.15.165:0/2827728398 learned_addr learned my addr 172.21.15.165:0/2827728398 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.973+0000 7f7b924b5640 1 -- 172.21.15.165:0/2827728398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1e859080 con 0x55ed1e858720 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.973+0000 7f7b924b5640 1 --2- 172.21.15.165:0/2827728398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e858720 0x55ed1e858b40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7b84052a30 tx=0x7f7b84008160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.974+0000 7f7a77fff640 1 -- 172.21.15.165:0/2827728398 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84077a70 con 0x55ed1e858720 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.974+0000 7f7a77fff640 1 -- 172.21.15.165:0/2827728398 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b84077c10 con 0x55ed1e858720 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.974+0000 7f7b949af800 1 -- 172.21.15.165:0/2827728398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ed1e859af0 con 0x55ed1e858720 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.974+0000 7f7a77fff640 1 -- 172.21.15.165:0/2827728398 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b8408d290 con 0x55ed1e858720 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.974+0000 7f7b949af800 1 -- 172.21.15.165:0/2827728398 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed1e859ff0 con 0x55ed1e858720 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.974+0000 7f7a77fff640 1 -- 172.21.15.165:0/2827728398 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f7b8408d430 con 0x55ed1e858720 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.975+0000 7f7a77fff640 1 --2- 172.21.15.165:0/2827728398 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a6c036900 0x7f7a6c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.985 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.975+0000 7f7a77fff640 1 -- 172.21.15.165:0/2827728398 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f7b84083ad0 con 0x55ed1e858720 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.975+0000 7f7b91cb4640 1 --2- 172.21.15.165:0/2827728398 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a6c036900 0x7f7a6c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.975+0000 7f7b949af800 1 Processor -- start 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.975+0000 7f7b949af800 1 -- start start 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.975+0000 7f7b91cb4640 1 --2- 172.21.15.165:0/2827728398 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a6c036900 0x7f7a6c038dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55ed1e83e4c0 tx=0x7f7b88006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.975+0000 7f7b949af800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 0x55ed1e865c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.975+0000 7f7b949af800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ed1e73db60 con 0x55ed1e8657e0 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.975+0000 7f7b924b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 0x55ed1e865c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.975+0000 7f7b924b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 0x55ed1e865c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49388/0 (socket says 172.21.15.165:49388) 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.975+0000 7f7b924b5640 1 -- 172.21.15.165:0/1540182197 learned_addr learned my addr 172.21.15.165:0/1540182197 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.976+0000 7f7b924b5640 1 -- 172.21.15.165:0/1540182197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1e8661d0 con 0x55ed1e8657e0 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.976+0000 7f7b924b5640 1 --2- 172.21.15.165:0/1540182197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 0x55ed1e865c00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7b840066d0 tx=0x7f7b8406e9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4a2ca8ee4ec3d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.976+0000 7f7a75ffb640 1 -- 172.21.15.165:0/1540182197 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84081e60 con 0x55ed1e8657e0 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.976+0000 7f7a75ffb640 1 -- 172.21.15.165:0/1540182197 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b8408d900 con 0x55ed1e8657e0 2024-04-27T03:30:52.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.976+0000 7f7a75ffb640 1 -- 172.21.15.165:0/1540182197 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84079d30 con 0x55ed1e8657e0 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.976+0000 7f7b949af800 1 -- 172.21.15.165:0/1540182197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 msgr2=0x55ed1e865c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.976+0000 7f7b949af800 1 --2- 172.21.15.165:0/1540182197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 0x55ed1e865c00 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7f7b840066d0 tx=0x7f7b8406e9b0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.976+0000 7f7b949af800 1 -- 172.21.15.165:0/1540182197 shutdown_connections 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.976+0000 7f7b949af800 1 --2- 172.21.15.165:0/1540182197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 0x55ed1e865c00 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.976+0000 7f7b949af800 1 -- 172.21.15.165:0/1540182197 >> 172.21.15.165:0/1540182197 conn(0x55ed1e861b40 msgr2=0x55ed1e85fba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.976+0000 7f7b949af800 1 -- 172.21.15.165:0/1540182197 shutdown_connections 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.976+0000 7f7b949af800 1 -- 172.21.15.165:0/1540182197 wait complete. 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.977+0000 7f7b949af800 1 Processor -- start 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.977+0000 7f7b949af800 1 -- start start 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.977+0000 7f7b949af800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 0x55ed1e86cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.977+0000 7f7b949af800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ed1e866880 con 0x55ed1e8657e0 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.977+0000 7f7b924b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 0x55ed1e86cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.977+0000 7f7b924b5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 0x55ed1e86cbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49394/0 (socket says 172.21.15.165:49394) 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.977+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 learned_addr learned my addr 172.21.15.165:0/1866505401 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:52.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.977+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed1e86d130 con 0x55ed1e8657e0 2024-04-27T03:30:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.978+0000 7f7b924b5640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 0x55ed1e86cbf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7b84063fd0 tx=0x7f7b84063f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.978+0000 7f7a63fff640 1 -- 172.21.15.165:0/1866505401 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b84081e60 con 0x55ed1e8657e0 2024-04-27T03:30:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.978+0000 7f7a63fff640 1 -- 172.21.15.165:0/1866505401 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7b84076880 con 0x55ed1e8657e0 2024-04-27T03:30:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.978+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ed1e86d130 con 0x55ed1e8657e0 2024-04-27T03:30:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.978+0000 7f7a63fff640 1 -- 172.21.15.165:0/1866505401 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b840815b0 con 0x55ed1e8657e0 2024-04-27T03:30:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.978+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed1e86d130 con 0x55ed1e8657e0 2024-04-27T03:30:52.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.979+0000 7f7a63fff640 1 -- 172.21.15.165:0/1866505401 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f7b84081750 con 0x55ed1e8657e0 2024-04-27T03:30:53.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.979+0000 7f7a63fff640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a5c036950 0x7f7a5c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.979+0000 7f7a63fff640 1 -- 172.21.15.165:0/1866505401 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f7b840a9070 con 0x55ed1e8657e0 2024-04-27T03:30:53.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.979+0000 7f7b91cb4640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a5c036950 0x7f7a5c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.979+0000 7f7b949af800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.979+0000 7f7b949af800 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ed1e878390 0x55ed1e87a850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.979+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87afa0 con 0x55ed1e878390 2024-04-27T03:30:53.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.979+0000 7f7b91cb4640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a5c036950 0x7f7a5c038e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55ed1e84a6c0 tx=0x7f7b88011000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.979+0000 7f7b92cb6640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ed1e878390 0x55ed1e87a850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.000 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b92cb6640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ed1e878390 0x55ed1e87a850 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b949af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b949af800 20 realm 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b949af800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87b3b0 con 0x55ed1e878390 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b949af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b949af800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b949af800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87bc80 con 0x55ed1e878390 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b949af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b949af800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.980+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e878390 2024-04-27T03:30:53.001 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.981+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.981+0000 7f7b949af800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.981+0000 7f7b949af800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.981+0000 7f7b949af800 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ed1e87cd80 0x55ed1e87f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.981+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87f780 con 0x55ed1e87cd80 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.981+0000 7f7b924b5640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ed1e87cd80 0x55ed1e87f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.981+0000 7f7b924b5640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ed1e87cd80 0x55ed1e87f170 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.981+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.981+0000 7f7b949af800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.981+0000 7f7b949af800 20 searching for the correct realm 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.981+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8810b0 con 0x55ed1e87cd80 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.982+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.982+0000 7f7b924b5640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7b84078060 0x7f7b840a1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.982+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8810b0 con 0x7f7b84078060 2024-04-27T03:30:53.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.982+0000 7f7b91cb4640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7b84078060 0x7f7b840a1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.982+0000 7f7b91cb4640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7b84078060 0x7f7b840a1410 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.982+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.982+0000 7f7b91cb4640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7b880393c0 0x7f7b880397e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.982+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8810b0 con 0x7f7b880393c0 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.983+0000 7f7b92cb6640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7b880393c0 0x7f7b880397e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.983+0000 7f7b92cb6640 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7b880393c0 0x7f7b880397e0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.983+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.983+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e878390 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.984+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.984+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e878390 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.984+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.984+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x7f7b880393c0 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.984+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.984+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x7f7b880393c0 2024-04-27T03:30:53.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.984+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.984+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x7f7b84078060 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.985+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.985+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e87cd80 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.985+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.985+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e878390 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.985+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.985+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x7f7b880393c0 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.985+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.985+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e878390 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.985+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.986+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x7f7b84078060 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.986+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.004 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.986+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e87cd80 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.986+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.986+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e87cd80 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.986+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.986+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x7f7b880393c0 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.987+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.987+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e87cd80 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.987+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.987+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e87cd80 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.987+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.987+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e878390 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.987+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.987+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e878390 2024-04-27T03:30:53.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.987+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.987+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x7f7b880393c0 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.988+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.988+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e87cd80 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.988+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.988+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x7f7b84078060 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.988+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.988+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e878390 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.988+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.989+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x7f7b880393c0 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.989+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.989+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e87cd80 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.989+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.989+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x7f7b880393c0 2024-04-27T03:30:53.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.989+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.989+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x7f7b84078060 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.990+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.990+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e87c020 con 0x55ed1e87cd80 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.990+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.990+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f7b840cf040 con 0x7f7b880393c0 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.990+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.990+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f7b840cf040 con 0x55ed1e878390 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f7b8c038040 con 0x7f7b84078060 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:53.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e880040 con 0x55ed1e878390 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e880430 con 0x55ed1e878390 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.991+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e880430 con 0x7f7b880393c0 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b949af800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b949af800 20 zone default found 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b949af800 4 Realm: () 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b949af800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:53.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b949af800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b949af800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b949af800 20 zonegroup default 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b949af800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e881530 con 0x7f7b880393c0 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b949af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b949af800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e882f10 con 0x55ed1e87cd80 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.992+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.993+0000 7f7b949af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.993+0000 7f7b949af800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.993+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8832b0 con 0x55ed1e87cd80 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.993+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.993+0000 7f7b949af800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.993+0000 7f7b949af800 20 started sync module instance, tier type = 2024-04-27T03:30:53.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.993+0000 7f7b949af800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:53.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.993+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8823b0 con 0x7f7b84078060 2024-04-27T03:30:53.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.994+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.994+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94476907847424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e883650 con 0x7f7b84078060 2024-04-27T03:30:53.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.994+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e883f00 con 0x55ed1e878390 2024-04-27T03:30:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.995+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94476907847424] v25'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.995+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.995+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94476907854320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8842f0 con 0x55ed1e878390 2024-04-27T03:30:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.995+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e884d70 con 0x55ed1e87cd80 2024-04-27T03:30:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.997+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94476907854320] v25'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.997+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.997+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94476907858016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e884d30 con 0x55ed1e87cd80 2024-04-27T03:30:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.997+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e885820 con 0x7f7b84078060 2024-04-27T03:30:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.998+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.998+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94476907860784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8860f0 con 0x7f7b84078060 2024-04-27T03:30:53.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.998+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94476907858016] v25'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.998+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e886bb0 con 0x55ed1e878390 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.999+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94476907860784] v25'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.999+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.999+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94476907865728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e885c40 con 0x55ed1e878390 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:52.999+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e887450 con 0x7f7b880393c0 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.000+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94476907865728] v25'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.000+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.000+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94476907849680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8872f0 con 0x7f7b880393c0 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.000+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e887b40 con 0x7f7b880393c0 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.001+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.001+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94476907869392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e887e60 con 0x7f7b880393c0 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.001+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e888980 con 0x7f7b880393c0 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.001+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94476907849680] v25'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.002+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94476907869392] v25'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.002+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'162 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.002+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94476907870448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e886df0 con 0x7f7b880393c0 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.002+0000 7f7b949af800 20 add_watcher() i=0 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.002+0000 7f7b949af800 20 add_watcher() i=1 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.002+0000 7f7b949af800 20 add_watcher() i=2 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.002+0000 7f7b949af800 20 add_watcher() i=3 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.002+0000 7f7b949af800 20 add_watcher() i=4 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.002+0000 7f7b949af800 20 add_watcher() i=5 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.002+0000 7f7b949af800 20 add_watcher() i=6 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.003+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94476907870448] v25'163 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.003+0000 7f7b949af800 20 add_watcher() i=7 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.003+0000 7f7b949af800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:53.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.003+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e88b970 con 0x55ed1e878390 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.004+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.004+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94476907873360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e888df0 con 0x55ed1e878390 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.005+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94476907873360] v25'163 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.005+0000 7f7a48fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.005+0000 7f7b949af800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.005+0000 7f7b949af800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.005+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8941c0 con 0x55ed1e878390 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.006+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'49 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.006+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e894890 con 0x7f7b880393c0 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.007+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'25 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.007+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e894cb0 con 0x55ed1e87cd80 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.008+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'26 uv7 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.008+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e894c90 con 0x55ed1e878390 2024-04-27T03:30:53.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.009+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'164 uv15 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.009+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e895200 con 0x7f7b84078060 2024-04-27T03:30:53.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.010+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'53 uv17 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.010+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e895770 con 0x55ed1e87cd80 2024-04-27T03:30:53.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.011+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'44 uv16 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.011+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e895770 con 0x55ed1e878390 2024-04-27T03:30:53.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.013+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'27 uv11 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.013+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e895770 con 0x7f7b880393c0 2024-04-27T03:30:53.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.014+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'37 uv12 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.014+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e895790 con 0x7f7b84078060 2024-04-27T03:30:53.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.015+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'19 uv3 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.015+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e895de0 con 0x55ed1e87cd80 2024-04-27T03:30:53.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.016+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'25 uv9 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.016+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e895de0 con 0x7f7b84078060 2024-04-27T03:30:53.022 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.017+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'54 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.017+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896350 con 0x7f7b880393c0 2024-04-27T03:30:53.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.018+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'22 uv7 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.018+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896350 con 0x7f7b84078060 2024-04-27T03:30:53.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.019+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'25 uv9 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.019+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896630 con 0x7f7b880393c0 2024-04-27T03:30:53.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.020+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'49 uv22 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.020+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896630 con 0x55ed1e878390 2024-04-27T03:30:53.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.021+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'165 uv19 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.021+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896630 con 0x55ed1e87cd80 2024-04-27T03:30:53.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.023+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'48 uv15 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.023+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8966c0 con 0x7f7b84078060 2024-04-27T03:30:53.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.023+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'55 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.029 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.023+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896750 con 0x55ed1e87cd80 2024-04-27T03:30:53.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.024+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'31 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.030 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.024+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896750 con 0x55ed1e878390 2024-04-27T03:30:53.031 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.026+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'27 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.026+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e893bc0 con 0x55ed1e878390 2024-04-27T03:30:53.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.027+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'166 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.032 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.027+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e893bc0 con 0x55ed1e87cd80 2024-04-27T03:30:53.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.029+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'49 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.029+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e893bc0 con 0x7f7b84078060 2024-04-27T03:30:53.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.030+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'35 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.030+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8969b0 con 0x55ed1e87cd80 2024-04-27T03:30:53.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.031+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'27 uv3 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.031+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896920 con 0x55ed1e87cd80 2024-04-27T03:30:53.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.032+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'38 uv12 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.032+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896920 con 0x55ed1e87cd80 2024-04-27T03:30:53.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.033+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'21 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.033+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896920 con 0x55ed1e87cd80 2024-04-27T03:30:53.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.033+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'45 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.034+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896920 con 0x7f7b880393c0 2024-04-27T03:30:53.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.035+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'28 uv26 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.035+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896d20 con 0x55ed1e87cd80 2024-04-27T03:30:53.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.036+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'31 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.036+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896d20 con 0x7f7b880393c0 2024-04-27T03:30:53.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.037+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'38 uv8 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.037+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896d20 con 0x55ed1e87cd80 2024-04-27T03:30:53.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.038+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'50 uv11 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.038+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896d20 con 0x7f7b880393c0 2024-04-27T03:30:53.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.039+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'50 uv10 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.039+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e896d20 con 0x7f7b84078060 2024-04-27T03:30:53.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.040+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'25 uv5 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.040+0000 7f79c37fe640 20 reqs_thread_entry: start 2024-04-27T03:30:53.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.040+0000 7f7b949af800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:53.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.040+0000 7f7b949af800 20 Filter name: none 2024-04-27T03:30:53.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.040+0000 7f79c17fa640 20 reqs_thread_entry: start 2024-04-27T03:30:53.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.040+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=1 2024-04-27T03:30:53.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.040+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:94 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b880393c0 2024-04-27T03:30:53.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.041+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(94 gc.13 [call] v25'51 uv51 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.041+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:95 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b880393c0 2024-04-27T03:30:53.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.042+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(95 gc.13 [call out=15b] v0'0 uv51 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.042+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.042+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:96 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x7f7b880393c0 2024-04-27T03:30:53.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.042+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(96 gc.13 [call out=24b] v0'0 uv51 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.042+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:97 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b880393c0 2024-04-27T03:30:53.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.042+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(97 gc.13 [call out=15b] v0'0 uv51 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.042+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.042+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:98 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b880393c0 2024-04-27T03:30:53.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.042+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== osd_op_reply(98 gc.13 [call out=15b] v0'0 uv51 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.042+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.042+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:99 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b880393c0 2024-04-27T03:30:53.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.043+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 30 ==== osd_op_reply(99 gc.13 [call] v25'52 uv52 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.043+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=1 2024-04-27T03:30:53.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.044+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:100 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.044+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(100 gc.14 [call] v25'167 uv167 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.044+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:101 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.045+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(101 gc.14 [call out=15b] v0'0 uv167 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.045+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.045+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:102 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x55ed1e878390 2024-04-27T03:30:53.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.045+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(102 gc.14 [call out=24b] v0'0 uv167 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.045+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:103 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.045+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(103 gc.14 [call out=15b] v0'0 uv167 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.045+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.045+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:104 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.045+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(104 gc.14 [call out=15b] v0'0 uv167 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.045+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.052 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.045+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:105 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.046+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(105 gc.14 [call] v25'168 uv168 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.046+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=1 2024-04-27T03:30:53.053 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.046+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:106 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.048+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(106 gc.15 [call] v25'51 uv51 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.048+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:107 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.048+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(107 gc.15 [call out=15b] v0'0 uv51 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.048+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.048+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:108 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x55ed1e87cd80 2024-04-27T03:30:53.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.048+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 29 ==== osd_op_reply(108 gc.15 [call out=24b] v0'0 uv51 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.048+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:109 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.048+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 30 ==== osd_op_reply(109 gc.15 [call out=15b] v0'0 uv51 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.048+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.048+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:110 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.049+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 31 ==== osd_op_reply(110 gc.15 [call out=15b] v0'0 uv51 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.049+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.049+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:111 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.050+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 32 ==== osd_op_reply(111 gc.15 [call] v25'52 uv52 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.050+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=1 2024-04-27T03:30:53.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.050+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:112 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b84078060 2024-04-27T03:30:53.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.051+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(112 gc.16 [call] v25'56 uv56 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.051+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:113 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b84078060 2024-04-27T03:30:53.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.051+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(113 gc.16 [call out=15b] v0'0 uv56 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.051+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.051+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:114 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x7f7b84078060 2024-04-27T03:30:53.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.051+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(114 gc.16 [call out=24b] v0'0 uv56 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.051+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:115 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b84078060 2024-04-27T03:30:53.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.051+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== osd_op_reply(115 gc.16 [call out=15b] v0'0 uv56 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.051+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.051+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:116 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b84078060 2024-04-27T03:30:53.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.052+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(116 gc.16 [call out=15b] v0'0 uv56 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.052+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.052+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:117 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b84078060 2024-04-27T03:30:53.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.053+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 23 ==== osd_op_reply(117 gc.16 [call] v25'57 uv57 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.053+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=1 2024-04-27T03:30:53.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.053+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:118 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.053+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 33 ==== osd_op_reply(118 gc.17 [call] v25'32 uv32 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.054+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:119 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.054+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 34 ==== osd_op_reply(119 gc.17 [call out=15b] v0'0 uv32 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.054+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.054+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:120 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x55ed1e87cd80 2024-04-27T03:30:53.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.054+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 35 ==== osd_op_reply(120 gc.17 [call out=24b] v0'0 uv32 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.054+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:121 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.054+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 36 ==== osd_op_reply(121 gc.17 [call out=15b] v0'0 uv32 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.054+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.054+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:122 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.054+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 37 ==== osd_op_reply(122 gc.17 [call out=15b] v0'0 uv32 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.054+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.054+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:123 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.055+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 38 ==== osd_op_reply(123 gc.17 [call] v25'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.055+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=1 2024-04-27T03:30:53.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.055+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:124 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.056+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(124 gc.18 [call] v25'28 uv28 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.057+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:125 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.057+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(125 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.057+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.063 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.057+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:126 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x55ed1e878390 2024-04-27T03:30:53.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.057+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(126 gc.18 [call out=24b] v0'0 uv28 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.057+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:127 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.057+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 36 ==== osd_op_reply(127 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.057+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.057+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:128 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.057+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 37 ==== osd_op_reply(128 gc.18 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.057+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.064 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.057+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:129 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.058+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 38 ==== osd_op_reply(129 gc.18 [call] v25'29 uv29 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.058+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=1 2024-04-27T03:30:53.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.058+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:130 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.059+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 39 ==== osd_op_reply(130 gc.19 [call] v25'169 uv169 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.059+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:131 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.059+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 40 ==== osd_op_reply(131 gc.19 [call out=15b] v0'0 uv169 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.059+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.059+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:132 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x55ed1e878390 2024-04-27T03:30:53.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.060+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 41 ==== osd_op_reply(132 gc.19 [call out=24b] v0'0 uv169 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.060+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:133 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.060+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 42 ==== osd_op_reply(133 gc.19 [call out=15b] v0'0 uv169 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.060+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.060+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:134 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.060+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 43 ==== osd_op_reply(134 gc.19 [call out=15b] v0'0 uv169 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.060+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.060+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:135 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e878390 2024-04-27T03:30:53.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.061+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 44 ==== osd_op_reply(135 gc.19 [call] v25'170 uv170 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.061+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=1 2024-04-27T03:30:53.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.061+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:136 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.062+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 39 ==== osd_op_reply(136 gc.20 [call] v25'53 uv53 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.062+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:137 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.063+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 40 ==== osd_op_reply(137 gc.20 [call out=15b] v0'0 uv53 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.063+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.063+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:138 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x55ed1e87cd80 2024-04-27T03:30:53.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.063+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 41 ==== osd_op_reply(138 gc.20 [call out=24b] v0'0 uv53 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.063+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:139 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.063+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 42 ==== osd_op_reply(139 gc.20 [call out=15b] v0'0 uv53 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.063+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.063+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:140 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.063+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 43 ==== osd_op_reply(140 gc.20 [call out=15b] v0'0 uv53 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.063+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.063+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:141 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.064+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 44 ==== osd_op_reply(141 gc.20 [call] v25'54 uv54 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.064+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=1 2024-04-27T03:30:53.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.064+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:142 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b84078060 2024-04-27T03:30:53.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.065+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 24 ==== osd_op_reply(142 gc.21 [call] v25'36 uv36 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.065+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:143 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b84078060 2024-04-27T03:30:53.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.066+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 25 ==== osd_op_reply(143 gc.21 [call out=15b] v0'0 uv36 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.066+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.066+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:144 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x7f7b84078060 2024-04-27T03:30:53.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.066+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 26 ==== osd_op_reply(144 gc.21 [call out=24b] v0'0 uv36 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.066+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:145 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b84078060 2024-04-27T03:30:53.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.066+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 27 ==== osd_op_reply(145 gc.21 [call out=15b] v0'0 uv36 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.066+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.066+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:146 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b84078060 2024-04-27T03:30:53.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.066+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 28 ==== osd_op_reply(146 gc.21 [call out=15b] v0'0 uv36 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.066+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.066+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:147 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b84078060 2024-04-27T03:30:53.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.067+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 29 ==== osd_op_reply(147 gc.21 [call] v25'37 uv37 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.067+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=1 2024-04-27T03:30:53.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.067+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:148 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.068+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 45 ==== osd_op_reply(148 gc.22 [call] v25'28 uv28 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.068+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:149 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.068+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 46 ==== osd_op_reply(149 gc.22 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.068+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.069+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:150 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x55ed1e87cd80 2024-04-27T03:30:53.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.069+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 47 ==== osd_op_reply(150 gc.22 [call out=24b] v0'0 uv28 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.069+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:151 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.069+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 48 ==== osd_op_reply(151 gc.22 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.069+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.069+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:152 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.069+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 49 ==== osd_op_reply(152 gc.22 [call out=15b] v0'0 uv28 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.069+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.069+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:153 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.070+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 50 ==== osd_op_reply(153 gc.22 [call] v25'29 uv29 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.070+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=1 2024-04-27T03:30:53.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.070+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:154 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.071+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 51 ==== osd_op_reply(154 gc.23 [call] v25'39 uv39 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.071+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:155 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.071+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 52 ==== osd_op_reply(155 gc.23 [call out=15b] v0'0 uv39 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.071+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.071+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:156 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x55ed1e87cd80 2024-04-27T03:30:53.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.072+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 53 ==== osd_op_reply(156 gc.23 [call out=24b] v0'0 uv39 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.072+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:157 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.072+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 54 ==== osd_op_reply(157 gc.23 [call out=15b] v0'0 uv39 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.072+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.072+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:158 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.072+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 55 ==== osd_op_reply(158 gc.23 [call out=15b] v0'0 uv39 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.072+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.072+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:159 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.073+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 56 ==== osd_op_reply(159 gc.23 [call] v25'40 uv40 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.073+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=1 2024-04-27T03:30:53.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.073+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:160 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.074+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 57 ==== osd_op_reply(160 gc.24 [call] v25'22 uv22 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.074+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:161 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.074+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 58 ==== osd_op_reply(161 gc.24 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.074+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.074+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:162 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x55ed1e87cd80 2024-04-27T03:30:53.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.075+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 59 ==== osd_op_reply(162 gc.24 [call out=24b] v0'0 uv22 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.075+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:163 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.075+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 60 ==== osd_op_reply(163 gc.24 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.075+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.075+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:164 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.075+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 61 ==== osd_op_reply(164 gc.24 [call out=15b] v0'0 uv22 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.075+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.075+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:165 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.076+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 62 ==== osd_op_reply(165 gc.24 [call] v25'23 uv23 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.076+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=1 2024-04-27T03:30:53.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.076+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:166 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.077+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 63 ==== osd_op_reply(166 gc.25 [call] v25'46 uv46 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.077+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:167 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.077+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 64 ==== osd_op_reply(167 gc.25 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.077+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.077+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:168 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x55ed1e87cd80 2024-04-27T03:30:53.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.078+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 65 ==== osd_op_reply(168 gc.25 [call out=24b] v0'0 uv46 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.078+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:169 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.078+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 66 ==== osd_op_reply(169 gc.25 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.078+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.078+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:170 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.078+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 67 ==== osd_op_reply(170 gc.25 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.078+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.078+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:171 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x55ed1e87cd80 2024-04-27T03:30:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.079+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 68 ==== osd_op_reply(171 gc.25 [call] v25'47 uv47 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.079+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=1 2024-04-27T03:30:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.079+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b880393c0 2024-04-27T03:30:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.080+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 31 ==== osd_op_reply(172 gc.26 [call] v25'29 uv29 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.080+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b880393c0 2024-04-27T03:30:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.081+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 32 ==== osd_op_reply(173 gc.26 [call out=15b] v0'0 uv29 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.081+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.081+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:174 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c1c00 con 0x7f7b880393c0 2024-04-27T03:30:53.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.081+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 33 ==== osd_op_reply(174 gc.26 [call out=24b] v0'0 uv29 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.081+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:175 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b880393c0 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.081+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 34 ==== osd_op_reply(175 gc.26 [call out=15b] v0'0 uv29 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.081+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.081+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:176 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c2c10 con 0x7f7b880393c0 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.081+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 35 ==== osd_op_reply(176 gc.26 [call out=1699b] v0'0 uv29 ondisk = 0) ==== 149+0+1699 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 20 garbage collection: RGWGC::process iterating over entry tag='708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.44429703447353615', time=2024-04-27T03:30:52.710093+0000, chain.objs.size()=7 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_1 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:177 3.20 3:07e5f0c9:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_1:head [call refcount.put in=82b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c0ef0 con 0x7f7b84078060 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_2 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:178 3.18 3:186afd53:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_2:head [call refcount.put in=82b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c5d40 con 0x55ed1e87cd80 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_3 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:179 3.33 3:ce0a46f2:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_3:head [call refcount.put in=82b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c8650 con 0x7f7b880393c0 2024-04-27T03:30:53.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_4 2024-04-27T03:30:53.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:180 3.1 3:826a0f39:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_4:head [call refcount.put in=82b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8caec0 con 0x7f7b880393c0 2024-04-27T03:30:53.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_5 2024-04-27T03:30:53.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:181 3.21 3:859b370e:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_5:head [call refcount.put in=82b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c8650 con 0x7f7b880393c0 2024-04-27T03:30:53.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_6 2024-04-27T03:30:53.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:182 3.23 3:c556ee94:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_6:head [call refcount.put in=82b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8cfc60 con 0x7f7b880393c0 2024-04-27T03:30:53.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_7 2024-04-27T03:30:53.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.082+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:183 3.20 3:07e23023:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_7:head [call refcount.put in=82b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8d24d0 con 0x7f7b84078060 2024-04-27T03:30:53.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.084+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 36 ==== osd_op_reply(181 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_5 [call] v25'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.084+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 69 ==== osd_op_reply(178 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_2 [call] v25'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.084+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 37 ==== osd_op_reply(179 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_3 [call] v25'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.086+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 38 ==== osd_op_reply(180 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_4 [call] v25'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.086+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 30 ==== osd_op_reply(177 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_1 [call] v25'5 uv5 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.087+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 39 ==== osd_op_reply(182 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_6 [call] v25'2 uv2 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.089+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 31 ==== osd_op_reply(183 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.1__shadow_.3yuFYpspnez5XeagXoG7CepVdYq0vnZ_7 [call] v25'6 uv6 ondisk = 0) ==== 230+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.089+0000 7f7b949af800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-27T03:30:53.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.089+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:184 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c08e0 con 0x7f7b880393c0 2024-04-27T03:30:53.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.091+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 40 ==== osd_op_reply(184 gc.26 [call] v25'30 uv30 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.091+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:185 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c08e0 con 0x7f7b880393c0 2024-04-27T03:30:53.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.092+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 41 ==== osd_op_reply(185 gc.26 [call] v25'31 uv31 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.092+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=1 2024-04-27T03:30:53.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.092+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:186 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c08e0 con 0x55ed1e87cd80 2024-04-27T03:30:53.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.093+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 70 ==== osd_op_reply(186 gc.27 [call] v25'32 uv32 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.093+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:187 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c08e0 con 0x55ed1e87cd80 2024-04-27T03:30:53.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.093+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 71 ==== osd_op_reply(187 gc.27 [call out=15b] v0'0 uv32 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.093+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.093+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:188 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c08e0 con 0x55ed1e87cd80 2024-04-27T03:30:53.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.093+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 72 ==== osd_op_reply(188 gc.27 [call out=24b] v0'0 uv32 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.094+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:189 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8cd250 con 0x55ed1e87cd80 2024-04-27T03:30:53.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.094+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 73 ==== osd_op_reply(189 gc.27 [call out=15b] v0'0 uv32 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.094+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.094+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:190 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8cd250 con 0x55ed1e87cd80 2024-04-27T03:30:53.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.094+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 74 ==== osd_op_reply(190 gc.27 [call out=15b] v0'0 uv32 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.094+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.094+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:191 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8cd250 con 0x55ed1e87cd80 2024-04-27T03:30:53.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.096+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 75 ==== osd_op_reply(191 gc.27 [call] v25'33 uv33 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.096+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=1 2024-04-27T03:30:53.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.096+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:192 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8cd250 con 0x7f7b880393c0 2024-04-27T03:30:53.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.097+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 42 ==== osd_op_reply(192 gc.28 [call] v25'39 uv39 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.097+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:193 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8cd250 con 0x7f7b880393c0 2024-04-27T03:30:53.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.097+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 43 ==== osd_op_reply(193 gc.28 [call out=15b] v0'0 uv39 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.097+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.097+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:194 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8cc240 con 0x7f7b880393c0 2024-04-27T03:30:53.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.097+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 44 ==== osd_op_reply(194 gc.28 [call out=24b] v0'0 uv39 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.097+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:195 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8cd250 con 0x7f7b880393c0 2024-04-27T03:30:53.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.098+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 45 ==== osd_op_reply(195 gc.28 [call out=15b] v0'0 uv39 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.098+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.098+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:196 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8cd250 con 0x7f7b880393c0 2024-04-27T03:30:53.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.098+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 46 ==== osd_op_reply(196 gc.28 [call out=15b] v0'0 uv39 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.098+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.098+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:197 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c8150 con 0x7f7b880393c0 2024-04-27T03:30:53.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.099+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 47 ==== osd_op_reply(197 gc.28 [call] v25'40 uv40 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.099+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=1 2024-04-27T03:30:53.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.099+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:198 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c8150 con 0x55ed1e87cd80 2024-04-27T03:30:53.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.100+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 76 ==== osd_op_reply(198 gc.29 [call] v25'55 uv55 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.100+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:199 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80e0 con 0x55ed1e87cd80 2024-04-27T03:30:53.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.100+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 77 ==== osd_op_reply(199 gc.29 [call out=15b] v0'0 uv55 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.100+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.100+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:200 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70d0 con 0x55ed1e87cd80 2024-04-27T03:30:53.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.101+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 78 ==== osd_op_reply(200 gc.29 [call out=24b] v0'0 uv55 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.101+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:201 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80e0 con 0x55ed1e87cd80 2024-04-27T03:30:53.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.101+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 79 ==== osd_op_reply(201 gc.29 [call out=15b] v0'0 uv55 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.101+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.101+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:202 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80e0 con 0x55ed1e87cd80 2024-04-27T03:30:53.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.101+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 80 ==== osd_op_reply(202 gc.29 [call out=15b] v0'0 uv55 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.101+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.101+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:203 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80e0 con 0x55ed1e87cd80 2024-04-27T03:30:53.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.102+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 81 ==== osd_op_reply(203 gc.29 [call] v25'56 uv56 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.102+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=1 2024-04-27T03:30:53.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.102+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:204 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80e0 con 0x7f7b880393c0 2024-04-27T03:30:53.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.103+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 48 ==== osd_op_reply(204 gc.30 [call] v25'53 uv53 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.103+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:205 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80e0 con 0x7f7b880393c0 2024-04-27T03:30:53.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.104+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 49 ==== osd_op_reply(205 gc.30 [call out=15b] v0'0 uv53 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.104+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.104+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:206 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70d0 con 0x7f7b880393c0 2024-04-27T03:30:53.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.104+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 50 ==== osd_op_reply(206 gc.30 [call out=24b] v0'0 uv53 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.104+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:207 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80e0 con 0x7f7b880393c0 2024-04-27T03:30:53.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.104+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 51 ==== osd_op_reply(207 gc.30 [call out=15b] v0'0 uv53 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.104+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.104+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:208 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80e0 con 0x7f7b880393c0 2024-04-27T03:30:53.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.104+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 52 ==== osd_op_reply(208 gc.30 [call out=15b] v0'0 uv53 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.104+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.104+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:209 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80e0 con 0x7f7b880393c0 2024-04-27T03:30:53.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.105+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 53 ==== osd_op_reply(209 gc.30 [call] v25'54 uv54 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.105+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=1 2024-04-27T03:30:53.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.105+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:210 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80e0 con 0x7f7b84078060 2024-04-27T03:30:53.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.107+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 32 ==== osd_op_reply(210 gc.31 [call] v25'26 uv26 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.107+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:211 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.107+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 33 ==== osd_op_reply(211 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.107+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.107+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:212 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.107+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 34 ==== osd_op_reply(212 gc.31 [call out=24b] v0'0 uv26 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.107+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:213 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.107+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 35 ==== osd_op_reply(213 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.107+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.107+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:214 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.108+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 36 ==== osd_op_reply(214 gc.31 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.108+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.108+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:215 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.109+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 37 ==== osd_op_reply(215 gc.31 [call] v25'27 uv27 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.109+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=1 2024-04-27T03:30:53.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.109+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:216 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.110+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 45 ==== osd_op_reply(216 gc.0 [call] v25'50 uv50 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.110+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:217 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.110+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 46 ==== osd_op_reply(217 gc.0 [call out=15b] v0'0 uv50 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.110+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.110+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:218 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x55ed1e878390 2024-04-27T03:30:53.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.111+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 47 ==== osd_op_reply(218 gc.0 [call out=24b] v0'0 uv50 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.111+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:219 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.111+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 48 ==== osd_op_reply(219 gc.0 [call out=15b] v0'0 uv50 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.111+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.111+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:220 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.111+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 49 ==== osd_op_reply(220 gc.0 [call out=15b] v0'0 uv50 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.111+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.111+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:221 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.112+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 50 ==== osd_op_reply(221 gc.0 [call] v25'51 uv51 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.112+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=1 2024-04-27T03:30:53.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.112+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:222 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.113+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 54 ==== osd_op_reply(222 gc.1 [call] v25'26 uv26 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.113+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:223 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.113+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 55 ==== osd_op_reply(223 gc.1 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.113+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.113+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:224 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b880393c0 2024-04-27T03:30:53.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.114+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 56 ==== osd_op_reply(224 gc.1 [call out=24b] v0'0 uv26 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.114+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:225 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.114+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 57 ==== osd_op_reply(225 gc.1 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.114+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.114+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:226 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.114+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 58 ==== osd_op_reply(226 gc.1 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.114+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.114+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:227 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.115+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 59 ==== osd_op_reply(227 gc.1 [call] v25'27 uv27 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.115+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=1 2024-04-27T03:30:53.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.115+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:228 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.116+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 82 ==== osd_op_reply(228 gc.2 [call] v25'27 uv27 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.116+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:229 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.117+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 83 ==== osd_op_reply(229 gc.2 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.117+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.117+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:230 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x55ed1e87cd80 2024-04-27T03:30:53.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.117+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 84 ==== osd_op_reply(230 gc.2 [call out=24b] v0'0 uv27 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.117+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:231 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.117+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 85 ==== osd_op_reply(231 gc.2 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.117+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.117+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:232 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.117+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 86 ==== osd_op_reply(232 gc.2 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.117+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.117+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:233 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.118+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 87 ==== osd_op_reply(233 gc.2 [call] v25'28 uv28 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.118+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=1 2024-04-27T03:30:53.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.118+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:234 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.119+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 51 ==== osd_op_reply(234 gc.3 [call] v25'171 uv171 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.119+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:235 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.120+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 52 ==== osd_op_reply(235 gc.3 [call out=15b] v0'0 uv171 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.120+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.120+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:236 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x55ed1e878390 2024-04-27T03:30:53.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.120+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 53 ==== osd_op_reply(236 gc.3 [call out=24b] v0'0 uv171 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.120+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:237 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.120+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 54 ==== osd_op_reply(237 gc.3 [call out=15b] v0'0 uv171 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.120+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.120+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:238 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.120+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 55 ==== osd_op_reply(238 gc.3 [call out=15b] v0'0 uv171 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.120+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.120+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:239 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.121+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 56 ==== osd_op_reply(239 gc.3 [call] v25'172 uv172 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.121+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=1 2024-04-27T03:30:53.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.121+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:240 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.122+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 38 ==== osd_op_reply(240 gc.4 [call] v25'58 uv58 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.122+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:241 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.123+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 39 ==== osd_op_reply(241 gc.4 [call out=15b] v0'0 uv58 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.123+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.123+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:242 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.123+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 40 ==== osd_op_reply(242 gc.4 [call out=24b] v0'0 uv58 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.123+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:243 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.123+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 41 ==== osd_op_reply(243 gc.4 [call out=15b] v0'0 uv58 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.123+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.123+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:244 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.123+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 42 ==== osd_op_reply(244 gc.4 [call out=15b] v0'0 uv58 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.123+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.123+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:245 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.124+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 43 ==== osd_op_reply(245 gc.4 [call] v25'59 uv59 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.124+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=1 2024-04-27T03:30:53.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.124+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:246 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.125+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 88 ==== osd_op_reply(246 gc.5 [call] v25'48 uv48 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.125+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:247 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.126+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 89 ==== osd_op_reply(247 gc.5 [call out=15b] v0'0 uv48 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.126+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.126+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:248 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x55ed1e87cd80 2024-04-27T03:30:53.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.126+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 90 ==== osd_op_reply(248 gc.5 [call out=24b] v0'0 uv48 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.126+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:249 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.126+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 91 ==== osd_op_reply(249 gc.5 [call out=15b] v0'0 uv48 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.126+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.126+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:250 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.126+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 92 ==== osd_op_reply(250 gc.5 [call out=15b] v0'0 uv48 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.126+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.126+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:251 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.133 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.127+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 93 ==== osd_op_reply(251 gc.5 [call] v25'49 uv49 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.127+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=1 2024-04-27T03:30:53.134 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.127+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:252 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.128+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 57 ==== osd_op_reply(252 gc.6 [call] v25'28 uv28 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.129+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:253 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.129+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 58 ==== osd_op_reply(253 gc.6 [call out=15b] v0'0 uv28 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.129+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.135 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.129+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:254 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x55ed1e878390 2024-04-27T03:30:53.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.129+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 59 ==== osd_op_reply(254 gc.6 [call out=24b] v0'0 uv28 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.129+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:255 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.129+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 60 ==== osd_op_reply(255 gc.6 [call out=15b] v0'0 uv28 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.129+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.129+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:256 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.129+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 61 ==== osd_op_reply(256 gc.6 [call out=15b] v0'0 uv28 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.129+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.129+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:257 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e878390 2024-04-27T03:30:53.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.130+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 62 ==== osd_op_reply(257 gc.6 [call] v25'29 uv29 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.130+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=1 2024-04-27T03:30:53.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.130+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:258 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.132+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 60 ==== osd_op_reply(258 gc.7 [call] v25'41 uv41 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.132+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:259 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.132+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 61 ==== osd_op_reply(259 gc.7 [call out=15b] v0'0 uv41 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.132+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.132+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:260 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b880393c0 2024-04-27T03:30:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.132+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 62 ==== osd_op_reply(260 gc.7 [call out=24b] v0'0 uv41 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.132+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:261 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.133+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 63 ==== osd_op_reply(261 gc.7 [call out=15b] v0'0 uv41 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.133+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.133+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:262 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.133+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 64 ==== osd_op_reply(262 gc.7 [call out=15b] v0'0 uv41 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.133+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.133+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:263 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.134+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 65 ==== osd_op_reply(263 gc.7 [call] v25'42 uv42 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.134+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=1 2024-04-27T03:30:53.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.134+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:264 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.135+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 44 ==== osd_op_reply(264 gc.8 [call] v25'20 uv20 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.135+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:265 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.135+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 45 ==== osd_op_reply(265 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.135+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.135+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:266 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.135+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 46 ==== osd_op_reply(266 gc.8 [call out=24b] v0'0 uv20 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.135+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:267 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.136+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 47 ==== osd_op_reply(267 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.136+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.136+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:268 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.136+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 48 ==== osd_op_reply(268 gc.8 [call out=15b] v0'0 uv20 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.136+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.136+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:269 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.137+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 49 ==== osd_op_reply(269 gc.8 [call] v25'21 uv21 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.137+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=1 2024-04-27T03:30:53.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.137+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:270 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.138+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 94 ==== osd_op_reply(270 gc.9 [call] v25'26 uv26 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.138+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:271 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.138+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 95 ==== osd_op_reply(271 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.138+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.138+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:272 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x55ed1e87cd80 2024-04-27T03:30:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.138+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 96 ==== osd_op_reply(272 gc.9 [call out=24b] v0'0 uv26 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.138+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:273 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.139+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 97 ==== osd_op_reply(273 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.139+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.145 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.139+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:274 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.139+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 98 ==== osd_op_reply(274 gc.9 [call out=15b] v0'0 uv26 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.139+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.146 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.139+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:275 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x55ed1e87cd80 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.140+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 99 ==== osd_op_reply(275 gc.9 [call] v25'27 uv27 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.140+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=1 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.140+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:276 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.141+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 50 ==== osd_op_reply(276 gc.10 [call] v25'60 uv60 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.141+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:277 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.141+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 51 ==== osd_op_reply(277 gc.10 [call out=15b] v0'0 uv60 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.141+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.141+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:278 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.141+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 52 ==== osd_op_reply(278 gc.10 [call out=24b] v0'0 uv60 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.142+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:279 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.142+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 53 ==== osd_op_reply(279 gc.10 [call out=15b] v0'0 uv60 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.142+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.148 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.142+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:280 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.142+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 54 ==== osd_op_reply(280 gc.10 [call out=15b] v0'0 uv60 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.142+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.142+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:281 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.143+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 55 ==== osd_op_reply(281 gc.10 [call] v25'61 uv61 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.143+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=1 2024-04-27T03:30:53.149 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.143+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:282 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.144+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 66 ==== osd_op_reply(282 gc.11 [call] v25'23 uv23 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.144+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:283 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.145+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 67 ==== osd_op_reply(283 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.145+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.145+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:284 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b880393c0 2024-04-27T03:30:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.145+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 68 ==== osd_op_reply(284 gc.11 [call out=24b] v0'0 uv23 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.151 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.145+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:285 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.145+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 69 ==== osd_op_reply(285 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.145+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.145+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:286 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.145+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 70 ==== osd_op_reply(286 gc.11 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.145+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.145+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:287 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b880393c0 2024-04-27T03:30:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.146+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 71 ==== osd_op_reply(287 gc.11 [call] v25'24 uv24 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.146+0000 7f7b949af800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=1 2024-04-27T03:30:53.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.146+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:288 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.147+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 56 ==== osd_op_reply(288 gc.12 [call] v25'26 uv26 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.147+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:289 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.148+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 57 ==== osd_op_reply(289 gc.12 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.148+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.148+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:290 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.148+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 58 ==== osd_op_reply(290 gc.12 [call out=24b] v0'0 uv26 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.148+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:291 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.148+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 59 ==== osd_op_reply(291 gc.12 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.148+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:30:53.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.148+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:292 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c70b0 con 0x7f7b84078060 2024-04-27T03:30:53.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.148+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 60 ==== osd_op_reply(292 gc.12 [call out=15b] v0'0 uv26 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.148+0000 7f7b949af800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:30:53.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.148+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:293 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c80c0 con 0x7f7b84078060 2024-04-27T03:30:53.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.149+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 61 ==== osd_op_reply(293 gc.12 [call] v25'27 uv27 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.150+0000 7f7a48fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:53.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.150+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:294 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94476907873360] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8c0130 con 0x55ed1e878390 2024-04-27T03:30:53.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.151+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 63 ==== osd_op_reply(294 data_loggenerations_metadata [watch unwatch cookie 94476907873360] v25'173 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.151+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:295 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94476907847424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e888450 con 0x7f7b84078060 2024-04-27T03:30:53.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.152+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 62 ==== osd_op_reply(295 notify.0 [watch unwatch cookie 94476907847424] v25'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.152+0000 7f7b949af800 20 remove_watcher() i=0 2024-04-27T03:30:53.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.152+0000 7f7b949af800 2 removed watcher, disabling cache 2024-04-27T03:30:53.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.152+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:296 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94476907854320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e888450 con 0x55ed1e878390 2024-04-27T03:30:53.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.153+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 64 ==== osd_op_reply(296 notify.1 [watch unwatch cookie 94476907854320] v25'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.153+0000 7f7b949af800 20 remove_watcher() i=1 2024-04-27T03:30:53.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.153+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:297 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94476907858016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e888450 con 0x55ed1e87cd80 2024-04-27T03:30:53.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.154+0000 7f7b924b5640 1 -- 172.21.15.165:0/1866505401 <== osd.3 v2:172.21.15.165:6808/2380918637 100 ==== osd_op_reply(297 notify.2 [watch unwatch cookie 94476907858016] v25'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b840bf040 con 0x55ed1e87cd80 2024-04-27T03:30:53.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.154+0000 7f7b949af800 20 remove_watcher() i=2 2024-04-27T03:30:53.160 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.154+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:298 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94476907860784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e888450 con 0x7f7b84078060 2024-04-27T03:30:53.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.155+0000 7f7b91cb4640 1 -- 172.21.15.165:0/1866505401 <== osd.2 v2:172.21.15.165:6816/975457786 63 ==== osd_op_reply(298 notify.3 [watch unwatch cookie 94476907860784] v25'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b88036040 con 0x7f7b84078060 2024-04-27T03:30:53.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.155+0000 7f7b949af800 20 remove_watcher() i=3 2024-04-27T03:30:53.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.155+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:299 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94476907865728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e888450 con 0x55ed1e878390 2024-04-27T03:30:53.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.156+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.1 v2:172.21.15.165:6800/79879041 65 ==== osd_op_reply(299 notify.4 [watch unwatch cookie 94476907865728] v25'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x55ed1e878390 2024-04-27T03:30:53.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.156+0000 7f7b949af800 20 remove_watcher() i=4 2024-04-27T03:30:53.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.156+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:300 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94476907849680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e888450 con 0x7f7b880393c0 2024-04-27T03:30:53.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.157+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 72 ==== osd_op_reply(300 notify.5 [watch unwatch cookie 94476907849680] v25'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.157+0000 7f7b949af800 20 remove_watcher() i=5 2024-04-27T03:30:53.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.157+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:301 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94476907869392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e8827d0 con 0x7f7b880393c0 2024-04-27T03:30:53.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.159+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 73 ==== osd_op_reply(301 notify.6 [watch unwatch cookie 94476907869392] v25'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.159+0000 7f7b949af800 20 remove_watcher() i=6 2024-04-27T03:30:53.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.159+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:302 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94476907870448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55ed1e888450 con 0x7f7b880393c0 2024-04-27T03:30:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 <== osd.0 v2:172.21.15.89:6800/2353668432 74 ==== osd_op_reply(302 notify.7 [watch unwatch cookie 94476907870448] v25'164 uv162 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7b8c00a4a0 con 0x7f7b880393c0 2024-04-27T03:30:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 20 remove_watcher() i=7 2024-04-27T03:30:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7b880393c0 msgr2=0x7f7b880397e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7b880393c0 0x7f7b880397e0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ed1e878390 msgr2=0x55ed1e87a850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ed1e878390 0x55ed1e87a850 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7b84078060 msgr2=0x7f7b840a1410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7b84078060 0x7f7b840a1410 crc :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ed1e87cd80 msgr2=0x55ed1e87f170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ed1e87cd80 0x55ed1e87f170 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a5c036950 msgr2=0x7f7a5c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a5c036950 0x7f7a5c038e10 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x55ed1e84a6c0 tx=0x7f7b88011000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 msgr2=0x55ed1e86cbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b92cb6640 1 -- 172.21.15.165:0/1866505401 reap_dead start 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 0x55ed1e86cbf0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f7b84063fd0 tx=0x7f7b84063f60 comp rx=0 tx=0).stop 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 shutdown_connections 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 --2- 172.21.15.165:0/1866505401 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e8657e0 0x55ed1e86cbf0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 >> 172.21.15.165:0/1866505401 conn(0x55ed1e861b40 msgr2=0x55ed1e85f1a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 shutdown_connections 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/1866505401 wait complete. 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/2827728398 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a6c036900 msgr2=0x7f7a6c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 --2- 172.21.15.165:0/2827728398 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a6c036900 0x7f7a6c038dc0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x55ed1e83e4c0 tx=0x7f7b88006000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/2827728398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e858720 msgr2=0x55ed1e858b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 --2- 172.21.15.165:0/2827728398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e858720 0x55ed1e858b40 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f7b84052a30 tx=0x7f7b84008160 comp rx=0 tx=0).stop 2024-04-27T03:30:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/2827728398 shutdown_connections 2024-04-27T03:30:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 --2- 172.21.15.165:0/2827728398 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a6c036900 0x7f7a6c038dc0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 --2- 172.21.15.165:0/2827728398 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e858720 0x55ed1e858b40 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.160+0000 7f7b949af800 1 -- 172.21.15.165:0/2827728398 >> 172.21.15.165:0/2827728398 conn(0x55ed1e84c7c0 msgr2=0x55ed1e84cbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 -- 172.21.15.165:0/2827728398 shutdown_connections 2024-04-27T03:30:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 -- 172.21.15.165:0/2827728398 wait complete. 2024-04-27T03:30:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ed1e8476f0 msgr2=0x55ed1e849b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55ed1e8476f0 0x55ed1e849b60 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ed1e741f50 msgr2=0x55ed1e840eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ed1e741f50 0x55ed1e840eb0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ed1e843620 msgr2=0x55ed1e845ab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ed1e843620 0x55ed1e845ab0 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a78036950 msgr2=0x7f7a78038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7a78036950 0x7f7a78038e10 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f7b88001e10 tx=0x7f7b88001390 comp rx=0 tx=0).stop 2024-04-27T03:30:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 msgr2=0x55ed1e839c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 --2- 172.21.15.165:0/4236632037 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ed1e6b21d0 0x55ed1e839c70 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f7b8401cfe0 tx=0x7f7b84003020 comp rx=0 tx=0).stop 2024-04-27T03:30:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b92cb6640 1 -- 172.21.15.165:0/4236632037 reap_dead start 2024-04-27T03:30:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 shutdown_connections 2024-04-27T03:30:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 >> 172.21.15.165:0/4236632037 conn(0x55ed1e739370 msgr2=0x55ed1e73e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 shutdown_connections 2024-04-27T03:30:53.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.161+0000 7f7b949af800 1 -- 172.21.15.165:0/4236632037 wait complete. 2024-04-27T03:30:53.175 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'list', '--include-all'] 2024-04-27T03:30:53.175 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'list', '--include-all'] 2024-04-27T03:30:53.175 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc list --include-all 2024-04-27T03:30:53.257 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:53.257 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:53.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.252+0000 7f9bc7c11800 1 Processor -- start 2024-04-27T03:30:53.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.252+0000 7f9bc7c11800 1 -- start start 2024-04-27T03:30:53.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.252+0000 7f9bc7c11800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55bcc6d41970 0x55bcc6d43d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.252+0000 7f9bc7c11800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55bcc6bffd40 con 0x55bcc6d442a0 2024-04-27T03:30:53.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.252+0000 7f9bc7c11800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55bcc6bd24d0 con 0x55bcc6d41970 2024-04-27T03:30:53.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.253+0000 7f9bc3347640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55bcc6d41970 0x55bcc6d43d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.253+0000 7f9bbbfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55bcc6d442a0 0x55bcc6d46690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:56558/0 (socket says 172.21.15.165:56558) 2024-04-27T03:30:53.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.253+0000 7f9bbbfff640 1 -- 172.21.15.165:0/1142179244 learned_addr learned my addr 172.21.15.165:0/1142179244 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:53.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.253+0000 7f9bc3347640 1 -- 172.21.15.165:0/1142179244 >> v1:172.21.15.89:6789/0 conn(0x55bcc6d442a0 legacy=0x55bcc6d46690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.253+0000 7f9bc3347640 1 -- 172.21.15.165:0/1142179244 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc6bfe590 con 0x55bcc6d41970 2024-04-27T03:30:53.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.255+0000 7f9bc3347640 1 --2- 172.21.15.165:0/1142179244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6d43d60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400a470 tx=0x7f9bb401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abcab883e5079f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.255+0000 7f9bc2b46640 1 -- 172.21.15.165:0/1142179244 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb4057070 con 0x55bcc6d41970 2024-04-27T03:30:53.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.255+0000 7f9bc2b46640 1 -- 172.21.15.165:0/1142179244 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb4052570 con 0x55bcc6d41970 2024-04-27T03:30:53.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.255+0000 7f9bc2b46640 1 -- 172.21.15.165:0/1142179244 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb4052870 con 0x55bcc6d41970 2024-04-27T03:30:53.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.256+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1142179244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 msgr2=0x55bcc6d43d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.256+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/1142179244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6d43d60 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f9bb400a470 tx=0x7f9bb401ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:53.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.256+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1142179244 shutdown_connections 2024-04-27T03:30:53.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.256+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/1142179244 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6d43d60 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.256+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1142179244 >> 172.21.15.165:0/1142179244 conn(0x55bcc6d3b320 msgr2=0x55bcc6d3d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.256+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1142179244 shutdown_connections 2024-04-27T03:30:53.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.256+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1142179244 wait complete. 2024-04-27T03:30:53.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.267+0000 7f9bc7c11800 1 Processor -- start 2024-04-27T03:30:53.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.268+0000 7f9bc7c11800 1 -- start start 2024-04-27T03:30:53.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.268+0000 7f9bc7c11800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6e32110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.268+0000 7f9bc7c11800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bcc6c8c3c0 con 0x55bcc6d41970 2024-04-27T03:30:53.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.268+0000 7f9bc3347640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6e32110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.268+0000 7f9bc3347640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6e32110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49418/0 (socket says 172.21.15.165:49418) 2024-04-27T03:30:53.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.268+0000 7f9bc3347640 1 -- 172.21.15.165:0/2385953655 learned_addr learned my addr 172.21.15.165:0/2385953655 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.268+0000 7f9bc3347640 1 -- 172.21.15.165:0/2385953655 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc6e326e0 con 0x55bcc6d41970 2024-04-27T03:30:53.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.269+0000 7f9bc3347640 1 --2- 172.21.15.165:0/2385953655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6e32110 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4055040 tx=0x7f9bb404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18aa69a9d554e418 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.269+0000 7f9ab160a640 1 -- 172.21.15.165:0/2385953655 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb4057070 con 0x55bcc6d41970 2024-04-27T03:30:53.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.269+0000 7f9ab160a640 1 -- 172.21.15.165:0/2385953655 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb4063070 con 0x55bcc6d41970 2024-04-27T03:30:53.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.269+0000 7f9ab160a640 1 -- 172.21.15.165:0/2385953655 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb4052720 con 0x55bcc6d41970 2024-04-27T03:30:53.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.269+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2385953655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 msgr2=0x55bcc6e32110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.269+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2385953655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6e32110 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4055040 tx=0x7f9bb404c000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.269+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2385953655 shutdown_connections 2024-04-27T03:30:53.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.269+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2385953655 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6e32110 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.269+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2385953655 >> 172.21.15.165:0/2385953655 conn(0x55bcc6d3b320 msgr2=0x55bcc6e2da30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.269+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2385953655 shutdown_connections 2024-04-27T03:30:53.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.269+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2385953655 wait complete. 2024-04-27T03:30:53.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.270+0000 7f9bc7c11800 1 Processor -- start 2024-04-27T03:30:53.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.270+0000 7f9bc7c11800 1 -- start start 2024-04-27T03:30:53.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.270+0000 7f9bc7c11800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6e39b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.270+0000 7f9bc7c11800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bcc6d4a0c0 con 0x55bcc6d41970 2024-04-27T03:30:53.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.270+0000 7f9bc3347640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6e39b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.270+0000 7f9bc3347640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6e39b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49426/0 (socket says 172.21.15.165:49426) 2024-04-27T03:30:53.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.270+0000 7f9bc3347640 1 -- 172.21.15.165:0/3688263457 learned_addr learned my addr 172.21.15.165:0/3688263457 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.270+0000 7f9bc3347640 1 -- 172.21.15.165:0/3688263457 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc6e3a0a0 con 0x55bcc6d41970 2024-04-27T03:30:53.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.271+0000 7f9bc3347640 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6e39b60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4060040 tx=0x7f9bb4002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.271+0000 7f9aafe07640 1 -- 172.21.15.165:0/3688263457 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb400a2d0 con 0x55bcc6d41970 2024-04-27T03:30:53.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.271+0000 7f9aafe07640 1 -- 172.21.15.165:0/3688263457 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb406c070 con 0x55bcc6d41970 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.271+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bcc6e3a330 con 0x55bcc6d41970 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.271+0000 7f9aafe07640 1 -- 172.21.15.165:0/3688263457 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb405e5d0 con 0x55bcc6d41970 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.271+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc6e3cf40 con 0x55bcc6d41970 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.272+0000 7f9aafe07640 1 -- 172.21.15.165:0/3688263457 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9bb405e770 con 0x55bcc6d41970 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.272+0000 7f9aafe07640 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9aa8036950 0x7f9aa8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.272+0000 7f9aafe07640 1 -- 172.21.15.165:0/3688263457 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f9bb408e900 con 0x55bcc6d41970 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.272+0000 7f9bbbfff640 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9aa8036950 0x7f9aa8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.272+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bcc6d442a0 0x55bcc6e40d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.272+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e41280 con 0x55bcc6d442a0 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.272+0000 7f9bc3b48640 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bcc6d442a0 0x55bcc6e40d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.272+0000 7f9bbbfff640 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9aa8036950 0x7f9aa8038e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9bac000f60 tx=0x7f9bac0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.273+0000 7f9bc3b48640 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bcc6d442a0 0x55bcc6e40d00 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.273+0000 7f9bc3b48640 1 -- 172.21.15.165:0/3688263457 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6d442a0 2024-04-27T03:30:53.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.273+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e3e2d0 con 0x55bcc6d442a0 2024-04-27T03:30:53.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.273+0000 7f9bc3b48640 1 -- 172.21.15.165:0/3688263457 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6d442a0 2024-04-27T03:30:53.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.273+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bcc6e43590 0x55bcc6e45980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.273+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e46020 con 0x55bcc6e43590 2024-04-27T03:30:53.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.273+0000 7f9bc3347640 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bcc6e43590 0x55bcc6e45980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.273+0000 7f9bc3347640 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bcc6e43590 0x55bcc6e45980 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.274+0000 7f9bc3347640 1 -- 172.21.15.165:0/3688263457 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9bb4071990 con 0x55bcc6e43590 2024-04-27T03:30:53.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.274+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e42540 con 0x55bcc6d442a0 2024-04-27T03:30:53.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.274+0000 7f9bc3b48640 1 -- 172.21.15.165:0/3688263457 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6d442a0 2024-04-27T03:30:53.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.274+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55bcc6e474d0 0x55bcc6e49920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.274+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e4a080 con 0x55bcc6e474d0 2024-04-27T03:30:53.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.274+0000 7f9bbbfff640 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55bcc6e474d0 0x55bcc6e49920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.275+0000 7f9bbbfff640 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55bcc6e474d0 0x55bcc6e49920 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.291 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.275+0000 7f9bbbfff640 1 -- 172.21.15.165:0/3688263457 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9bac009050 con 0x55bcc6e474d0 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.275+0000 7f9bc7c11800 1 Processor -- start 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.275+0000 7f9bc7c11800 1 -- start start 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.276+0000 7f9bc7c11800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6e584e0 0x55bcc6e58900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.276+0000 7f9bc7c11800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bcc6c326b0 con 0x55bcc6e584e0 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.276+0000 7f9bc3347640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6e584e0 0x55bcc6e58900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.276+0000 7f9bc3347640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6e584e0 0x55bcc6e58900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49432/0 (socket says 172.21.15.165:49432) 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.276+0000 7f9bc3347640 1 -- 172.21.15.165:0/1025240535 learned_addr learned my addr 172.21.15.165:0/1025240535 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.276+0000 7f9bc3347640 1 -- 172.21.15.165:0/1025240535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc6e58e40 con 0x55bcc6e584e0 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.276+0000 7f9bc3347640 1 --2- 172.21.15.165:0/1025240535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6e584e0 0x55bcc6e58900 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4052540 tx=0x7f9bb40716a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.276+0000 7f9aad602640 1 -- 172.21.15.165:0/1025240535 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb4092070 con 0x55bcc6e584e0 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.276+0000 7f9aad602640 1 -- 172.21.15.165:0/1025240535 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb4058040 con 0x55bcc6e584e0 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.276+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1025240535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bcc6e598b0 con 0x55bcc6e584e0 2024-04-27T03:30:53.292 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.276+0000 7f9aad602640 1 -- 172.21.15.165:0/1025240535 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb408eba0 con 0x55bcc6e584e0 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.276+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1025240535 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc6e59db0 con 0x55bcc6e584e0 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.277+0000 7f9aad602640 1 -- 172.21.15.165:0/1025240535 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9bb408e260 con 0x55bcc6e584e0 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.277+0000 7f9aad602640 1 --2- 172.21.15.165:0/1025240535 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9a9c0369a0 0x7f9a9c038e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.277+0000 7f9aad602640 1 -- 172.21.15.165:0/1025240535 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f9bb4097ab0 con 0x55bcc6e584e0 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.277+0000 7f9bbbfff640 1 --2- 172.21.15.165:0/1025240535 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9a9c0369a0 0x7f9a9c038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.278+0000 7f9bc7c11800 1 Processor -- start 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.278+0000 7f9bc7c11800 1 -- start start 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.278+0000 7f9bbbfff640 1 --2- 172.21.15.165:0/1025240535 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9a9c0369a0 0x7f9a9c038e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00bf50 tx=0x7f9bac00f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.278+0000 7f9bc7c11800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 0x55bcc6d41380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.278+0000 7f9bc7c11800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bcc6e329a0 con 0x55bcc6d40f60 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.278+0000 7f9bc3347640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 0x55bcc6d41380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.278+0000 7f9bc3347640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 0x55bcc6d41380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49434/0 (socket says 172.21.15.165:49434) 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.278+0000 7f9bc3347640 1 -- 172.21.15.165:0/2095160953 learned_addr learned my addr 172.21.15.165:0/2095160953 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.278+0000 7f9bc3347640 1 -- 172.21.15.165:0/2095160953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc6d3f760 con 0x55bcc6d40f60 2024-04-27T03:30:53.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.279+0000 7f9bc3347640 1 --2- 172.21.15.165:0/2095160953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 0x55bcc6d41380 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4009ff0 tx=0x7f9bb408eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51bedacc4b233f10 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.279+0000 7f9aa6ffd640 1 -- 172.21.15.165:0/2095160953 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb409b040 con 0x55bcc6d40f60 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.279+0000 7f9aa6ffd640 1 -- 172.21.15.165:0/2095160953 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb407afe0 con 0x55bcc6d40f60 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.279+0000 7f9aa6ffd640 1 -- 172.21.15.165:0/2095160953 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb408b130 con 0x55bcc6d40f60 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.279+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2095160953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 msgr2=0x55bcc6d41380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.279+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2095160953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 0x55bcc6d41380 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4009ff0 tx=0x7f9bb408eeb0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.279+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2095160953 shutdown_connections 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.279+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2095160953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 0x55bcc6d41380 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.279+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2095160953 >> 172.21.15.165:0/2095160953 conn(0x55bcc6e61930 msgr2=0x55bcc6e5f990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.279+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2095160953 shutdown_connections 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.279+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2095160953 wait complete. 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.279+0000 7f9bc7c11800 1 Processor -- start 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.280+0000 7f9bc7c11800 1 -- start start 2024-04-27T03:30:53.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.280+0000 7f9bc7c11800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 0x55bcc6e6ec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.280+0000 7f9bc7c11800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bcc6d3fb70 con 0x55bcc6d40f60 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.280+0000 7f9bc3347640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 0x55bcc6e6ec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.280+0000 7f9bc3347640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 0x55bcc6e6ec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49446/0 (socket says 172.21.15.165:49446) 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.280+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 learned_addr learned my addr 172.21.15.165:0/2928187073 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.280+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bcc6e6f1a0 con 0x55bcc6d40f60 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.280+0000 7f9bc3347640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 0x55bcc6e6ec60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9bb408bbc0 tx=0x7f9bb406cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.281+0000 7f9aa57fa640 1 -- 172.21.15.165:0/2928187073 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb40bb070 con 0x55bcc6d40f60 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.281+0000 7f9aa57fa640 1 -- 172.21.15.165:0/2928187073 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9bb4066960 con 0x55bcc6d40f60 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.281+0000 7f9aa57fa640 1 -- 172.21.15.165:0/2928187073 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9bb409b050 con 0x55bcc6d40f60 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.281+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bcc6e6f1a0 con 0x55bcc6d40f60 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.281+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bcc6e6f1a0 con 0x55bcc6d40f60 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.282+0000 7f9aa57fa640 1 -- 172.21.15.165:0/2928187073 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9bb40be070 con 0x55bcc6d40f60 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.282+0000 7f9bc7c11800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.282+0000 7f9aa57fa640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9a8c036900 0x7f9a8c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.282+0000 7f9aa57fa640 1 -- 172.21.15.165:0/2928187073 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f9bb40a7070 con 0x55bcc6d40f60 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.282+0000 7f9bbbfff640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9a8c036900 0x7f9a8c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.282+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bcc6e7a300 0x55bcc6e7c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.282+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7cef0 con 0x55bcc6e7a300 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.282+0000 7f9bc3b48640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bcc6e7a300 0x55bcc6e7c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.282+0000 7f9bbbfff640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9a8c036900 0x7f9a8c038dc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9bac001b60 tx=0x7f9bac010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc3b48640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bcc6e7a300 0x55bcc6e7c7a0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc7c11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc7c11800 20 realm 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc7c11800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7d300 con 0x55bcc6e7a300 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc7c11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc7c11800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc7c11800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7dba0 con 0x55bcc6e7a300 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc7c11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc7c11800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.283+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7a300 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.284+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.284+0000 7f9bc7c11800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.284+0000 7f9bc7c11800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.284+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bcc6e7eca0 0x55bcc6e81090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.284+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e815d0 con 0x55bcc6e7eca0 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.284+0000 7f9bc3347640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bcc6e7eca0 0x55bcc6e81090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.284+0000 7f9bc3347640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bcc6e7eca0 0x55bcc6e81090 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.284+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.284+0000 7f9bc7c11800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.284+0000 7f9bc7c11800 20 searching for the correct realm 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.284+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e82f80 con 0x55bcc6e7eca0 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.285+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.285+0000 7f9bc3347640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9bb40c0060 0x7f9bb40974d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.285+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e82f80 con 0x7f9bb40c0060 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.285+0000 7f9bbbfff640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9bb40c0060 0x7f9bb40974d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.285+0000 7f9bbbfff640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9bb40c0060 0x7f9bb40974d0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.285+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bac009050 con 0x7f9bb40c0060 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.285+0000 7f9bbbfff640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9bac038040 0x7f9bac01bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.285+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e82f80 con 0x7f9bac038040 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.286+0000 7f9bc3b48640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9bac038040 0x7f9bac01bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.286+0000 7f9bc3b48640 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9bac038040 0x7f9bac01bab0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.286+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.286+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7a300 2024-04-27T03:30:53.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.286+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.286+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7a300 2024-04-27T03:30:53.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.287+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.287+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x7f9bac038040 2024-04-27T03:30:53.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.287+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.287+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x7f9bac038040 2024-04-27T03:30:53.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.287+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.287+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x7f9bb40c0060 2024-04-27T03:30:53.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.287+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bac009050 con 0x7f9bb40c0060 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.287+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7eca0 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.288+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.288+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7a300 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.288+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.288+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x7f9bac038040 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.288+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.288+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7a300 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.288+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.288+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x7f9bb40c0060 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.289+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9bac009050 con 0x7f9bb40c0060 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.289+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7eca0 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.289+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.289+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7eca0 2024-04-27T03:30:53.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.289+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.289+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x7f9bac038040 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.290+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.290+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7eca0 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.290+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.290+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7eca0 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.290+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.290+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7a300 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.290+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.290+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7a300 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.291+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.291+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x7f9bac038040 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.291+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.291+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7eca0 2024-04-27T03:30:53.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.291+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.291+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x7f9bb40c0060 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.291+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bac009050 con 0x7f9bb40c0060 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.291+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7a300 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.292+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.292+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x7f9bac038040 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.292+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.292+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7eca0 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.292+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.292+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x7f9bac038040 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.293+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.293+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x7f9bb40c0060 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.293+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bac009050 con 0x7f9bb40c0060 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.293+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7eca0 2024-04-27T03:30:53.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.293+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.293+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x7f9bac038040 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.293+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.293+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7a300 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9bbc038040 con 0x7f9bb40c0060 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc7c11800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc7c11800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc7c11800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc7c11800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc7c11800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc7c11800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:53.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc7c11800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e7df40 con 0x55bcc6e7a300 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc7c11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc7c11800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.294+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e81f50 con 0x55bcc6e7a300 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e81f50 con 0x7f9bac038040 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 20 zone default found 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 4 Realm: () 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:53.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 20 zonegroup default 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e82370 con 0x7f9bac038040 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e827b0 con 0x55bcc6e7eca0 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.295+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.296+0000 7f9bc7c11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.296+0000 7f9bc7c11800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.296+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e84e30 con 0x55bcc6e7eca0 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.296+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.296+0000 7f9bc7c11800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.296+0000 7f9bc7c11800 20 started sync module instance, tier type = 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.296+0000 7f9bc7c11800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.296+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e842d0 con 0x7f9bb40c0060 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.297+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.297+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94269279321632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e851d0 con 0x7f9bb40c0060 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.297+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e85a20 con 0x55bcc6e7a300 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.298+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94269279321632] v25'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.298+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.298+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94269279327600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e85e40 con 0x55bcc6e7a300 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.299+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e866f0 con 0x55bcc6e7eca0 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.300+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94269279327600] v25'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.300+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.300+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94269279329136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e86b10 con 0x55bcc6e7eca0 2024-04-27T03:30:53.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.300+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e87510 con 0x7f9bb40c0060 2024-04-27T03:30:53.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.301+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94269279329136] v25'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.301+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.301+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94269279330288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e87de0 con 0x7f9bb40c0060 2024-04-27T03:30:53.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.301+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e887a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.302+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94269279330288] v25'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.302+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.302+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94269279339248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e88bc0 con 0x55bcc6e7a300 2024-04-27T03:30:53.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.302+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e89580 con 0x7f9bac038040 2024-04-27T03:30:53.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.303+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94269279339248] v25'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.303+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.303+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94269279340784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e89580 con 0x7f9bac038040 2024-04-27T03:30:53.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.303+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e89f40 con 0x7f9bac038040 2024-04-27T03:30:53.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.304+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.304+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94269279345296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e8a360 con 0x7f9bac038040 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.304+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e8ae80 con 0x7f9bac038040 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.305+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94269279340784] v25'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.305+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94269279345296] v25'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.305+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'165 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.305+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94269279349072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e89eb0 con 0x7f9bac038040 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.305+0000 7f9bc7c11800 20 add_watcher() i=0 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.305+0000 7f9bc7c11800 20 add_watcher() i=1 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.305+0000 7f9bc7c11800 20 add_watcher() i=2 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.305+0000 7f9bc7c11800 20 add_watcher() i=3 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.305+0000 7f9bc7c11800 20 add_watcher() i=4 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.305+0000 7f9bc7c11800 20 add_watcher() i=5 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.305+0000 7f9bc7c11800 20 add_watcher() i=6 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.307+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94269279349072] v25'166 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.307+0000 7f9bc7c11800 20 add_watcher() i=7 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.307+0000 7f9bc7c11800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.307+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e8a360 con 0x55bcc6e7a300 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.307+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.307+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94269279342800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e8a360 con 0x55bcc6e7a300 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.308+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94269279342800] v25'174 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.308+0000 7f9a79fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.308+0000 7f9bc7c11800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.308+0000 7f9bc7c11800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.308+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e96380 con 0x55bcc6e7a300 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.309+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'52 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.309+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e96a30 con 0x7f9bac038040 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.310+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.311+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e96e10 con 0x55bcc6e7eca0 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.311+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'29 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.311+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e96dd0 con 0x55bcc6e7a300 2024-04-27T03:30:53.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.312+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'175 uv172 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.312+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e974a0 con 0x7f9bb40c0060 2024-04-27T03:30:53.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.313+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'62 uv59 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.313+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e97ec0 con 0x55bcc6e7eca0 2024-04-27T03:30:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.314+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'50 uv49 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.314+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e97ec0 con 0x55bcc6e7a300 2024-04-27T03:30:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.316+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'30 uv29 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.316+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e97ec0 con 0x7f9bac038040 2024-04-27T03:30:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.316+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'43 uv42 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.317+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e97ec0 con 0x7f9bb40c0060 2024-04-27T03:30:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.318+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'22 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.318+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98430 con 0x55bcc6e7eca0 2024-04-27T03:30:53.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.319+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.319+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e984c0 con 0x7f9bb40c0060 2024-04-27T03:30:53.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.320+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'63 uv61 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.320+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98b90 con 0x7f9bac038040 2024-04-27T03:30:53.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.321+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'25 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.321+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98b90 con 0x7f9bb40c0060 2024-04-27T03:30:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.322+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.322+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a00 con 0x7f9bac038040 2024-04-27T03:30:53.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.323+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'55 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.323+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a00 con 0x55bcc6e7a300 2024-04-27T03:30:53.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.324+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'176 uv168 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.324+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a00 con 0x55bcc6e7eca0 2024-04-27T03:30:53.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.325+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'57 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.325+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a00 con 0x7f9bb40c0060 2024-04-27T03:30:53.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.326+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'64 uv57 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.326+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x55bcc6e7eca0 2024-04-27T03:30:53.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.327+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'34 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.327+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x55bcc6e7a300 2024-04-27T03:30:53.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.328+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'30 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.328+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x55bcc6e7a300 2024-04-27T03:30:53.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.329+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'177 uv170 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.329+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x55bcc6e7eca0 2024-04-27T03:30:53.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.330+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'58 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.330+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x7f9bb40c0060 2024-04-27T03:30:53.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.331+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'38 uv37 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.331+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x55bcc6e7eca0 2024-04-27T03:30:53.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.332+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'30 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.332+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x55bcc6e7eca0 2024-04-27T03:30:53.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.333+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'41 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.333+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x55bcc6e7eca0 2024-04-27T03:30:53.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.334+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'24 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.334+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x55bcc6e7eca0 2024-04-27T03:30:53.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.335+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'51 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.340 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.335+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x7f9bac038040 2024-04-27T03:30:53.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.336+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'32 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.341 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.336+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x55bcc6e7eca0 2024-04-27T03:30:53.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.337+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'34 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.337+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x7f9bac038040 2024-04-27T03:30:53.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.338+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'44 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.338+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x55bcc6e7eca0 2024-04-27T03:30:53.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.339+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'59 uv56 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.339+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x7f9bac038040 2024-04-27T03:30:53.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.340+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'56 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.340+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e98a90 con 0x7f9bb40c0060 2024-04-27T03:30:53.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.341+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'28 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.342+0000 7f9a78fe1640 20 reqs_thread_entry: start 2024-04-27T03:30:53.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.342+0000 7f9bc7c11800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:53.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.342+0000 7f9bc7c11800 20 Filter name: none 2024-04-27T03:30:53.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.342+0000 7f99f27fc640 20 reqs_thread_entry: start 2024-04-27T03:30:53.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.342+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:94 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.342+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(94 gc.0 [call out=15b] v0'0 uv51 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.342+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:95 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.343+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(95 gc.0 [call out=24b] v0'0 uv51 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.343+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:96 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.343+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(96 gc.0 [call out=15b] v0'0 uv51 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.343+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:97 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.343+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(97 gc.1 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.343+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:98 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.343+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(98 gc.1 [call out=24b] v0'0 uv27 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.343+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:99 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.344+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(99 gc.1 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.344+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:100 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.344+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(100 gc.2 [call out=15b] v0'0 uv28 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.344+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:101 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.344+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(101 gc.2 [call out=24b] v0'0 uv28 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.344+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:102 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.344+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 29 ==== osd_op_reply(102 gc.2 [call out=15b] v0'0 uv28 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.344+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:103 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.345+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(103 gc.3 [call out=15b] v0'0 uv172 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.345+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:104 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.345+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(104 gc.3 [call out=24b] v0'0 uv172 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.345+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:105 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.345+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(105 gc.3 [call out=15b] v0'0 uv172 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.345+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:106 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.345+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(106 gc.4 [call out=15b] v0'0 uv59 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.345+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:107 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.346+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(107 gc.4 [call out=24b] v0'0 uv59 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.346+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:108 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.346+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(108 gc.4 [call out=15b] v0'0 uv59 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.346+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:109 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.346+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 30 ==== osd_op_reply(109 gc.5 [call out=15b] v0'0 uv49 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.346+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:110 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.347+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 31 ==== osd_op_reply(110 gc.5 [call out=24b] v0'0 uv49 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.347+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:111 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.347+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 32 ==== osd_op_reply(111 gc.5 [call out=15b] v0'0 uv49 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.347+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:112 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.347+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(112 gc.6 [call out=15b] v0'0 uv29 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.347+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:113 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.347+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(113 gc.6 [call out=24b] v0'0 uv29 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.347+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:114 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.348+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(114 gc.6 [call out=15b] v0'0 uv29 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.348+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:115 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.348+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(115 gc.7 [call out=15b] v0'0 uv42 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.348+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:116 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.348+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== osd_op_reply(116 gc.7 [call out=24b] v0'0 uv42 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.348+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:117 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.349+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 30 ==== osd_op_reply(117 gc.7 [call out=15b] v0'0 uv42 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.349+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:118 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.349+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== osd_op_reply(118 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.349+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:119 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.349+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(119 gc.8 [call out=24b] v0'0 uv21 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.349+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:120 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.349+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 23 ==== osd_op_reply(120 gc.8 [call out=15b] v0'0 uv21 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.349+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:121 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.350+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 33 ==== osd_op_reply(121 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.350+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:122 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.350+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 34 ==== osd_op_reply(122 gc.9 [call out=24b] v0'0 uv27 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.350+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:123 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.350+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 35 ==== osd_op_reply(123 gc.9 [call out=15b] v0'0 uv27 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.350+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:124 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.351+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 24 ==== osd_op_reply(124 gc.10 [call out=15b] v0'0 uv61 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.351+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:125 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.351+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 25 ==== osd_op_reply(125 gc.10 [call out=24b] v0'0 uv61 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.351+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:126 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.351+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 26 ==== osd_op_reply(126 gc.10 [call out=15b] v0'0 uv61 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.351+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:127 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.352+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 31 ==== osd_op_reply(127 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.352+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:128 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.352+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 32 ==== osd_op_reply(128 gc.11 [call out=24b] v0'0 uv24 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.352+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:129 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.353+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 33 ==== osd_op_reply(129 gc.11 [call out=15b] v0'0 uv24 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.353+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:130 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.353+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 27 ==== osd_op_reply(130 gc.12 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.353+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:131 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.353+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 28 ==== osd_op_reply(131 gc.12 [call out=24b] v0'0 uv27 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.353+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:132 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.354+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 29 ==== osd_op_reply(132 gc.12 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.354+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:133 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.354+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 34 ==== osd_op_reply(133 gc.13 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.354+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:134 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.354+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 35 ==== osd_op_reply(134 gc.13 [call out=24b] v0'0 uv52 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.354+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:135 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.355+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 36 ==== osd_op_reply(135 gc.13 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.355+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:136 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.355+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 36 ==== osd_op_reply(136 gc.14 [call out=15b] v0'0 uv168 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.355+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:137 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.355+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 37 ==== osd_op_reply(137 gc.14 [call out=24b] v0'0 uv168 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.355+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:138 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.355+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 38 ==== osd_op_reply(138 gc.14 [call out=15b] v0'0 uv168 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.355+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:139 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.356+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 36 ==== osd_op_reply(139 gc.15 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.356+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:140 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.356+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 37 ==== osd_op_reply(140 gc.15 [call out=24b] v0'0 uv52 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.356+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:141 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.356+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 38 ==== osd_op_reply(141 gc.15 [call out=15b] v0'0 uv52 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.356+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:142 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.356+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 30 ==== osd_op_reply(142 gc.16 [call out=15b] v0'0 uv57 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.356+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:143 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.357+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 31 ==== osd_op_reply(143 gc.16 [call out=24b] v0'0 uv57 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.357+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:144 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.357+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 32 ==== osd_op_reply(144 gc.16 [call out=15b] v0'0 uv57 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.357+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:145 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.357+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 39 ==== osd_op_reply(145 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.357+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:146 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.357+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 40 ==== osd_op_reply(146 gc.17 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.357+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:147 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.358+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 41 ==== osd_op_reply(147 gc.17 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.358+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:148 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.358+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 39 ==== osd_op_reply(148 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.358+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:149 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.358+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 40 ==== osd_op_reply(149 gc.18 [call out=24b] v0'0 uv29 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.358+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:150 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.358+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 41 ==== osd_op_reply(150 gc.18 [call out=15b] v0'0 uv29 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.358+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:151 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.359+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 42 ==== osd_op_reply(151 gc.19 [call out=15b] v0'0 uv170 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.359+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:152 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.359+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 43 ==== osd_op_reply(152 gc.19 [call out=24b] v0'0 uv170 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.359+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:153 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7a300 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.359+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 44 ==== osd_op_reply(153 gc.19 [call out=15b] v0'0 uv170 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.359+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:154 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.359+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 42 ==== osd_op_reply(154 gc.20 [call out=15b] v0'0 uv54 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.359+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:155 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.359+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 43 ==== osd_op_reply(155 gc.20 [call out=24b] v0'0 uv54 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.359+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:156 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.360+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 44 ==== osd_op_reply(156 gc.20 [call out=15b] v0'0 uv54 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.360+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:157 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.360+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 33 ==== osd_op_reply(157 gc.21 [call out=15b] v0'0 uv37 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.360+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:158 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.360+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 34 ==== osd_op_reply(158 gc.21 [call out=24b] v0'0 uv37 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.360+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:159 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.360+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 35 ==== osd_op_reply(159 gc.21 [call out=15b] v0'0 uv37 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.360+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:160 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.361+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 45 ==== osd_op_reply(160 gc.22 [call out=15b] v0'0 uv29 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.361+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:161 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.361+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 46 ==== osd_op_reply(161 gc.22 [call out=24b] v0'0 uv29 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.361+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:162 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.361+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 47 ==== osd_op_reply(162 gc.22 [call out=15b] v0'0 uv29 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.361+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:163 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.361+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 48 ==== osd_op_reply(163 gc.23 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.362+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:164 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.362+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 49 ==== osd_op_reply(164 gc.23 [call out=24b] v0'0 uv40 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.362+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:165 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.362+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 50 ==== osd_op_reply(165 gc.23 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.362+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:166 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.362+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 51 ==== osd_op_reply(166 gc.24 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.362+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:167 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.362+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 52 ==== osd_op_reply(167 gc.24 [call out=24b] v0'0 uv23 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.362+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:168 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.363+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 53 ==== osd_op_reply(168 gc.24 [call out=15b] v0'0 uv23 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.363+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:169 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.363+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 54 ==== osd_op_reply(169 gc.25 [call out=15b] v0'0 uv47 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.363+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:170 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.363+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 55 ==== osd_op_reply(170 gc.25 [call out=24b] v0'0 uv47 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.363+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:171 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.363+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 56 ==== osd_op_reply(171 gc.25 [call out=15b] v0'0 uv47 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.363+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:172 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.364+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 37 ==== osd_op_reply(172 gc.26 [call out=15b] v0'0 uv31 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.364+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:173 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.364+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 38 ==== osd_op_reply(173 gc.26 [call out=24b] v0'0 uv31 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.364+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:174 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.364+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 39 ==== osd_op_reply(174 gc.26 [call out=15b] v0'0 uv31 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.364+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:175 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.365+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 57 ==== osd_op_reply(175 gc.27 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.365+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:176 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.365+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 58 ==== osd_op_reply(176 gc.27 [call out=24b] v0'0 uv33 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.365+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:177 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.365+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 59 ==== osd_op_reply(177 gc.27 [call out=15b] v0'0 uv33 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.365+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:178 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.366+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 40 ==== osd_op_reply(178 gc.28 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.366+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:179 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.366+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 41 ==== osd_op_reply(179 gc.28 [call out=24b] v0'0 uv40 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.366+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:180 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.366+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 42 ==== osd_op_reply(180 gc.28 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.366+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:181 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.366+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 60 ==== osd_op_reply(181 gc.29 [call out=15b] v0'0 uv56 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.366+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:182 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.367+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 61 ==== osd_op_reply(182 gc.29 [call out=24b] v0'0 uv56 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.367+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:183 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x55bcc6e7eca0 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.367+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 62 ==== osd_op_reply(183 gc.29 [call out=15b] v0'0 uv56 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.367+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:184 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.367+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 43 ==== osd_op_reply(184 gc.30 [call out=15b] v0'0 uv54 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.367+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:185 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.367+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 44 ==== osd_op_reply(185 gc.30 [call out=24b] v0'0 uv54 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.367+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:186 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bac038040 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.368+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 45 ==== osd_op_reply(186 gc.30 [call out=15b] v0'0 uv54 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.368+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:187 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.368+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 36 ==== osd_op_reply(187 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.368+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:188 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.368+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 37 ==== osd_op_reply(188 gc.31 [call out=24b] v0'0 uv27 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.368+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:189 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e94810 con 0x7f9bb40c0060 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.368+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 38 ==== osd_op_reply(189 gc.31 [call out=15b] v0'0 uv27 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.369+0000 7f9a79fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.369+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:190 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94269279342800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e947b0 con 0x55bcc6e7a300 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.370+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 45 ==== osd_op_reply(190 data_loggenerations_metadata [watch unwatch cookie 94269279342800] v25'178 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.370+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:191 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94269279321632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e890d0 con 0x7f9bb40c0060 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.371+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 39 ==== osd_op_reply(191 notify.0 [watch unwatch cookie 94269279321632] v25'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.371+0000 7f9bc7c11800 20 remove_watcher() i=0 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.371+0000 7f9bc7c11800 2 removed watcher, disabling cache 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.371+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:192 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94269279327600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e83e20 con 0x55bcc6e7a300 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.373+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 46 ==== osd_op_reply(192 notify.1 [watch unwatch cookie 94269279327600] v25'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.373+0000 7f9bc7c11800 20 remove_watcher() i=1 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.373+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:193 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94269279329136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e83e20 con 0x55bcc6e7eca0 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.374+0000 7f9bc3347640 1 -- 172.21.15.165:0/2928187073 <== osd.3 v2:172.21.15.165:6808/2380918637 63 ==== osd_op_reply(193 notify.2 [watch unwatch cookie 94269279329136] v25'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bb40bd030 con 0x55bcc6e7eca0 2024-04-27T03:30:53.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.374+0000 7f9bc7c11800 20 remove_watcher() i=2 2024-04-27T03:30:53.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.374+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:194 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94269279330288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e85b70 con 0x7f9bb40c0060 2024-04-27T03:30:53.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.375+0000 7f9bbbfff640 1 -- 172.21.15.165:0/2928187073 <== osd.2 v2:172.21.15.165:6816/975457786 40 ==== osd_op_reply(194 notify.3 [watch unwatch cookie 94269279330288] v25'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bac045040 con 0x7f9bb40c0060 2024-04-27T03:30:53.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.375+0000 7f9bc7c11800 20 remove_watcher() i=3 2024-04-27T03:30:53.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.375+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:195 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94269279339248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e83e20 con 0x55bcc6e7a300 2024-04-27T03:30:53.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.376+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.1 v2:172.21.15.165:6800/79879041 47 ==== osd_op_reply(195 notify.4 [watch unwatch cookie 94269279339248] v25'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x55bcc6e7a300 2024-04-27T03:30:53.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.376+0000 7f9bc7c11800 20 remove_watcher() i=4 2024-04-27T03:30:53.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.376+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:196 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94269279340784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e85b70 con 0x7f9bac038040 2024-04-27T03:30:53.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.377+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 46 ==== osd_op_reply(196 notify.5 [watch unwatch cookie 94269279340784] v25'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.377+0000 7f9bc7c11800 20 remove_watcher() i=5 2024-04-27T03:30:53.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.377+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:197 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94269279345296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e882f0 con 0x7f9bac038040 2024-04-27T03:30:53.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.378+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 47 ==== osd_op_reply(197 notify.6 [watch unwatch cookie 94269279345296] v25'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.378+0000 7f9bc7c11800 20 remove_watcher() i=6 2024-04-27T03:30:53.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.378+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:198 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94269279349072] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55bcc6e83e20 con 0x7f9bac038040 2024-04-27T03:30:53.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 <== osd.0 v2:172.21.15.89:6800/2353668432 48 ==== osd_op_reply(198 notify.7 [watch unwatch cookie 94269279349072] v25'167 uv165 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9bbc00a4a0 con 0x7f9bac038040 2024-04-27T03:30:53.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 20 remove_watcher() i=7 2024-04-27T03:30:53.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9bac038040 msgr2=0x7f9bac01bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9bac038040 0x7f9bac01bab0 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bcc6e7a300 msgr2=0x55bcc6e7c7a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bcc6e7a300 0x55bcc6e7c7a0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9bb40c0060 msgr2=0x7f9bb40974d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9bb40c0060 0x7f9bb40974d0 crc :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bcc6e7eca0 msgr2=0x55bcc6e81090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bcc6e7eca0 0x55bcc6e81090 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9a8c036900 msgr2=0x7f9a8c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9a8c036900 0x7f9a8c038dc0 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f9bac001b60 tx=0x7f9bac010000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc3b48640 1 -- 172.21.15.165:0/2928187073 reap_dead start 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 msgr2=0x55bcc6e6ec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 0x55bcc6e6ec60 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f9bb408bbc0 tx=0x7f9bb406cee0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.379+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 shutdown_connections 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/2928187073 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d40f60 0x55bcc6e6ec60 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 >> 172.21.15.165:0/2928187073 conn(0x55bcc6e61930 msgr2=0x55bcc6e5ef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 shutdown_connections 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 -- 172.21.15.165:0/2928187073 wait complete. 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1025240535 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9a9c0369a0 msgr2=0x7f9a9c038e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/1025240535 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9a9c0369a0 0x7f9a9c038e60 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f9bac00bf50 tx=0x7f9bac00f000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1025240535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6e584e0 msgr2=0x55bcc6e58900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/1025240535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6e584e0 0x55bcc6e58900 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4052540 tx=0x7f9bb40716a0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1025240535 shutdown_connections 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/1025240535 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9a9c0369a0 0x7f9a9c038e60 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/1025240535 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6e584e0 0x55bcc6e58900 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1025240535 >> 172.21.15.165:0/1025240535 conn(0x55bcc6e4c580 msgr2=0x55bcc6e4c980 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1025240535 shutdown_connections 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 -- 172.21.15.165:0/1025240535 wait complete. 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55bcc6e474d0 msgr2=0x55bcc6e49920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55bcc6e474d0 0x55bcc6e49920 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.380+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bcc6d442a0 msgr2=0x55bcc6e40d00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.381+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bcc6d442a0 0x55bcc6e40d00 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.381+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bcc6e43590 msgr2=0x55bcc6e45980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.381+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bcc6e43590 0x55bcc6e45980 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.381+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9aa8036950 msgr2=0x7f9aa8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.381+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9aa8036950 0x7f9aa8038e10 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f9bac000f60 tx=0x7f9bac0012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.381+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 msgr2=0x55bcc6e39b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.381+0000 7f9bc7c11800 1 --2- 172.21.15.165:0/3688263457 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bcc6d41970 0x55bcc6e39b60 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7f9bb4060040 tx=0x7f9bb4002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.381+0000 7f9bc3b48640 1 -- 172.21.15.165:0/3688263457 reap_dead start 2024-04-27T03:30:53.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.381+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 shutdown_connections 2024-04-27T03:30:53.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.381+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 >> 172.21.15.165:0/3688263457 conn(0x55bcc6d3b320 msgr2=0x55bcc6e2c230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.381+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 shutdown_connections 2024-04-27T03:30:53.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.381+0000 7f9bc7c11800 1 -- 172.21.15.165:0/3688263457 wait complete. 2024-04-27T03:30:53.395 INFO:teuthology.orchestra.run.smithi165.stdout:[] 2024-04-27T03:30:53.395 DEBUG:tasks.util.rgw: json result: [] 2024-04-27T03:30:53.396 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-04-27T03:30:53.396 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-04-27T03:30:53.396 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-04-27T03:30:53.479 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:53.479 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:53.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.474+0000 7f9f11614800 1 Processor -- start 2024-04-27T03:30:53.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.474+0000 7f9f11614800 1 -- start start 2024-04-27T03:30:53.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.474+0000 7f9f11614800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x557477f712e0 0x557477f79de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.474+0000 7f9f11614800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x557477e2fd40 con 0x557477f6f7a0 2024-04-27T03:30:53.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.474+0000 7f9f11614800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x557477e024d0 con 0x557477f712e0 2024-04-27T03:30:53.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.475+0000 7f9f0c55b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x557477f712e0 0x557477f79de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.475+0000 7f9f0cd5c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x557477f6f7a0 0x557477f6fbc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:38058/0 (socket says 172.21.15.165:38058) 2024-04-27T03:30:53.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.475+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3089593367 learned_addr learned my addr 172.21.15.165:0/3089593367 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:53.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.475+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3089593367 >> v1:172.21.15.89:6789/0 conn(0x557477f6f7a0 legacy=0x557477f6fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.475+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3089593367 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x557477e2e590 con 0x557477f712e0 2024-04-27T03:30:53.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.477+0000 7f9f0c55b640 1 --2- 172.21.15.165:0/3089593367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f712e0 0x557477f79de0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000a470 tx=0x7f9f0001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19a74c91068ab0e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.477+0000 7f9f0bd5a640 1 -- 172.21.15.165:0/3089593367 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f00057070 con 0x557477f712e0 2024-04-27T03:30:53.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.477+0000 7f9f0bd5a640 1 -- 172.21.15.165:0/3089593367 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9f00052440 con 0x557477f712e0 2024-04-27T03:30:53.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.477+0000 7f9f0bd5a640 1 -- 172.21.15.165:0/3089593367 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9f00052740 con 0x557477f712e0 2024-04-27T03:30:53.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.477+0000 7f9f11614800 1 -- 172.21.15.165:0/3089593367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f712e0 msgr2=0x557477f79de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.477+0000 7f9f11614800 1 --2- 172.21.15.165:0/3089593367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f712e0 0x557477f79de0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f9f0000a470 tx=0x7f9f0001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:53.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.477+0000 7f9f11614800 1 -- 172.21.15.165:0/3089593367 shutdown_connections 2024-04-27T03:30:53.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.477+0000 7f9f11614800 1 --2- 172.21.15.165:0/3089593367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f712e0 0x557477f79de0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.477+0000 7f9f11614800 1 -- 172.21.15.165:0/3089593367 >> 172.21.15.165:0/3089593367 conn(0x557477f6b320 msgr2=0x557477f6d740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.478+0000 7f9f11614800 1 -- 172.21.15.165:0/3089593367 shutdown_connections 2024-04-27T03:30:53.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.478+0000 7f9f11614800 1 -- 172.21.15.165:0/3089593367 wait complete. 2024-04-27T03:30:53.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.489+0000 7f9f11614800 1 Processor -- start 2024-04-27T03:30:53.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.489+0000 7f9f11614800 1 -- start start 2024-04-27T03:30:53.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.489+0000 7f9f11614800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 0x557478064500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.489+0000 7f9f11614800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557477ebc3c0 con 0x557477f6f7a0 2024-04-27T03:30:53.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.490+0000 7f9f0cd5c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 0x557478064500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.490+0000 7f9f0cd5c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 0x557478064500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51250/0 (socket says 172.21.15.165:51250) 2024-04-27T03:30:53.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.490+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/931980825 learned_addr learned my addr 172.21.15.165:0/931980825 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.490+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/931980825 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557478064ad0 con 0x557477f6f7a0 2024-04-27T03:30:53.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.490+0000 7f9f0cd5c640 1 --2- 172.21.15.165:0/931980825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 0x557478064500 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9efc001de0 tx=0x7f9efc00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6695c210c99a3bbd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.490+0000 7f9dfae0d640 1 -- 172.21.15.165:0/931980825 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9efc012020 con 0x557477f6f7a0 2024-04-27T03:30:53.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.490+0000 7f9dfae0d640 1 -- 172.21.15.165:0/931980825 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9efc00e5a0 con 0x557477f6f7a0 2024-04-27T03:30:53.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.490+0000 7f9dfae0d640 1 -- 172.21.15.165:0/931980825 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9efc00e8c0 con 0x557477f6f7a0 2024-04-27T03:30:53.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.491+0000 7f9f11614800 1 -- 172.21.15.165:0/931980825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 msgr2=0x557478064500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.491+0000 7f9f11614800 1 --2- 172.21.15.165:0/931980825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 0x557478064500 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f9efc001de0 tx=0x7f9efc00b6c0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.491+0000 7f9f11614800 1 -- 172.21.15.165:0/931980825 shutdown_connections 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.491+0000 7f9f11614800 1 --2- 172.21.15.165:0/931980825 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 0x557478064500 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.491+0000 7f9f11614800 1 -- 172.21.15.165:0/931980825 >> 172.21.15.165:0/931980825 conn(0x557477f6b320 msgr2=0x55747805fe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.491+0000 7f9f11614800 1 -- 172.21.15.165:0/931980825 shutdown_connections 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.491+0000 7f9f11614800 1 -- 172.21.15.165:0/931980825 wait complete. 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.491+0000 7f9f11614800 1 Processor -- start 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.491+0000 7f9f11614800 1 -- start start 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.492+0000 7f9f11614800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 0x55747806c0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.492+0000 7f9f11614800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557477f7c600 con 0x557477f6f7a0 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.492+0000 7f9f0cd5c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 0x55747806c0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.492+0000 7f9f0cd5c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 0x55747806c0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51266/0 (socket says 172.21.15.165:51266) 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.492+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3145753826 learned_addr learned my addr 172.21.15.165:0/3145753826 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.492+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3145753826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747806c600 con 0x557477f6f7a0 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.492+0000 7f9f0cd5c640 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 0x55747806c0c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9efc001de0 tx=0x7f9efc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.492+0000 7f9df960a640 1 -- 172.21.15.165:0/3145753826 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9efc012030 con 0x557477f6f7a0 2024-04-27T03:30:53.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.492+0000 7f9df960a640 1 -- 172.21.15.165:0/3145753826 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9efc01a7a0 con 0x557477f6f7a0 2024-04-27T03:30:53.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.492+0000 7f9df960a640 1 -- 172.21.15.165:0/3145753826 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9efc01aac0 con 0x557477f6f7a0 2024-04-27T03:30:53.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.493+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55747806c890 con 0x557477f6f7a0 2024-04-27T03:30:53.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.493+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747806f4a0 con 0x557477f6f7a0 2024-04-27T03:30:53.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.493+0000 7f9df960a640 1 -- 172.21.15.165:0/3145753826 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9efc015020 con 0x557477f6f7a0 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.493+0000 7f9df960a640 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9df4036950 0x7f9df4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.493+0000 7f9df960a640 1 -- 172.21.15.165:0/3145753826 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f9efc020070 con 0x557477f6f7a0 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.494+0000 7f9f0c55b640 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9df4036950 0x7f9df4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.494+0000 7f9f11614800 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557477f712e0 0x557478073260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.494+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780737a0 con 0x557477f712e0 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.494+0000 7f9f0d55d640 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557477f712e0 0x557478073260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.494+0000 7f9f0c55b640 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9df4036950 0x7f9df4038e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9f00054000 tx=0x7f9f0004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.494+0000 7f9f0d55d640 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557477f712e0 0x557478073260 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.494+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3145753826 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9f0400a4a0 con 0x557477f712e0 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.494+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557478070830 con 0x557477f712e0 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.495+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3145753826 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9f0400a4a0 con 0x557477f712e0 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.495+0000 7f9f11614800 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557478075a90 0x557478077e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.495+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557478078520 con 0x557478075a90 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.495+0000 7f9f0cd5c640 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557478075a90 0x557478077e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.495+0000 7f9f0cd5c640 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557478075a90 0x557478077e80 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.495+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3145753826 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9efc024980 con 0x557478075a90 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.495+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x557478074a40 con 0x557477f712e0 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.496+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3145753826 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9f0400a4a0 con 0x557477f712e0 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.496+0000 7f9f11614800 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5574780799d0 0x55747807be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.496+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55747807c580 con 0x5574780799d0 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.496+0000 7f9f0c55b640 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5574780799d0 0x55747807be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.496+0000 7f9f0c55b640 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5574780799d0 0x55747807be20 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.496+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3145753826 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9f000525f0 con 0x5574780799d0 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.497+0000 7f9f11614800 1 Processor -- start 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.497+0000 7f9f11614800 1 -- start start 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.497+0000 7f9f11614800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55747808a9e0 0x55747808ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.497+0000 7f9f11614800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557477e626b0 con 0x55747808a9e0 2024-04-27T03:30:53.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.497+0000 7f9f0cd5c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55747808a9e0 0x55747808ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.497+0000 7f9f0cd5c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55747808a9e0 0x55747808ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51278/0 (socket says 172.21.15.165:51278) 2024-04-27T03:30:53.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.497+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/1519036650 learned_addr learned my addr 172.21.15.165:0/1519036650 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.497+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/1519036650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747808b340 con 0x55747808a9e0 2024-04-27T03:30:53.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.498+0000 7f9f0cd5c640 1 --2- 172.21.15.165:0/1519036650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55747808a9e0 0x55747808ae00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9efc04afd0 tx=0x7f9efc044500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.498+0000 7f9df27fc640 1 -- 172.21.15.165:0/1519036650 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9efc03c070 con 0x55747808a9e0 2024-04-27T03:30:53.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.498+0000 7f9df27fc640 1 -- 172.21.15.165:0/1519036650 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9efc047260 con 0x55747808a9e0 2024-04-27T03:30:53.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.498+0000 7f9f11614800 1 -- 172.21.15.165:0/1519036650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55747808bdb0 con 0x55747808a9e0 2024-04-27T03:30:53.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.498+0000 7f9df27fc640 1 -- 172.21.15.165:0/1519036650 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9efc04c520 con 0x55747808a9e0 2024-04-27T03:30:53.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.498+0000 7f9f11614800 1 -- 172.21.15.165:0/1519036650 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747808c2b0 con 0x55747808a9e0 2024-04-27T03:30:53.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.499+0000 7f9df27fc640 1 -- 172.21.15.165:0/1519036650 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9efc03f070 con 0x55747808a9e0 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.499+0000 7f9df27fc640 1 --2- 172.21.15.165:0/1519036650 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9de4036950 0x7f9de4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.499+0000 7f9df27fc640 1 -- 172.21.15.165:0/1519036650 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f9efc019030 con 0x55747808a9e0 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.499+0000 7f9f0c55b640 1 --2- 172.21.15.165:0/1519036650 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9de4036950 0x7f9de4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.499+0000 7f9f11614800 1 Processor -- start 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.499+0000 7f9f11614800 1 -- start start 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.499+0000 7f9f0c55b640 1 --2- 172.21.15.165:0/1519036650 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9de4036950 0x7f9de4038e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x557478070750 tx=0x7f9f00005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.499+0000 7f9f11614800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 0x557478097f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.499+0000 7f9f11614800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557478064e50 con 0x557478097af0 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.499+0000 7f9f0cd5c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 0x557478097f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.499+0000 7f9f0cd5c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 0x557478097f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51284/0 (socket says 172.21.15.165:51284) 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.499+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3899147386 learned_addr learned my addr 172.21.15.165:0/3899147386 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.500+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3899147386 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557478098500 con 0x557478097af0 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.500+0000 7f9f0cd5c640 1 --2- 172.21.15.165:0/3899147386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 0x557478097f10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9efc047970 tx=0x7f9efc0479a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21b6e13dd4203a28 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.500+0000 7f9de3fff640 1 -- 172.21.15.165:0/3899147386 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9efc044600 con 0x557478097af0 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.500+0000 7f9de3fff640 1 -- 172.21.15.165:0/3899147386 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9efc04cb30 con 0x557478097af0 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.500+0000 7f9de3fff640 1 -- 172.21.15.165:0/3899147386 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9efc0447a0 con 0x557478097af0 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.500+0000 7f9f11614800 1 -- 172.21.15.165:0/3899147386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 msgr2=0x557478097f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.500+0000 7f9f11614800 1 --2- 172.21.15.165:0/3899147386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 0x557478097f10 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f9efc047970 tx=0x7f9efc0479a0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.500+0000 7f9f11614800 1 -- 172.21.15.165:0/3899147386 shutdown_connections 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.500+0000 7f9f11614800 1 --2- 172.21.15.165:0/3899147386 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 0x557478097f10 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.500+0000 7f9f11614800 1 -- 172.21.15.165:0/3899147386 >> 172.21.15.165:0/3899147386 conn(0x557478093e30 msgr2=0x557478091e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.500+0000 7f9f11614800 1 -- 172.21.15.165:0/3899147386 shutdown_connections 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.500+0000 7f9f11614800 1 -- 172.21.15.165:0/3899147386 wait complete. 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.501+0000 7f9f11614800 1 Processor -- start 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.501+0000 7f9f11614800 1 -- start start 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.501+0000 7f9f11614800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 0x55747809eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.501+0000 7f9f11614800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x557478098c20 con 0x557478097af0 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.501+0000 7f9f0cd5c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 0x55747809eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.501+0000 7f9f0cd5c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 0x55747809eee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51300/0 (socket says 172.21.15.165:51300) 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.501+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 learned_addr learned my addr 172.21.15.165:0/3377547357 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.501+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55747809f420 con 0x557478097af0 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.502+0000 7f9f0cd5c640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 0x55747809eee0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9efc000fb0 tx=0x7f9efc01ff10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.502+0000 7f9de27fc640 1 -- 172.21.15.165:0/3377547357 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9efc06e020 con 0x557478097af0 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.502+0000 7f9de27fc640 1 -- 172.21.15.165:0/3377547357 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9efc047c80 con 0x557478097af0 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.502+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55747809f420 con 0x557478097af0 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.502+0000 7f9de27fc640 1 -- 172.21.15.165:0/3377547357 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9efc066040 con 0x557478097af0 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.502+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55747809f420 con 0x557478097af0 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.502+0000 7f9de27fc640 1 -- 172.21.15.165:0/3377547357 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9efc071020 con 0x557478097af0 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.503+0000 7f9de27fc640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9dd8036950 0x7f9dd8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.503+0000 7f9de27fc640 1 -- 172.21.15.165:0/3377547357 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f9efc074ab0 con 0x557478097af0 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.503+0000 7f9f0c55b640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9dd8036950 0x7f9dd8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.503+0000 7f9f11614800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.503+0000 7f9f11614800 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574780aa6b0 0x5574780acb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.503+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780ad2c0 con 0x5574780aa6b0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.503+0000 7f9f0c55b640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9dd8036950 0x7f9dd8038e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9f00000c00 tx=0x7f9f0004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.503+0000 7f9f0d55d640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574780aa6b0 0x5574780acb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f0d55d640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574780aa6b0 0x5574780acb70 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9f0400a4a0 con 0x5574780aa6b0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f11614800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f11614800 20 realm 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f11614800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780ad6d0 con 0x5574780aa6b0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9f0400a4a0 con 0x5574780aa6b0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f11614800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f11614800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f11614800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780adfa0 con 0x5574780aa6b0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9f0400a4a0 con 0x5574780aa6b0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f11614800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f11614800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780ae340 con 0x5574780aa6b0 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9f0400a4a0 con 0x5574780aa6b0 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.504+0000 7f9f11614800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.505+0000 7f9f11614800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.505+0000 7f9f11614800 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574780af0a0 0x5574780b1490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.505+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780b1aa0 con 0x5574780af0a0 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.505+0000 7f9f0cd5c640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574780af0a0 0x5574780b1490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.505+0000 7f9f0cd5c640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574780af0a0 0x5574780b1490 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.505+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.505+0000 7f9f11614800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.505+0000 7f9f11614800 20 searching for the correct realm 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.505+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780b33b0 con 0x5574780af0a0 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.506+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.506+0000 7f9f0cd5c640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9efc073020 0x7f9efc017430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.506+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780b33b0 con 0x7f9efc073020 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.506+0000 7f9f0c55b640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9efc073020 0x7f9efc017430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.506+0000 7f9f0c55b640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9efc073020 0x7f9efc017430 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.506+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.506+0000 7f9f0c55b640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9f00076040 0x7f9f00083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.506+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780b33b0 con 0x7f9f00076040 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.507+0000 7f9f0d55d640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9f00076040 0x7f9f00083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.507+0000 7f9f0d55d640 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9f00076040 0x7f9f00083410 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.507+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f9f0400a4a0 con 0x7f9f00076040 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.507+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780ae340 con 0x5574780aa6b0 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.507+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f0400a4a0 con 0x5574780aa6b0 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.507+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780ae340 con 0x5574780aa6b0 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.508+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f0400a4a0 con 0x5574780aa6b0 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.508+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780ae340 con 0x7f9f00076040 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.508+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f0400a4a0 con 0x7f9f00076040 2024-04-27T03:30:53.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.508+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780ae340 con 0x7f9f00076040 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.508+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f0400a4a0 con 0x7f9f00076040 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.508+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780ae340 con 0x7f9efc073020 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.508+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.508+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780ae340 con 0x5574780af0a0 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.509+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.509+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780ae340 con 0x5574780aa6b0 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.509+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f0400a4a0 con 0x5574780aa6b0 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.509+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780ae340 con 0x7f9f00076040 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.509+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f0400a4a0 con 0x7f9f00076040 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.509+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780ae340 con 0x5574780aa6b0 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.510+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f0400a4a0 con 0x5574780aa6b0 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.510+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5574780ae340 con 0x7f9efc073020 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.510+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.510+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x5574780af0a0 2024-04-27T03:30:53.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.510+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.510+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x5574780af0a0 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.510+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.510+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x7f9f00076040 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.511+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f0400a4a0 con 0x7f9f00076040 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.511+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x5574780af0a0 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.511+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.511+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x5574780af0a0 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.511+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.511+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x5574780aa6b0 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.511+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f0400a4a0 con 0x5574780aa6b0 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.511+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x5574780aa6b0 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.511+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9f0400a4a0 con 0x5574780aa6b0 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.511+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x7f9f00076040 2024-04-27T03:30:53.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.512+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f04033040 con 0x7f9f00076040 2024-04-27T03:30:53.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.512+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x5574780af0a0 2024-04-27T03:30:53.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.512+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.512+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x7f9efc073020 2024-04-27T03:30:53.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.512+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.512+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x5574780aa6b0 2024-04-27T03:30:53.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.512+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.512+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x7f9f00076040 2024-04-27T03:30:53.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.513+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.513+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x5574780af0a0 2024-04-27T03:30:53.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.513+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.513+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x7f9f00076040 2024-04-27T03:30:53.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.513+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.513+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x7f9efc073020 2024-04-27T03:30:53.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.514+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.514+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x5574780af0a0 2024-04-27T03:30:53.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.514+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.514+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x7f9f00076040 2024-04-27T03:30:53.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.514+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.514+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x5574780aa6b0 2024-04-27T03:30:53.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f9f00090060 con 0x7f9efc073020 2024-04-27T03:30:53.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780b2360 con 0x5574780aa6b0 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780b2750 con 0x5574780aa6b0 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:53.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.515+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780b2750 con 0x7f9f00076040 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 20 zone default found 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 4 Realm: () 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 20 zonegroup default 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780b3830 con 0x7f9f00076040 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780b5210 con 0x5574780af0a0 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.516+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780b55b0 con 0x5574780af0a0 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.517+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.517+0000 7f9f11614800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.517+0000 7f9f11614800 20 started sync module instance, tier type = 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.517+0000 7f9f11614800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.517+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b46b0 con 0x7f9efc073020 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.518+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.518+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93958718571008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b5950 con 0x7f9efc073020 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.518+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b6200 con 0x5574780aa6b0 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.519+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93958718571008] v25'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.519+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.519+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93958718577904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b65f0 con 0x5574780aa6b0 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.519+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b7050 con 0x5574780af0a0 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.520+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93958718577904] v25'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.520+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.520+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93958718573216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b7440 con 0x5574780af0a0 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.520+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b7e80 con 0x7f9efc073020 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.521+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93958718573216] v25'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.521+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.522+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93958718581552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b8750 con 0x7f9efc073020 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.522+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b9110 con 0x5574780aa6b0 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.523+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.523+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93958718590528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b98b0 con 0x5574780aa6b0 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.523+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780ba3d0 con 0x7f9f00076040 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.523+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93958718581552] v25'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.538 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.524+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.524+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93958718594720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780ba390 con 0x7f9f00076040 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.524+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93958718590528] v25'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.524+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780bae00 con 0x7f9f00076040 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.525+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93958718594720] v25'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.525+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.525+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93958718591936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b82a0 con 0x7f9f00076040 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.525+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780bb2b0 con 0x7f9f00076040 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.526+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 93958718591936] v25'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.526+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'168 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.526+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93958718597424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780bb2b0 con 0x7f9f00076040 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.526+0000 7f9f11614800 20 add_watcher() i=0 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.526+0000 7f9f11614800 20 add_watcher() i=1 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.526+0000 7f9f11614800 20 add_watcher() i=2 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.526+0000 7f9f11614800 20 add_watcher() i=3 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.526+0000 7f9f11614800 20 add_watcher() i=4 2024-04-27T03:30:53.539 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.526+0000 7f9f11614800 20 add_watcher() i=5 2024-04-27T03:30:53.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.526+0000 7f9f11614800 20 add_watcher() i=6 2024-04-27T03:30:53.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.527+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93958718597424] v25'169 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.527+0000 7f9f11614800 20 add_watcher() i=7 2024-04-27T03:30:53.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.527+0000 7f9f11614800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:53.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.527+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780be710 con 0x5574780aa6b0 2024-04-27T03:30:53.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.528+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.540 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.528+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93958718611616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780bb4b0 con 0x5574780aa6b0 2024-04-27T03:30:53.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.529+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93958718611616] v25'179 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.529+0000 7f9dc2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:53.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.529+0000 7f9f11614800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:53.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.529+0000 7f9f11614800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:53.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.529+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780bd880 con 0x5574780aa6b0 2024-04-27T03:30:53.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.530+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'53 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.530+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c77d0 con 0x7f9f00076040 2024-04-27T03:30:53.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.531+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.531+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c7c80 con 0x5574780af0a0 2024-04-27T03:30:53.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.532+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'30 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.532+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c8020 con 0x5574780aa6b0 2024-04-27T03:30:53.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.533+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'180 uv172 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.533+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c8590 con 0x7f9efc073020 2024-04-27T03:30:53.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.534+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'65 uv59 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.534+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c8a90 con 0x5574780af0a0 2024-04-27T03:30:53.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.535+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'52 uv49 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.535+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c8a90 con 0x5574780aa6b0 2024-04-27T03:30:53.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.536+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'31 uv29 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.536+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c8a90 con 0x7f9f00076040 2024-04-27T03:30:53.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.538+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'45 uv42 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.538+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c8a90 con 0x7f9efc073020 2024-04-27T03:30:53.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.539+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'23 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.539+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c90a0 con 0x5574780af0a0 2024-04-27T03:30:53.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.540+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.540+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c9550 con 0x7f9efc073020 2024-04-27T03:30:53.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.541+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'66 uv61 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.541+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c9550 con 0x7f9f00076040 2024-04-27T03:30:53.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.542+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'26 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.542+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c9550 con 0x7f9efc073020 2024-04-27T03:30:53.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.543+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.543+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c9550 con 0x7f9f00076040 2024-04-27T03:30:53.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.544+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'57 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.544+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c9550 con 0x5574780aa6b0 2024-04-27T03:30:53.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.545+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'181 uv168 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.545+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c9550 con 0x5574780af0a0 2024-04-27T03:30:53.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.546+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'60 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.546+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c9550 con 0x7f9efc073020 2024-04-27T03:30:53.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.547+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'67 uv57 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.547+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c9550 con 0x5574780af0a0 2024-04-27T03:30:53.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.548+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'35 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.548+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c9550 con 0x5574780aa6b0 2024-04-27T03:30:53.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.549+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'31 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.549+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x5574780aa6b0 2024-04-27T03:30:53.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.550+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'182 uv170 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.550+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x5574780af0a0 2024-04-27T03:30:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.551+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'61 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.551+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x7f9efc073020 2024-04-27T03:30:53.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.552+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'39 uv37 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.552+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x5574780af0a0 2024-04-27T03:30:53.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.553+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'31 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.553+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x5574780af0a0 2024-04-27T03:30:53.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.554+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'42 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.554+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x5574780af0a0 2024-04-27T03:30:53.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.555+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'25 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.555+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x5574780af0a0 2024-04-27T03:30:53.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.556+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'53 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.556+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x7f9f00076040 2024-04-27T03:30:53.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.557+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'33 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.557+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x5574780af0a0 2024-04-27T03:30:53.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.558+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'35 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.558+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x7f9f00076040 2024-04-27T03:30:53.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.559+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'46 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.559+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x5574780af0a0 2024-04-27T03:30:53.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.560+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'62 uv56 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.560+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x7f9f00076040 2024-04-27T03:30:53.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.561+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'58 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.561+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780c95e0 con 0x7f9efc073020 2024-04-27T03:30:53.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.562+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'29 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.562+0000 7f9dc1fe3640 20 reqs_thread_entry: start 2024-04-27T03:30:53.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.563+0000 7f9f11614800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:53.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.563+0000 7f9f11614800 20 Filter name: none 2024-04-27T03:30:53.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.563+0000 7f9d337fe640 20 reqs_thread_entry: start 2024-04-27T03:30:53.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.563+0000 7f9f11614800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:53.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.563+0000 7f9f11614800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.563+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780f4580 con 0x5574780aa6b0 2024-04-27T03:30:53.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.563+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(94 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv13 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.563+0000 7f9f11614800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:53.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.563+0000 7f9f11614800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:53.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.563+0000 7f9f11614800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:53.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.563+0000 7f9f11614800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:53.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.563+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5574780f4540 con 0x7f9efc073020 2024-04-27T03:30:53.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.564+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv26 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.570 INFO:teuthology.orchestra.run.smithi165.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-04-27T03:30:53.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.564+0000 7f9dc2fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:53.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.564+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93958718611616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780a7000 con 0x5574780aa6b0 2024-04-27T03:30:53.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.565+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 93958718611616] v25'183 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.566+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93958718571008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780be0a0 con 0x7f9efc073020 2024-04-27T03:30:53.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.566+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 93958718571008] v25'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.566+0000 7f9f11614800 20 remove_watcher() i=0 2024-04-27T03:30:53.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.566+0000 7f9f11614800 2 removed watcher, disabling cache 2024-04-27T03:30:53.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.566+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93958718577904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b4200 con 0x5574780aa6b0 2024-04-27T03:30:53.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.568+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 93958718577904] v25'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.568+0000 7f9f11614800 20 remove_watcher() i=1 2024-04-27T03:30:53.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.568+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93958718573216] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b4200 con 0x5574780af0a0 2024-04-27T03:30:53.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.569+0000 7f9f0cd5c640 1 -- 172.21.15.165:0/3377547357 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 93958718573216] v25'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9efc03e030 con 0x5574780af0a0 2024-04-27T03:30:53.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.569+0000 7f9f11614800 20 remove_watcher() i=2 2024-04-27T03:30:53.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.569+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93958718581552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b4aa0 con 0x7f9efc073020 2024-04-27T03:30:53.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.570+0000 7f9f0c55b640 1 -- 172.21.15.165:0/3377547357 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 93958718581552] v25'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f00081070 con 0x7f9efc073020 2024-04-27T03:30:53.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.570+0000 7f9f11614800 20 remove_watcher() i=3 2024-04-27T03:30:53.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.570+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93958718590528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b4200 con 0x5574780aa6b0 2024-04-27T03:30:53.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.571+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 93958718590528] v25'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x5574780aa6b0 2024-04-27T03:30:53.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.571+0000 7f9f11614800 20 remove_watcher() i=4 2024-04-27T03:30:53.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.571+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93958718594720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b4aa0 con 0x7f9f00076040 2024-04-27T03:30:53.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.572+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 93958718594720] v25'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.572+0000 7f9f11614800 20 remove_watcher() i=5 2024-04-27T03:30:53.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.572+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93958718591936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b8e40 con 0x7f9f00076040 2024-04-27T03:30:53.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.573+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 93958718591936] v25'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.573+0000 7f9f11614800 20 remove_watcher() i=6 2024-04-27T03:30:53.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.573+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93958718597424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5574780b4200 con 0x7f9f00076040 2024-04-27T03:30:53.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 93958718597424] v25'170 uv168 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9f04031040 con 0x7f9f00076040 2024-04-27T03:30:53.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 20 remove_watcher() i=7 2024-04-27T03:30:53.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9f00076040 msgr2=0x7f9f00083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9f00076040 0x7f9f00083410 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574780aa6b0 msgr2=0x5574780acb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5574780aa6b0 0x5574780acb70 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9efc073020 msgr2=0x7f9efc017430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9efc073020 0x7f9efc017430 crc :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574780af0a0 msgr2=0x5574780b1490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5574780af0a0 0x5574780b1490 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9dd8036950 msgr2=0x7f9dd8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9dd8036950 0x7f9dd8038e10 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f9f00000c00 tx=0x7f9f0004f000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 msgr2=0x55747809eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3377547357 reap_dead start 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.574+0000 7f9f11614800 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 0x55747809eee0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f9efc000fb0 tx=0x7f9efc01ff10 comp rx=0 tx=0).stop 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 shutdown_connections 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 --2- 172.21.15.165:0/3377547357 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557478097af0 0x55747809eee0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 >> 172.21.15.165:0/3377547357 conn(0x557478093e30 msgr2=0x557478091490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 shutdown_connections 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/3377547357 wait complete. 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/1519036650 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9de4036950 msgr2=0x7f9de4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 --2- 172.21.15.165:0/1519036650 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9de4036950 0x7f9de4038e10 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x557478070750 tx=0x7f9f00005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:53.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/1519036650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55747808a9e0 msgr2=0x55747808ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 --2- 172.21.15.165:0/1519036650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55747808a9e0 0x55747808ae00 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f9efc04afd0 tx=0x7f9efc044500 comp rx=0 tx=0).stop 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/1519036650 shutdown_connections 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 --2- 172.21.15.165:0/1519036650 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9de4036950 0x7f9de4038e10 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 --2- 172.21.15.165:0/1519036650 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55747808a9e0 0x55747808ae00 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/1519036650 >> 172.21.15.165:0/1519036650 conn(0x55747807ea80 msgr2=0x55747807ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/1519036650 shutdown_connections 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/1519036650 wait complete. 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5574780799d0 msgr2=0x55747807be20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5574780799d0 0x55747807be20 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557477f712e0 msgr2=0x557478073260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x557477f712e0 0x557478073260 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557478075a90 msgr2=0x557478077e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x557478075a90 0x557478077e80 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9df4036950 msgr2=0x7f9df4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9df4036950 0x7f9df4038e10 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f9f00054000 tx=0x7f9f0004e000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 msgr2=0x55747806c0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 --2- 172.21.15.165:0/3145753826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x557477f6f7a0 0x55747806c0c0 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f9efc001de0 tx=0x7f9efc000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f0d55d640 1 -- 172.21.15.165:0/3145753826 reap_dead start 2024-04-27T03:30:53.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 shutdown_connections 2024-04-27T03:30:53.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.575+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 >> 172.21.15.165:0/3145753826 conn(0x557477f6b320 msgr2=0x55747805e620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.576+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 shutdown_connections 2024-04-27T03:30:53.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.576+0000 7f9f11614800 1 -- 172.21.15.165:0/3145753826 wait complete. 2024-04-27T03:30:53.589 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-27T03:30:53.597 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=236 failed=True 2024-04-27T03:30:53.601 INFO:tasks.radosgw_admin:RL: delete_obj myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:53.628 INFO:tasks.radosgw_admin:RL: delete_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:53.638 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:53.682 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-04-27T03:30:53.687 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:53.687 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-27T03:30:53.687 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-27T03:30:53.687 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-04-27T03:30:53.730 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:53.730 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:53.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.725+0000 7f94acb2b800 1 Processor -- start 2024-04-27T03:30:53.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.725+0000 7f94acb2b800 1 -- start start 2024-04-27T03:30:53.731 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.725+0000 7f94acb2b800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5568da06bd40 0x5568da06c160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.725+0000 7f94acb2b800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5568d9f2bd40 con 0x5568da06cfa0 2024-04-27T03:30:53.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.725+0000 7f94acb2b800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5568d9efe4d0 con 0x5568da06bd40 2024-04-27T03:30:53.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.726+0000 7f94a87ab640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5568da06bd40 0x5568da06c160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.726+0000 7f949ffaa640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5568da06cfa0 0x5568da06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:38064/0 (socket says 172.21.15.165:38064) 2024-04-27T03:30:53.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.726+0000 7f949ffaa640 1 -- 172.21.15.165:0/1917497373 learned_addr learned my addr 172.21.15.165:0/1917497373 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:53.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.726+0000 7f94a87ab640 1 -- 172.21.15.165:0/1917497373 >> v1:172.21.15.89:6789/0 conn(0x5568da06cfa0 legacy=0x5568da06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.726+0000 7f94a87ab640 1 -- 172.21.15.165:0/1917497373 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568d9f2a590 con 0x5568da06bd40 2024-04-27T03:30:53.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.728+0000 7f94a87ab640 1 --2- 172.21.15.165:0/1917497373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da06c160 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f949800a470 tx=0x7f949801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e479496f1ba6c394 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.728+0000 7f94a7faa640 1 -- 172.21.15.165:0/1917497373 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9498057070 con 0x5568da06bd40 2024-04-27T03:30:53.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.728+0000 7f94a7faa640 1 -- 172.21.15.165:0/1917497373 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9498052570 con 0x5568da06bd40 2024-04-27T03:30:53.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.728+0000 7f94a7faa640 1 -- 172.21.15.165:0/1917497373 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9498052870 con 0x5568da06bd40 2024-04-27T03:30:53.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.728+0000 7f94acb2b800 1 -- 172.21.15.165:0/1917497373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 msgr2=0x5568da06c160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.728+0000 7f94acb2b800 1 --2- 172.21.15.165:0/1917497373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da06c160 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f949800a470 tx=0x7f949801ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:53.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.728+0000 7f94acb2b800 1 -- 172.21.15.165:0/1917497373 shutdown_connections 2024-04-27T03:30:53.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.728+0000 7f94acb2b800 1 --2- 172.21.15.165:0/1917497373 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da06c160 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.729+0000 7f94acb2b800 1 -- 172.21.15.165:0/1917497373 >> 172.21.15.165:0/1917497373 conn(0x5568da0673a0 msgr2=0x5568da0697c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.729+0000 7f94acb2b800 1 -- 172.21.15.165:0/1917497373 shutdown_connections 2024-04-27T03:30:53.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.729+0000 7f94acb2b800 1 -- 172.21.15.165:0/1917497373 wait complete. 2024-04-27T03:30:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.740+0000 7f94acb2b800 1 Processor -- start 2024-04-27T03:30:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.740+0000 7f94acb2b800 1 -- start start 2024-04-27T03:30:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.740+0000 7f94acb2b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da15c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.740+0000 7f94acb2b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5568d9fb83b0 con 0x5568da06bd40 2024-04-27T03:30:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.741+0000 7f94a87ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da15c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.741+0000 7f94a87ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da15c3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51312/0 (socket says 172.21.15.165:51312) 2024-04-27T03:30:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.741+0000 7f94a87ab640 1 -- 172.21.15.165:0/1622037999 learned_addr learned my addr 172.21.15.165:0/1622037999 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.741+0000 7f94a87ab640 1 -- 172.21.15.165:0/1622037999 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568da15c9a0 con 0x5568da06bd40 2024-04-27T03:30:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.741+0000 7f94a87ab640 1 --2- 172.21.15.165:0/1622037999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da15c3d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9498055040 tx=0x7f949804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=273cca44731430d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.741+0000 7f939660c640 1 -- 172.21.15.165:0/1622037999 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9498057070 con 0x5568da06bd40 2024-04-27T03:30:53.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.741+0000 7f939660c640 1 -- 172.21.15.165:0/1622037999 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9498063070 con 0x5568da06bd40 2024-04-27T03:30:53.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.741+0000 7f939660c640 1 -- 172.21.15.165:0/1622037999 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9498052720 con 0x5568da06bd40 2024-04-27T03:30:53.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.742+0000 7f94acb2b800 1 -- 172.21.15.165:0/1622037999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 msgr2=0x5568da15c3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.742+0000 7f94acb2b800 1 --2- 172.21.15.165:0/1622037999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da15c3d0 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9498055040 tx=0x7f949804c000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.742+0000 7f94acb2b800 1 -- 172.21.15.165:0/1622037999 shutdown_connections 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.742+0000 7f94acb2b800 1 --2- 172.21.15.165:0/1622037999 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da15c3d0 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.742+0000 7f94acb2b800 1 -- 172.21.15.165:0/1622037999 >> 172.21.15.165:0/1622037999 conn(0x5568da0673a0 msgr2=0x5568da157cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.742+0000 7f94acb2b800 1 -- 172.21.15.165:0/1622037999 shutdown_connections 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.742+0000 7f94acb2b800 1 -- 172.21.15.165:0/1622037999 wait complete. 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.742+0000 7f94acb2b800 1 Processor -- start 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.742+0000 7f94acb2b800 1 -- start start 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.742+0000 7f94acb2b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da168320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.742+0000 7f94acb2b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5568d9fdf710 con 0x5568da06bd40 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.743+0000 7f94a87ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da168320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.743+0000 7f94a87ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da168320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51322/0 (socket says 172.21.15.165:51322) 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.743+0000 7f94a87ab640 1 -- 172.21.15.165:0/3248932632 learned_addr learned my addr 172.21.15.165:0/3248932632 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.743+0000 7f94a87ab640 1 -- 172.21.15.165:0/3248932632 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568da168860 con 0x5568da06bd40 2024-04-27T03:30:53.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.743+0000 7f94a87ab640 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da168320 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9498060040 tx=0x7f9498002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.743+0000 7f9394e09640 1 -- 172.21.15.165:0/3248932632 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9498052c70 con 0x5568da06bd40 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.743+0000 7f9394e09640 1 -- 172.21.15.165:0/3248932632 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949806c070 con 0x5568da06bd40 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.743+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5568da168af0 con 0x5568da06bd40 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.743+0000 7f9394e09640 1 -- 172.21.15.165:0/3248932632 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f949805e480 con 0x5568da06bd40 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.743+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568da16b750 con 0x5568da06bd40 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.744+0000 7f9394e09640 1 -- 172.21.15.165:0/3248932632 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f949805e620 con 0x5568da06bd40 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.744+0000 7f9394e09640 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9390036900 0x7f9390038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.744+0000 7f9394e09640 1 -- 172.21.15.165:0/3248932632 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f949808e5c0 con 0x5568da06bd40 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.744+0000 7f949ffaa640 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9390036900 0x7f9390038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.744+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568da06cfa0 0x5568da16f510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.744+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da16fae0 con 0x5568da06cfa0 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.745+0000 7f94a8fac640 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568da06cfa0 0x5568da16f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.745+0000 7f949ffaa640 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9390036900 0x7f9390038dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9490000f60 tx=0x7f94900012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.745+0000 7f94a8fac640 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568da06cfa0 0x5568da16f510 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.745+0000 7f94a8fac640 1 -- 172.21.15.165:0/3248932632 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da06cfa0 2024-04-27T03:30:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.745+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da16cae0 con 0x5568da06cfa0 2024-04-27T03:30:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.745+0000 7f94a8fac640 1 -- 172.21.15.165:0/3248932632 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da06cfa0 2024-04-27T03:30:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.745+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5568da171e10 0x5568da174200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.745+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1748a0 con 0x5568da171e10 2024-04-27T03:30:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.746+0000 7f94a87ab640 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5568da171e10 0x5568da174200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.746+0000 7f94a87ab640 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5568da171e10 0x5568da174200 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.746+0000 7f94a87ab640 1 -- 172.21.15.165:0/3248932632 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9498009d10 con 0x5568da171e10 2024-04-27T03:30:53.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.746+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da170dc0 con 0x5568da06cfa0 2024-04-27T03:30:53.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.746+0000 7f94a8fac640 1 -- 172.21.15.165:0/3248932632 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da06cfa0 2024-04-27T03:30:53.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.747+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5568da175d50 0x5568da1781a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.747+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da178900 con 0x5568da175d50 2024-04-27T03:30:53.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.747+0000 7f949ffaa640 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5568da175d50 0x5568da1781a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.747+0000 7f949ffaa640 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5568da175d50 0x5568da1781a0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.747+0000 7f949ffaa640 1 -- 172.21.15.165:0/3248932632 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9490009050 con 0x5568da175d50 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.748+0000 7f94acb2b800 1 Processor -- start 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.748+0000 7f94acb2b800 1 -- start start 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.748+0000 7f94acb2b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da186cf0 0x5568da187110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.748+0000 7f94acb2b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5568d9f5e6b0 con 0x5568da186cf0 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.748+0000 7f94a87ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da186cf0 0x5568da187110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.748+0000 7f94a87ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da186cf0 0x5568da187110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51334/0 (socket says 172.21.15.165:51334) 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.748+0000 7f94a87ab640 1 -- 172.21.15.165:0/3416607454 learned_addr learned my addr 172.21.15.165:0/3416607454 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.749+0000 7f94a87ab640 1 -- 172.21.15.165:0/3416607454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568da187650 con 0x5568da186cf0 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.749+0000 7f94a87ab640 1 --2- 172.21.15.165:0/3416607454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da186cf0 0x5568da187110 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9498001eb0 tx=0x7f94980793a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.749+0000 7f938dffb640 1 -- 172.21.15.165:0/3416607454 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9498081040 con 0x5568da186cf0 2024-04-27T03:30:53.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.749+0000 7f938dffb640 1 -- 172.21.15.165:0/3416607454 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9498058040 con 0x5568da186cf0 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.749+0000 7f938dffb640 1 -- 172.21.15.165:0/3416607454 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f949808e0b0 con 0x5568da186cf0 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.749+0000 7f94acb2b800 1 -- 172.21.15.165:0/3416607454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5568da1880c0 con 0x5568da186cf0 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.749+0000 7f94acb2b800 1 -- 172.21.15.165:0/3416607454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568da1885c0 con 0x5568da186cf0 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.750+0000 7f938dffb640 1 -- 172.21.15.165:0/3416607454 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9498095060 con 0x5568da186cf0 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.750+0000 7f938dffb640 1 --2- 172.21.15.165:0/3416607454 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9380036900 0x7f9380038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.750+0000 7f938dffb640 1 -- 172.21.15.165:0/3416607454 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f949804b100 con 0x5568da186cf0 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.750+0000 7f949ffaa640 1 --2- 172.21.15.165:0/3416607454 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9380036900 0x7f9380038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.750+0000 7f94acb2b800 1 Processor -- start 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.750+0000 7f94acb2b800 1 -- start start 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.750+0000 7f949ffaa640 1 --2- 172.21.15.165:0/3416607454 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9380036900 0x7f9380038dc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f949000bf50 tx=0x7f949000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.750+0000 7f94acb2b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 0x5568da194230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.750+0000 7f94acb2b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5568da15cc60 con 0x5568da193e10 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.751+0000 7f94a87ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 0x5568da194230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.751+0000 7f94a87ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 0x5568da194230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51344/0 (socket says 172.21.15.165:51344) 2024-04-27T03:30:53.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.751+0000 7f94a87ab640 1 -- 172.21.15.165:0/3826737740 learned_addr learned my addr 172.21.15.165:0/3826737740 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.751+0000 7f94a87ab640 1 -- 172.21.15.165:0/3826737740 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568da194800 con 0x5568da193e10 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.751+0000 7f94a87ab640 1 --2- 172.21.15.165:0/3826737740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 0x5568da194230 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9498068a90 tx=0x7f949808af80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91614ada4f6fd354 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.751+0000 7f937f7fe640 1 -- 172.21.15.165:0/3826737740 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f949807adf0 con 0x5568da193e10 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.751+0000 7f937f7fe640 1 -- 172.21.15.165:0/3826737740 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949808a0d0 con 0x5568da193e10 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.751+0000 7f937f7fe640 1 -- 172.21.15.165:0/3826737740 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94980678a0 con 0x5568da193e10 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.751+0000 7f94acb2b800 1 -- 172.21.15.165:0/3826737740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 msgr2=0x5568da194230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.751+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3826737740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 0x5568da194230 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f9498068a90 tx=0x7f949808af80 comp rx=0 tx=0).stop 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.752+0000 7f94acb2b800 1 -- 172.21.15.165:0/3826737740 shutdown_connections 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.752+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3826737740 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 0x5568da194230 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.752+0000 7f94acb2b800 1 -- 172.21.15.165:0/3826737740 >> 172.21.15.165:0/3826737740 conn(0x5568da190170 msgr2=0x5568da18e1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.752+0000 7f94acb2b800 1 -- 172.21.15.165:0/3826737740 shutdown_connections 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.752+0000 7f94acb2b800 1 -- 172.21.15.165:0/3826737740 wait complete. 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.752+0000 7f94acb2b800 1 Processor -- start 2024-04-27T03:30:53.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.752+0000 7f94acb2b800 1 -- start start 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.752+0000 7f94acb2b800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 0x5568da19b220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.752+0000 7f94acb2b800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5568da194eb0 con 0x5568da193e10 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.752+0000 7f94a87ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 0x5568da19b220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.752+0000 7f94a87ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 0x5568da19b220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51348/0 (socket says 172.21.15.165:51348) 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.752+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 learned_addr learned my addr 172.21.15.165:0/3697488659 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.753+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568da19b760 con 0x5568da193e10 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.753+0000 7f94a87ab640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 0x5568da19b220 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9498066d60 tx=0x7f9498095f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.753+0000 7f937dffb640 1 -- 172.21.15.165:0/3697488659 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f94980b2020 con 0x5568da193e10 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.753+0000 7f937dffb640 1 -- 172.21.15.165:0/3697488659 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f949808b550 con 0x5568da193e10 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.753+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5568da19b760 con 0x5568da193e10 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.753+0000 7f937dffb640 1 -- 172.21.15.165:0/3697488659 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9498066590 con 0x5568da193e10 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.753+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568da19b760 con 0x5568da193e10 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.754+0000 7f937dffb640 1 -- 172.21.15.165:0/3697488659 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f94980b5020 con 0x5568da193e10 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.754+0000 7f937dffb640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9370036900 0x7f9370038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.754+0000 7f937dffb640 1 -- 172.21.15.165:0/3697488659 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f94980b8ab0 con 0x5568da193e10 2024-04-27T03:30:53.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.754+0000 7f949ffaa640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9370036900 0x7f9370038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.754+0000 7f949ffaa640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9370036900 0x7f9370038dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9490001b60 tx=0x7f9490010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568da1a69d0 0x5568da1a8e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1a95e0 con 0x5568da1a69d0 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94a8fac640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568da1a69d0 0x5568da1a8e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94a8fac640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568da1a69d0 0x5568da1a8e90 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da1a69d0 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94acb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94acb2b800 20 realm 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1a99f0 con 0x5568da1a69d0 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da1a69d0 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94acb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94acb2b800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:53.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.755+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa2c0 con 0x5568da1a69d0 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.756+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da1a69d0 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.756+0000 7f94acb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.756+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.756+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1a69d0 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.756+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da1a69d0 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.756+0000 7f94acb2b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.756+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.756+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5568da1ab3c0 0x5568da1ad7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.756+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1addc0 con 0x5568da1ab3c0 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.756+0000 7f94a87ab640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5568da1ab3c0 0x5568da1ad7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.756+0000 7f94a87ab640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5568da1ab3c0 0x5568da1ad7b0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.757+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.757+0000 7f94acb2b800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.757+0000 7f94acb2b800 20 searching for the correct realm 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.757+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1af6f0 con 0x5568da1ab3c0 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.757+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.757+0000 7f94a87ab640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f94980b7020 0x7f94980716a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.757+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1af6f0 con 0x7f94980b7020 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.757+0000 7f949ffaa640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f94980b7020 0x7f94980716a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.758+0000 7f949ffaa640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f94980b7020 0x7f94980716a0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.758+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9490009050 con 0x7f94980b7020 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.758+0000 7f949ffaa640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9490038040 0x7f949001bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.758+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1af6f0 con 0x7f9490038040 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.758+0000 7f94a8fac640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9490038040 0x7f949001bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.759+0000 7f94a8fac640 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9490038040 0x7f949001bab0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.759+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f94a000a4a0 con 0x7f9490038040 2024-04-27T03:30:53.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.759+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1a69d0 2024-04-27T03:30:53.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.759+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da1a69d0 2024-04-27T03:30:53.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.759+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1a69d0 2024-04-27T03:30:53.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.759+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da1a69d0 2024-04-27T03:30:53.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.759+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f9490038040 2024-04-27T03:30:53.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.760+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a000a4a0 con 0x7f9490038040 2024-04-27T03:30:53.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.760+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f9490038040 2024-04-27T03:30:53.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.760+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a000a4a0 con 0x7f9490038040 2024-04-27T03:30:53.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.760+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f94980b7020 2024-04-27T03:30:53.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.760+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9490009050 con 0x7f94980b7020 2024-04-27T03:30:53.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.760+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1ab3c0 2024-04-27T03:30:53.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.760+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.760+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1a69d0 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.761+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da1a69d0 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.761+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f9490038040 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.761+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a000a4a0 con 0x7f9490038040 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.761+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1a69d0 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.761+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da1a69d0 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.761+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f94980b7020 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.761+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9490009050 con 0x7f94980b7020 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.761+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1ab3c0 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.761+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.761+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1ab3c0 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.762+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.762+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f9490038040 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.762+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a000a4a0 con 0x7f9490038040 2024-04-27T03:30:53.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.762+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1ab3c0 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.762+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.762+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1ab3c0 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.763+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.763+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1a69d0 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.763+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da1a69d0 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.763+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1a69d0 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.763+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f94a000a4a0 con 0x5568da1a69d0 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.763+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f9490038040 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.764+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a0033040 con 0x7f9490038040 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.764+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1ab3c0 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.764+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.764+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f94980b7020 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.764+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9490009050 con 0x7f94980b7020 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.764+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1a69d0 2024-04-27T03:30:53.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.764+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.764+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f9490038040 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.765+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.765+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1ab3c0 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.765+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.765+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f9490038040 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.765+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.765+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f94980b7020 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.765+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9490009050 con 0x7f94980b7020 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.766+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1ab3c0 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.766+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.766+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f9490038040 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.766+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.766+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x5568da1a69d0 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aa660 con 0x7f94980b7020 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9490045040 con 0x7f94980b7020 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1ae680 con 0x5568da1a69d0 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aea70 con 0x5568da1a69d0 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.767+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1aea70 con 0x7f9490038040 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 20 zone default found 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 4 Realm: () 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 20 zonegroup default 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1afb70 con 0x7f9490038040 2024-04-27T03:30:53.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b1550 con 0x5568da1ab3c0 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.768+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b18f0 con 0x5568da1ab3c0 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.769+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.769+0000 7f94acb2b800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.769+0000 7f94acb2b800 20 started sync module instance, tier type = 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.769+0000 7f94acb2b800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.769+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b09f0 con 0x7f94980b7020 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.770+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9490045040 con 0x7f94980b7020 2024-04-27T03:30:53.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.770+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93908824163648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b1c90 con 0x7f94980b7020 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.770+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b2520 con 0x5568da1a69d0 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.771+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 93908824163648] v25'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9490045040 con 0x7f94980b7020 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.771+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.771+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93908824165856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b2910 con 0x5568da1a69d0 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.771+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b3370 con 0x5568da1ab3c0 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.772+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93908824165856] v25'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.772+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.772+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93908824170544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b3760 con 0x5568da1ab3c0 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.772+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b4190 con 0x7f94980b7020 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.773+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93908824170544] v25'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.773+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9490045040 con 0x7f94980b7020 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.773+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93908824177824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b4530 con 0x7f94980b7020 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.773+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b4fd0 con 0x5568da1a69d0 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.774+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.774+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93908824181440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b58a0 con 0x5568da1a69d0 2024-04-27T03:30:53.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.774+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b6350 con 0x7f9490038040 2024-04-27T03:30:53.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.774+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 93908824177824] v25'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9490045040 con 0x7f94980b7020 2024-04-27T03:30:53.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.775+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93908824181440] v25'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.776+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.776+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93908824174160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b52f0 con 0x7f9490038040 2024-04-27T03:30:53.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.776+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b6bf0 con 0x7f9490038040 2024-04-27T03:30:53.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.776+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 93908824174160] v25'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.777+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.777+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93908824186416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b52f0 con 0x7f9490038040 2024-04-27T03:30:53.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.777+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b7170 con 0x7f9490038040 2024-04-27T03:30:53.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.778+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'171 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.778+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93908824190112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b7620 con 0x7f9490038040 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.778+0000 7f94acb2b800 20 add_watcher() i=0 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.778+0000 7f94acb2b800 20 add_watcher() i=1 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.778+0000 7f94acb2b800 20 add_watcher() i=2 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.778+0000 7f94acb2b800 20 add_watcher() i=3 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.778+0000 7f94acb2b800 20 add_watcher() i=4 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.778+0000 7f94acb2b800 20 add_watcher() i=5 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.780+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 93908824186416] v25'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.780+0000 7f94acb2b800 20 add_watcher() i=6 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.780+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 93908824190112] v25'172 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.780+0000 7f94acb2b800 20 add_watcher() i=7 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.780+0000 7f94acb2b800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:53.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.780+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b65d0 con 0x5568da1a69d0 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.781+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.781+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93908824191344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1b65d0 con 0x5568da1a69d0 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.782+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93908824191344] v25'184 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.782+0000 7f935e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.782+0000 7f94acb2b800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.782+0000 7f94acb2b800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.782+0000 7f94acb2b800 5 note: GC not initialized 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.782+0000 7f935d7e2640 20 reqs_thread_entry: start 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.782+0000 7f94acb2b800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.782+0000 7f94acb2b800 20 Filter name: none 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.782+0000 7f92ceffd640 20 reqs_thread_entry: start 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.782+0000 7f94acb2b800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.782+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1edfb0 con 0x7f94980b7020 2024-04-27T03:30:53.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f9490045040 con 0x7f94980b7020 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 : miss 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.1f 8:f8c3518a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1efde0 con 0x7f94980b7020 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(63 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f9490045040 con 0x7f94980b7020 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 info.flags=0x17 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 to cache LRU end 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.783+0000 7f94acb2b800 20 get_obj_state: octx=0x7f94a000aaf0 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x5568da1f24a8 s->prefetch_data=0 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.784+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 3.35 3:ad937139:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x5568da1f3570 con 0x5568da1a69d0 2024-04-27T03:30:53.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.784+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(64 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=849b,stat out=16b] v0'0 uv2 ondisk = 0) ==== 262+0+865 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.784+0000 7f94acb2b800 10 manifest: total_size = 5 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.784+0000 7f94acb2b800 20 get_obj_state: setting s->obj_tag to _ygz-MLUR6NaLnjHrAbe5ezPRS4N725h 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.785+0000 7f935e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.785+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93908824191344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1eb900 con 0x5568da1a69d0 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.785+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 93908824191344] v25'185 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.786+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93908824163648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1f3570 con 0x7f94980b7020 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.786+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 93908824163648] v25'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9490045040 con 0x7f94980b7020 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.786+0000 7f94acb2b800 20 remove_watcher() i=0 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.786+0000 7f94acb2b800 2 removed watcher, disabling cache 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.787+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93908824165856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1f3570 con 0x5568da1a69d0 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.788+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 93908824165856] v25'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.788+0000 7f94acb2b800 20 remove_watcher() i=1 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.788+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93908824170544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1f3570 con 0x5568da1ab3c0 2024-04-27T03:30:53.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.789+0000 7f94a87ab640 1 -- 172.21.15.165:0/3697488659 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 93908824170544] v25'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94980ab040 con 0x5568da1ab3c0 2024-04-27T03:30:53.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.789+0000 7f94acb2b800 20 remove_watcher() i=2 2024-04-27T03:30:53.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.789+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93908824177824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1f3570 con 0x7f94980b7020 2024-04-27T03:30:53.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.790+0000 7f949ffaa640 1 -- 172.21.15.165:0/3697488659 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 93908824177824] v25'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9490045040 con 0x7f94980b7020 2024-04-27T03:30:53.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.790+0000 7f94acb2b800 20 remove_watcher() i=3 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.790+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93908824181440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1f3570 con 0x5568da1a69d0 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.791+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 93908824181440] v25'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x5568da1a69d0 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.791+0000 7f94acb2b800 20 remove_watcher() i=4 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.791+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93908824174160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1f3570 con 0x7f9490038040 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.792+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 93908824174160] v25'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.792+0000 7f94acb2b800 20 remove_watcher() i=5 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.792+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93908824186416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1f3570 con 0x7f9490038040 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.793+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 93908824186416] v25'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.793+0000 7f94acb2b800 20 remove_watcher() i=6 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.793+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93908824190112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x5568da1f3570 con 0x7f9490038040 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 93908824190112] v25'173 uv171 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f94a0031040 con 0x7f9490038040 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 20 remove_watcher() i=7 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9490038040 msgr2=0x7f949001bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9490038040 0x7f949001bab0 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568da1a69d0 msgr2=0x5568da1a8e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568da1a69d0 0x5568da1a8e90 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f94980b7020 msgr2=0x7f94980716a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f94980b7020 0x7f94980716a0 crc :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5568da1ab3c0 msgr2=0x5568da1ad7b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5568da1ab3c0 0x5568da1ad7b0 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9370036900 msgr2=0x7f9370038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9370036900 0x7f9370038dc0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f9490001b60 tx=0x7f9490010000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 msgr2=0x5568da19b220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94a8fac640 1 -- 172.21.15.165:0/3697488659 reap_dead start 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 0x5568da19b220 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f9498066d60 tx=0x7f9498095f60 comp rx=0 tx=0).stop 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 shutdown_connections 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3697488659 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da193e10 0x5568da19b220 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 >> 172.21.15.165:0/3697488659 conn(0x5568da190170 msgr2=0x5568da18d7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 shutdown_connections 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.794+0000 7f94acb2b800 1 -- 172.21.15.165:0/3697488659 wait complete. 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3416607454 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9380036900 msgr2=0x7f9380038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3416607454 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9380036900 0x7f9380038dc0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f949000bf50 tx=0x7f949000f000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3416607454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da186cf0 msgr2=0x5568da187110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3416607454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da186cf0 0x5568da187110 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f9498001eb0 tx=0x7f94980793a0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3416607454 shutdown_connections 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3416607454 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9380036900 0x7f9380038dc0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3416607454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da186cf0 0x5568da187110 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3416607454 >> 172.21.15.165:0/3416607454 conn(0x5568da17ad90 msgr2=0x5568da17b190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3416607454 shutdown_connections 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3416607454 wait complete. 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5568da175d50 msgr2=0x5568da1781a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x5568da175d50 0x5568da1781a0 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568da06cfa0 msgr2=0x5568da16f510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5568da06cfa0 0x5568da16f510 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5568da171e10 msgr2=0x5568da174200 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5568da171e10 0x5568da174200 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9390036900 msgr2=0x7f9390038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9390036900 0x7f9390038dc0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f9490000f60 tx=0x7f94900012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 msgr2=0x5568da168320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 --2- 172.21.15.165:0/3248932632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5568da06bd40 0x5568da168320 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f9498060040 tx=0x7f9498002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94a8fac640 1 -- 172.21.15.165:0/3248932632 reap_dead start 2024-04-27T03:30:53.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 shutdown_connections 2024-04-27T03:30:53.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 >> 172.21.15.165:0/3248932632 conn(0x5568da0673a0 msgr2=0x5568da156530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 shutdown_connections 2024-04-27T03:30:53.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.795+0000 7f94acb2b800 1 -- 172.21.15.165:0/3248932632 wait complete. 2024-04-27T03:30:53.808 INFO:teuthology.orchestra.run.smithi165.stdout:fooFoofooFooFULL_CONTROL 2024-04-27T03:30:53.809 DEBUG:tasks.util.rgw: raw result: fooFoofooFooFULL_CONTROL 2024-04-27T03:30:53.811 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=417 failed=False 2024-04-27T03:30:53.815 INFO:tasks.radosgw_admin:RL: put_acls myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:53.815 INFO:tasks.util.rgw:rgwadmin: client.0 : ['policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-27T03:30:53.815 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'xml', '-n', 'client.0', '--cluster', 'ceph', 'policy', '--bucket', 'myfoo', '--object', 'bb3aec0fdcdbc2974890f805c585d432'] 2024-04-27T03:30:53.816 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format xml -n client.0 --cluster ceph policy --bucket myfoo --object bb3aec0fdcdbc2974890f805c585d432 2024-04-27T03:30:53.859 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:53.859 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:53.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.854+0000 7f2fa2925800 1 Processor -- start 2024-04-27T03:30:53.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.854+0000 7f2fa2925800 1 -- start start 2024-04-27T03:30:53.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.854+0000 7f2fa2925800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562fa2474990 0x562fa2476d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.854+0000 7f2fa2925800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562fa2332d40 con 0x562fa24772c0 2024-04-27T03:30:53.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.854+0000 7f2fa2925800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x562fa23054d0 con 0x562fa2474990 2024-04-27T03:30:53.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.855+0000 7f2f9e6b7640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562fa2474990 0x562fa2476d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.855+0000 7f2f95eb6640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x562fa24772c0 0x562fa24796b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:38068/0 (socket says 172.21.15.165:38068) 2024-04-27T03:30:53.860 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.855+0000 7f2f95eb6640 1 -- 172.21.15.165:0/3823827297 learned_addr learned my addr 172.21.15.165:0/3823827297 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:53.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.855+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/3823827297 >> v1:172.21.15.89:6789/0 conn(0x562fa24772c0 legacy=0x562fa24796b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.855+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/3823827297 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa2331590 con 0x562fa2474990 2024-04-27T03:30:53.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.857+0000 7f2f9e6b7640 1 --2- 172.21.15.165:0/3823827297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa2476d80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000a470 tx=0x7f2f9001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6325e73f8e4c04cd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.857+0000 7f2f9deb6640 1 -- 172.21.15.165:0/3823827297 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f90057070 con 0x562fa2474990 2024-04-27T03:30:53.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.857+0000 7f2f9deb6640 1 -- 172.21.15.165:0/3823827297 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f90052570 con 0x562fa2474990 2024-04-27T03:30:53.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.857+0000 7f2f9deb6640 1 -- 172.21.15.165:0/3823827297 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f90052870 con 0x562fa2474990 2024-04-27T03:30:53.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.858+0000 7f2fa2925800 1 -- 172.21.15.165:0/3823827297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 msgr2=0x562fa2476d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.858+0000 7f2fa2925800 1 --2- 172.21.15.165:0/3823827297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa2476d80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7f2f9000a470 tx=0x7f2f9001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:53.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.858+0000 7f2fa2925800 1 -- 172.21.15.165:0/3823827297 shutdown_connections 2024-04-27T03:30:53.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.858+0000 7f2fa2925800 1 --2- 172.21.15.165:0/3823827297 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa2476d80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.858+0000 7f2fa2925800 1 -- 172.21.15.165:0/3823827297 >> 172.21.15.165:0/3823827297 conn(0x562fa246e340 msgr2=0x562fa2470760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.858+0000 7f2fa2925800 1 -- 172.21.15.165:0/3823827297 shutdown_connections 2024-04-27T03:30:53.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.858+0000 7f2fa2925800 1 -- 172.21.15.165:0/3823827297 wait complete. 2024-04-27T03:30:53.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.869+0000 7f2fa2925800 1 Processor -- start 2024-04-27T03:30:53.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.869+0000 7f2fa2925800 1 -- start start 2024-04-27T03:30:53.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.870+0000 7f2fa2925800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa24742d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.870+0000 7f2fa2925800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562fa23bf3b0 con 0x562fa2474990 2024-04-27T03:30:53.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.870+0000 7f2f9e6b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa24742d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.870+0000 7f2f9e6b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa24742d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51364/0 (socket says 172.21.15.165:51364) 2024-04-27T03:30:53.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.870+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/1552642358 learned_addr learned my addr 172.21.15.165:0/1552642358 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.870+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/1552642358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa2472890 con 0x562fa2474990 2024-04-27T03:30:53.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.870+0000 7f2f9e6b7640 1 --2- 172.21.15.165:0/1552642358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa24742d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2f90055040 tx=0x7f2f9004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5cc795d2f39bbcb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.870+0000 7f2e8c608640 1 -- 172.21.15.165:0/1552642358 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f90057070 con 0x562fa2474990 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.870+0000 7f2e8c608640 1 -- 172.21.15.165:0/1552642358 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f90063070 con 0x562fa2474990 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.871+0000 7f2e8c608640 1 -- 172.21.15.165:0/1552642358 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f90052720 con 0x562fa2474990 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.871+0000 7f2fa2925800 1 -- 172.21.15.165:0/1552642358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 msgr2=0x562fa24742d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.871+0000 7f2fa2925800 1 --2- 172.21.15.165:0/1552642358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa24742d0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f2f90055040 tx=0x7f2f9004c000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.871+0000 7f2fa2925800 1 -- 172.21.15.165:0/1552642358 shutdown_connections 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.871+0000 7f2fa2925800 1 --2- 172.21.15.165:0/1552642358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa24742d0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.871+0000 7f2fa2925800 1 -- 172.21.15.165:0/1552642358 >> 172.21.15.165:0/1552642358 conn(0x562fa246e340 msgr2=0x562fa2560ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.871+0000 7f2fa2925800 1 -- 172.21.15.165:0/1552642358 shutdown_connections 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.871+0000 7f2fa2925800 1 -- 172.21.15.165:0/1552642358 wait complete. 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.872+0000 7f2fa2925800 1 Processor -- start 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.872+0000 7f2fa2925800 1 -- start start 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.872+0000 7f2fa2925800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa256ee30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.872+0000 7f2fa2925800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562fa247d0e0 con 0x562fa2474990 2024-04-27T03:30:53.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.872+0000 7f2f9e6b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa256ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.872+0000 7f2f9e6b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa256ee30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51368/0 (socket says 172.21.15.165:51368) 2024-04-27T03:30:53.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.872+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/3991286454 learned_addr learned my addr 172.21.15.165:0/3991286454 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.872+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/3991286454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa256f370 con 0x562fa2474990 2024-04-27T03:30:53.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.872+0000 7f2f9e6b7640 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa256ee30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2f90060040 tx=0x7f2f90002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.873+0000 7f2e8ae05640 1 -- 172.21.15.165:0/3991286454 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f90052e30 con 0x562fa2474990 2024-04-27T03:30:53.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.873+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562fa256f600 con 0x562fa2474990 2024-04-27T03:30:53.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.873+0000 7f2e8ae05640 1 -- 172.21.15.165:0/3991286454 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f9006c070 con 0x562fa2474990 2024-04-27T03:30:53.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.873+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa25722b0 con 0x562fa2474990 2024-04-27T03:30:53.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.873+0000 7f2e8ae05640 1 -- 172.21.15.165:0/3991286454 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f9005e5b0 con 0x562fa2474990 2024-04-27T03:30:53.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.873+0000 7f2e8ae05640 1 -- 172.21.15.165:0/3991286454 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2f9005e750 con 0x562fa2474990 2024-04-27T03:30:53.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.874+0000 7f2e8ae05640 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e84036900 0x7f2e84038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.874+0000 7f2e8ae05640 1 -- 172.21.15.165:0/3991286454 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2f9008e760 con 0x562fa2474990 2024-04-27T03:30:53.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.874+0000 7f2f95eb6640 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e84036900 0x7f2e84038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.874+0000 7f2fa2925800 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562fa24772c0 0x562fa25760d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.874+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa2576650 con 0x562fa24772c0 2024-04-27T03:30:53.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.874+0000 7f2f9eeb8640 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562fa24772c0 0x562fa25760d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.874+0000 7f2f95eb6640 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e84036900 0x7f2e84038dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2f88000f60 tx=0x7f2f880012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.874+0000 7f2f9eeb8640 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562fa24772c0 0x562fa25760d0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.875+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/3991286454 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa24772c0 2024-04-27T03:30:53.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.875+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25736a0 con 0x562fa24772c0 2024-04-27T03:30:53.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.875+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/3991286454 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa24772c0 2024-04-27T03:30:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.875+0000 7f2fa2925800 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562fa2578960 0x562fa257ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.875+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa257b440 con 0x562fa2578960 2024-04-27T03:30:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.875+0000 7f2f9e6b7640 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562fa2578960 0x562fa257ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.875+0000 7f2f9e6b7640 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562fa2578960 0x562fa257ad50 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.875+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/3991286454 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2f90071990 con 0x562fa2578960 2024-04-27T03:30:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.875+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa2577910 con 0x562fa24772c0 2024-04-27T03:30:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.876+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/3991286454 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa24772c0 2024-04-27T03:30:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.876+0000 7f2fa2925800 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562fa257c8a0 0x562fa257ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.876+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa257f470 con 0x562fa257c8a0 2024-04-27T03:30:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.876+0000 7f2f95eb6640 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562fa257c8a0 0x562fa257ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.876+0000 7f2f95eb6640 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562fa257c8a0 0x562fa257ed10 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.877+0000 7f2f95eb6640 1 -- 172.21.15.165:0/3991286454 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2f88009050 con 0x562fa257c8a0 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.877+0000 7f2fa2925800 1 Processor -- start 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.877+0000 7f2fa2925800 1 -- start start 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.877+0000 7f2fa2925800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa258d920 0x562fa258dd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.877+0000 7f2fa2925800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562fa23656b0 con 0x562fa258d920 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.877+0000 7f2f9e6b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa258d920 0x562fa258dd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.877+0000 7f2f9e6b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa258d920 0x562fa258dd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51382/0 (socket says 172.21.15.165:51382) 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.877+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/4204035358 learned_addr learned my addr 172.21.15.165:0/4204035358 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.878+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/4204035358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa258e280 con 0x562fa258d920 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.878+0000 7f2f9e6b7640 1 --2- 172.21.15.165:0/4204035358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa258d920 0x562fa258dd40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2f9008cd60 tx=0x7f2f90067490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.878+0000 7f2e83fff640 1 -- 172.21.15.165:0/4204035358 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f9008ec70 con 0x562fa258d920 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.878+0000 7f2e83fff640 1 -- 172.21.15.165:0/4204035358 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f90093070 con 0x562fa258d920 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.878+0000 7f2fa2925800 1 -- 172.21.15.165:0/4204035358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562fa258ecf0 con 0x562fa258d920 2024-04-27T03:30:53.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.878+0000 7f2e83fff640 1 -- 172.21.15.165:0/4204035358 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f9005ea40 con 0x562fa258d920 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.878+0000 7f2fa2925800 1 -- 172.21.15.165:0/4204035358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa258f1f0 con 0x562fa258d920 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.879+0000 7f2e83fff640 1 -- 172.21.15.165:0/4204035358 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2f9005ebe0 con 0x562fa258d920 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.879+0000 7f2e83fff640 1 --2- 172.21.15.165:0/4204035358 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e78036900 0x7f2e78038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.879+0000 7f2e83fff640 1 -- 172.21.15.165:0/4204035358 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2f9008b050 con 0x562fa258d920 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.879+0000 7f2f95eb6640 1 --2- 172.21.15.165:0/4204035358 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e78036900 0x7f2e78038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.879+0000 7f2fa2925800 1 Processor -- start 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.879+0000 7f2fa2925800 1 -- start start 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.879+0000 7f2f95eb6640 1 --2- 172.21.15.165:0/4204035358 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e78036900 0x7f2e78038dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800bf50 tx=0x7f2f8800f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.879+0000 7f2fa2925800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 0x562fa259afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.879+0000 7f2fa2925800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562fa2472b50 con 0x562fa259ab90 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.879+0000 7f2f9e6b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 0x562fa259afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.880+0000 7f2f9e6b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 0x562fa259afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51396/0 (socket says 172.21.15.165:51396) 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.880+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/566013435 learned_addr learned my addr 172.21.15.165:0/566013435 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.880+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/566013435 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa259b580 con 0x562fa259ab90 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.880+0000 7f2f9e6b7640 1 --2- 172.21.15.165:0/566013435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 0x562fa259afb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009720 tx=0x7f2f90079cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5af78264f346c41a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.880+0000 7f2e81ffb640 1 -- 172.21.15.165:0/566013435 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f90098460 con 0x562fa259ab90 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.880+0000 7f2e81ffb640 1 -- 172.21.15.165:0/566013435 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f900a8070 con 0x562fa259ab90 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.880+0000 7f2e81ffb640 1 -- 172.21.15.165:0/566013435 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f90098600 con 0x562fa259ab90 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.880+0000 7f2fa2925800 1 -- 172.21.15.165:0/566013435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 msgr2=0x562fa259afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.880+0000 7f2fa2925800 1 --2- 172.21.15.165:0/566013435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 0x562fa259afb0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009720 tx=0x7f2f90079cf0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.881+0000 7f2fa2925800 1 -- 172.21.15.165:0/566013435 shutdown_connections 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.881+0000 7f2fa2925800 1 --2- 172.21.15.165:0/566013435 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 0x562fa259afb0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.881+0000 7f2fa2925800 1 -- 172.21.15.165:0/566013435 >> 172.21.15.165:0/566013435 conn(0x562fa2596e60 msgr2=0x562fa2594ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.881+0000 7f2fa2925800 1 -- 172.21.15.165:0/566013435 shutdown_connections 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.881+0000 7f2fa2925800 1 -- 172.21.15.165:0/566013435 wait complete. 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.881+0000 7f2fa2925800 1 Processor -- start 2024-04-27T03:30:53.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.881+0000 7f2fa2925800 1 -- start start 2024-04-27T03:30:53.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.881+0000 7f2fa2925800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 0x562fa25a1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.881+0000 7f2fa2925800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562fa259bbe0 con 0x562fa259ab90 2024-04-27T03:30:53.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.881+0000 7f2f9e6b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 0x562fa25a1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.881+0000 7f2f9e6b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 0x562fa25a1e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51398/0 (socket says 172.21.15.165:51398) 2024-04-27T03:30:53.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.881+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 learned_addr learned my addr 172.21.15.165:0/829488254 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:53.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.882+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562fa25a23b0 con 0x562fa259ab90 2024-04-27T03:30:53.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.882+0000 7f2f9e6b7640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 0x562fa25a1e70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009720 tx=0x7f2f9007e290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.882+0000 7f2e6ffff640 1 -- 172.21.15.165:0/829488254 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f900bc070 con 0x562fa259ab90 2024-04-27T03:30:53.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.882+0000 7f2e6ffff640 1 -- 172.21.15.165:0/829488254 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2f90058040 con 0x562fa259ab90 2024-04-27T03:30:53.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.882+0000 7f2e6ffff640 1 -- 172.21.15.165:0/829488254 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2f9008b6f0 con 0x562fa259ab90 2024-04-27T03:30:53.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.882+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562fa25a23b0 con 0x562fa259ab90 2024-04-27T03:30:53.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.882+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562fa25a23b0 con 0x562fa259ab90 2024-04-27T03:30:53.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.883+0000 7f2e6ffff640 1 -- 172.21.15.165:0/829488254 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2f900bf070 con 0x562fa259ab90 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.883+0000 7f2e6ffff640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e68036900 0x7f2e68038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.883+0000 7f2e6ffff640 1 -- 172.21.15.165:0/829488254 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f2f900b4070 con 0x562fa259ab90 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.883+0000 7f2f95eb6640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e68036900 0x7f2e68038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.883+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.883+0000 7f2fa2925800 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562fa25ad640 0x562fa25afb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.883+0000 7f2f95eb6640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e68036900 0x7f2e68038dc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2f88001b60 tx=0x7f2f88010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.883+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b0250 con 0x562fa25ad640 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2f9eeb8640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562fa25ad640 0x562fa25afb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2f9eeb8640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562fa25ad640 0x562fa25afb00 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa25ad640 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2fa2925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2fa2925800 20 realm 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b0660 con 0x562fa25ad640 2024-04-27T03:30:53.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa25ad640 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2fa2925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2fa2925800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b0f30 con 0x562fa25ad640 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa25ad640 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2fa2925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.884+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25ad640 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.885+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa25ad640 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.885+0000 7f2fa2925800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.885+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.885+0000 7f2fa2925800 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562fa25b2030 0x562fa25b4420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.885+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b49e0 con 0x562fa25b2030 2024-04-27T03:30:53.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.885+0000 7f2f9e6b7640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562fa25b2030 0x562fa25b4420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.885+0000 7f2f9e6b7640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562fa25b2030 0x562fa25b4420 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.885+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.885+0000 7f2fa2925800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.885+0000 7f2fa2925800 20 searching for the correct realm 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.885+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b6310 con 0x562fa25b2030 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.886+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.886+0000 7f2f9e6b7640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2f900c1060 0x7f2f90083700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.886+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b6310 con 0x7f2f900c1060 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.886+0000 7f2f95eb6640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2f900c1060 0x7f2f90083700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.886+0000 7f2f95eb6640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2f900c1060 0x7f2f90083700 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.887+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.887+0000 7f2f95eb6640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2f88037040 0x7f2f88036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.887+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b6310 con 0x7f2f88037040 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.887+0000 7f2f9eeb8640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2f88037040 0x7f2f88036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:53.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.887+0000 7f2f9eeb8640 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2f88037040 0x7f2f88036410 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.887+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f2f9800a4a0 con 0x7f2f88037040 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.887+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25ad640 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.888+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa25ad640 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.888+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25ad640 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.888+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa25ad640 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.888+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f88037040 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.888+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f9800a4a0 con 0x7f2f88037040 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.888+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f88037040 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.888+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f9800a4a0 con 0x7f2f88037040 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.888+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f900c1060 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.888+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.888+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25b2030 2024-04-27T03:30:53.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.889+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.889+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25ad640 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.889+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa25ad640 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.889+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f88037040 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.889+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f9800a4a0 con 0x7f2f88037040 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.889+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25ad640 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.889+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa25ad640 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.889+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f900c1060 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.890+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.890+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25b2030 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.890+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.890+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25b2030 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.890+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.890+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f88037040 2024-04-27T03:30:53.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.891+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f9800a4a0 con 0x7f2f88037040 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.891+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25b2030 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.891+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.891+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25b2030 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.891+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.891+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25ad640 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.891+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa25ad640 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.891+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25ad640 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.891+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2f9800a4a0 con 0x562fa25ad640 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.891+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f88037040 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.892+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f98033040 con 0x7f2f88037040 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.892+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25b2030 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.892+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.892+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f900c1060 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.892+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.892+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25ad640 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.892+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.892+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f88037040 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.893+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.893+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25b2030 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.893+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.893+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f88037040 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.893+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.893+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f900c1060 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.893+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.893+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25b2030 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.894+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.894+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f88037040 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.894+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.894+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x562fa25ad640 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.894+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.894+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b12d0 con 0x7f2f900c1060 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b52a0 con 0x562fa25ad640 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b5690 con 0x562fa25ad640 2024-04-27T03:30:53.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b5690 con 0x7f2f88037040 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 zone default found 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 4 Realm: () 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 zonegroup default 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.895+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b6790 con 0x7f2f88037040 2024-04-27T03:30:53.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2fa2925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b8170 con 0x562fa25b2030 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2fa2925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b8510 con 0x562fa25b2030 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2fa2925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2fa2925800 20 started sync module instance, tier type = 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2fa2925800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.896+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b7610 con 0x7f2f900c1060 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.897+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.897+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94762587355648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b7270 con 0x7f2f900c1060 2024-04-27T03:30:53.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.897+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b9210 con 0x562fa25ad640 2024-04-27T03:30:53.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.898+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94762587355648] v25'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.899+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.899+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94762587359888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b9460 con 0x562fa25ad640 2024-04-27T03:30:53.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.899+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b9f00 con 0x562fa25b2030 2024-04-27T03:30:53.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.900+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.900+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94762587363824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b9f00 con 0x562fa25b2030 2024-04-27T03:30:53.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.900+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25baf00 con 0x7f2f900c1060 2024-04-27T03:30:53.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.900+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94762587359888] v25'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.902+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94762587363824] v25'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.902+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.902+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94762587366176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25b8eb0 con 0x7f2f900c1060 2024-04-27T03:30:53.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.903+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25bb910 con 0x562fa25ad640 2024-04-27T03:30:53.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.903+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94762587366176] v25'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.904+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.904+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94762587370592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25bbd30 con 0x562fa25ad640 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.904+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25bc6f0 con 0x7f2f88037040 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.905+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.905+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94762587374144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25bcb10 con 0x7f2f88037040 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.905+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94762587370592] v25'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.905+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25bb880 con 0x7f2f88037040 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.906+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94762587374144] v25'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.906+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.906+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94762587377792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25bcb10 con 0x7f2f88037040 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.906+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25bda00 con 0x7f2f88037040 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.907+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94762587377792] v25'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.907+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'174 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.907+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94762587375200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25bdbe0 con 0x7f2f88037040 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.907+0000 7f2fa2925800 20 add_watcher() i=0 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.907+0000 7f2fa2925800 20 add_watcher() i=1 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.907+0000 7f2fa2925800 20 add_watcher() i=2 2024-04-27T03:30:53.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.907+0000 7f2fa2925800 20 add_watcher() i=3 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.907+0000 7f2fa2925800 20 add_watcher() i=4 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.907+0000 7f2fa2925800 20 add_watcher() i=5 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.907+0000 7f2fa2925800 20 add_watcher() i=6 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.908+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94762587375200] v25'175 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.908+0000 7f2fa2925800 20 add_watcher() i=7 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.908+0000 7f2fa2925800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.908+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25c0d90 con 0x562fa25ad640 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.908+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.908+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94762587379600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25be060 con 0x562fa25ad640 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.910+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94762587379600] v25'186 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.910+0000 7f2e54fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.910+0000 7f2fa2925800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.910+0000 7f2fa2925800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.910+0000 7f2fa2925800 5 note: GC not initialized 2024-04-27T03:30:53.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.910+0000 7f2dcf7fe640 20 reqs_thread_entry: start 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.910+0000 7f2fa2925800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.910+0000 7f2fa2925800 20 Filter name: none 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.910+0000 7f2dcd7fa640 20 reqs_thread_entry: start 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.910+0000 7f2fa2925800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.910+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.910+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:62 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25f4720 con 0x7f2f900c1060 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(62 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 : miss 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 20 rados->read ofs=0 len=0 2024-04-27T03:30:53.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.1f 8:f8c3518a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25f65b0 con 0x7f2f900c1060 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(63 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 info.flags=0x17 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 to cache LRU end 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 20 get_obj_state: octx=0x7f2f9800ab10 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x562fa25f8c78 s->prefetch_data=0 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 3.35 3:ad937139:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x562fa25f9de0 con 0x562fa25ad640 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(64 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=911b,stat out=16b] v0'0 uv3 ondisk = 0) ==== 262+0+927 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.911+0000 7f2fa2925800 10 manifest: total_size = 5 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.912+0000 7f2fa2925800 20 get_obj_state: setting s->obj_tag to _4a9Ygr0EyToT1PzsBoKuufNP4wHWPBO 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.912+0000 7f2e54fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:53.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.912+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94762587379600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25f2070 con 0x562fa25ad640 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.913+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94762587379600] v25'187 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.913+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94762587355648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25f9de0 con 0x7f2f900c1060 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.914+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94762587355648] v25'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.914+0000 7f2fa2925800 20 remove_watcher() i=0 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.914+0000 7f2fa2925800 2 removed watcher, disabling cache 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.914+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94762587359888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25f9de0 con 0x562fa25ad640 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.915+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94762587359888] v25'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.915+0000 7f2fa2925800 20 remove_watcher() i=1 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.916+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94762587363824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25f9de0 con 0x562fa25b2030 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.916+0000 7f2f9e6b7640 1 -- 172.21.15.165:0/829488254 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94762587363824] v25'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f900be030 con 0x562fa25b2030 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.917+0000 7f2fa2925800 20 remove_watcher() i=2 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.917+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94762587366176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25f9de0 con 0x7f2f900c1060 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.917+0000 7f2f95eb6640 1 -- 172.21.15.165:0/829488254 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94762587366176] v25'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f88009050 con 0x7f2f900c1060 2024-04-27T03:30:53.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.918+0000 7f2fa2925800 20 remove_watcher() i=3 2024-04-27T03:30:53.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.918+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94762587370592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25f9de0 con 0x562fa25ad640 2024-04-27T03:30:53.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.919+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94762587370592] v25'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x562fa25ad640 2024-04-27T03:30:53.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.919+0000 7f2fa2925800 20 remove_watcher() i=4 2024-04-27T03:30:53.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.919+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94762587374144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25f9de0 con 0x7f2f88037040 2024-04-27T03:30:53.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.920+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94762587374144] v25'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.920+0000 7f2fa2925800 20 remove_watcher() i=5 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.920+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94762587377792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25f9de0 con 0x7f2f88037040 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.920+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94762587377792] v25'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.921+0000 7f2fa2925800 20 remove_watcher() i=6 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.921+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94762587375200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x562fa25f9de0 con 0x7f2f88037040 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.921+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94762587375200] v25'176 uv174 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2f98031040 con 0x7f2f88037040 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 20 remove_watcher() i=7 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2f88037040 msgr2=0x7f2f88036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2f88037040 0x7f2f88036410 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562fa25ad640 msgr2=0x562fa25afb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562fa25ad640 0x562fa25afb00 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2f900c1060 msgr2=0x7f2f90083700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2f900c1060 0x7f2f90083700 crc :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562fa25b2030 msgr2=0x562fa25b4420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562fa25b2030 0x562fa25b4420 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e68036900 msgr2=0x7f2e68038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e68036900 0x7f2e68038dc0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f2f88001b60 tx=0x7f2f88010000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 msgr2=0x562fa25a1e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 --2- 172.21.15.165:0/829488254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa259ab90 0x562fa25a1e70 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f2f90009720 tx=0x7f2f9007e290 comp rx=0 tx=0).stop 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 reap_dead start 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/829488254 reap_dead start 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 shutdown_connections 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 >> 172.21.15.165:0/829488254 conn(0x562fa2596e60 msgr2=0x562fa25944c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 shutdown_connections 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/829488254 wait complete. 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/4204035358 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e78036900 msgr2=0x7f2e78038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 --2- 172.21.15.165:0/4204035358 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e78036900 0x7f2e78038dc0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2f8800bf50 tx=0x7f2f8800f000 comp rx=0 tx=0).stop 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/4204035358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa258d920 msgr2=0x562fa258dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 --2- 172.21.15.165:0/4204035358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa258d920 0x562fa258dd40 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f2f9008cd60 tx=0x7f2f90067490 comp rx=0 tx=0).stop 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/4204035358 shutdown_connections 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 --2- 172.21.15.165:0/4204035358 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e78036900 0x7f2e78038dc0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 --2- 172.21.15.165:0/4204035358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa258d920 0x562fa258dd40 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/4204035358 >> 172.21.15.165:0/4204035358 conn(0x562fa25819c0 msgr2=0x562fa2581dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/4204035358 shutdown_connections 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.922+0000 7f2fa2925800 1 -- 172.21.15.165:0/4204035358 wait complete. 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562fa257c8a0 msgr2=0x562fa257ed10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x562fa257c8a0 0x562fa257ed10 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562fa24772c0 msgr2=0x562fa25760d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562fa24772c0 0x562fa25760d0 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562fa2578960 msgr2=0x562fa257ad50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562fa2578960 0x562fa257ad50 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e84036900 msgr2=0x7f2e84038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2e84036900 0x7f2e84038dc0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f2f88000f60 tx=0x7f2f880012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 msgr2=0x562fa256ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:53.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 --2- 172.21.15.165:0/3991286454 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562fa2474990 0x562fa256ee30 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f2f90060040 tx=0x7f2f90002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:53.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2f9eeb8640 1 -- 172.21.15.165:0/3991286454 reap_dead start 2024-04-27T03:30:53.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 shutdown_connections 2024-04-27T03:30:53.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 >> 172.21.15.165:0/3991286454 conn(0x562fa246e340 msgr2=0x562fa2560690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:53.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 shutdown_connections 2024-04-27T03:30:53.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:53.923+0000 7f2fa2925800 1 -- 172.21.15.165:0/3991286454 wait complete. 2024-04-27T03:30:53.937 INFO:teuthology.orchestra.run.smithi165.stdout:fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-04-27T03:30:53.937 DEBUG:tasks.util.rgw: raw result: fooFoohttp://acs.amazonaws.com/groups/global/AllUsersREADfooFooFULL_CONTROL 2024-04-27T03:30:53.939 INFO:tasks.radosgw_admin:RL: get_acls myfoo foo bytes_out=0 bytes_in=609 failed=False 2024-04-27T03:30:53.959 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:54.055 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=5 bytes_in=0 failed=False 2024-04-27T03:30:54.055 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=4 bytes_in=0 failed=False 2024-04-27T03:30:54.055 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=3 bytes_in=0 failed=False 2024-04-27T03:30:54.055 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-04-27T03:30:54.055 INFO:tasks.util.rgw:rgwadmin: client.0 : ['bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-04-27T03:30:54.055 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'bucket', 'rm', '--bucket', 'myfoo', '--purge-objects'] 2024-04-27T03:30:54.055 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph bucket rm --bucket myfoo --purge-objects 2024-04-27T03:30:54.099 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:54.099 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:54.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.094+0000 7f1b47a3f800 1 Processor -- start 2024-04-27T03:30:54.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.094+0000 7f1b47a3f800 1 -- start start 2024-04-27T03:30:54.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.094+0000 7f1b47a3f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55c3a47a01d0 0x55c3a47a25c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.094+0000 7f1b47a3f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55c3a465bd40 con 0x55c3a479bd40 2024-04-27T03:30:54.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.094+0000 7f1b47a3f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55c3a462e4d0 con 0x55c3a47a01d0 2024-04-27T03:30:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.095+0000 7f1b3cd3d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55c3a47a01d0 0x55c3a47a25c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.095+0000 7f1b4553e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55c3a479bd40 0x55c3a479c160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:38080/0 (socket says 172.21.15.165:38080) 2024-04-27T03:30:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.095+0000 7f1b4553e640 1 -- 172.21.15.165:0/661493616 learned_addr learned my addr 172.21.15.165:0/661493616 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.095+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/661493616 >> v1:172.21.15.89:6789/0 conn(0x55c3a479bd40 legacy=0x55c3a479c160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.095+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/661493616 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a465a590 con 0x55c3a47a01d0 2024-04-27T03:30:54.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.097+0000 7f1b3cd3d640 1 --2- 172.21.15.165:0/661493616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a47a01d0 0x55c3a47a25c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000a470 tx=0x7f1b3001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63fa9823a24bd38f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.097+0000 7f1b44d3d640 1 -- 172.21.15.165:0/661493616 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b30057070 con 0x55c3a47a01d0 2024-04-27T03:30:54.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.097+0000 7f1b44d3d640 1 -- 172.21.15.165:0/661493616 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b30052570 con 0x55c3a47a01d0 2024-04-27T03:30:54.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.097+0000 7f1b44d3d640 1 -- 172.21.15.165:0/661493616 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b30052870 con 0x55c3a47a01d0 2024-04-27T03:30:54.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.097+0000 7f1b47a3f800 1 -- 172.21.15.165:0/661493616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a47a01d0 msgr2=0x55c3a47a25c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.097+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/661493616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a47a01d0 0x55c3a47a25c0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f1b3000a470 tx=0x7f1b3001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:54.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.098+0000 7f1b47a3f800 1 -- 172.21.15.165:0/661493616 shutdown_connections 2024-04-27T03:30:54.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.098+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/661493616 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a47a01d0 0x55c3a47a25c0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.098+0000 7f1b47a3f800 1 -- 172.21.15.165:0/661493616 >> 172.21.15.165:0/661493616 conn(0x55c3a47973a0 msgr2=0x55c3a47997c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.098+0000 7f1b47a3f800 1 -- 172.21.15.165:0/661493616 shutdown_connections 2024-04-27T03:30:54.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.098+0000 7f1b47a3f800 1 -- 172.21.15.165:0/661493616 wait complete. 2024-04-27T03:30:54.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.110+0000 7f1b47a3f800 1 Processor -- start 2024-04-27T03:30:54.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.110+0000 7f1b47a3f800 1 -- start start 2024-04-27T03:30:54.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.110+0000 7f1b47a3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 0x55c3a4710010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.110+0000 7f1b47a3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c3a46e83b0 con 0x55c3a479bd40 2024-04-27T03:30:54.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.110+0000 7f1b4553e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 0x55c3a4710010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.110+0000 7f1b4553e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 0x55c3a4710010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51424/0 (socket says 172.21.15.165:51424) 2024-04-27T03:30:54.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.110+0000 7f1b4553e640 1 -- 172.21.15.165:0/2821805661 learned_addr learned my addr 172.21.15.165:0/2821805661 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.111+0000 7f1b4553e640 1 -- 172.21.15.165:0/2821805661 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a4890e20 con 0x55c3a479bd40 2024-04-27T03:30:54.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.111+0000 7f1b4553e640 1 --2- 172.21.15.165:0/2821805661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 0x55c3a4710010 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1b38001e10 tx=0x7f1b3800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a3a94c14f23476 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.111+0000 7f1a33606640 1 -- 172.21.15.165:0/2821805661 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b38012020 con 0x55c3a479bd40 2024-04-27T03:30:54.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.111+0000 7f1a33606640 1 -- 172.21.15.165:0/2821805661 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3800e5a0 con 0x55c3a479bd40 2024-04-27T03:30:54.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.111+0000 7f1a33606640 1 -- 172.21.15.165:0/2821805661 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3800e8e0 con 0x55c3a479bd40 2024-04-27T03:30:54.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.112+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2821805661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 msgr2=0x55c3a4710010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.112+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/2821805661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 0x55c3a4710010 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f1b38001e10 tx=0x7f1b3800b780 comp rx=0 tx=0).stop 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.112+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2821805661 shutdown_connections 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.112+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/2821805661 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 0x55c3a4710010 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.112+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2821805661 >> 172.21.15.165:0/2821805661 conn(0x55c3a47973a0 msgr2=0x55c3a4888910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.112+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2821805661 shutdown_connections 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.112+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2821805661 wait complete. 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.112+0000 7f1b47a3f800 1 Processor -- start 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.112+0000 7f1b47a3f800 1 -- start start 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.112+0000 7f1b47a3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 0x55c3a488e750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.113+0000 7f1b47a3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c3a47a2e90 con 0x55c3a479bd40 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.113+0000 7f1b4553e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 0x55c3a488e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.113+0000 7f1b4553e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 0x55c3a488e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51428/0 (socket says 172.21.15.165:51428) 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.113+0000 7f1b4553e640 1 -- 172.21.15.165:0/1303159927 learned_addr learned my addr 172.21.15.165:0/1303159927 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.113+0000 7f1b4553e640 1 -- 172.21.15.165:0/1303159927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a488ce40 con 0x55c3a479bd40 2024-04-27T03:30:54.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.113+0000 7f1b4553e640 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 0x55c3a488e750 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1b38001e10 tx=0x7f1b38000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.113+0000 7f1a31e03640 1 -- 172.21.15.165:0/1303159927 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b38012030 con 0x55c3a479bd40 2024-04-27T03:30:54.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.113+0000 7f1a31e03640 1 -- 172.21.15.165:0/1303159927 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3801a7a0 con 0x55c3a479bd40 2024-04-27T03:30:54.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.113+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c3a488d0d0 con 0x55c3a479bd40 2024-04-27T03:30:54.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.113+0000 7f1a31e03640 1 -- 172.21.15.165:0/1303159927 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3801aac0 con 0x55c3a479bd40 2024-04-27T03:30:54.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.113+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a488d5f0 con 0x55c3a479bd40 2024-04-27T03:30:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.114+0000 7f1a31e03640 1 -- 172.21.15.165:0/1303159927 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f1b38015020 con 0x55c3a479bd40 2024-04-27T03:30:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.114+0000 7f1a31e03640 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a2c036900 0x7f1a2c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.114+0000 7f1a31e03640 1 -- 172.21.15.165:0/1303159927 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f1b38020030 con 0x55c3a479bd40 2024-04-27T03:30:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.114+0000 7f1b3cd3d640 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a2c036900 0x7f1a2c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.115+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c3a47a01d0 0x55c3a489f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.115+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a489f650 con 0x55c3a47a01d0 2024-04-27T03:30:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.115+0000 7f1b45d3f640 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c3a47a01d0 0x55c3a489f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.115+0000 7f1b3cd3d640 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a2c036900 0x7f1a2c038dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1b30054000 tx=0x7f1b3004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.115+0000 7f1b45d3f640 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c3a47a01d0 0x55c3a489f0c0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.115+0000 7f1b45d3f640 1 -- 172.21.15.165:0/1303159927 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a47a01d0 2024-04-27T03:30:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.115+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a489ff60 con 0x55c3a47a01d0 2024-04-27T03:30:54.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.116+0000 7f1b45d3f640 1 -- 172.21.15.165:0/1303159927 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a47a01d0 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.116+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c3a48a1990 0x55c3a48a3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.116+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48a45e0 con 0x55c3a48a1990 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.116+0000 7f1b4553e640 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c3a48a1990 0x55c3a48a3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.116+0000 7f1b4553e640 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c3a48a1990 0x55c3a48a3e50 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.116+0000 7f1b4553e640 1 -- 172.21.15.165:0/1303159927 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48a1990 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.116+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48a07e0 con 0x55c3a47a01d0 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.117+0000 7f1b45d3f640 1 -- 172.21.15.165:0/1303159927 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a47a01d0 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.117+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c3a48a5a90 0x55c3a48a7f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.117+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48a8680 con 0x55c3a48a5a90 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.117+0000 7f1b3cd3d640 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c3a48a5a90 0x55c3a48a7f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.117+0000 7f1b3cd3d640 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c3a48a5a90 0x55c3a48a7f20 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.118+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/1303159927 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f1b30052940 con 0x55c3a48a5a90 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.118+0000 7f1b47a3f800 1 Processor -- start 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.118+0000 7f1b47a3f800 1 -- start start 2024-04-27T03:30:54.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.118+0000 7f1b47a3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48b69f0 0x55c3a48b6e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.118+0000 7f1b47a3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c3a488ec90 con 0x55c3a48b69f0 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.118+0000 7f1b4553e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48b69f0 0x55c3a48b6e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.118+0000 7f1b4553e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48b69f0 0x55c3a48b6e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51436/0 (socket says 172.21.15.165:51436) 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.118+0000 7f1b4553e640 1 -- 172.21.15.165:0/3610493118 learned_addr learned my addr 172.21.15.165:0/3610493118 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.119+0000 7f1b4553e640 1 -- 172.21.15.165:0/3610493118 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a48b7350 con 0x55c3a48b69f0 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.119+0000 7f1b4553e640 1 --2- 172.21.15.165:0/3610493118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48b69f0 0x55c3a48b6e10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1b3804ad50 tx=0x7f1b38018bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.119+0000 7f1a2affd640 1 -- 172.21.15.165:0/3610493118 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b38045bc0 con 0x55c3a48b69f0 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.119+0000 7f1a2affd640 1 -- 172.21.15.165:0/3610493118 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b38045d60 con 0x55c3a48b69f0 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.119+0000 7f1a2affd640 1 -- 172.21.15.165:0/3610493118 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3804c1c0 con 0x55c3a48b69f0 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.119+0000 7f1b47a3f800 1 -- 172.21.15.165:0/3610493118 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c3a48b7dc0 con 0x55c3a48b69f0 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.119+0000 7f1b47a3f800 1 -- 172.21.15.165:0/3610493118 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a48b8260 con 0x55c3a48b69f0 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.120+0000 7f1a2affd640 1 -- 172.21.15.165:0/3610493118 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f1b3804c360 con 0x55c3a48b69f0 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.120+0000 7f1a2affd640 1 --2- 172.21.15.165:0/3610493118 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a1c036900 0x7f1a1c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.120+0000 7f1a2affd640 1 -- 172.21.15.165:0/3610493118 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f1b38037ad0 con 0x55c3a48b69f0 2024-04-27T03:30:54.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.120+0000 7f1b3cd3d640 1 --2- 172.21.15.165:0/3610493118 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a1c036900 0x7f1a1c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.120+0000 7f1b47a3f800 1 Processor -- start 2024-04-27T03:30:54.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.120+0000 7f1b47a3f800 1 -- start start 2024-04-27T03:30:54.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.120+0000 7f1b3cd3d640 1 --2- 172.21.15.165:0/3610493118 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a1c036900 0x7f1a1c038dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55c3a48a07b0 tx=0x7f1b30005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.120+0000 7f1b47a3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 0x55c3a48c3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.120+0000 7f1b47a3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c3a468e6b0 con 0x55c3a48c3ba0 2024-04-27T03:30:54.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.121+0000 7f1b4553e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 0x55c3a48c3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.121+0000 7f1b4553e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 0x55c3a48c3fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51454/0 (socket says 172.21.15.165:51454) 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.121+0000 7f1b4553e640 1 -- 172.21.15.165:0/2572963468 learned_addr learned my addr 172.21.15.165:0/2572963468 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.121+0000 7f1b4553e640 1 -- 172.21.15.165:0/2572963468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a48c4590 con 0x55c3a48c3ba0 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.121+0000 7f1b4553e640 1 --2- 172.21.15.165:0/2572963468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 0x55c3a48c3fc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1b380166c0 tx=0x7f1b38024d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4052f20b09d6e0c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.121+0000 7f1a28ff9640 1 -- 172.21.15.165:0/2572963468 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3805c070 con 0x55c3a48c3ba0 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.121+0000 7f1a28ff9640 1 -- 172.21.15.165:0/2572963468 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3804c8d0 con 0x55c3a48c3ba0 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.121+0000 7f1a28ff9640 1 -- 172.21.15.165:0/2572963468 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3803b070 con 0x55c3a48c3ba0 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.122+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2572963468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 msgr2=0x55c3a48c3fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.122+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/2572963468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 0x55c3a48c3fc0 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7f1b380166c0 tx=0x7f1b38024d70 comp rx=0 tx=0).stop 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.122+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2572963468 shutdown_connections 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.122+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/2572963468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 0x55c3a48c3fc0 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.122+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2572963468 >> 172.21.15.165:0/2572963468 conn(0x55c3a48bfe70 msgr2=0x55c3a48bded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.122+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2572963468 shutdown_connections 2024-04-27T03:30:54.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.122+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2572963468 wait complete. 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.122+0000 7f1b47a3f800 1 Processor -- start 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.122+0000 7f1b47a3f800 1 -- start start 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.122+0000 7f1b47a3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 0x55c3a48cadb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.122+0000 7f1b47a3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55c3a48c4ae0 con 0x55c3a48c3ba0 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.123+0000 7f1b4553e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 0x55c3a48cadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.123+0000 7f1b4553e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 0x55c3a48cadb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51468/0 (socket says 172.21.15.165:51468) 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.123+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 learned_addr learned my addr 172.21.15.165:0/2036576120 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.123+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c3a48cb2f0 con 0x55c3a48c3ba0 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.123+0000 7f1b4553e640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 0x55c3a48cadb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1b380166c0 tx=0x7f1b38010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.123+0000 7f1a1affd640 1 -- 172.21.15.165:0/2036576120 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b3805c030 con 0x55c3a48c3ba0 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.123+0000 7f1a1affd640 1 -- 172.21.15.165:0/2036576120 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f1b3802c960 con 0x55c3a48c3ba0 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.123+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55c3a48cb2f0 con 0x55c3a48c3ba0 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.123+0000 7f1a1affd640 1 -- 172.21.15.165:0/2036576120 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1b38024420 con 0x55c3a48c3ba0 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.123+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c3a48cb7f0 con 0x55c3a48c3ba0 2024-04-27T03:30:54.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.124+0000 7f1a1affd640 1 -- 172.21.15.165:0/2036576120 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f1b3806a020 con 0x55c3a48c3ba0 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.124+0000 7f1a1affd640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a100368b0 0x7f1a10038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.124+0000 7f1a1affd640 1 -- 172.21.15.165:0/2036576120 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f1b38068020 con 0x55c3a48c3ba0 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.124+0000 7f1b3cd3d640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a100368b0 0x7f1a10038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.125+0000 7f1b3cd3d640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a100368b0 0x7f1a10038d70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b30052fd0 tx=0x7f1b3004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.125+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.125+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c3a48d6560 0x55c3a48d8a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.125+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48d9170 con 0x55c3a48d6560 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.125+0000 7f1b45d3f640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c3a48d6560 0x55c3a48d8a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.125+0000 7f1b45d3f640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c3a48d6560 0x55c3a48d8a20 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.125+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a48d6560 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.125+0000 7f1b47a3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.125+0000 7f1b47a3f800 20 realm 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.125+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.125+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48d9580 con 0x55c3a48d6560 2024-04-27T03:30:54.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.125+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a48d6560 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b47a3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b47a3f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48d9e50 con 0x55c3a48d6560 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a48d6560 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b47a3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48da1f0 con 0x55c3a48d6560 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a48d6560 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b47a3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c3a48daf50 0x55c3a48dd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48dd950 con 0x55c3a48daf50 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b4553e640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c3a48daf50 0x55c3a48dd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.141 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.126+0000 7f1b4553e640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c3a48daf50 0x55c3a48dd340 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.127+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.127+0000 7f1b47a3f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.127+0000 7f1b47a3f800 20 searching for the correct realm 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.127+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48df210 con 0x55c3a48daf50 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.127+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.127+0000 7f1b4553e640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1b38017050 0x7f1b380245c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.127+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48df210 con 0x7f1b38017050 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.127+0000 7f1b3cd3d640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1b38017050 0x7f1b380245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.128+0000 7f1b3cd3d640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1b38017050 0x7f1b380245c0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.128+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.128+0000 7f1b3cd3d640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1b30076040 0x7f1b30083410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.128+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48df210 con 0x7f1b30076040 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.128+0000 7f1b45d3f640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1b30076040 0x7f1b30083410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.142 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.129+0000 7f1b45d3f640 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1b30076040 0x7f1b30083410 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.129+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f1b4000a4a0 con 0x7f1b30076040 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.129+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48da1f0 con 0x55c3a48d6560 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.129+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a48d6560 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.129+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48da1f0 con 0x55c3a48d6560 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.129+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a48d6560 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.129+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48da1f0 con 0x7f1b30076040 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.130+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b4000a4a0 con 0x7f1b30076040 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.130+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48da1f0 con 0x7f1b30076040 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.130+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b4000a4a0 con 0x7f1b30076040 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.130+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48da1f0 con 0x7f1b38017050 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.130+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.130+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48da1f0 con 0x55c3a48daf50 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.130+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.130+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48da1f0 con 0x55c3a48d6560 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.131+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a48d6560 2024-04-27T03:30:54.143 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.131+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48da1f0 con 0x7f1b30076040 2024-04-27T03:30:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.131+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b4000a4a0 con 0x7f1b30076040 2024-04-27T03:30:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.131+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48da1f0 con 0x55c3a48d6560 2024-04-27T03:30:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.131+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a48d6560 2024-04-27T03:30:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.131+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48da1f0 con 0x7f1b38017050 2024-04-27T03:30:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.131+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.131+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x55c3a48daf50 2024-04-27T03:30:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.132+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.132+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x55c3a48daf50 2024-04-27T03:30:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.132+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.132+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x7f1b30076040 2024-04-27T03:30:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.132+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b4000a4a0 con 0x7f1b30076040 2024-04-27T03:30:54.144 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.132+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x55c3a48daf50 2024-04-27T03:30:54.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.132+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.132+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x55c3a48daf50 2024-04-27T03:30:54.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.133+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.133+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x55c3a48d6560 2024-04-27T03:30:54.152 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.133+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a48d6560 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.133+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x55c3a48d6560 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.133+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f1b4000a4a0 con 0x55c3a48d6560 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.133+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x7f1b30076040 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.134+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b40033040 con 0x7f1b30076040 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.134+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x55c3a48daf50 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.134+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.134+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x7f1b38017050 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.134+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.134+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x55c3a48d6560 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.135+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.135+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x7f1b30076040 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.135+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.135+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x55c3a48daf50 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.135+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.135+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x7f1b30076040 2024-04-27T03:30:54.153 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.135+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.135+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x7f1b38017050 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.136+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.136+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b30090060 con 0x55c3a48daf50 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.136+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.136+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b38089040 con 0x7f1b30076040 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.136+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.136+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b38089040 con 0x55c3a48d6560 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.136+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.136+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x7f1b38089040 con 0x7f1b38017050 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:54.154 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48de1a0 con 0x55c3a48d6560 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48de540 con 0x55c3a48d6560 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.137+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48de540 con 0x7f1b30076040 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.155 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 20 zone default found 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 4 Realm: () 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 20 zonegroup default 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48df690 con 0x7f1b30076040 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e1070 con 0x55c3a48daf50 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.156 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e1410 con 0x55c3a48daf50 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 20 started sync module instance, tier type = 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.138+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e0510 con 0x7f1b38017050 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.139+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.139+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94298767753312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e17b0 con 0x7f1b38017050 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.139+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e2080 con 0x55c3a48d6560 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.140+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94298767753312] v25'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.141+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.141+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94298767760208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e2470 con 0x55c3a48d6560 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.141+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e2ef0 con 0x55c3a48daf50 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.142+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94298767760208] v25'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.157 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.142+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.142+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94298767763936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e32e0 con 0x55c3a48daf50 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.142+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e3d20 con 0x7f1b38017050 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.143+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94298767763936] v25'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.143+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.143+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94298767767536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e4110 con 0x7f1b38017050 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.143+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e4b70 con 0x55c3a48d6560 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.144+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94298767767536] v25'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.144+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.144+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94298767755520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e4f90 con 0x55c3a48d6560 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.144+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e59e0 con 0x7f1b30076040 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.145+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.145+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94298767774928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e59a0 con 0x7f1b30076040 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.145+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e69c0 con 0x7f1b30076040 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.145+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94298767755520] v25'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.158 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.146+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94298767774928] v25'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.146+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.146+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94298767777232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e4680 con 0x7f1b30076040 2024-04-27T03:30:54.159 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.146+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e6e70 con 0x7f1b30076040 2024-04-27T03:30:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.147+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94298767777232] v25'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.147+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'177 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.147+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94298767779024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e6e70 con 0x7f1b30076040 2024-04-27T03:30:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.147+0000 7f1b47a3f800 20 add_watcher() i=0 2024-04-27T03:30:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.147+0000 7f1b47a3f800 20 add_watcher() i=1 2024-04-27T03:30:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.147+0000 7f1b47a3f800 20 add_watcher() i=2 2024-04-27T03:30:54.161 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.147+0000 7f1b47a3f800 20 add_watcher() i=3 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.147+0000 7f1b47a3f800 20 add_watcher() i=4 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.147+0000 7f1b47a3f800 20 add_watcher() i=5 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.147+0000 7f1b47a3f800 20 add_watcher() i=6 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.149+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94298767779024] v25'178 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.149+0000 7f1b47a3f800 20 add_watcher() i=7 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.149+0000 7f1b47a3f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.149+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48ea300 con 0x55c3a48d6560 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.149+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.149+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94298767780960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e76f0 con 0x55c3a48d6560 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.150+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94298767780960] v25'188 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.150+0000 7f19fb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.150+0000 7f1b47a3f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.150+0000 7f1b47a3f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.150+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f2c20 con 0x55c3a48d6560 2024-04-27T03:30:54.162 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.151+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'54 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.151+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f32d0 con 0x7f1b30076040 2024-04-27T03:30:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.152+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.152+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f3710 con 0x55c3a48daf50 2024-04-27T03:30:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.153+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'31 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.153+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f36a0 con 0x55c3a48d6560 2024-04-27T03:30:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.154+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'189 uv172 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.154+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f3d30 con 0x7f1b38017050 2024-04-27T03:30:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.155+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'68 uv59 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.163 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.155+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f42a0 con 0x55c3a48daf50 2024-04-27T03:30:54.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.156+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'54 uv49 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.156+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f4750 con 0x55c3a48d6560 2024-04-27T03:30:54.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.157+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'32 uv29 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.157+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f4750 con 0x7f1b30076040 2024-04-27T03:30:54.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.159+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'47 uv42 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.164 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.159+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f4750 con 0x7f1b38017050 2024-04-27T03:30:54.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.160+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'24 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.165 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.160+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f0a10 con 0x55c3a48daf50 2024-04-27T03:30:54.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.161+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.166 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.161+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f0a10 con 0x7f1b38017050 2024-04-27T03:30:54.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.161+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'69 uv61 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.167 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.161+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f0a10 con 0x7f1b30076040 2024-04-27T03:30:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.163+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'27 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.168 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.163+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f0a10 con 0x7f1b38017050 2024-04-27T03:30:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.164+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.169 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.164+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f0b00 con 0x7f1b30076040 2024-04-27T03:30:54.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.165+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'59 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.170 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.165+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f0b00 con 0x55c3a48d6560 2024-04-27T03:30:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.166+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'190 uv168 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.171 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.166+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f0aa0 con 0x55c3a48daf50 2024-04-27T03:30:54.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.167+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'63 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.172 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.167+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x7f1b38017050 2024-04-27T03:30:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.168+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'70 uv57 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.173 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.168+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x55c3a48daf50 2024-04-27T03:30:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.169+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'36 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.174 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.169+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x55c3a48d6560 2024-04-27T03:30:54.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.170+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'32 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.175 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.170+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x55c3a48d6560 2024-04-27T03:30:54.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.171+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'191 uv170 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.176 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.171+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x55c3a48daf50 2024-04-27T03:30:54.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.172+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'64 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.172+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x7f1b38017050 2024-04-27T03:30:54.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.173+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'40 uv37 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.173+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x55c3a48daf50 2024-04-27T03:30:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.174+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'32 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.174+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x55c3a48daf50 2024-04-27T03:30:54.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.175+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'43 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.175+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x55c3a48daf50 2024-04-27T03:30:54.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.175+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'26 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.176+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x55c3a48daf50 2024-04-27T03:30:54.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.176+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'55 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.182 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.176+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x7f1b30076040 2024-04-27T03:30:54.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.178+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'34 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.183 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.178+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x55c3a48daf50 2024-04-27T03:30:54.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.179+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'36 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.184 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.179+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x7f1b30076040 2024-04-27T03:30:54.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.180+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'48 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.185 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.180+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x55c3a48daf50 2024-04-27T03:30:54.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.181+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'65 uv56 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.186 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.181+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x7f1b30076040 2024-04-27T03:30:54.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.182+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'60 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.187 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.182+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48f52b0 con 0x7f1b38017050 2024-04-27T03:30:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.183+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'30 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.183+0000 7f19fa7e4640 20 reqs_thread_entry: start 2024-04-27T03:30:54.190 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.183+0000 7f1b47a3f800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.183+0000 7f1b47a3f800 20 Filter name: none 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.183+0000 7f196bfff640 20 reqs_thread_entry: start 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:94 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a491fe00 con 0x7f1b38017050 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(94 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv33 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 : miss 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:95 8.1f 8:f8c3518a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a49216f0 con 0x7f1b38017050 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(95 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv1 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:30:54.191 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 info.flags=0x17 2024-04-27T03:30:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 to cache LRU end 2024-04-27T03:30:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:30:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 2024-04-27T03:30:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.184+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:96 4.9 4:93d623a9:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a499edb0 con 0x55c3a48d6560 2024-04-27T03:30:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.185+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(96 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.0 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.185+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:97 4.16 4:69a17937:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a499edb0 con 0x55c3a48daf50 2024-04-27T03:30:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.185+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(97 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.185+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:98 4.e 4:70f2b889:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a499edb0 con 0x7f1b38017050 2024-04-27T03:30:54.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.185+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(98 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2 [call out=426b] v0'0 uv5 ondisk = 0) ==== 194+0+426 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.185+0000 7f1b47a3f800 10 cls_bucket_list_unordered: got 9c8454ddf7aa50116496bac348d7550d 2024-04-27T03:30:54.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.185+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:99 4.5 4:a04b284e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a499f300 con 0x55c3a48daf50 2024-04-27T03:30:54.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.186+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(99 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.3 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.186+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:100 4.c 4:3258eacf:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a499f300 con 0x55c3a48d6560 2024-04-27T03:30:54.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.186+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(100 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.195 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.186+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 4.7 4:e2b2c60d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a499f300 con 0x7f1b30076040 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.187+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(101 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.5 [call out=85b] v0'0 uv7 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.187+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:102 4.32 4:4e93d832:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a499f300 con 0x55c3a48d6560 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.187+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(102 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6 [call out=425b] v0'0 uv3 ondisk = 0) ==== 194+0+425 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.187+0000 7f1b47a3f800 10 cls_bucket_list_unordered: got b1b9a972ccd8c962a473909b97007eb4 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.187+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:103 4.29 4:94a9bb84:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a499f300 con 0x55c3a48daf50 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.187+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 29 ==== osd_op_reply(103 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b3800dcd0 con 0x55c3a48daf50 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.187+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:104 4.27 4:e56221ec:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a499f300 con 0x55c3a48d6560 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.188+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(104 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8 [call out=426b] v0'0 uv5 ondisk = 0) ==== 194+0+426 (crc 0 0 0) 0x7f1b40031040 con 0x55c3a48d6560 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.188+0000 7f1b47a3f800 10 cls_bucket_list_unordered: got c785e1ed2950e3e36b1e2ca01f299a54 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.188+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:105 4.a 4:51fc66b8:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a499f300 con 0x7f1b38017050 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.188+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== osd_op_reply(105 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.9 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.188+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:106 4.3b 4:df276851:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a499f300 con 0x55c3a48daf50 2024-04-27T03:30:54.196 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.188+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 30 ==== osd_op_reply(106 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10 [call out=698b] v0'0 uv9 ondisk = 0) ==== 195+0+698 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.188+0000 7f1b47a3f800 10 cls_bucket_list_unordered: got 24d27c169c2c881eb09a065116f2aa5c 2024-04-27T03:30:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.188+0000 7f1b47a3f800 10 cls_bucket_list_unordered: got bb3aec0fdcdbc2974890f805c585d432 2024-04-27T03:30:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.189+0000 7f1b47a3f800 20 get_obj_state: octx=0x7f1b4000aa90 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x55c3a4921f78 s->prefetch_data=0 2024-04-27T03:30:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.189+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:107 3.31 3:8f8e69da:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_9c8454ddf7aa50116496bac348d7550d:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a499c910 con 0x7f1b30076040 2024-04-27T03:30:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.189+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(107 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_9c8454ddf7aa50116496bac348d7550d [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+895 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.189+0000 7f1b47a3f800 10 manifest: total_size = 6 2024-04-27T03:30:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.189+0000 7f1b47a3f800 20 get_obj_state: setting s->obj_tag to 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11242738628429672480 2024-04-27T03:30:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.189+0000 7f1b47a3f800 20 get_obj_state: octx=0x7f1b4000aa90 obj=myfoo:9c8454ddf7aa50116496bac348d7550d state=0x55c3a4921f78 s->prefetch_data=0 2024-04-27T03:30:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.189+0000 7f1b47a3f800 20 prepare_atomic_modification: state is not atomic. state=0x55c3a4921f78 2024-04-27T03:30:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.189+0000 7f1b47a3f800 20 bucket index object: :.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2 2024-04-27T03:30:54.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.189+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:108 4.e 4:70f2b889:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4926de0 con 0x7f1b38017050 2024-04-27T03:30:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.190+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(108 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2 [stat,call,call] v25'6 uv6 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.190+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:109 3.31 3:8f8e69da:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_9c8454ddf7aa50116496bac348d7550d:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4924dc0 con 0x7f1b30076040 2024-04-27T03:30:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.191+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(109 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_9c8454ddf7aa50116496bac348d7550d [call] v25'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f1b40031040 con 0x7f1b30076040 2024-04-27T03:30:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.191+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:110 4.e 4:70f2b889:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4926dc0 con 0x7f1b38017050 2024-04-27T03:30:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.191+0000 7f1b47a3f800 20 get_obj_state: octx=0x7f1b4000aa90 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x55c3a4921f78 s->prefetch_data=0 2024-04-27T03:30:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.191+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:111 3.c 3:33f451b9:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_b1b9a972ccd8c962a473909b97007eb4:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4926dc0 con 0x55c3a48daf50 2024-04-27T03:30:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.192+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 31 ==== osd_op_reply(111 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_b1b9a972ccd8c962a473909b97007eb4 [getxattrs out=877b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+893 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.192+0000 7f1b47a3f800 10 manifest: total_size = 3 2024-04-27T03:30:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.192+0000 7f1b47a3f800 20 get_obj_state: setting s->obj_tag to 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5638545531796596379 2024-04-27T03:30:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.192+0000 7f1b47a3f800 20 get_obj_state: octx=0x7f1b4000aa90 obj=myfoo:b1b9a972ccd8c962a473909b97007eb4 state=0x55c3a4921f78 s->prefetch_data=0 2024-04-27T03:30:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.192+0000 7f1b47a3f800 20 prepare_atomic_modification: state is not atomic. state=0x55c3a4921f78 2024-04-27T03:30:54.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.192+0000 7f1b47a3f800 20 bucket index object: :.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6 2024-04-27T03:30:54.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.192+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:112 4.32 4:4e93d832:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a492c110 con 0x55c3a48d6560 2024-04-27T03:30:54.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.193+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 23 ==== osd_op_reply(110 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2 [stat,call,call] v25'7 uv7 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.193+0000 7f1a28ff9640 20 handle_completion(): completion ok for obj=9c8454ddf7aa50116496bac348d7550d 2024-04-27T03:30:54.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.193+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(112 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6 [stat,call,call] v25'4 uv4 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.193+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:113 3.c 3:33f451b9:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_b1b9a972ccd8c962a473909b97007eb4:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4924910 con 0x55c3a48daf50 2024-04-27T03:30:54.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.194+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 32 ==== osd_op_reply(113 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_b1b9a972ccd8c962a473909b97007eb4 [call] v25'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.194+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:114 4.32 4:4e93d832:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4926190 con 0x55c3a48d6560 2024-04-27T03:30:54.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.194+0000 7f1b47a3f800 20 get_obj_state: octx=0x7f1b4000aa90 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x55c3a4921f78 s->prefetch_data=0 2024-04-27T03:30:54.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.194+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:115 3.29 3:96319110:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_c785e1ed2950e3e36b1e2ca01f299a54:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a49290e0 con 0x7f1b30076040 2024-04-27T03:30:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.195+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(115 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_c785e1ed2950e3e36b1e2ca01f299a54 [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+895 (crc 0 0 0) 0x7f1b40043040 con 0x7f1b30076040 2024-04-27T03:30:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.195+0000 7f1b47a3f800 10 manifest: total_size = 4 2024-04-27T03:30:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.195+0000 7f1b47a3f800 20 get_obj_state: setting s->obj_tag to 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.16947250222292875552 2024-04-27T03:30:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.195+0000 7f1b47a3f800 20 get_obj_state: octx=0x7f1b4000aa90 obj=myfoo:c785e1ed2950e3e36b1e2ca01f299a54 state=0x55c3a4921f78 s->prefetch_data=0 2024-04-27T03:30:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.195+0000 7f1b47a3f800 20 prepare_atomic_modification: state is not atomic. state=0x55c3a4921f78 2024-04-27T03:30:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.195+0000 7f1b47a3f800 20 bucket index object: :.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8 2024-04-27T03:30:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.195+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:116 4.27 4:e56221ec:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4926190 con 0x55c3a48d6560 2024-04-27T03:30:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.195+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(114 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6 [stat,call,call] v25'5 uv5 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.195+0000 7f1a1bfff640 20 handle_completion(): completion ok for obj=b1b9a972ccd8c962a473909b97007eb4 2024-04-27T03:30:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.196+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(116 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8 [stat,call,call] v25'6 uv6 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.196+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:117 3.29 3:96319110:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_c785e1ed2950e3e36b1e2ca01f299a54:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a492c110 con 0x7f1b30076040 2024-04-27T03:30:54.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.197+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== osd_op_reply(117 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_c785e1ed2950e3e36b1e2ca01f299a54 [call] v25'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x7f1b30076040 2024-04-27T03:30:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.197+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:118 4.27 4:e56221ec:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4926110 con 0x55c3a48d6560 2024-04-27T03:30:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.197+0000 7f1b47a3f800 20 get_obj_state: octx=0x7f1b4000aa90 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x55c3a4921f78 s->prefetch_data=0 2024-04-27T03:30:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.197+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:119 3.15 3:ab349723:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_24d27c169c2c881eb09a065116f2aa5c:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4926110 con 0x7f1b38017050 2024-04-27T03:30:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.197+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 24 ==== osd_op_reply(119 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_24d27c169c2c881eb09a065116f2aa5c [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+895 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.197+0000 7f1b47a3f800 10 manifest: total_size = 5 2024-04-27T03:30:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.197+0000 7f1b47a3f800 20 get_obj_state: setting s->obj_tag to 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.14548135381027152049 2024-04-27T03:30:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.197+0000 7f1b47a3f800 20 get_obj_state: octx=0x7f1b4000aa90 obj=myfoo:24d27c169c2c881eb09a065116f2aa5c state=0x55c3a4921f78 s->prefetch_data=0 2024-04-27T03:30:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.197+0000 7f1b47a3f800 20 prepare_atomic_modification: state is not atomic. state=0x55c3a4921f78 2024-04-27T03:30:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.197+0000 7f1b47a3f800 20 bucket index object: :.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10 2024-04-27T03:30:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.197+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:120 4.3b 4:df276851:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a492c110 con 0x55c3a48daf50 2024-04-27T03:30:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.198+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(118 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8 [stat,call,call] v25'7 uv7 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.204 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.198+0000 7f1a28ff9640 20 handle_completion(): completion ok for obj=c785e1ed2950e3e36b1e2ca01f299a54 2024-04-27T03:30:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.199+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 33 ==== osd_op_reply(120 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10 [stat,call,call] v25'10 uv10 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.199+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:121 3.15 3:ab349723:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_24d27c169c2c881eb09a065116f2aa5c:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4923db0 con 0x7f1b38017050 2024-04-27T03:30:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.200+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 25 ==== osd_op_reply(121 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_24d27c169c2c881eb09a065116f2aa5c [call] v25'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.200+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:122 4.3b 4:df276851:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a49294e0 con 0x55c3a48daf50 2024-04-27T03:30:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.200+0000 7f1b47a3f800 20 get_obj_state: octx=0x7f1b4000aa90 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x55c3a4921f78 s->prefetch_data=0 2024-04-27T03:30:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.200+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:123 3.35 3:ad937139:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_bb3aec0fdcdbc2974890f805c585d432:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a492a1e0 con 0x55c3a48d6560 2024-04-27T03:30:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.200+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(123 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_bb3aec0fdcdbc2974890f805c585d432 [getxattrs out=911b,stat out=16b] v0'0 uv3 ondisk = 0) ==== 262+0+927 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.200+0000 7f1b47a3f800 10 manifest: total_size = 5 2024-04-27T03:30:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.200+0000 7f1b47a3f800 20 get_obj_state: setting s->obj_tag to _4a9Ygr0EyToT1PzsBoKuufNP4wHWPBO 2024-04-27T03:30:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.200+0000 7f1b47a3f800 20 get_obj_state: octx=0x7f1b4000aa90 obj=myfoo:bb3aec0fdcdbc2974890f805c585d432 state=0x55c3a4921f78 s->prefetch_data=0 2024-04-27T03:30:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.200+0000 7f1b47a3f800 20 prepare_atomic_modification: state is not atomic. state=0x55c3a4921f78 2024-04-27T03:30:54.206 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.200+0000 7f1b47a3f800 20 bucket index object: :.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10 2024-04-27T03:30:54.207 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.200+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:124 4.3b 4:df276851:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a492e6b0 con 0x55c3a48daf50 2024-04-27T03:30:54.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.201+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 34 ==== osd_op_reply(122 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10 [stat,call,call] v25'11 uv11 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.201+0000 7f1a1bfff640 20 handle_completion(): completion ok for obj=24d27c169c2c881eb09a065116f2aa5c 2024-04-27T03:30:54.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.201+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 35 ==== osd_op_reply(124 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10 [stat,call,call] v25'12 uv12 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.201+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:125 3.35 3:ad937139:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_bb3aec0fdcdbc2974890f805c585d432:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4929640 con 0x55c3a48d6560 2024-04-27T03:30:54.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 36 ==== osd_op_reply(125 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5_bb3aec0fdcdbc2974890f805c585d432 [call] v25'4 uv4 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:126 4.3b 4:df276851:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4929640 con 0x55c3a48daf50 2024-04-27T03:30:54.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 20 list_objects_ordered: starting attempt 1 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:127 4.9 4:93d623a9:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4929640 con 0x55c3a48d6560 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:128 4.16 4:69a17937:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4921d90 con 0x55c3a48daf50 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:129 4.e 4:70f2b889:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4930cc0 con 0x7f1b38017050 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:130 4.5 4:a04b284e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4930cc0 con 0x55c3a48daf50 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:131 4.c 4:3258eacf:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4935a00 con 0x55c3a48d6560 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:132 4.7 4:e2b2c60d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4938360 con 0x7f1b30076040 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:133 4.32 4:4e93d832:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a493af10 con 0x55c3a48d6560 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:134 4.29 4:94a9bb84:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a493d980 con 0x55c3a48daf50 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:135 4.27 4:e56221ec:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4940450 con 0x55c3a48d6560 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:136 4.a 4:51fc66b8:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4943020 con 0x7f1b38017050 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.202+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:137 4.3b 4:df276851:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4945d40 con 0x55c3a48daf50 2024-04-27T03:30:54.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.203+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 36 ==== osd_op_reply(128 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.203+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 26 ==== osd_op_reply(129 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2 [call out=124b] v0'0 uv7 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.203+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 37 ==== osd_op_reply(127 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.0 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.203+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 38 ==== osd_op_reply(131 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.203+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 30 ==== osd_op_reply(132 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.5 [call out=85b] v0'0 uv7 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b40043040 con 0x7f1b30076040 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.203+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 37 ==== osd_op_reply(134 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.203+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 39 ==== osd_op_reply(133 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.203+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 27 ==== osd_op_reply(136 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.9 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.203+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 40 ==== osd_op_reply(135 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8 [call out=124b] v0'0 uv7 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.203+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 38 ==== osd_op_reply(130 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.3 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 39 ==== osd_op_reply(126 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10 [stat,call,call] v25'13 uv13 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1a28ff9640 20 handle_completion(): completion ok for obj=bb3aec0fdcdbc2974890f805c585d432 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 40 ==== osd_op_reply(137 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10 [call out=124b] v0'0 uv13 ondisk = 0) ==== 195+0+124 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-27T03:30:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-27T03:30:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-27T03:30:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-27T03:30:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5])"; uploads.size()=0; is_truncated=0 2024-04-27T03:30:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:138 4.9 4:93d623a9:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a49293d0 con 0x55c3a48d6560 2024-04-27T03:30:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:139 4.16 4:69a17937:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a49456b0 con 0x55c3a48daf50 2024-04-27T03:30:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:140 4.e 4:70f2b889:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4935e90 con 0x7f1b38017050 2024-04-27T03:30:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:141 4.5 4:a04b284e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4945670 con 0x55c3a48daf50 2024-04-27T03:30:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:142 4.c 4:3258eacf:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a49293d0 con 0x55c3a48d6560 2024-04-27T03:30:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:143 4.7 4:e2b2c60d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4945670 con 0x7f1b30076040 2024-04-27T03:30:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:144 4.32 4:4e93d832:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4935e90 con 0x55c3a48d6560 2024-04-27T03:30:54.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:145 4.29 4:94a9bb84:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a493b3a0 con 0x55c3a48daf50 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:146 4.27 4:e56221ec:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a493dd10 con 0x55c3a48d6560 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:147 4.a 4:51fc66b8:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a492f130 con 0x7f1b38017050 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:148 4.3b 4:df276851:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4931ab0 con 0x55c3a48daf50 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 28 ==== osd_op_reply(140 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2 [call out=124b] v0'0 uv7 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.204+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 29 ==== osd_op_reply(147 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.9 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.205+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 41 ==== osd_op_reply(138 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.0 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.205+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 42 ==== osd_op_reply(142 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.4 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.205+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 43 ==== osd_op_reply(146 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8 [call out=124b] v0'0 uv7 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.205+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 44 ==== osd_op_reply(144 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6 [call out=124b] v0'0 uv5 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.205+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 41 ==== osd_op_reply(139 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.1 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.205+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 31 ==== osd_op_reply(143 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.5 [call out=85b] v0'0 uv7 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b40043040 con 0x7f1b30076040 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.205+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 42 ==== osd_op_reply(141 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.3 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.205+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 43 ==== osd_op_reply(145 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.205+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 44 ==== osd_op_reply(148 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10 [call out=124b] v0'0 uv13 ondisk = 0) ==== 195+0+124 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.205+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:149 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a493b3a0 con 0x7f1b38017050 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.206+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 30 ==== osd_op_reply(149 foo.buckets [call] v25'37 uv37 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.206+0000 7f1b47a3f800 10 cache get: name=default.rgw.meta+root+myfoo : hit (requested=0x11, cached=0x11) 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.206+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:150 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a493b3a0 con 0x7f1b38017050 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.207+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 31 ==== osd_op_reply(150 myfoo [call,call,delete] v25'38 uv38 ondisk = 0) ==== 233+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.207+0000 7f1b47a3f800 10 removing default.rgw.meta+root+myfoo from cache 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.207+0000 7f1b47a3f800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.207+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:151 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94298768126848 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4922040 con 0x7f1b30076040 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.207+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 32 ==== watch-notify(notify (1) cookie 94298767777232 notify 107374182412 ret 0) ==== 191+0+0 (crc 0 0 0) 0x55c3a48f1110 con 0x7f1b30076040 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.207+0000 7f1a28ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182412 cookie 94298767777232 notifier 4637 bl.length()=149 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.207+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 33 ==== osd_op_reply(151 notify.6 [notify cookie 94298768126848 out=8b] v0'0 uv177 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f1b40043040 con 0x7f1b30076040 2024-04-27T03:30:54.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.207+0000 7f1a28ff9640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:152 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f1a08003d60 con 0x7f1b30076040 2024-04-27T03:30:54.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.207+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 34 ==== osd_op_reply(152 notify.6 [notify-ack] v0'0 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x7f1b30076040 2024-04-27T03:30:54.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.208+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 35 ==== watch-notify(notify_complete (2) cookie 94298768126848 notify 107374182412 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55c3a48b7dc0 con 0x7f1b30076040 2024-04-27T03:30:54.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.208+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:153 8.1f 8:f8c3518a:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a493b2a0 con 0x7f1b38017050 2024-04-27T03:30:54.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 32 ==== osd_op_reply(153 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 [call,call,delete] v25'2 uv2 ondisk = 0) ==== 290+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 10 removing default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 from cache 2024-04-27T03:30:54.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5, ofs0, ns] 2024-04-27T03:30:54.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:154 7.0 7:05bf5b68:::notify.1:head [notify cookie 94298768525952 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a493b070 con 0x55c3a48d6560 2024-04-27T03:30:54.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 45 ==== watch-notify(notify (1) cookie 94298767760208 notify 107374182433 ret 0) ==== 248+0+0 (crc 0 0 0) 0x55c3a48b7dc0 con 0x55c3a48d6560 2024-04-27T03:30:54.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1a28ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182433 cookie 94298767760208 notifier 4637 bl.length()=206 2024-04-27T03:30:54.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1a28ff9640 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:155 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f1a08003d60 con 0x55c3a48d6560 2024-04-27T03:30:54.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 46 ==== osd_op_reply(154 notify.1 [notify cookie 94298768525952 out=8b] v0'0 uv177 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 47 ==== watch-notify(notify_complete (2) cookie 94298768525952 notify 107374182433 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55c3a48f2db0 con 0x55c3a48d6560 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 48 ==== osd_op_reply(155 notify.1 [notify-ack] v0'0 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:156 4.9 4:93d623a9:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4920fc0 con 0x55c3a48d6560 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:157 4.16 4:69a17937:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a49293d0 con 0x55c3a48daf50 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:158 4.e 4:70f2b889:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a492f130 con 0x7f1b38017050 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:159 4.5 4:a04b284e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4929770 con 0x55c3a48daf50 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:160 4.c 4:3258eacf:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a49293d0 con 0x55c3a48d6560 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:161 4.7 4:e2b2c60d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4929770 con 0x7f1b30076040 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:162 4.32 4:4e93d832:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a49293d0 con 0x55c3a48d6560 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:163 4.29 4:94a9bb84:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a492f130 con 0x55c3a48daf50 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:164 4.27 4:e56221ec:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a491edf0 con 0x55c3a48d6560 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:165 4.a 4:51fc66b8:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4927c20 con 0x7f1b38017050 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.209+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:166 4.3b 4:df276851:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a49286a0 con 0x55c3a48daf50 2024-04-27T03:30:54.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.211+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 49 ==== osd_op_reply(156 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.0 [delete] v25'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.211+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 45 ==== osd_op_reply(157 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.1 [delete] v25'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.211+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 50 ==== osd_op_reply(162 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.6 [delete] v25'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.211+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 46 ==== osd_op_reply(166 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.10 [delete] v25'14 uv14 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 36 ==== osd_op_reply(161 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.5 [delete] v25'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x7f1b30076040 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 33 ==== osd_op_reply(158 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.2 [delete] v25'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 51 ==== osd_op_reply(160 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.4 [delete] v25'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 34 ==== osd_op_reply(165 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.9 [delete] v25'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 47 ==== osd_op_reply(159 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.3 [delete] v25'6 uv6 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 48 ==== osd_op_reply(163 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.7 [delete] v25'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 52 ==== osd_op_reply(164 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5.8 [delete] v25'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b47a3f800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 : miss 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b47a3f800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:167 6.9 6:91e132b1:::pubsub..bucket.myfoo%2f708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4927c20 con 0x55c3a48daf50 2024-04-27T03:30:54.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 49 ==== osd_op_reply(167 pubsub..bucket.myfoo/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b47a3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b47a3f800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 info.flags=0x0 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b47a3f800 10 adding default.rgw.log++pubsub..bucket.myfoo/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 to cache LRU end 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.212+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:168 6.9 6:91e132b1:::pubsub..bucket.myfoo%2f708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4927c20 con 0x55c3a48daf50 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.213+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 50 ==== osd_op_reply(168 pubsub..bucket.myfoo/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.5 [delete] v25'56 uv49 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.213+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:169 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4927c20 con 0x7f1b38017050 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.214+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 35 ==== osd_op_reply(169 foo.buckets [call] v25'39 uv39 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.215+0000 7f19fb7e6640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.215+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:170 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94298767780960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4927c20 con 0x55c3a48d6560 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.216+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 53 ==== osd_op_reply(170 data_loggenerations_metadata [watch unwatch cookie 94298767780960] v25'192 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.216+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:171 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94298767753312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a4927c20 con 0x7f1b38017050 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.217+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 36 ==== osd_op_reply(171 notify.0 [watch unwatch cookie 94298767753312] v25'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.217+0000 7f1b47a3f800 20 remove_watcher() i=0 2024-04-27T03:30:54.226 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.217+0000 7f1b47a3f800 2 removed watcher, disabling cache 2024-04-27T03:30:54.227 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.217+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:172 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94298767760208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e0060 con 0x55c3a48d6560 2024-04-27T03:30:54.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.218+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 54 ==== osd_op_reply(172 notify.1 [watch unwatch cookie 94298767760208] v25'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.218+0000 7f1b47a3f800 20 remove_watcher() i=1 2024-04-27T03:30:54.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.218+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:173 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94298767763936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e1b50 con 0x55c3a48daf50 2024-04-27T03:30:54.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.219+0000 7f1b4553e640 1 -- 172.21.15.165:0/2036576120 <== osd.3 v2:172.21.15.165:6808/2380918637 51 ==== osd_op_reply(173 notify.2 [watch unwatch cookie 94298767763936] v25'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b380902d0 con 0x55c3a48daf50 2024-04-27T03:30:54.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.219+0000 7f1b47a3f800 20 remove_watcher() i=2 2024-04-27T03:30:54.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.219+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:174 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94298767767536] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e1b50 con 0x7f1b38017050 2024-04-27T03:30:54.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.220+0000 7f1b3cd3d640 1 -- 172.21.15.165:0/2036576120 <== osd.2 v2:172.21.15.165:6816/975457786 37 ==== osd_op_reply(174 notify.3 [watch unwatch cookie 94298767767536] v25'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b30081070 con 0x7f1b38017050 2024-04-27T03:30:54.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.220+0000 7f1b47a3f800 20 remove_watcher() i=3 2024-04-27T03:30:54.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.220+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:175 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94298767755520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e1b50 con 0x55c3a48d6560 2024-04-27T03:30:54.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.221+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.1 v2:172.21.15.165:6800/79879041 55 ==== osd_op_reply(175 notify.4 [watch unwatch cookie 94298767755520] v25'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x55c3a48d6560 2024-04-27T03:30:54.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.221+0000 7f1b47a3f800 20 remove_watcher() i=4 2024-04-27T03:30:54.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.221+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:176 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94298767774928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e0900 con 0x7f1b30076040 2024-04-27T03:30:54.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.222+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 37 ==== osd_op_reply(176 notify.5 [watch unwatch cookie 94298767774928] v25'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x7f1b30076040 2024-04-27T03:30:54.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.222+0000 7f1b47a3f800 20 remove_watcher() i=5 2024-04-27T03:30:54.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.222+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:177 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94298767777232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e1b50 con 0x7f1b30076040 2024-04-27T03:30:54.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.223+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 38 ==== osd_op_reply(177 notify.6 [watch unwatch cookie 94298767777232] v25'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x7f1b30076040 2024-04-27T03:30:54.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.223+0000 7f1b47a3f800 20 remove_watcher() i=6 2024-04-27T03:30:54.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.223+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:178 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94298767779024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x55c3a48e54d0 con 0x7f1b30076040 2024-04-27T03:30:54.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.224+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 <== osd.0 v2:172.21.15.89:6800/2353668432 39 ==== osd_op_reply(178 notify.7 [watch unwatch cookie 94298767779024] v25'179 uv177 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f1b40043040 con 0x7f1b30076040 2024-04-27T03:30:54.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.224+0000 7f1b47a3f800 20 remove_watcher() i=7 2024-04-27T03:30:54.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.224+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1b30076040 msgr2=0x7f1b30083410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.224+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f1b30076040 0x7f1b30083410 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.224+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c3a48d6560 msgr2=0x55c3a48d8a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.224+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c3a48d6560 0x55c3a48d8a20 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.224+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1b38017050 msgr2=0x7f1b380245c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.224+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f1b38017050 0x7f1b380245c0 crc :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.224+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c3a48daf50 msgr2=0x55c3a48dd340 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.224+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c3a48daf50 0x55c3a48dd340 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a100368b0 msgr2=0x7f1a10038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a100368b0 0x7f1a10038d70 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f1b30052fd0 tx=0x7f1b3004f000 comp rx=0 tx=0).stop 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 msgr2=0x55c3a48cadb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/2036576120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48c3ba0 0x55c3a48cadb0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7f1b380166c0 tx=0x7f1b38010e80 comp rx=0 tx=0).stop 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 reap_dead start 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b45d3f640 1 -- 172.21.15.165:0/2036576120 reap_dead start 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 shutdown_connections 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 >> 172.21.15.165:0/2036576120 conn(0x55c3a48bfe70 msgr2=0x55c3a48bd4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 shutdown_connections 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/2036576120 wait complete. 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/3610493118 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a1c036900 msgr2=0x7f1a1c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/3610493118 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a1c036900 0x7f1a1c038dc0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x55c3a48a07b0 tx=0x7f1b30005a00 comp rx=0 tx=0).stop 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/3610493118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48b69f0 msgr2=0x55c3a48b6e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/3610493118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48b69f0 0x55c3a48b6e10 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7f1b3804ad50 tx=0x7f1b38018bf0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/3610493118 shutdown_connections 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/3610493118 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a1c036900 0x7f1a1c038dc0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/3610493118 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a48b69f0 0x55c3a48b6e10 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/3610493118 >> 172.21.15.165:0/3610493118 conn(0x55c3a48aaa90 msgr2=0x55c3a48aae90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/3610493118 shutdown_connections 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/3610493118 wait complete. 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c3a48a5a90 msgr2=0x55c3a48a7f20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x55c3a48a5a90 0x55c3a48a7f20 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c3a47a01d0 msgr2=0x55c3a489f0c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55c3a47a01d0 0x55c3a489f0c0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c3a48a1990 msgr2=0x55c3a48a3e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55c3a48a1990 0x55c3a48a3e50 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a2c036900 msgr2=0x7f1a2c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f1a2c036900 0x7f1a2c038dc0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f1b30054000 tx=0x7f1b3004e000 comp rx=0 tx=0).stop 2024-04-27T03:30:54.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 msgr2=0x55c3a488e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b47a3f800 1 --2- 172.21.15.165:0/1303159927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55c3a479bd40 0x55c3a488e750 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f1b38001e10 tx=0x7f1b38000ed0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.225+0000 7f1b45d3f640 1 -- 172.21.15.165:0/1303159927 reap_dead start 2024-04-27T03:30:54.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.226+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 shutdown_connections 2024-04-27T03:30:54.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.226+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 >> 172.21.15.165:0/1303159927 conn(0x55c3a47973a0 msgr2=0x55c3a48871d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.226+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 shutdown_connections 2024-04-27T03:30:54.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.226+0000 7f1b47a3f800 1 -- 172.21.15.165:0/1303159927 wait complete. 2024-04-27T03:30:54.239 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-04-27T03:30:54.239 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'foo', '--caps', 'user=read'] 2024-04-27T03:30:54.239 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid foo --caps user=read 2024-04-27T03:30:54.283 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:54.284 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:54.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.279+0000 7fba99d2d800 1 Processor -- start 2024-04-27T03:30:54.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.279+0000 7fba99d2d800 1 -- start start 2024-04-27T03:30:54.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.279+0000 7fba99d2d800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560ae22c3d40 0x560ae22c4160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.279+0000 7fba99d2d800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x560ae2183d40 con 0x560ae22c4fa0 2024-04-27T03:30:54.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.279+0000 7fba99d2d800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x560ae21564d0 con 0x560ae22c3d40 2024-04-27T03:30:54.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.279+0000 7fba95978640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560ae22c3d40 0x560ae22c4160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.279+0000 7fba95978640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560ae22c3d40 0x560ae22c4160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51478/0 (socket says 172.21.15.165:51478) 2024-04-27T03:30:54.285 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.279+0000 7fba95177640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x560ae22c4fa0 0x560ae22c5420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:38086/0 (socket says 172.21.15.165:38086) 2024-04-27T03:30:54.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.279+0000 7fba95978640 1 -- 172.21.15.165:0/276494607 learned_addr learned my addr 172.21.15.165:0/276494607 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.280+0000 7fba95978640 1 -- 172.21.15.165:0/276494607 >> v1:172.21.15.89:6789/0 conn(0x560ae22c4fa0 legacy=0x560ae22c5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.286 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.280+0000 7fba95978640 1 -- 172.21.15.165:0/276494607 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae2182590 con 0x560ae22c3d40 2024-04-27T03:30:54.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.282+0000 7fba95978640 1 --2- 172.21.15.165:0/276494607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae22c4160 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fba9000a6b0 tx=0x7fba9001eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daaaad608c11f02e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.282+0000 7fba94976640 1 -- 172.21.15.165:0/276494607 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba9000a200 con 0x560ae22c3d40 2024-04-27T03:30:54.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.282+0000 7fba94976640 1 -- 172.21.15.165:0/276494607 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9000a3a0 con 0x560ae22c3d40 2024-04-27T03:30:54.287 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.282+0000 7fba94976640 1 -- 172.21.15.165:0/276494607 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba900086e0 con 0x560ae22c3d40 2024-04-27T03:30:54.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.282+0000 7fba99d2d800 1 -- 172.21.15.165:0/276494607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 msgr2=0x560ae22c4160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.282+0000 7fba99d2d800 1 --2- 172.21.15.165:0/276494607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae22c4160 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fba9000a6b0 tx=0x7fba9001eed0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.288 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.282+0000 7fba99d2d800 1 -- 172.21.15.165:0/276494607 shutdown_connections 2024-04-27T03:30:54.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.282+0000 7fba99d2d800 1 --2- 172.21.15.165:0/276494607 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae22c4160 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.282+0000 7fba99d2d800 1 -- 172.21.15.165:0/276494607 >> 172.21.15.165:0/276494607 conn(0x560ae22bf3a0 msgr2=0x560ae22c17c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.282+0000 7fba99d2d800 1 -- 172.21.15.165:0/276494607 shutdown_connections 2024-04-27T03:30:54.289 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.282+0000 7fba99d2d800 1 -- 172.21.15.165:0/276494607 wait complete. 2024-04-27T03:30:54.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.294+0000 7fba99d2d800 1 Processor -- start 2024-04-27T03:30:54.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.294+0000 7fba99d2d800 1 -- start start 2024-04-27T03:30:54.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.294+0000 7fba99d2d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae23b8790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.294+0000 7fba99d2d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560ae22103b0 con 0x560ae22c3d40 2024-04-27T03:30:54.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.295+0000 7fba95978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae23b8790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.295+0000 7fba95978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae23b8790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51494/0 (socket says 172.21.15.165:51494) 2024-04-27T03:30:54.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.295+0000 7fba95978640 1 -- 172.21.15.165:0/1732968180 learned_addr learned my addr 172.21.15.165:0/1732968180 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.295+0000 7fba95978640 1 -- 172.21.15.165:0/1732968180 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae23b8d60 con 0x560ae22c3d40 2024-04-27T03:30:54.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.295+0000 7fba95978640 1 --2- 172.21.15.165:0/1732968180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae23b8790 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fba90000c00 tx=0x7fba9004e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c741f1f4d7e36b0d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.295+0000 7fb983e07640 1 -- 172.21.15.165:0/1732968180 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba9000a200 con 0x560ae22c3d40 2024-04-27T03:30:54.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.295+0000 7fb983e07640 1 -- 172.21.15.165:0/1732968180 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba90063070 con 0x560ae22c3d40 2024-04-27T03:30:54.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.295+0000 7fb983e07640 1 -- 172.21.15.165:0/1732968180 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba90007540 con 0x560ae22c3d40 2024-04-27T03:30:54.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.296+0000 7fba99d2d800 1 -- 172.21.15.165:0/1732968180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 msgr2=0x560ae23b8790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.296+0000 7fba99d2d800 1 --2- 172.21.15.165:0/1732968180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae23b8790 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7fba90000c00 tx=0x7fba9004e000 comp rx=0 tx=0).stop 2024-04-27T03:30:54.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.296+0000 7fba99d2d800 1 -- 172.21.15.165:0/1732968180 shutdown_connections 2024-04-27T03:30:54.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.296+0000 7fba99d2d800 1 --2- 172.21.15.165:0/1732968180 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae23b8790 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.296+0000 7fba99d2d800 1 -- 172.21.15.165:0/1732968180 >> 172.21.15.165:0/1732968180 conn(0x560ae22bf3a0 msgr2=0x560ae23afca0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.296+0000 7fba99d2d800 1 -- 172.21.15.165:0/1732968180 shutdown_connections 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.296+0000 7fba99d2d800 1 -- 172.21.15.165:0/1732968180 wait complete. 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.296+0000 7fba99d2d800 1 Processor -- start 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.297+0000 7fba99d2d800 1 -- start start 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.297+0000 7fba99d2d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae23c01e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.297+0000 7fba99d2d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560ae2237710 con 0x560ae22c3d40 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.297+0000 7fba95978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae23c01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.297+0000 7fba95978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae23c01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51510/0 (socket says 172.21.15.165:51510) 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.297+0000 7fba95978640 1 -- 172.21.15.165:0/2581688609 learned_addr learned my addr 172.21.15.165:0/2581688609 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.297+0000 7fba95978640 1 -- 172.21.15.165:0/2581688609 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae23c0720 con 0x560ae22c3d40 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.297+0000 7fba95978640 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae23c01e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fba90060040 tx=0x7fba90002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.298+0000 7fb982604640 1 -- 172.21.15.165:0/2581688609 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba90008900 con 0x560ae22c3d40 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.298+0000 7fb982604640 1 -- 172.21.15.165:0/2581688609 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba9006e070 con 0x560ae22c3d40 2024-04-27T03:30:54.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.298+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ae23c09b0 con 0x560ae22c3d40 2024-04-27T03:30:54.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.298+0000 7fb982604640 1 -- 172.21.15.165:0/2581688609 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba90008bd0 con 0x560ae22c3d40 2024-04-27T03:30:54.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.298+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae23c3660 con 0x560ae22c3d40 2024-04-27T03:30:54.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.298+0000 7fb982604640 1 -- 172.21.15.165:0/2581688609 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fba90068400 con 0x560ae22c3d40 2024-04-27T03:30:54.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.299+0000 7fb982604640 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb97c036900 0x7fb97c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.299+0000 7fb982604640 1 -- 172.21.15.165:0/2581688609 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fba9008e360 con 0x560ae22c3d40 2024-04-27T03:30:54.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.299+0000 7fba95177640 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb97c036900 0x7fb97c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.299+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560ae22c4fa0 0x560ae23c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.299+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae23c79f0 con 0x560ae22c4fa0 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.299+0000 7fba96179640 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560ae22c4fa0 0x560ae23c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.299+0000 7fba95177640 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb97c036900 0x7fb97c038dc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba80001e40 tx=0x7fba800013a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.299+0000 7fba96179640 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560ae22c4fa0 0x560ae23c7420 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.299+0000 7fba96179640 1 -- 172.21.15.165:0/2581688609 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae22c4fa0 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.299+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae23c49d0 con 0x560ae22c4fa0 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.300+0000 7fba96179640 1 -- 172.21.15.165:0/2581688609 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae22c4fa0 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.300+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560ae23c9d70 0x560ae23cc160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.300+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae23cc800 con 0x560ae23c9d70 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.300+0000 7fba95978640 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560ae23c9d70 0x560ae23cc160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.300+0000 7fba95978640 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560ae23c9d70 0x560ae23cc160 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.300+0000 7fba95978640 1 -- 172.21.15.165:0/2581688609 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fba9007a070 con 0x560ae23c9d70 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.300+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae23c8d20 con 0x560ae22c4fa0 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.301+0000 7fba96179640 1 -- 172.21.15.165:0/2581688609 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae22c4fa0 2024-04-27T03:30:54.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.301+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560ae23cdcb0 0x560ae23d0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.301+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae23d0860 con 0x560ae23cdcb0 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.301+0000 7fba95177640 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560ae23cdcb0 0x560ae23d0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.301+0000 7fba95177640 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560ae23cdcb0 0x560ae23d0100 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.302+0000 7fba95177640 1 -- 172.21.15.165:0/2581688609 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fba8000f6d0 con 0x560ae23cdcb0 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.302+0000 7fba99d2d800 1 Processor -- start 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.302+0000 7fba99d2d800 1 -- start start 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.302+0000 7fba99d2d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23decc0 0x560ae23df0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.302+0000 7fba99d2d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560ae21b66b0 con 0x560ae23decc0 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.302+0000 7fba95978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23decc0 0x560ae23df0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.302+0000 7fba95978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23decc0 0x560ae23df0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51518/0 (socket says 172.21.15.165:51518) 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.302+0000 7fba95978640 1 -- 172.21.15.165:0/1405826419 learned_addr learned my addr 172.21.15.165:0/1405826419 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.302+0000 7fba95978640 1 -- 172.21.15.165:0/1405826419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae23df620 con 0x560ae23decc0 2024-04-27T03:30:54.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.303+0000 7fba95978640 1 --2- 172.21.15.165:0/1405826419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23decc0 0x560ae23df0e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fba90054fb0 tx=0x7fba90068920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.303+0000 7fb97b7fe640 1 -- 172.21.15.165:0/1405826419 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba900534d0 con 0x560ae23decc0 2024-04-27T03:30:54.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.303+0000 7fb97b7fe640 1 -- 172.21.15.165:0/1405826419 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba90074ba0 con 0x560ae23decc0 2024-04-27T03:30:54.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.303+0000 7fba99d2d800 1 -- 172.21.15.165:0/1405826419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ae23e0090 con 0x560ae23decc0 2024-04-27T03:30:54.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.303+0000 7fb97b7fe640 1 -- 172.21.15.165:0/1405826419 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba90074e80 con 0x560ae23decc0 2024-04-27T03:30:54.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.303+0000 7fba99d2d800 1 -- 172.21.15.165:0/1405826419 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae23e0590 con 0x560ae23decc0 2024-04-27T03:30:54.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.304+0000 7fb97b7fe640 1 -- 172.21.15.165:0/1405826419 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fba90095060 con 0x560ae23decc0 2024-04-27T03:30:54.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.304+0000 7fb97b7fe640 1 --2- 172.21.15.165:0/1405826419 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb970036af0 0x7fb970038fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.304+0000 7fb97b7fe640 1 -- 172.21.15.165:0/1405826419 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fba90087050 con 0x560ae23decc0 2024-04-27T03:30:54.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.304+0000 7fba95177640 1 --2- 172.21.15.165:0/1405826419 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb970036af0 0x7fb970038fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.304+0000 7fba99d2d800 1 Processor -- start 2024-04-27T03:30:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.304+0000 7fba99d2d800 1 -- start start 2024-04-27T03:30:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.304+0000 7fba95177640 1 --2- 172.21.15.165:0/1405826419 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb970036af0 0x7fb970038fb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x560ae23c4910 tx=0x7fba8000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.304+0000 7fba99d2d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 0x560ae23ec190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.304+0000 7fba99d2d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560ae23b9020 con 0x560ae23ebd70 2024-04-27T03:30:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.304+0000 7fba95978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 0x560ae23ec190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.305+0000 7fba95978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 0x560ae23ec190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51524/0 (socket says 172.21.15.165:51524) 2024-04-27T03:30:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.305+0000 7fba95978640 1 -- 172.21.15.165:0/2125037144 learned_addr learned my addr 172.21.15.165:0/2125037144 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.305+0000 7fba95978640 1 -- 172.21.15.165:0/2125037144 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae23ec760 con 0x560ae23ebd70 2024-04-27T03:30:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.305+0000 7fba95978640 1 --2- 172.21.15.165:0/2125037144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 0x560ae23ec190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fba90053750 tx=0x7fba90097630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415a29ca0773bc06 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.305+0000 7fb9797fa640 1 -- 172.21.15.165:0/2125037144 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba900539a0 con 0x560ae23ebd70 2024-04-27T03:30:54.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.305+0000 7fb9797fa640 1 -- 172.21.15.165:0/2125037144 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba90074440 con 0x560ae23ebd70 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.305+0000 7fb9797fa640 1 -- 172.21.15.165:0/2125037144 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba9008b020 con 0x560ae23ebd70 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.305+0000 7fba99d2d800 1 -- 172.21.15.165:0/2125037144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 msgr2=0x560ae23ec190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.305+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2125037144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 0x560ae23ec190 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7fba90053750 tx=0x7fba90097630 comp rx=0 tx=0).stop 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba99d2d800 1 -- 172.21.15.165:0/2125037144 shutdown_connections 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2125037144 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 0x560ae23ec190 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba99d2d800 1 -- 172.21.15.165:0/2125037144 >> 172.21.15.165:0/2125037144 conn(0x560ae23e80d0 msgr2=0x560ae23e6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba99d2d800 1 -- 172.21.15.165:0/2125037144 shutdown_connections 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba99d2d800 1 -- 172.21.15.165:0/2125037144 wait complete. 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba99d2d800 1 Processor -- start 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba99d2d800 1 -- start start 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba99d2d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 0x560ae23f3110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba99d2d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560ae23ece10 con 0x560ae23ebd70 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba95978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 0x560ae23f3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba95978640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 0x560ae23f3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51534/0 (socket says 172.21.15.165:51534) 2024-04-27T03:30:54.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 learned_addr learned my addr 172.21.15.165:0/2770379994 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.306+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ae23f3650 con 0x560ae23ebd70 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.307+0000 7fba95978640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 0x560ae23f3110 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fba90097670 tx=0x7fba9006cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.307+0000 7fb9677fe640 1 -- 172.21.15.165:0/2770379994 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba900aa040 con 0x560ae23ebd70 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.307+0000 7fb9677fe640 1 -- 172.21.15.165:0/2770379994 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fba90097d70 con 0x560ae23ebd70 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.307+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560ae23f3650 con 0x560ae23ebd70 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.307+0000 7fb9677fe640 1 -- 172.21.15.165:0/2770379994 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fba9005ebc0 con 0x560ae23ebd70 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.307+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ae23f3650 con 0x560ae23ebd70 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.308+0000 7fb9677fe640 1 -- 172.21.15.165:0/2770379994 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fba90076a10 con 0x560ae23ebd70 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.308+0000 7fb9677fe640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb960036950 0x7fb960038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.308+0000 7fb9677fe640 1 -- 172.21.15.165:0/2770379994 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7fba900b47d0 con 0x560ae23ebd70 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.308+0000 7fba95177640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb960036950 0x7fb960038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.308+0000 7fba95177640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb960036950 0x7fb960038e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x560ae23d0c60 tx=0x7fba80010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.308+0000 7fba99d2d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.308+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560ae23fe880 0x560ae2400d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.308+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae2401490 con 0x560ae23fe880 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba96179640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560ae23fe880 0x560ae2400d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba96179640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560ae23fe880 0x560ae2400d40 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae23fe880 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba99d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba99d2d800 20 realm 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba99d2d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae24018a0 con 0x560ae23fe880 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae23fe880 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba99d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba99d2d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba99d2d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402170 con 0x560ae23fe880 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae23fe880 2024-04-27T03:30:54.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba99d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba99d2d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.309+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae23fe880 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.310+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae23fe880 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.310+0000 7fba99d2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.310+0000 7fba99d2d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.310+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560ae2403270 0x560ae2405660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.310+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae2405c70 con 0x560ae2403270 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.310+0000 7fba95978640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560ae2403270 0x560ae2405660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.310+0000 7fba95978640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560ae2403270 0x560ae2405660 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.310+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.310+0000 7fba99d2d800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.310+0000 7fba99d2d800 20 searching for the correct realm 2024-04-27T03:30:54.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.310+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2407580 con 0x560ae2403270 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.311+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.311+0000 7fba95978640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fba90097070 0x7fba900b4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.311+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2407580 con 0x7fba90097070 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.311+0000 7fba95177640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fba90097070 0x7fba900b4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.311+0000 7fba95177640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fba90097070 0x7fba900b4a20 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.312+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba80037040 con 0x7fba90097070 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.312+0000 7fba95177640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fba8000f070 0x7fba8003a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.312+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2407580 con 0x7fba8000f070 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.312+0000 7fba96179640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fba8000f070 0x7fba8003a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.312+0000 7fba96179640 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fba8000f070 0x7fba8003a3c0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.312+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fba8800a4a0 con 0x7fba8000f070 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.312+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae23fe880 2024-04-27T03:30:54.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.313+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae23fe880 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.313+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae23fe880 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.313+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae23fe880 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.313+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba8000f070 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.313+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x7fba8000f070 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.313+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba8000f070 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.313+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x7fba8000f070 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.313+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba90097070 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.314+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba80037040 con 0x7fba90097070 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.314+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae2403270 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.314+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.314+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae23fe880 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.314+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae23fe880 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.314+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba8000f070 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.314+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x7fba8000f070 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.314+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae23fe880 2024-04-27T03:30:54.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.315+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae23fe880 2024-04-27T03:30:54.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.315+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba90097070 2024-04-27T03:30:54.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.315+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fba80037040 con 0x7fba90097070 2024-04-27T03:30:54.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.315+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae2403270 2024-04-27T03:30:54.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.315+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.315+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae2403270 2024-04-27T03:30:54.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.315+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.315+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba8000f070 2024-04-27T03:30:54.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.316+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x7fba8000f070 2024-04-27T03:30:54.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.316+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae2403270 2024-04-27T03:30:54.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.316+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.316+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae2403270 2024-04-27T03:30:54.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.316+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.316+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae23fe880 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.316+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae23fe880 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.316+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae23fe880 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.316+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fba8800a4a0 con 0x560ae23fe880 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.316+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba8000f070 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.317+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba88033040 con 0x7fba8000f070 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.317+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae2403270 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.317+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.317+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba90097070 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.317+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba80037040 con 0x7fba90097070 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.317+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae23fe880 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.317+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.317+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba8000f070 2024-04-27T03:30:54.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.318+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.318+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae2403270 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.318+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.318+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba8000f070 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.318+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.318+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba90097070 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.318+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba80037040 con 0x7fba90097070 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.318+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae2403270 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba8000f070 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x560ae23fe880 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x560ae2402510 con 0x7fba90097070 2024-04-27T03:30:54.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fba80037040 con 0x7fba90097070 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba99d2d800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba99d2d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba99d2d800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba99d2d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba99d2d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba99d2d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba99d2d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.319+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae2406530 con 0x560ae23fe880 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.320+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.320+0000 7fba99d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.320+0000 7fba99d2d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.320+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae2406920 con 0x560ae23fe880 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.320+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.320+0000 7fba99d2d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.320+0000 7fba99d2d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.320+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae2406920 con 0x7fba8000f070 2024-04-27T03:30:54.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:54.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 20 zone default found 2024-04-27T03:30:54.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 4 Realm: () 2024-04-27T03:30:54.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:54.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 20 zonegroup default 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae2407a00 con 0x7fba8000f070 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae24093e0 con 0x560ae2403270 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.321+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae2409780 con 0x560ae2403270 2024-04-27T03:30:54.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.322+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.322+0000 7fba99d2d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.322+0000 7fba99d2d800 20 started sync module instance, tier type = 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.322+0000 7fba99d2d800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.322+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae2408880 con 0x7fba90097070 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.323+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba80037040 con 0x7fba90097070 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.323+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94604745540560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae2409b20 con 0x7fba90097070 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.323+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240a3b0 con 0x560ae23fe880 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.324+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94604745540560] v25'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba80037040 con 0x7fba90097070 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.324+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.324+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94604745542816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240a7a0 con 0x560ae23fe880 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.324+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240b240 con 0x560ae2403270 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.325+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94604745542816] v25'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.325+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.325+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94604745551120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240b630 con 0x560ae2403270 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.325+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240c070 con 0x7fba90097070 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.326+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba80037040 con 0x7fba90097070 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.326+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94604745557088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240cd30 con 0x7fba90097070 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.326+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240d7a0 con 0x560ae23fe880 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.326+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94604745551120] v25'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.327+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94604745557088] v25'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba80037040 con 0x7fba90097070 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.328+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.328+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94604745558144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae2409ec0 con 0x560ae23fe880 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.328+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240e080 con 0x7fba8000f070 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.329+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94604745558144] v25'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.329+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.329+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94604745560688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240dff0 con 0x7fba8000f070 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.329+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240ea90 con 0x7fba8000f070 2024-04-27T03:30:54.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.330+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(56 notify.6 [create] v25'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.330+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94604745565664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240eeb0 con 0x7fba8000f070 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.330+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240f9b0 con 0x7fba8000f070 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.330+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(55 notify.5 [watch watch cookie 94604745560688] v25'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.331+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94604745565664] v25'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.331+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'180 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.331+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94604745569440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240ea00 con 0x7fba8000f070 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.331+0000 7fba99d2d800 20 add_watcher() i=0 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.331+0000 7fba99d2d800 20 add_watcher() i=1 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.331+0000 7fba99d2d800 20 add_watcher() i=2 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.331+0000 7fba99d2d800 20 add_watcher() i=3 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.331+0000 7fba99d2d800 20 add_watcher() i=4 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.331+0000 7fba99d2d800 20 add_watcher() i=5 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.331+0000 7fba99d2d800 20 add_watcher() i=6 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.332+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94604745569440] v25'181 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.332+0000 7fba99d2d800 20 add_watcher() i=7 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.332+0000 7fba99d2d800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.332+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae2412ab0 con 0x560ae23fe880 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.332+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.332+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94604745562944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240ff30 con 0x560ae23fe880 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.333+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94604745562944] v25'193 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.333+0000 7fb9477fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.333+0000 7fba99d2d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.333+0000 7fba99d2d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.333+0000 7fba99d2d800 5 note: GC not initialized 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.333+0000 7fb9467fc640 20 reqs_thread_entry: start 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.333+0000 7fba99d2d800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.334+0000 7fba99d2d800 20 Filter name: none 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.334+0000 7fb8b7fff640 20 reqs_thread_entry: start 2024-04-27T03:30:54.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.334+0000 7fba99d2d800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.334+0000 7fba99d2d800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.334+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae2445b90 con 0x560ae23fe880 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.334+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv13 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.334+0000 7fba99d2d800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.334+0000 7fba99d2d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.334+0000 7fba99d2d800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.334+0000 7fba99d2d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.334+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~403 in=403b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae244bd30 con 0x560ae23fe880 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.335+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~403] v25'14 uv14 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.335+0000 7fba99d2d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.335+0000 7fba99d2d800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.335+0000 7fba99d2d800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.335+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94604745816000 in=591b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x560ae2447610 con 0x560ae23fe880 2024-04-27T03:30:54.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.336+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify (1) cookie 94604745558144 notify 107374182434 ret 0) ==== 621+0+0 (crc 0 0 0) 0x7fba8801e040 con 0x560ae23fe880 2024-04-27T03:30:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.336+0000 7fb9797fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182434 cookie 94604745558144 notifier 4644 bl.length()=579 2024-04-27T03:30:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.336+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(64 notify.4 [notify cookie 94604745816000 out=8b] v0'0 uv180 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.336+0000 7fb9797fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.336+0000 7fb9797fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.336+0000 7fb9797fa640 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7fb958003ea0 con 0x560ae23fe880 2024-04-27T03:30:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.336+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.336+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== watch-notify(notify_complete (2) cookie 94604745816000 notify 107374182434 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fba88018db0 con 0x560ae23fe880 2024-04-27T03:30:54.351 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[],"caps":[{"type":"user","perm":"read"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:54.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.337+0000 7fb9477fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:54.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.337+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94604745562944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae244abc0 con 0x560ae23fe880 2024-04-27T03:30:54.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.338+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94604745562944] v25'194 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.338+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94604745540560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240db40 con 0x7fba90097070 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.339+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94604745540560] v25'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba80037040 con 0x7fba90097070 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.339+0000 7fba99d2d800 20 remove_watcher() i=0 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.339+0000 7fba99d2d800 2 removed watcher, disabling cache 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.339+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94604745542816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae24083d0 con 0x560ae23fe880 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.340+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94604745542816] v25'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.340+0000 7fba99d2d800 20 remove_watcher() i=1 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.340+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94604745551120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae24083d0 con 0x560ae2403270 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.341+0000 7fba95978640 1 -- 172.21.15.165:0/2770379994 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94604745551120] v25'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba9007a070 con 0x560ae2403270 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.341+0000 7fba99d2d800 20 remove_watcher() i=2 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.341+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94604745557088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae24083d0 con 0x7fba90097070 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.342+0000 7fba95177640 1 -- 172.21.15.165:0/2770379994 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94604745557088] v25'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba80037040 con 0x7fba90097070 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.342+0000 7fba99d2d800 20 remove_watcher() i=3 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.342+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94604745558144] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240c460 con 0x560ae23fe880 2024-04-27T03:30:54.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.343+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94604745558144] v25'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x560ae23fe880 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.343+0000 7fba99d2d800 20 remove_watcher() i=4 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.343+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94604745560688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae24083d0 con 0x7fba8000f070 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.344+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94604745560688] v25'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.344+0000 7fba99d2d800 20 remove_watcher() i=5 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.344+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94604745565664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae240c910 con 0x7fba8000f070 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.345+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94604745565664] v25'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.345+0000 7fba99d2d800 20 remove_watcher() i=6 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.345+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94604745569440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x560ae24083d0 con 0x7fba8000f070 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.346+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94604745569440] v25'182 uv180 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fba88031040 con 0x7fba8000f070 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.346+0000 7fba99d2d800 20 remove_watcher() i=7 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fba8000f070 msgr2=0x7fba8003a3c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fba8000f070 0x7fba8003a3c0 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560ae23fe880 msgr2=0x560ae2400d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560ae23fe880 0x560ae2400d40 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fba90097070 msgr2=0x7fba900b4a20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fba90097070 0x7fba900b4a20 crc :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560ae2403270 msgr2=0x560ae2405660 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560ae2403270 0x560ae2405660 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb960036950 msgr2=0x7fb960038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb960036950 0x7fb960038e10 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x560ae23d0c60 tx=0x7fba80010000 comp rx=0 tx=0).stop 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 msgr2=0x560ae23f3110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba96179640 1 -- 172.21.15.165:0/2770379994 reap_dead start 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 0x560ae23f3110 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7fba90097670 tx=0x7fba9006cf60 comp rx=0 tx=0).stop 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 shutdown_connections 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2770379994 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23ebd70 0x560ae23f3110 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 >> 172.21.15.165:0/2770379994 conn(0x560ae23e80d0 msgr2=0x560ae23e5730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 shutdown_connections 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/2770379994 wait complete. 2024-04-27T03:30:54.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/1405826419 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb970036af0 msgr2=0x7fb970038fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 --2- 172.21.15.165:0/1405826419 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb970036af0 0x7fb970038fb0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x560ae23c4910 tx=0x7fba8000f000 comp rx=0 tx=0).stop 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/1405826419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23decc0 msgr2=0x560ae23df0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 --2- 172.21.15.165:0/1405826419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23decc0 0x560ae23df0e0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7fba90054fb0 tx=0x7fba90068920 comp rx=0 tx=0).stop 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/1405826419 shutdown_connections 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 --2- 172.21.15.165:0/1405826419 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb970036af0 0x7fb970038fb0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 --2- 172.21.15.165:0/1405826419 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae23decc0 0x560ae23df0e0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/1405826419 >> 172.21.15.165:0/1405826419 conn(0x560ae23d2d60 msgr2=0x560ae23d3160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/1405826419 shutdown_connections 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.347+0000 7fba99d2d800 1 -- 172.21.15.165:0/1405826419 wait complete. 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560ae23cdcb0 msgr2=0x560ae23d0100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x560ae23cdcb0 0x560ae23d0100 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560ae22c4fa0 msgr2=0x560ae23c7420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560ae22c4fa0 0x560ae23c7420 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560ae23c9d70 msgr2=0x560ae23cc160 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560ae23c9d70 0x560ae23cc160 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb97c036900 msgr2=0x7fb97c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fb97c036900 0x7fb97c038dc0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fba80001e40 tx=0x7fba800013a0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 msgr2=0x560ae23c01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 --2- 172.21.15.165:0/2581688609 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560ae22c3d40 0x560ae23c01e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7fba90060040 tx=0x7fba90002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba96179640 1 -- 172.21.15.165:0/2581688609 reap_dead start 2024-04-27T03:30:54.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 shutdown_connections 2024-04-27T03:30:54.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 >> 172.21.15.165:0/2581688609 conn(0x560ae22bf3a0 msgr2=0x560ae23ae520 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 shutdown_connections 2024-04-27T03:30:54.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.348+0000 7fba99d2d800 1 -- 172.21.15.165:0/2581688609 wait complete. 2024-04-27T03:30:54.362 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [], 'caps': [{'type': 'user', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:54.362 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-04-27T03:30:54.362 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'foo', '--caps', 'user=read'] 2024-04-27T03:30:54.362 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid foo --caps user=read 2024-04-27T03:30:54.407 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:54.407 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:54.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.402+0000 7f83d5016800 1 Processor -- start 2024-04-27T03:30:54.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.402+0000 7f83d5016800 1 -- start start 2024-04-27T03:30:54.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.402+0000 7f83d5016800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x561ac09e2fb0 0x561ac09e33d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.402+0000 7f83d5016800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x561ac08a1d40 con 0x561ac09e4120 2024-04-27T03:30:54.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.402+0000 7f83d5016800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x561ac08744d0 con 0x561ac09e2fb0 2024-04-27T03:30:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.403+0000 7f83d075c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x561ac09e2fb0 0x561ac09e33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.403+0000 7f83c7fff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x561ac09e4120 0x561ac09e45a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:38100/0 (socket says 172.21.15.165:38100) 2024-04-27T03:30:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.403+0000 7f83c7fff640 1 -- 172.21.15.165:0/3079915747 learned_addr learned my addr 172.21.15.165:0/3079915747 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.403+0000 7f83d075c640 1 -- 172.21.15.165:0/3079915747 >> v1:172.21.15.89:6789/0 conn(0x561ac09e4120 legacy=0x561ac09e45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.403+0000 7f83d075c640 1 -- 172.21.15.165:0/3079915747 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ac08a0590 con 0x561ac09e2fb0 2024-04-27T03:30:54.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.405+0000 7f83d075c640 1 --2- 172.21.15.165:0/3079915747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e2fb0 0x561ac09e33d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f83c000a470 tx=0x7f83c001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e18f2f0b79bcd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.405+0000 7f83cff5b640 1 -- 172.21.15.165:0/3079915747 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c0057070 con 0x561ac09e2fb0 2024-04-27T03:30:54.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.405+0000 7f83cff5b640 1 -- 172.21.15.165:0/3079915747 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c0052570 con 0x561ac09e2fb0 2024-04-27T03:30:54.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.405+0000 7f83cff5b640 1 -- 172.21.15.165:0/3079915747 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c0052870 con 0x561ac09e2fb0 2024-04-27T03:30:54.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.405+0000 7f83d5016800 1 -- 172.21.15.165:0/3079915747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e2fb0 msgr2=0x561ac09e33d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.405+0000 7f83d5016800 1 --2- 172.21.15.165:0/3079915747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e2fb0 0x561ac09e33d0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f83c000a470 tx=0x7f83c001ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:54.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.405+0000 7f83d5016800 1 -- 172.21.15.165:0/3079915747 shutdown_connections 2024-04-27T03:30:54.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.405+0000 7f83d5016800 1 --2- 172.21.15.165:0/3079915747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e2fb0 0x561ac09e33d0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.405+0000 7f83d5016800 1 -- 172.21.15.165:0/3079915747 >> 172.21.15.165:0/3079915747 conn(0x561ac09de550 msgr2=0x561ac09e0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.405+0000 7f83d5016800 1 -- 172.21.15.165:0/3079915747 shutdown_connections 2024-04-27T03:30:54.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.405+0000 7f83d5016800 1 -- 172.21.15.165:0/3079915747 wait complete. 2024-04-27T03:30:54.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.417+0000 7f83d5016800 1 Processor -- start 2024-04-27T03:30:54.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.417+0000 7f83d5016800 1 -- start start 2024-04-27T03:30:54.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.417+0000 7f83d5016800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e2fb0 0x561ac0ad4520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.417+0000 7f83d5016800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561ac092e3b0 con 0x561ac09e2fb0 2024-04-27T03:30:54.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.417+0000 7f83d075c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e2fb0 0x561ac0ad4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.417+0000 7f83d075c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e2fb0 0x561ac0ad4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51538/0 (socket says 172.21.15.165:51538) 2024-04-27T03:30:54.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.417+0000 7f83d075c640 1 -- 172.21.15.165:0/2316437954 learned_addr learned my addr 172.21.15.165:0/2316437954 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.418+0000 7f83d075c640 1 -- 172.21.15.165:0/2316437954 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ac0ad4af0 con 0x561ac09e2fb0 2024-04-27T03:30:54.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.418+0000 7f83d075c640 1 --2- 172.21.15.165:0/2316437954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e2fb0 0x561ac0ad4520 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f83c0055040 tx=0x7f83c004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222a9232a91ec071 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.419+0000 7f82be60c640 1 -- 172.21.15.165:0/2316437954 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c0057070 con 0x561ac09e2fb0 2024-04-27T03:30:54.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.419+0000 7f82be60c640 1 -- 172.21.15.165:0/2316437954 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c0063070 con 0x561ac09e2fb0 2024-04-27T03:30:54.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.419+0000 7f82be60c640 1 -- 172.21.15.165:0/2316437954 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c0052720 con 0x561ac09e2fb0 2024-04-27T03:30:54.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.420+0000 7f83d5016800 1 -- 172.21.15.165:0/2316437954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e2fb0 msgr2=0x561ac0ad4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.420+0000 7f83d5016800 1 --2- 172.21.15.165:0/2316437954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e2fb0 0x561ac0ad4520 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f83c0055040 tx=0x7f83c004c000 comp rx=0 tx=0).stop 2024-04-27T03:30:54.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.420+0000 7f83d5016800 1 -- 172.21.15.165:0/2316437954 shutdown_connections 2024-04-27T03:30:54.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.420+0000 7f83d5016800 1 --2- 172.21.15.165:0/2316437954 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e2fb0 0x561ac0ad4520 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.420+0000 7f83d5016800 1 -- 172.21.15.165:0/2316437954 >> 172.21.15.165:0/2316437954 conn(0x561ac09de550 msgr2=0x561ac0acfe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.420+0000 7f83d5016800 1 -- 172.21.15.165:0/2316437954 shutdown_connections 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.420+0000 7f83d5016800 1 -- 172.21.15.165:0/2316437954 wait complete. 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.420+0000 7f83d5016800 1 Processor -- start 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.420+0000 7f83d5016800 1 -- start start 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.421+0000 7f83d5016800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e4120 0x561ac0adc210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.421+0000 7f83d5016800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561ac09ec590 con 0x561ac09e4120 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.421+0000 7f83d075c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e4120 0x561ac0adc210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.421+0000 7f83d075c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e4120 0x561ac0adc210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51540/0 (socket says 172.21.15.165:51540) 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.421+0000 7f83d075c640 1 -- 172.21.15.165:0/569759662 learned_addr learned my addr 172.21.15.165:0/569759662 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.421+0000 7f83d075c640 1 -- 172.21.15.165:0/569759662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ac0adc750 con 0x561ac09e4120 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.421+0000 7f83d075c640 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e4120 0x561ac0adc210 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f83c0055040 tx=0x7f83c0002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.421+0000 7f82bce09640 1 -- 172.21.15.165:0/569759662 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c0052b60 con 0x561ac09e4120 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.421+0000 7f82bce09640 1 -- 172.21.15.165:0/569759662 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c006c070 con 0x561ac09e4120 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.421+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561ac0adc9e0 con 0x561ac09e4120 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.421+0000 7f82bce09640 1 -- 172.21.15.165:0/569759662 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c005e480 con 0x561ac09e4120 2024-04-27T03:30:54.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.421+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ac0adf640 con 0x561ac09e4120 2024-04-27T03:30:54.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.422+0000 7f82bce09640 1 -- 172.21.15.165:0/569759662 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f83c0077060 con 0x561ac09e4120 2024-04-27T03:30:54.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.422+0000 7f82bce09640 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f82b8036900 0x7f82b8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.432 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.422+0000 7f82bce09640 1 -- 172.21.15.165:0/569759662 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f83c008dfe0 con 0x561ac09e4120 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.422+0000 7f83c7fff640 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f82b8036900 0x7f82b8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.423+0000 7f83d5016800 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561ac0ae13f0 0x561ac0ae37e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.423+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0ae3d70 con 0x561ac0ae13f0 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.423+0000 7f83d0f5d640 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561ac0ae13f0 0x561ac0ae37e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.423+0000 7f83c7fff640 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f82b8036900 0x7f82b8038dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f83b8000f60 tx=0x7f83b80012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.423+0000 7f83d0f5d640 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561ac0ae13f0 0x561ac0ae37e0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.423+0000 7f83d0f5d640 1 -- 172.21.15.165:0/569759662 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0ae13f0 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.423+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0ae09d0 con 0x561ac0ae13f0 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.424+0000 7f83d0f5d640 1 -- 172.21.15.165:0/569759662 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0ae13f0 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.424+0000 7f83d5016800 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561ac0ae60c0 0x561ac0ae84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.424+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0ae8b50 con 0x561ac0ae60c0 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.424+0000 7f83d075c640 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561ac0ae60c0 0x561ac0ae84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.424+0000 7f83d075c640 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561ac0ae60c0 0x561ac0ae84b0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.433 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.424+0000 7f83d075c640 1 -- 172.21.15.165:0/569759662 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f83c0079450 con 0x561ac0ae60c0 2024-04-27T03:30:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.424+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0ae5070 con 0x561ac0ae13f0 2024-04-27T03:30:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.425+0000 7f83d0f5d640 1 -- 172.21.15.165:0/569759662 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0ae13f0 2024-04-27T03:30:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.425+0000 7f83d5016800 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561ac0aea000 0x561ac0aec450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.425+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0aecbb0 con 0x561ac0aea000 2024-04-27T03:30:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.425+0000 7f83c7fff640 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561ac0aea000 0x561ac0aec450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.425+0000 7f83c7fff640 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561ac0aea000 0x561ac0aec450 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.425+0000 7f83c7fff640 1 -- 172.21.15.165:0/569759662 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f83b8009050 con 0x561ac0aea000 2024-04-27T03:30:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.426+0000 7f83d5016800 1 Processor -- start 2024-04-27T03:30:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.426+0000 7f83d5016800 1 -- start start 2024-04-27T03:30:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.426+0000 7f83d5016800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0afb010 0x561ac0afb430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.426+0000 7f83d5016800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561ac08d46b0 con 0x561ac0afb010 2024-04-27T03:30:54.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.426+0000 7f83d075c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0afb010 0x561ac0afb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.426+0000 7f83d075c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0afb010 0x561ac0afb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51554/0 (socket says 172.21.15.165:51554) 2024-04-27T03:30:54.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.426+0000 7f83d075c640 1 -- 172.21.15.165:0/3164564834 learned_addr learned my addr 172.21.15.165:0/3164564834 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.426+0000 7f83d075c640 1 -- 172.21.15.165:0/3164564834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ac0afb970 con 0x561ac0afb010 2024-04-27T03:30:54.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.427+0000 7f83d075c640 1 --2- 172.21.15.165:0/3164564834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0afb010 0x561ac0afb430 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f83c006f940 tx=0x7f83c006fbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.427+0000 7f82b5ffb640 1 -- 172.21.15.165:0/3164564834 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c0068de0 con 0x561ac0afb010 2024-04-27T03:30:54.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.427+0000 7f82b5ffb640 1 -- 172.21.15.165:0/3164564834 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c0058040 con 0x561ac0afb010 2024-04-27T03:30:54.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.427+0000 7f82b5ffb640 1 -- 172.21.15.165:0/3164564834 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c0068530 con 0x561ac0afb010 2024-04-27T03:30:54.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.427+0000 7f83d5016800 1 -- 172.21.15.165:0/3164564834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561ac0afc3e0 con 0x561ac0afb010 2024-04-27T03:30:54.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.427+0000 7f83d5016800 1 -- 172.21.15.165:0/3164564834 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ac0afc3e0 con 0x561ac0afb010 2024-04-27T03:30:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.427+0000 7f82b5ffb640 1 -- 172.21.15.165:0/3164564834 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f83c006e030 con 0x561ac0afb010 2024-04-27T03:30:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.428+0000 7f82b5ffb640 1 --2- 172.21.15.165:0/3164564834 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f82a8036900 0x7f82a8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.428+0000 7f82b5ffb640 1 -- 172.21.15.165:0/3164564834 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f83c004b100 con 0x561ac0afb010 2024-04-27T03:30:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.428+0000 7f83c7fff640 1 --2- 172.21.15.165:0/3164564834 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f82a8036900 0x7f82a8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.428+0000 7f83d5016800 1 Processor -- start 2024-04-27T03:30:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.428+0000 7f83d5016800 1 -- start start 2024-04-27T03:30:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.428+0000 7f83c7fff640 1 --2- 172.21.15.165:0/3164564834 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f82a8036900 0x7f82a8038dc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f83b800bf50 tx=0x7f83b800f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.428+0000 7f83d5016800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 0x561ac0b083b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.428+0000 7f83d5016800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561ac0ad4db0 con 0x561ac0b07f90 2024-04-27T03:30:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.428+0000 7f83d075c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 0x561ac0b083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.428+0000 7f83d075c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 0x561ac0b083b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51570/0 (socket says 172.21.15.165:51570) 2024-04-27T03:30:54.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.428+0000 7f83d075c640 1 -- 172.21.15.165:0/4105895719 learned_addr learned my addr 172.21.15.165:0/4105895719 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.429+0000 7f83d075c640 1 -- 172.21.15.165:0/4105895719 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ac0b08980 con 0x561ac0b07f90 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.429+0000 7f83d075c640 1 --2- 172.21.15.165:0/4105895719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 0x561ac0b083b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f83c008f800 tx=0x7f83c0009b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c17b9d302f370ee server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.429+0000 7f82a77fe640 1 -- 172.21.15.165:0/4105895719 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c00a9070 con 0x561ac0b07f90 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.429+0000 7f82a77fe640 1 -- 172.21.15.165:0/4105895719 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c009d070 con 0x561ac0b07f90 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.429+0000 7f82a77fe640 1 -- 172.21.15.165:0/4105895719 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c0050b90 con 0x561ac0b07f90 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.429+0000 7f83d5016800 1 -- 172.21.15.165:0/4105895719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 msgr2=0x561ac0b083b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.429+0000 7f83d5016800 1 --2- 172.21.15.165:0/4105895719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 0x561ac0b083b0 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f83c008f800 tx=0x7f83c0009b70 comp rx=0 tx=0).stop 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.429+0000 7f83d5016800 1 -- 172.21.15.165:0/4105895719 shutdown_connections 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.429+0000 7f83d5016800 1 --2- 172.21.15.165:0/4105895719 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 0x561ac0b083b0 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.429+0000 7f83d5016800 1 -- 172.21.15.165:0/4105895719 >> 172.21.15.165:0/4105895719 conn(0x561ac0b04260 msgr2=0x561ac0b022c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.429+0000 7f83d5016800 1 -- 172.21.15.165:0/4105895719 shutdown_connections 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.429+0000 7f83d5016800 1 -- 172.21.15.165:0/4105895719 wait complete. 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.430+0000 7f83d5016800 1 Processor -- start 2024-04-27T03:30:54.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.430+0000 7f83d5016800 1 -- start start 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.430+0000 7f83d5016800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 0x561ac0b0f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.430+0000 7f83d5016800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x561ac0b08fe0 con 0x561ac0b07f90 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.430+0000 7f83d075c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 0x561ac0b0f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.430+0000 7f83d075c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 0x561ac0b0f350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51580/0 (socket says 172.21.15.165:51580) 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.430+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 learned_addr learned my addr 172.21.15.165:0/1356740060 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.430+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561ac0b0f890 con 0x561ac0b07f90 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.431+0000 7f83d075c640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 0x561ac0b0f350 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f83c0081fd0 tx=0x7f83c0059f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.431+0000 7f82a5ffb640 1 -- 172.21.15.165:0/1356740060 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c00b4070 con 0x561ac0b07f90 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.431+0000 7f82a5ffb640 1 -- 172.21.15.165:0/1356740060 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f83c008fd40 con 0x561ac0b07f90 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.431+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x561ac0b0f890 con 0x561ac0b07f90 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.431+0000 7f82a5ffb640 1 -- 172.21.15.165:0/1356740060 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f83c00a9040 con 0x561ac0b07f90 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.431+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561ac0b0f890 con 0x561ac0b07f90 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.431+0000 7f82a5ffb640 1 -- 172.21.15.165:0/1356740060 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f83c00b7070 con 0x561ac0b07f90 2024-04-27T03:30:54.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.432+0000 7f82a5ffb640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8298036900 0x7f8298038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.432+0000 7f82a5ffb640 1 -- 172.21.15.165:0/1356740060 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f83c008acf0 con 0x561ac0b07f90 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.432+0000 7f83c7fff640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8298036900 0x7f8298038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.432+0000 7f83d5016800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.432+0000 7f83c7fff640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8298036900 0x7f8298038dc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f83b8001b60 tx=0x7f83b8010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.432+0000 7f83d5016800 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561ac0b1aa30 0x561ac0b1cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.432+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1d640 con 0x561ac0b1aa30 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.432+0000 7f83d0f5d640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561ac0b1aa30 0x561ac0b1cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d0f5d640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561ac0b1aa30 0x561ac0b1cef0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0b1aa30 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 20 realm 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1da50 con 0x561ac0b1aa30 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0b1aa30 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e320 con 0x561ac0b1aa30 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0b1aa30 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1aa30 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0b1aa30 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561ac0b1f420 0x561ac0b21810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.433+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b21e20 con 0x561ac0b1f420 2024-04-27T03:30:54.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.434+0000 7f83d075c640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561ac0b1f420 0x561ac0b21810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.434+0000 7f83d075c640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561ac0b1f420 0x561ac0b21810 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.434+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.434+0000 7f83d5016800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.434+0000 7f83d5016800 20 searching for the correct realm 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.434+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b23750 con 0x561ac0b1f420 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.435+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.435+0000 7f83d075c640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f83c00b9070 0x7f83c00a0990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.435+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b23750 con 0x7f83c00b9070 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.435+0000 7f83c7fff640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f83c00b9070 0x7f83c00a0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.435+0000 7f83c7fff640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f83c00b9070 0x7f83c00a0990 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.435+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83b8009050 con 0x7f83c00b9070 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.435+0000 7f83c7fff640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f83b8038040 0x7f83b801bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.435+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b23750 con 0x7f83b8038040 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.436+0000 7f83d0f5d640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f83b8038040 0x7f83b801bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.436+0000 7f83d0f5d640 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f83b8038040 0x7f83b801bab0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.436+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f83c800a4a0 con 0x7f83b8038040 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.436+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1aa30 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.437+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0b1aa30 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.437+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1aa30 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.437+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0b1aa30 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.437+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83b8038040 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.437+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c800a4a0 con 0x7f83b8038040 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.437+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83b8038040 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.437+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c800a4a0 con 0x7f83b8038040 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.437+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83c00b9070 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.437+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83b8009050 con 0x7f83c00b9070 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.438+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1f420 2024-04-27T03:30:54.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.438+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.438+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1aa30 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.438+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0b1aa30 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.438+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83b8038040 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.438+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c800a4a0 con 0x7f83b8038040 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.438+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1aa30 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.439+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0b1aa30 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.439+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83c00b9070 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.439+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f83b8009050 con 0x7f83c00b9070 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.439+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1f420 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.439+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.439+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1f420 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.439+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.439+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83b8038040 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.440+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c800a4a0 con 0x7f83b8038040 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.440+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1f420 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.440+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.440+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1f420 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.440+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.440+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1aa30 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.440+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0b1aa30 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.440+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1aa30 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.440+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f83c800a4a0 con 0x561ac0b1aa30 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.440+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83b8038040 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.441+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c8033040 con 0x7f83b8038040 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.441+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1f420 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.441+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.441+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83c00b9070 2024-04-27T03:30:54.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.441+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83b8009050 con 0x7f83c00b9070 2024-04-27T03:30:54.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.441+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1aa30 2024-04-27T03:30:54.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.441+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.441+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83b8038040 2024-04-27T03:30:54.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.442+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.442+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1f420 2024-04-27T03:30:54.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.442+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.442+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83b8038040 2024-04-27T03:30:54.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.442+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.442+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83c00b9070 2024-04-27T03:30:54.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.443+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83b8009050 con 0x7f83c00b9070 2024-04-27T03:30:54.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.443+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1f420 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.443+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.443+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83b8038040 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.443+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.443+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x561ac0b1aa30 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b1e6c0 con 0x7f83c00b9070 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f83b8045040 con 0x7f83c00b9070 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:54.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b226e0 con 0x561ac0b1aa30 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b22ad0 con 0x561ac0b1aa30 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.444+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b22ad0 con 0x7f83b8038040 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d5016800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d5016800 20 zone default found 2024-04-27T03:30:54.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d5016800 4 Realm: () 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d5016800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d5016800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d5016800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d5016800 20 zonegroup default 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d5016800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b23bd0 con 0x7f83b8038040 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d5016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d5016800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.445+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b255b0 con 0x561ac0b1f420 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.446+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.446+0000 7f83d5016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.446+0000 7f83d5016800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.446+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b25950 con 0x561ac0b1f420 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.446+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.446+0000 7f83d5016800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.446+0000 7f83d5016800 20 started sync module instance, tier type = 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.446+0000 7f83d5016800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.446+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b24a50 con 0x7f83c00b9070 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.447+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83b8045040 con 0x7f83c00b9070 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.447+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94672902047136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b25cf0 con 0x7f83c00b9070 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.447+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b26580 con 0x561ac0b1aa30 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.448+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94672902047136] v25'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83b8045040 con 0x7f83c00b9070 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.448+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.448+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94672902049392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b26970 con 0x561ac0b1aa30 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.448+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b273b0 con 0x561ac0b1f420 2024-04-27T03:30:54.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.450+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94672902049392] v25'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.450+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.450+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94672902057600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b277a0 con 0x561ac0b1f420 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.450+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b281e0 con 0x7f83c00b9070 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.451+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94672902057600] v25'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.451+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83b8045040 con 0x7f83c00b9070 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.451+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94672902054032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b28a80 con 0x7f83c00b9070 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.451+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b29440 con 0x561ac0b1aa30 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.452+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.452+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94672902066544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b29c20 con 0x561ac0b1aa30 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.452+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b2a740 con 0x7f83b8038040 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.452+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94672902054032] v25'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83b8045040 con 0x7f83c00b9070 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.453+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.453+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94672902070800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b2a700 con 0x7f83b8038040 2024-04-27T03:30:54.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.453+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94672902066544] v25'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.453+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b296b0 con 0x7f83b8038040 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.454+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94672902070800] v25'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.454+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.454+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94672902063568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b2aca0 con 0x7f83b8038040 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.454+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b2b660 con 0x7f83b8038040 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.455+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(58 notify.7 [create] v25'183 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.455+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94672902074800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b2ba80 con 0x7f83b8038040 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.455+0000 7f83d5016800 20 add_watcher() i=0 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.455+0000 7f83d5016800 20 add_watcher() i=1 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.455+0000 7f83d5016800 20 add_watcher() i=2 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.455+0000 7f83d5016800 20 add_watcher() i=3 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.455+0000 7f83d5016800 20 add_watcher() i=4 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.455+0000 7f83d5016800 20 add_watcher() i=5 2024-04-27T03:30:54.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.455+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(57 notify.6 [watch watch cookie 94672902063568] v25'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.455+0000 7f83d5016800 20 add_watcher() i=6 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.456+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94672902074800] v25'184 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.456+0000 7f83d5016800 20 add_watcher() i=7 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.456+0000 7f83d5016800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.456+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b29670 con 0x561ac0b1aa30 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.456+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.456+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94672902075856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b29670 con 0x561ac0b1aa30 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.457+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94672902075856] v25'195 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.458+0000 7f82867e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.458+0000 7f83d5016800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.458+0000 7f83d5016800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.458+0000 7f83d5016800 5 note: GC not initialized 2024-04-27T03:30:54.473 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.458+0000 7f82857e2640 20 reqs_thread_entry: start 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.458+0000 7f83d5016800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.458+0000 7f83d5016800 20 Filter name: none 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.458+0000 7f81f6ffd640 20 reqs_thread_entry: start 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.458+0000 7f83d5016800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.458+0000 7f83d5016800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.458+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b61d60 con 0x561ac0b1aa30 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.459+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~403 out=403b,getxattrs out=75b] v0'0 uv14 ondisk = 0) ==== 231+0+526 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.459+0000 7f83d5016800 20 rados_obj.operate() r=0 bl.length=403 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.459+0000 7f83d5016800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.459+0000 7f83d5016800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.459+0000 7f83d5016800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.459+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.18 8:1e6ca5ed:users.uid::foo:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~391 in=391b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b67ee0 con 0x561ac0b1aa30 2024-04-27T03:30:54.474 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 foo [delete,create,call,call,writefull 0~391] v25'15 uv15 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f83d5016800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f83d5016800 10 moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f83d5016800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:30:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94672902322512 in=579b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b637e0 con 0x561ac0b1aa30 2024-04-27T03:30:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify (1) cookie 94672902066544 notify 107374182435 ret 0) ==== 609+0+0 (crc 0 0 0) 0x7f83c801e040 con 0x561ac0b1aa30 2024-04-27T03:30:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f82a77fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 107374182435 cookie 94672902066544 notifier 4651 bl.length()=567 2024-04-27T03:30:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(64 notify.4 [notify cookie 94672902322512 out=8b] v0'0 uv183 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f82a77fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+foo info.flags=0x17 2024-04-27T03:30:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f82a77fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f82a77fe640 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x7f8294003ea0 con 0x561ac0b1aa30 2024-04-27T03:30:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== watch-notify(notify_complete (2) cookie 94672902322512 notify 107374182435 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f83c8018db0 con 0x561ac0b1aa30 2024-04-27T03:30:54.475 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.460+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.476 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"foo","display_name":"Foo","email":"foo@foo.com","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"foo","access_key":"9te6NH5mcdcq0Tc5i8i1","secret_key":"Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu","active":true,"create_date":"2024-04-27T03:30:30.404920Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:30:30.404907Z","tags":[],"group_ids":[]} 2024-04-27T03:30:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.461+0000 7f82867e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.461+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94672902075856] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b67180 con 0x561ac0b1aa30 2024-04-27T03:30:54.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.462+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94672902075856] v25'196 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.462+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94672902047136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b2b5d0 con 0x7f83c00b9070 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.463+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94672902047136] v25'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83b8045040 con 0x7f83c00b9070 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.463+0000 7f83d5016800 20 remove_watcher() i=0 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.463+0000 7f83d5016800 2 removed watcher, disabling cache 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.463+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94672902049392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b245a0 con 0x561ac0b1aa30 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.465+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94672902049392] v25'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.465+0000 7f83d5016800 20 remove_watcher() i=1 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.465+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94672902057600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b24e70 con 0x561ac0b1f420 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.466+0000 7f83d075c640 1 -- 172.21.15.165:0/1356740060 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94672902057600] v25'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c00b6030 con 0x561ac0b1f420 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.466+0000 7f83d5016800 20 remove_watcher() i=2 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.466+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94672902054032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b26e80 con 0x7f83c00b9070 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.467+0000 7f83c7fff640 1 -- 172.21.15.165:0/1356740060 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94672902054032] v25'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83b8045040 con 0x7f83c00b9070 2024-04-27T03:30:54.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.467+0000 7f83d5016800 20 remove_watcher() i=3 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.467+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94672902066544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b26e80 con 0x561ac0b1aa30 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.468+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94672902066544] v25'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x561ac0b1aa30 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.468+0000 7f83d5016800 20 remove_watcher() i=4 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.468+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94672902070800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b26e80 con 0x7f83b8038040 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.469+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94672902070800] v25'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.469+0000 7f83d5016800 20 remove_watcher() i=5 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.469+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94672902063568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b2a210 con 0x7f83b8038040 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.470+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94672902063568] v25'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.470+0000 7f83d5016800 20 remove_watcher() i=6 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.470+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94672902074800] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x561ac0b26e80 con 0x7f83b8038040 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94672902074800] v25'185 uv183 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f83c8031040 con 0x7f83b8038040 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 20 remove_watcher() i=7 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f83b8038040 msgr2=0x7f83b801bab0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f83b8038040 0x7f83b801bab0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561ac0b1aa30 msgr2=0x561ac0b1cef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561ac0b1aa30 0x561ac0b1cef0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f83c00b9070 msgr2=0x7f83c00a0990 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f83c00b9070 0x7f83c00a0990 crc :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561ac0b1f420 msgr2=0x561ac0b21810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561ac0b1f420 0x561ac0b21810 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8298036900 msgr2=0x7f8298038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8298036900 0x7f8298038dc0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f83b8001b60 tx=0x7f83b8010000 comp rx=0 tx=0).stop 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 msgr2=0x561ac0b0f350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 0x561ac0b0f350 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f83c0081fd0 tx=0x7f83c0059f40 comp rx=0 tx=0).stop 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d0f5d640 1 -- 172.21.15.165:0/1356740060 reap_dead start 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 shutdown_connections 2024-04-27T03:30:54.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 --2- 172.21.15.165:0/1356740060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0b07f90 0x561ac0b0f350 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 >> 172.21.15.165:0/1356740060 conn(0x561ac0b04260 msgr2=0x561ac0b018c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 shutdown_connections 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.471+0000 7f83d5016800 1 -- 172.21.15.165:0/1356740060 wait complete. 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/3164564834 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f82a8036900 msgr2=0x7f82a8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 --2- 172.21.15.165:0/3164564834 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f82a8036900 0x7f82a8038dc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f83b800bf50 tx=0x7f83b800f000 comp rx=0 tx=0).stop 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/3164564834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0afb010 msgr2=0x561ac0afb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 --2- 172.21.15.165:0/3164564834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0afb010 0x561ac0afb430 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7f83c006f940 tx=0x7f83c006fbe0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/3164564834 shutdown_connections 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 --2- 172.21.15.165:0/3164564834 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f82a8036900 0x7f82a8038dc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 --2- 172.21.15.165:0/3164564834 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac0afb010 0x561ac0afb430 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/3164564834 >> 172.21.15.165:0/3164564834 conn(0x561ac0aef0b0 msgr2=0x561ac0aef4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/3164564834 shutdown_connections 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/3164564834 wait complete. 2024-04-27T03:30:54.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561ac0aea000 msgr2=0x561ac0aec450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x561ac0aea000 0x561ac0aec450 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561ac0ae13f0 msgr2=0x561ac0ae37e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x561ac0ae13f0 0x561ac0ae37e0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561ac0ae60c0 msgr2=0x561ac0ae84b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x561ac0ae60c0 0x561ac0ae84b0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f82b8036900 msgr2=0x7f82b8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f82b8036900 0x7f82b8038dc0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f83b8000f60 tx=0x7f83b80012b0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e4120 msgr2=0x561ac0adc210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 --2- 172.21.15.165:0/569759662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x561ac09e4120 0x561ac0adc210 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7f83c0055040 tx=0x7f83c0002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d0f5d640 1 -- 172.21.15.165:0/569759662 reap_dead start 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 shutdown_connections 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 >> 172.21.15.165:0/569759662 conn(0x561ac09de550 msgr2=0x561ac0ace640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 shutdown_connections 2024-04-27T03:30:54.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.472+0000 7f83d5016800 1 -- 172.21.15.165:0/569759662 wait complete. 2024-04-27T03:30:54.487 DEBUG:tasks.util.rgw: json result: {'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True, 'create_date': '2024-04-27T03:30:30.404920Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:30:30.404907Z', 'tags': [], 'group_ids': []} 2024-04-27T03:30:54.500 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:54.500 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo'] 2024-04-27T03:30:54.500 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo'] 2024-04-27T03:30:54.500 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo 2024-04-27T03:30:54.543 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:30:54.543 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:30:54.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.538+0000 7f81a9498800 1 Processor -- start 2024-04-27T03:30:54.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.538+0000 7f81a9498800 1 -- start start 2024-04-27T03:30:54.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.538+0000 7f81a9498800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x56138e0e9d20 0x56138e0ea140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.538+0000 7f81a9498800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56138dfa9d40 con 0x56138e0eaf80 2024-04-27T03:30:54.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.538+0000 7f81a9498800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x56138df7c4d0 con 0x56138e0e9d20 2024-04-27T03:30:54.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.539+0000 7f81a6f9e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x56138e0e9d20 0x56138e0ea140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.539+0000 7f81a679d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x56138e0eaf80 0x56138e0eb400 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:38104/0 (socket says 172.21.15.165:38104) 2024-04-27T03:30:54.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.539+0000 7f81a6f9e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x56138e0e9d20 0x56138e0ea140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51586/0 (socket says 172.21.15.165:51586) 2024-04-27T03:30:54.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.539+0000 7f81a679d640 1 -- 172.21.15.165:0/23633746 learned_addr learned my addr 172.21.15.165:0/23633746 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:30:54.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.539+0000 7f81a6f9e640 1 -- 172.21.15.165:0/23633746 >> v1:172.21.15.89:6789/0 conn(0x56138e0eaf80 legacy=0x56138e0eb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.539+0000 7f81a6f9e640 1 -- 172.21.15.165:0/23633746 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138dfa8590 con 0x56138e0e9d20 2024-04-27T03:30:54.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.541+0000 7f81a6f9e640 1 --2- 172.21.15.165:0/23633746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e0ea140 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f819800a470 tx=0x7f819801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28391834d7e4893b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.541+0000 7f81a5f9c640 1 -- 172.21.15.165:0/23633746 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8198057070 con 0x56138e0e9d20 2024-04-27T03:30:54.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.541+0000 7f81a5f9c640 1 -- 172.21.15.165:0/23633746 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8198052570 con 0x56138e0e9d20 2024-04-27T03:30:54.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.541+0000 7f81a5f9c640 1 -- 172.21.15.165:0/23633746 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8198052870 con 0x56138e0e9d20 2024-04-27T03:30:54.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.541+0000 7f81a9498800 1 -- 172.21.15.165:0/23633746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 msgr2=0x56138e0ea140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.541+0000 7f81a9498800 1 --2- 172.21.15.165:0/23633746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e0ea140 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f819800a470 tx=0x7f819801ce90 comp rx=0 tx=0).stop 2024-04-27T03:30:54.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.541+0000 7f81a9498800 1 -- 172.21.15.165:0/23633746 shutdown_connections 2024-04-27T03:30:54.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.541+0000 7f81a9498800 1 --2- 172.21.15.165:0/23633746 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e0ea140 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.541+0000 7f81a9498800 1 -- 172.21.15.165:0/23633746 >> 172.21.15.165:0/23633746 conn(0x56138e0e5380 msgr2=0x56138e0e77a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.541+0000 7f81a9498800 1 -- 172.21.15.165:0/23633746 shutdown_connections 2024-04-27T03:30:54.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.541+0000 7f81a9498800 1 -- 172.21.15.165:0/23633746 wait complete. 2024-04-27T03:30:54.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.553+0000 7f81a9498800 1 Processor -- start 2024-04-27T03:30:54.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.553+0000 7f81a9498800 1 -- start start 2024-04-27T03:30:54.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.553+0000 7f81a9498800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e05dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.553+0000 7f81a9498800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56138e0363c0 con 0x56138e0e9d20 2024-04-27T03:30:54.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.554+0000 7f81a6f9e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e05dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.554+0000 7f81a6f9e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e05dff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51590/0 (socket says 172.21.15.165:51590) 2024-04-27T03:30:54.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.554+0000 7f81a6f9e640 1 -- 172.21.15.165:0/3088374390 learned_addr learned my addr 172.21.15.165:0/3088374390 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.554+0000 7f81a6f9e640 1 -- 172.21.15.165:0/3088374390 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138e1df270 con 0x56138e0e9d20 2024-04-27T03:30:54.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.554+0000 7f81a6f9e640 1 --2- 172.21.15.165:0/3088374390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e05dff0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8198055040 tx=0x7f819804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff481d7d31f396c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.554+0000 7f8094e09640 1 -- 172.21.15.165:0/3088374390 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8198057070 con 0x56138e0e9d20 2024-04-27T03:30:54.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.554+0000 7f8094e09640 1 -- 172.21.15.165:0/3088374390 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8198063070 con 0x56138e0e9d20 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.554+0000 7f8094e09640 1 -- 172.21.15.165:0/3088374390 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8198052720 con 0x56138e0e9d20 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.555+0000 7f81a9498800 1 -- 172.21.15.165:0/3088374390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 msgr2=0x56138e05dff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.555+0000 7f81a9498800 1 --2- 172.21.15.165:0/3088374390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e05dff0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f8198055040 tx=0x7f819804c000 comp rx=0 tx=0).stop 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.555+0000 7f81a9498800 1 -- 172.21.15.165:0/3088374390 shutdown_connections 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.555+0000 7f81a9498800 1 --2- 172.21.15.165:0/3088374390 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e05dff0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.555+0000 7f81a9498800 1 -- 172.21.15.165:0/3088374390 >> 172.21.15.165:0/3088374390 conn(0x56138e0e5380 msgr2=0x56138e1d6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.555+0000 7f81a9498800 1 -- 172.21.15.165:0/3088374390 shutdown_connections 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.555+0000 7f81a9498800 1 -- 172.21.15.165:0/3088374390 wait complete. 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.555+0000 7f81a9498800 1 Processor -- start 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.555+0000 7f81a9498800 1 -- start start 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.556+0000 7f81a9498800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e1e3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.556+0000 7f81a9498800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56138e0f33f0 con 0x56138e0e9d20 2024-04-27T03:30:54.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.556+0000 7f81a6f9e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e1e3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.556+0000 7f81a6f9e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e1e3ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51598/0 (socket says 172.21.15.165:51598) 2024-04-27T03:30:54.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.556+0000 7f81a6f9e640 1 -- 172.21.15.165:0/4182386196 learned_addr learned my addr 172.21.15.165:0/4182386196 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.556+0000 7f81a6f9e640 1 -- 172.21.15.165:0/4182386196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138e1e43e0 con 0x56138e0e9d20 2024-04-27T03:30:54.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.556+0000 7f81a6f9e640 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e1e3ea0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8198055040 tx=0x7f8198002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.556+0000 7f8093606640 1 -- 172.21.15.165:0/4182386196 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8198052cd0 con 0x56138e0e9d20 2024-04-27T03:30:54.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.557+0000 7f8093606640 1 -- 172.21.15.165:0/4182386196 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819806c070 con 0x56138e0e9d20 2024-04-27T03:30:54.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.557+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56138e1e4670 con 0x56138e0e9d20 2024-04-27T03:30:54.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.557+0000 7f8093606640 1 -- 172.21.15.165:0/4182386196 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f819805e480 con 0x56138e0e9d20 2024-04-27T03:30:54.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.557+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56138e1e7320 con 0x56138e0e9d20 2024-04-27T03:30:54.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.557+0000 7f8093606640 1 -- 172.21.15.165:0/4182386196 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f819805e620 con 0x56138e0e9d20 2024-04-27T03:30:54.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.557+0000 7f8093606640 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f808c036900 0x7f808c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.558+0000 7f8093606640 1 -- 172.21.15.165:0/4182386196 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f819808e650 con 0x56138e0e9d20 2024-04-27T03:30:54.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.558+0000 7f81a679d640 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f808c036900 0x7f808c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.558+0000 7f81a9498800 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56138e0eaf80 0x56138e1eb0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.558+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e1eb6b0 con 0x56138e0eaf80 2024-04-27T03:30:54.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.558+0000 7f81a779f640 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56138e0eaf80 0x56138e1eb0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.558+0000 7f81a679d640 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f808c036900 0x7f808c038dc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f819c001d90 tx=0x7f819c0012f0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.558+0000 7f81a779f640 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56138e0eaf80 0x56138e1eb0e0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.558+0000 7f81a779f640 1 -- 172.21.15.165:0/4182386196 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e0eaf80 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.558+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e1ebfa0 con 0x56138e0eaf80 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.559+0000 7f81a779f640 1 -- 172.21.15.165:0/4182386196 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e0eaf80 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.559+0000 7f81a9498800 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56138e1ed8b0 0x56138e1efd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.559+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e1f04d0 con 0x56138e1ed8b0 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.559+0000 7f81a6f9e640 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56138e1ed8b0 0x56138e1efd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.559+0000 7f81a6f9e640 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56138e1ed8b0 0x56138e1efd40 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.559+0000 7f81a6f9e640 1 -- 172.21.15.165:0/4182386196 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f819808cd70 con 0x56138e1ed8b0 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.559+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e1ec820 con 0x56138e0eaf80 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.560+0000 7f81a779f640 1 -- 172.21.15.165:0/4182386196 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e0eaf80 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.560+0000 7f81a9498800 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x56138e1f1980 0x56138e1f3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.560+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e1f4550 con 0x56138e1f1980 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.560+0000 7f81a679d640 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x56138e1f1980 0x56138e1f3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.560+0000 7f81a679d640 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x56138e1f1980 0x56138e1f3df0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.561+0000 7f81a679d640 1 -- 172.21.15.165:0/4182386196 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f819c009050 con 0x56138e1f1980 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.561+0000 7f81a9498800 1 Processor -- start 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.561+0000 7f81a9498800 1 -- start start 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.561+0000 7f81a9498800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e2029b0 0x56138e202dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.561+0000 7f81a9498800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56138dfdc6b0 con 0x56138e2029b0 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.561+0000 7f81a6f9e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e2029b0 0x56138e202dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.561+0000 7f81a6f9e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e2029b0 0x56138e202dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51612/0 (socket says 172.21.15.165:51612) 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.561+0000 7f81a6f9e640 1 -- 172.21.15.165:0/1897091642 learned_addr learned my addr 172.21.15.165:0/1897091642 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.561+0000 7f81a6f9e640 1 -- 172.21.15.165:0/1897091642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138e203310 con 0x56138e2029b0 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.562+0000 7f81a6f9e640 1 --2- 172.21.15.165:0/1897091642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e2029b0 0x56138e202dd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8198060040 tx=0x7f8198068f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.562+0000 7f8090e01640 1 -- 172.21.15.165:0/1897091642 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8198092020 con 0x56138e2029b0 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.562+0000 7f8090e01640 1 -- 172.21.15.165:0/1897091642 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8198058040 con 0x56138e2029b0 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.562+0000 7f81a9498800 1 -- 172.21.15.165:0/1897091642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56138e203d80 con 0x56138e2029b0 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.562+0000 7f8090e01640 1 -- 172.21.15.165:0/1897091642 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f819808eb60 con 0x56138e2029b0 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.562+0000 7f81a9498800 1 -- 172.21.15.165:0/1897091642 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56138e204280 con 0x56138e2029b0 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.563+0000 7f8090e01640 1 -- 172.21.15.165:0/1897091642 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f8198095020 con 0x56138e2029b0 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.563+0000 7f8090e01640 1 --2- 172.21.15.165:0/1897091642 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8080036900 0x7f8080038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.563+0000 7f8090e01640 1 -- 172.21.15.165:0/1897091642 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f8198098ab0 con 0x56138e2029b0 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.563+0000 7f81a679d640 1 --2- 172.21.15.165:0/1897091642 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8080036900 0x7f8080038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.563+0000 7f81a9498800 1 Processor -- start 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.563+0000 7f81a9498800 1 -- start start 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.563+0000 7f81a679d640 1 --2- 172.21.15.165:0/1897091642 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8080036900 0x7f8080038dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x56138e1e85d0 tx=0x7f819c00f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.563+0000 7f81a9498800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 0x56138e20ff30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.563+0000 7f81a9498800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56138e1df500 con 0x56138e20fb10 2024-04-27T03:30:54.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.563+0000 7f81a6f9e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 0x56138e20ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.563+0000 7f81a6f9e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 0x56138e20ff30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51620/0 (socket says 172.21.15.165:51620) 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.563+0000 7f81a6f9e640 1 -- 172.21.15.165:0/1832410639 learned_addr learned my addr 172.21.15.165:0/1832410639 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.564+0000 7f81a6f9e640 1 -- 172.21.15.165:0/1832410639 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138e210500 con 0x56138e20fb10 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.564+0000 7f81a6f9e640 1 --2- 172.21.15.165:0/1832410639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 0x56138e20ff30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f819807adf0 tx=0x7f819807ae20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfe975e1f0d18b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.564+0000 7f808a7fc640 1 -- 172.21.15.165:0/1832410639 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f81980600d0 con 0x56138e20fb10 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.564+0000 7f808a7fc640 1 -- 172.21.15.165:0/1832410639 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f819805d3e0 con 0x56138e20fb10 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.564+0000 7f808a7fc640 1 -- 172.21.15.165:0/1832410639 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8198060270 con 0x56138e20fb10 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.564+0000 7f81a9498800 1 -- 172.21.15.165:0/1832410639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 msgr2=0x56138e20ff30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.564+0000 7f81a9498800 1 --2- 172.21.15.165:0/1832410639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 0x56138e20ff30 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f819807adf0 tx=0x7f819807ae20 comp rx=0 tx=0).stop 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.564+0000 7f81a9498800 1 -- 172.21.15.165:0/1832410639 shutdown_connections 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.564+0000 7f81a9498800 1 --2- 172.21.15.165:0/1832410639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 0x56138e20ff30 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.564+0000 7f81a9498800 1 -- 172.21.15.165:0/1832410639 >> 172.21.15.165:0/1832410639 conn(0x56138e20be70 msgr2=0x56138e209ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.564+0000 7f81a9498800 1 -- 172.21.15.165:0/1832410639 shutdown_connections 2024-04-27T03:30:54.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.564+0000 7f81a9498800 1 -- 172.21.15.165:0/1832410639 wait complete. 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.565+0000 7f81a9498800 1 Processor -- start 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.565+0000 7f81a9498800 1 -- start start 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.565+0000 7f81a9498800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 0x56138e1dcba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.565+0000 7f81a9498800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56138e210bb0 con 0x56138e20fb10 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.565+0000 7f81a6f9e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 0x56138e1dcba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.565+0000 7f81a6f9e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 0x56138e1dcba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:51632/0 (socket says 172.21.15.165:51632) 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.565+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 learned_addr learned my addr 172.21.15.165:0/2165491914 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.565+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56138e1db290 con 0x56138e20fb10 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.566+0000 7f81a6f9e640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 0x56138e1dcba0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8198067b90 tx=0x7f8198076f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.566+0000 7f8088ff9640 1 -- 172.21.15.165:0/2165491914 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f819805d590 con 0x56138e20fb10 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.566+0000 7f8088ff9640 1 -- 172.21.15.165:0/2165491914 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f81980bb070 con 0x56138e20fb10 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.566+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56138e1db290 con 0x56138e20fb10 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.566+0000 7f8088ff9640 1 -- 172.21.15.165:0/2165491914 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f819807bcf0 con 0x56138e20fb10 2024-04-27T03:30:54.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.566+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56138e1db290 con 0x56138e20fb10 2024-04-27T03:30:54.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.566+0000 7f8088ff9640 1 -- 172.21.15.165:0/2165491914 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f819805f9a0 con 0x56138e20fb10 2024-04-27T03:30:54.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.567+0000 7f8088ff9640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8070036900 0x7f8070038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.567+0000 7f8088ff9640 1 -- 172.21.15.165:0/2165491914 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(25..25 src has 1..25) ==== 5706+0+0 (secure 0 0 0) 0x7f81980900d0 con 0x56138e20fb10 2024-04-27T03:30:54.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.567+0000 7f81a679d640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8070036900 0x7f8070038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.567+0000 7f81a679d640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8070036900 0x7f8070038dc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x56138e1f4950 tx=0x7f819c010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.567+0000 7f81a9498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.567+0000 7f81a9498800 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56138e224890 0x56138e226d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.567+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e2274a0 con 0x56138e224890 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.567+0000 7f81a779f640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56138e224890 0x56138e226d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a779f640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56138e224890 0x56138e226d50 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e224890 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a9498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a9498800 20 realm 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a9498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e2278b0 con 0x56138e224890 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e224890 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a9498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a9498800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a9498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e228180 con 0x56138e224890 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e224890 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a9498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a9498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.568+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e224890 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.569+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e224890 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.569+0000 7f81a9498800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.569+0000 7f81a9498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.569+0000 7f81a9498800 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56138e229280 0x56138e22b670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.569+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e22bc80 con 0x56138e229280 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.569+0000 7f81a6f9e640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56138e229280 0x56138e22b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.569+0000 7f81a6f9e640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56138e229280 0x56138e22b670 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.569+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.569+0000 7f81a9498800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.569+0000 7f81a9498800 20 searching for the correct realm 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.569+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e22d5b0 con 0x56138e229280 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.570+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.570+0000 7f81a6f9e640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f81980a2040 0x7f819808e050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.570+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e22d5b0 con 0x7f81980a2040 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.570+0000 7f81a679d640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f81980a2040 0x7f819808e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.570+0000 7f81a679d640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f81980a2040 0x7f819808e050 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.570+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.570+0000 7f81a679d640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f819c037040 0x7f819c036410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.570+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e22d5b0 con 0x7f819c037040 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.571+0000 7f81a779f640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f819c037040 0x7f819c036410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.571+0000 7f81a779f640 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f819c037040 0x7f819c036410 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.571+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(8 [pgnls start_epoch 25 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f81a000a4a0 con 0x7f819c037040 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.571+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e224890 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.571+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e224890 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.572+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e224890 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.572+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e224890 2024-04-27T03:30:54.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.572+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f819c037040 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.572+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a000a4a0 con 0x7f819c037040 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.572+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f819c037040 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.572+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(12 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a000a4a0 con 0x7f819c037040 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.572+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f81980a2040 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.573+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(13 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.573+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e229280 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.573+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.573+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e224890 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.573+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e224890 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.573+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f819c037040 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.574+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(16 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a000a4a0 con 0x7f819c037040 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.574+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e224890 2024-04-27T03:30:54.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.574+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e224890 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.574+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f81980a2040 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.574+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(18 [pgnls start_epoch 25 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.574+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e229280 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.574+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.574+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e229280 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.575+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.575+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f819c037040 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.575+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(21 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a000a4a0 con 0x7f819c037040 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.575+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e229280 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.575+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 25 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.575+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e229280 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.575+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.575+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e224890 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.576+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e224890 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.576+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e224890 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.576+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 25 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f81a000a4a0 con 0x56138e224890 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.576+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f819c037040 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.576+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(26 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a0033040 con 0x7f819c037040 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.576+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e229280 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.576+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.576+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f81980a2040 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.577+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(28 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.577+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e224890 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.577+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.577+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f819c037040 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.577+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(30 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.577+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e229280 2024-04-27T03:30:54.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.577+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.577+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f819c037040 2024-04-27T03:30:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.578+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(32 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.578+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f81980a2040 2024-04-27T03:30:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.578+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(33 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.578+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e229280 2024-04-27T03:30:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.578+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.578+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f819c037040 2024-04-27T03:30:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.578+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(35 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.578+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x56138e224890 2024-04-27T03:30:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 25 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 25 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e25) -- 0x56138e228520 con 0x7f81980a2040 2024-04-27T03:30:54.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(37 [pgnls start_epoch 25 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:30:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:30:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:30:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:30:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:30:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:30:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e22c540 con 0x56138e224890 2024-04-27T03:30:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e22c930 con 0x56138e224890 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.579+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e22c930 con 0x7f819c037040 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 20 zone default found 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 4 Realm: () 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 20 zonegroup default 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e22c0c0 con 0x7f819c037040 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e22c110 con 0x56138e229280 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.580+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e22f520 con 0x56138e229280 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.581+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.581+0000 7f81a9498800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.581+0000 7f81a9498800 20 started sync module instance, tier type = 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.581+0000 7f81a9498800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.581+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e22e8b0 con 0x7f81980a2040 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.582+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(44 notify.0 [create] v25'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.582+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94641989020832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e22fe30 con 0x7f81980a2040 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.582+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e230680 con 0x56138e224890 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.583+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(45 notify.0 [watch watch cookie 94641989020832] v25'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.583+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v25'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.583+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94641989024128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e230720 con 0x56138e224890 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.583+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e231240 con 0x56138e229280 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.584+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94641989024128] v25'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.584+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v25'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.584+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94641989029136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e231630 con 0x56138e229280 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.584+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e232070 con 0x7f81980a2040 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.585+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(50 notify.3 [create] v25'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.585+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94641989035104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e232d30 con 0x7f81980a2040 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.585+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e233780 con 0x56138e224890 2024-04-27T03:30:54.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.585+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94641989029136] v25'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.586+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(51 notify.3 [watch watch cookie 94641989035104] v25'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.586+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v25'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.586+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94641989036160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2301d0 con 0x56138e224890 2024-04-27T03:30:54.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.586+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e234190 con 0x7f819c037040 2024-04-27T03:30:54.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.587+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(54 notify.5 [create] v25'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.587+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94641989041376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2345b0 con 0x7f819c037040 2024-04-27T03:30:54.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.587+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94641989036160] v25'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.587+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e234ff0 con 0x7f819c037040 2024-04-27T03:30:54.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.588+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(55 notify.5 [watch watch cookie 94641989041376] v25'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.589+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(56 notify.6 [create] v25'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.589+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94641989044928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e234100 con 0x7f819c037040 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.589+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2358c0 con 0x7f819c037040 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.590+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(57 notify.6 [watch watch cookie 94641989044928] v25'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.590+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(58 notify.7 [create] v25'186 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.590+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94641989038704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e234100 con 0x7f819c037040 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.590+0000 7f81a9498800 20 add_watcher() i=0 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.590+0000 7f81a9498800 20 add_watcher() i=1 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.590+0000 7f81a9498800 20 add_watcher() i=2 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.590+0000 7f81a9498800 20 add_watcher() i=3 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.590+0000 7f81a9498800 20 add_watcher() i=4 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.590+0000 7f81a9498800 20 add_watcher() i=5 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.590+0000 7f81a9498800 20 add_watcher() i=6 2024-04-27T03:30:54.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.591+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(59 notify.7 [watch watch cookie 94641989038704] v25'187 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.591+0000 7f81a9498800 20 add_watcher() i=7 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.591+0000 7f81a9498800 2 all 8 watchers are set, enabling cache 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.591+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e2388f0 con 0x56138e224890 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.591+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.591+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94641989047232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e235d60 con 0x56138e224890 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.592+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94641989047232] v25'197 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.592+0000 7f805d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.592+0000 7f81a9498800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.592+0000 7f81a9498800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.592+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2411b0 con 0x56138e224890 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.593+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v25'55 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.593+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e241880 con 0x7f819c037040 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.594+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(63 gc.1 [create,call,call,call] v25'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.595+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e241ca0 con 0x56138e229280 2024-04-27T03:30:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.595+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v25'32 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.595+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e241bf0 con 0x56138e224890 2024-04-27T03:30:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.596+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v25'198 uv172 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.596+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e242280 con 0x7f81980a2040 2024-04-27T03:30:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.597+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(66 gc.4 [create,call,call,call] v25'71 uv59 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.597+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e242ca0 con 0x56138e229280 2024-04-27T03:30:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.598+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v25'57 uv49 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.598+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e242ca0 con 0x56138e224890 2024-04-27T03:30:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.600+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v25'33 uv29 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.600+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e242ca0 con 0x7f819c037040 2024-04-27T03:30:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.601+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(69 gc.7 [create,call,call,call] v25'49 uv42 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.601+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e242d30 con 0x7f81980a2040 2024-04-27T03:30:54.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.602+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(70 gc.8 [create,call,call,call] v25'25 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.602+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2432f0 con 0x56138e229280 2024-04-27T03:30:54.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.603+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v25'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.603+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243380 con 0x7f81980a2040 2024-04-27T03:30:54.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.604+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(72 gc.10 [create,call,call,call] v25'72 uv61 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.604+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x7f819c037040 2024-04-27T03:30:54.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.605+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(73 gc.11 [create,call,call,call] v25'28 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.605+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x7f81980a2040 2024-04-27T03:30:54.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.606+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(74 gc.12 [create,call,call,call] v25'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.606+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x7f819c037040 2024-04-27T03:30:54.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.607+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(75 gc.13 [create,call,call,call] v25'61 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.607+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2437a0 con 0x56138e224890 2024-04-27T03:30:54.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.608+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v25'199 uv168 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.608+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2437a0 con 0x56138e229280 2024-04-27T03:30:54.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.610+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v25'66 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.610+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2437a0 con 0x7f81980a2040 2024-04-27T03:30:54.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.611+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(78 gc.16 [create,call,call,call] v25'73 uv57 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.611+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x56138e229280 2024-04-27T03:30:54.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.612+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v25'37 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.612+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x56138e224890 2024-04-27T03:30:54.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.613+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v25'33 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.613+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x56138e224890 2024-04-27T03:30:54.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.614+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v25'200 uv170 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.614+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243860 con 0x56138e229280 2024-04-27T03:30:54.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.615+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v25'67 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.615+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243860 con 0x7f81980a2040 2024-04-27T03:30:54.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.616+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 gc.21 [create,call,call,call] v25'41 uv37 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.616+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2438f0 con 0x56138e229280 2024-04-27T03:30:54.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.617+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v25'33 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.617+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2438f0 con 0x56138e229280 2024-04-27T03:30:54.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.618+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v25'44 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.618+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2438c0 con 0x56138e229280 2024-04-27T03:30:54.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.619+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v25'27 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.619+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x56138e229280 2024-04-27T03:30:54.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.620+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v25'58 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.620+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x7f819c037040 2024-04-27T03:30:54.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.621+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(88 gc.26 [create,call,call,call] v25'35 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.621+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x56138e229280 2024-04-27T03:30:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.622+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v25'37 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.622+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x7f819c037040 2024-04-27T03:30:54.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.623+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(90 gc.28 [create,call,call,call] v25'50 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.623+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x56138e229280 2024-04-27T03:30:54.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.624+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v25'68 uv56 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.625+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x7f819c037040 2024-04-27T03:30:54.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.626+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(92 gc.30 [create,call,call,call] v25'62 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.626+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e243830 con 0x7f81980a2040 2024-04-27T03:30:54.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.627+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(93 gc.31 [create,call,call,call] v25'31 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.627+0000 7f7fd7fff640 20 reqs_thread_entry: start 2024-04-27T03:30:54.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.627+0000 7f81a9498800 20 init_complete bucket index max shards: 11 2024-04-27T03:30:54.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.627+0000 7f81a9498800 20 Filter name: none 2024-04-27T03:30:54.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.627+0000 7f7fd5ffb640 20 reqs_thread_entry: start 2024-04-27T03:30:54.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.627+0000 7f81a9498800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:30:54.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.627+0000 7f81a9498800 20 rados->read ofs=0 len=0 2024-04-27T03:30:54.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.627+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e26e7b0 con 0x56138e224890 2024-04-27T03:30:54.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.628+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(94 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.628+0000 7f81a9498800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:30:54.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.628+0000 7f81a9498800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:30:54.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.628+0000 7f81a9498800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:30:54.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.628+0000 7f81a9498800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:30:54.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.628+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e25) -- 0x56138e26e720 con 0x7f81980a2040 2024-04-27T03:30:54.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.628+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv41 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.635 INFO:teuthology.orchestra.run.smithi165.stderr:could not remove user: unable to remove user, must specify purge data to remove user with buckets 2024-04-27T03:30:54.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.629+0000 7f805d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:30:54.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.629+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:96 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94641989047232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e240070 con 0x56138e224890 2024-04-27T03:30:54.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.630+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(96 data_loggenerations_metadata [watch unwatch cookie 94641989047232] v25'201 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.630+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:97 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94641989020832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2353c0 con 0x7f81980a2040 2024-04-27T03:30:54.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.631+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(97 notify.0 [watch unwatch cookie 94641989020832] v25'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.631+0000 7f81a9498800 20 remove_watcher() i=0 2024-04-27T03:30:54.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.631+0000 7f81a9498800 2 removed watcher, disabling cache 2024-04-27T03:30:54.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.631+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94641989024128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2353c0 con 0x56138e224890 2024-04-27T03:30:54.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.632+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(98 notify.1 [watch unwatch cookie 94641989024128] v25'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.632+0000 7f81a9498800 20 remove_watcher() i=1 2024-04-27T03:30:54.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.632+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:99 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94641989029136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e22f980 con 0x56138e229280 2024-04-27T03:30:54.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.633+0000 7f81a6f9e640 1 -- 172.21.15.165:0/2165491914 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(99 notify.2 [watch unwatch cookie 94641989029136] v25'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f819808cd70 con 0x56138e229280 2024-04-27T03:30:54.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.633+0000 7f81a9498800 20 remove_watcher() i=2 2024-04-27T03:30:54.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.633+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:100 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94641989035104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2353c0 con 0x7f81980a2040 2024-04-27T03:30:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.634+0000 7f81a679d640 1 -- 172.21.15.165:0/2165491914 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(100 notify.3 [watch unwatch cookie 94641989035104] v25'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f819c009050 con 0x7f81980a2040 2024-04-27T03:30:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.634+0000 7f81a9498800 20 remove_watcher() i=3 2024-04-27T03:30:54.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.634+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:101 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94641989036160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e230d10 con 0x56138e224890 2024-04-27T03:30:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.635+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(101 notify.4 [watch unwatch cookie 94641989036160] v25'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x56138e224890 2024-04-27T03:30:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.635+0000 7f81a9498800 20 remove_watcher() i=4 2024-04-27T03:30:54.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.635+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94641989041376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e2353c0 con 0x7f819c037040 2024-04-27T03:30:54.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.636+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(102 notify.5 [watch unwatch cookie 94641989041376] v25'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.636+0000 7f81a9498800 20 remove_watcher() i=5 2024-04-27T03:30:54.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.636+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:103 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94641989044928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e230d10 con 0x7f819c037040 2024-04-27T03:30:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.637+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(103 notify.6 [watch unwatch cookie 94641989044928] v25'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.637+0000 7f81a9498800 20 remove_watcher() i=6 2024-04-27T03:30:54.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.637+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:104 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94641989038704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e25) -- 0x56138e233ce0 con 0x7f819c037040 2024-04-27T03:30:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(104 notify.7 [watch unwatch cookie 94641989038704] v25'188 uv186 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f81a0031040 con 0x7f819c037040 2024-04-27T03:30:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 20 remove_watcher() i=7 2024-04-27T03:30:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f819c037040 msgr2=0x7f819c036410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f819c037040 0x7f819c036410 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56138e224890 msgr2=0x56138e226d50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56138e224890 0x56138e226d50 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f81980a2040 msgr2=0x7f819808e050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f81980a2040 0x7f819808e050 crc :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56138e229280 msgr2=0x56138e22b670 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56138e229280 0x56138e22b670 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8070036900 msgr2=0x7f8070038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8070036900 0x7f8070038dc0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x56138e1f4950 tx=0x7f819c010000 comp rx=0 tx=0).stop 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 msgr2=0x56138e1dcba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 --2- 172.21.15.165:0/2165491914 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e20fb10 0x56138e1dcba0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f8198067b90 tx=0x7f8198076f20 comp rx=0 tx=0).stop 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 reap_dead start 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a779f640 1 -- 172.21.15.165:0/2165491914 reap_dead start 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 shutdown_connections 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.638+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 >> 172.21.15.165:0/2165491914 conn(0x56138e20be70 msgr2=0x56138e2094d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 shutdown_connections 2024-04-27T03:30:54.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/2165491914 wait complete. 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/1897091642 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8080036900 msgr2=0x7f8080038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 --2- 172.21.15.165:0/1897091642 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8080036900 0x7f8080038dc0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x56138e1e85d0 tx=0x7f819c00f000 comp rx=0 tx=0).stop 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/1897091642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e2029b0 msgr2=0x56138e202dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 --2- 172.21.15.165:0/1897091642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e2029b0 0x56138e202dd0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f8198060040 tx=0x7f8198068f10 comp rx=0 tx=0).stop 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/1897091642 shutdown_connections 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 --2- 172.21.15.165:0/1897091642 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8080036900 0x7f8080038dc0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 --2- 172.21.15.165:0/1897091642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e2029b0 0x56138e202dd0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/1897091642 >> 172.21.15.165:0/1897091642 conn(0x56138e1f6a50 msgr2=0x56138e1f6e50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/1897091642 shutdown_connections 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/1897091642 wait complete. 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x56138e1f1980 msgr2=0x56138e1f3df0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x56138e1f1980 0x56138e1f3df0 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56138e0eaf80 msgr2=0x56138e1eb0e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56138e0eaf80 0x56138e1eb0e0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56138e1ed8b0 msgr2=0x56138e1efd40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56138e1ed8b0 0x56138e1efd40 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f808c036900 msgr2=0x7f808c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f808c036900 0x7f808c038dc0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f819c001d90 tx=0x7f819c0012f0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 msgr2=0x56138e1e3ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:30:54.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 --2- 172.21.15.165:0/4182386196 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56138e0e9d20 0x56138e1e3ea0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f8198055040 tx=0x7f8198002fb0 comp rx=0 tx=0).stop 2024-04-27T03:30:54.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a779f640 1 -- 172.21.15.165:0/4182386196 reap_dead start 2024-04-27T03:30:54.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 shutdown_connections 2024-04-27T03:30:54.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 >> 172.21.15.165:0/4182386196 conn(0x56138e0e5380 msgr2=0x56138e1d55e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:30:54.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 shutdown_connections 2024-04-27T03:30:54.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:30:54.639+0000 7f81a9498800 1 -- 172.21.15.165:0/4182386196 wait complete. 2024-04-27T03:30:54.653 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-27T03:30:54.667 INFO:tasks.radosgw_admin:RL: create_bucket myfoo foo bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:30:54.671 INFO:tasks.radosgw_admin:RL: put_obj myfoo foo bytes_out=6 bytes_in=0 failed=False 2024-04-27T03:31:29.672 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--categories', 'delete_obj'] 2024-04-27T03:31:29.672 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--categories', 'delete_obj'] 2024-04-27T03:31:29.672 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --categories delete_obj 2024-04-27T03:31:29.724 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.719+0000 7f476fe0e800 1 Processor -- start 2024-04-27T03:31:29.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.719+0000 7f476fe0e800 1 -- start start 2024-04-27T03:31:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.719+0000 7f476fe0e800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a4d92dd370 0x55a4d92df760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.719+0000 7f476fe0e800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a4d9197d40 con 0x55a4d92d7100 2024-04-27T03:31:29.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.719+0000 7f476fe0e800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55a4d916a4d0 con 0x55a4d92dd370 2024-04-27T03:31:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.719+0000 7f476d10c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a4d92dd370 0x55a4d92df760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.719+0000 7f476d90d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55a4d92d7100 0x55a4d92d7520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:54688/0 (socket says 172.21.15.165:54688) 2024-04-27T03:31:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.720+0000 7f476d90d640 1 -- 172.21.15.165:0/1871468383 learned_addr learned my addr 172.21.15.165:0/1871468383 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:31:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.720+0000 7f476d10c640 1 -- 172.21.15.165:0/1871468383 >> v1:172.21.15.89:6789/0 conn(0x55a4d92d7100 legacy=0x55a4d92d7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.720+0000 7f476c90b640 1 -- 172.21.15.165:0/1871468383 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2147140438 0 0) 0x55a4d9197d40 con 0x55a4d92d7100 2024-04-27T03:31:29.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.720+0000 7f476d10c640 1 -- 172.21.15.165:0/1871468383 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d9196590 con 0x55a4d92dd370 2024-04-27T03:31:29.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.722+0000 7f476d10c640 1 --2- 172.21.15.165:0/1871468383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92dd370 0x55a4d92df760 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f476400a470 tx=0x7f476401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed31e15c96c13fad server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.722+0000 7f476c90b640 1 -- 172.21.15.165:0/1871468383 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4764057070 con 0x55a4d92dd370 2024-04-27T03:31:29.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.722+0000 7f476c90b640 1 -- 172.21.15.165:0/1871468383 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4764052570 con 0x55a4d92dd370 2024-04-27T03:31:29.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.722+0000 7f476c90b640 1 -- 172.21.15.165:0/1871468383 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4764052870 con 0x55a4d92dd370 2024-04-27T03:31:29.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.723+0000 7f476fe0e800 1 -- 172.21.15.165:0/1871468383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92dd370 msgr2=0x55a4d92df760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.723+0000 7f476fe0e800 1 --2- 172.21.15.165:0/1871468383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92dd370 0x55a4d92df760 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7f476400a470 tx=0x7f476401ce90 comp rx=0 tx=0).stop 2024-04-27T03:31:29.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.723+0000 7f476fe0e800 1 -- 172.21.15.165:0/1871468383 shutdown_connections 2024-04-27T03:31:29.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.723+0000 7f476fe0e800 1 --2- 172.21.15.165:0/1871468383 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92dd370 0x55a4d92df760 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.723+0000 7f476fe0e800 1 -- 172.21.15.165:0/1871468383 >> 172.21.15.165:0/1871468383 conn(0x55a4d92d2b00 msgr2=0x55a4d92d4f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:29.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.723+0000 7f476fe0e800 1 -- 172.21.15.165:0/1871468383 shutdown_connections 2024-04-27T03:31:29.730 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.723+0000 7f476fe0e800 1 -- 172.21.15.165:0/1871468383 wait complete. 2024-04-27T03:31:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.736+0000 7f476fe0e800 1 Processor -- start 2024-04-27T03:31:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.737+0000 7f476fe0e800 1 -- start start 2024-04-27T03:31:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.737+0000 7f476fe0e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 0x55a4d92d9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.737+0000 7f476fe0e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a4d92243c0 con 0x55a4d92d7100 2024-04-27T03:31:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.737+0000 7f476d90d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 0x55a4d92d9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.737+0000 7f476d90d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 0x55a4d92d9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49202/0 (socket says 172.21.15.165:49202) 2024-04-27T03:31:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.737+0000 7f476d90d640 1 -- 172.21.15.165:0/2106468668 learned_addr learned my addr 172.21.15.165:0/2106468668 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.737+0000 7f476d90d640 1 -- 172.21.15.165:0/2106468668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d92d8040 con 0x55a4d92d7100 2024-04-27T03:31:29.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.738+0000 7f476d90d640 1 --2- 172.21.15.165:0/2106468668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 0x55a4d92d9a80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f476000b240 tx=0x7f476000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=821e94e3430983a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.738+0000 7f465be07640 1 -- 172.21.15.165:0/2106468668 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760008070 con 0x55a4d92d7100 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.738+0000 7f465be07640 1 -- 172.21.15.165:0/2106468668 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f476000d500 con 0x55a4d92d7100 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.738+0000 7f465be07640 1 -- 172.21.15.165:0/2106468668 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f476000d860 con 0x55a4d92d7100 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.738+0000 7f476fe0e800 1 -- 172.21.15.165:0/2106468668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 msgr2=0x55a4d92d9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.738+0000 7f476fe0e800 1 --2- 172.21.15.165:0/2106468668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 0x55a4d92d9a80 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7f476000b240 tx=0x7f476000b700 comp rx=0 tx=0).stop 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.739+0000 7f476fe0e800 1 -- 172.21.15.165:0/2106468668 shutdown_connections 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.739+0000 7f476fe0e800 1 --2- 172.21.15.165:0/2106468668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 0x55a4d92d9a80 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.739+0000 7f476fe0e800 1 -- 172.21.15.165:0/2106468668 >> 172.21.15.165:0/2106468668 conn(0x55a4d92d2b00 msgr2=0x55a4d93c5840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.739+0000 7f476fe0e800 1 -- 172.21.15.165:0/2106468668 shutdown_connections 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.739+0000 7f476fe0e800 1 -- 172.21.15.165:0/2106468668 wait complete. 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.739+0000 7f476fe0e800 1 Processor -- start 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.739+0000 7f476fe0e800 1 -- start start 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.739+0000 7f476fe0e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 0x55a4d93d3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.739+0000 7f476fe0e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a4d92dffa0 con 0x55a4d92d7100 2024-04-27T03:31:29.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.739+0000 7f476d90d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 0x55a4d93d3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.739+0000 7f476d90d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 0x55a4d93d3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49210/0 (socket says 172.21.15.165:49210) 2024-04-27T03:31:29.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.739+0000 7f476d90d640 1 -- 172.21.15.165:0/1890037706 learned_addr learned my addr 172.21.15.165:0/1890037706 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.740+0000 7f476d90d640 1 -- 172.21.15.165:0/1890037706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d93d4210 con 0x55a4d92d7100 2024-04-27T03:31:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.740+0000 7f476d90d640 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 0x55a4d93d3cd0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55a4d92d90b0 tx=0x7f4760000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.740+0000 7f465a604640 1 -- 172.21.15.165:0/1890037706 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760019870 con 0x55a4d92d7100 2024-04-27T03:31:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.740+0000 7f465a604640 1 -- 172.21.15.165:0/1890037706 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4760019a10 con 0x55a4d92d7100 2024-04-27T03:31:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.740+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4d93d44a0 con 0x55a4d92d7100 2024-04-27T03:31:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.740+0000 7f465a604640 1 -- 172.21.15.165:0/1890037706 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760019d30 con 0x55a4d92d7100 2024-04-27T03:31:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.740+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d93d7150 con 0x55a4d92d7100 2024-04-27T03:31:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.741+0000 7f465a604640 1 -- 172.21.15.165:0/1890037706 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f4760017040 con 0x55a4d92d7100 2024-04-27T03:31:29.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.741+0000 7f465a604640 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4654036950 0x7f4654038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.741+0000 7f465a604640 1 -- 172.21.15.165:0/1890037706 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f4760015020 con 0x55a4d92d7100 2024-04-27T03:31:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.741+0000 7f476d10c640 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4654036950 0x7f4654038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.742+0000 7f476fe0e800 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a4d92dd370 0x55a4d93daf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.742+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d93db450 con 0x55a4d92dd370 2024-04-27T03:31:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.742+0000 7f476e10e640 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a4d92dd370 0x55a4d93daf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.742+0000 7f476d10c640 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4654036950 0x7f4654038e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4764054000 tx=0x7f476404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.742+0000 7f476e10e640 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a4d92dd370 0x55a4d93daf10 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.742+0000 7f476e10e640 1 -- 172.21.15.165:0/1890037706 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d92dd370 2024-04-27T03:31:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.742+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d93d84e0 con 0x55a4d92dd370 2024-04-27T03:31:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.743+0000 7f476e10e640 1 -- 172.21.15.165:0/1890037706 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d92dd370 2024-04-27T03:31:29.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.743+0000 7f476fe0e800 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a4d93dd7d0 0x55a4d93dfbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.743+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d93e0260 con 0x55a4d93dd7d0 2024-04-27T03:31:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.743+0000 7f476d90d640 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a4d93dd7d0 0x55a4d93dfbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.743+0000 7f476d90d640 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a4d93dd7d0 0x55a4d93dfbc0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.744+0000 7f476d90d640 1 -- 172.21.15.165:0/1890037706 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f476002d070 con 0x55a4d93dd7d0 2024-04-27T03:31:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.744+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d93dc780 con 0x55a4d92dd370 2024-04-27T03:31:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.744+0000 7f476e10e640 1 -- 172.21.15.165:0/1890037706 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d92dd370 2024-04-27T03:31:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.744+0000 7f476fe0e800 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a4d93e1710 0x55a4d93e3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.744+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d93e42c0 con 0x55a4d93e1710 2024-04-27T03:31:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.744+0000 7f476d10c640 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a4d93e1710 0x55a4d93e3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.744+0000 7f476d10c640 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a4d93e1710 0x55a4d93e3b60 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.745+0000 7f476d10c640 1 -- 172.21.15.165:0/1890037706 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4764052940 con 0x55a4d93e1710 2024-04-27T03:31:29.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.745+0000 7f476fe0e800 1 Processor -- start 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.745+0000 7f476fe0e800 1 -- start start 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.745+0000 7f476fe0e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93f26d0 0x55a4d93f2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.745+0000 7f476fe0e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a4d91ca6b0 con 0x55a4d93f26d0 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.746+0000 7f476d90d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93f26d0 0x55a4d93f2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.746+0000 7f476d90d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93f26d0 0x55a4d93f2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49218/0 (socket says 172.21.15.165:49218) 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.746+0000 7f476d90d640 1 -- 172.21.15.165:0/3505019694 learned_addr learned my addr 172.21.15.165:0/3505019694 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.746+0000 7f476d90d640 1 -- 172.21.15.165:0/3505019694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d93f3030 con 0x55a4d93f26d0 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.746+0000 7f476d90d640 1 --2- 172.21.15.165:0/3505019694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93f26d0 0x55a4d93f2af0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f476004b4b0 tx=0x7f476000f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.746+0000 7f46537fe640 1 -- 172.21.15.165:0/3505019694 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4760041e40 con 0x55a4d93f26d0 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.747+0000 7f46537fe640 1 -- 172.21.15.165:0/3505019694 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f476001a5d0 con 0x55a4d93f26d0 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.747+0000 7f476fe0e800 1 -- 172.21.15.165:0/3505019694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4d93f3aa0 con 0x55a4d93f26d0 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.747+0000 7f46537fe640 1 -- 172.21.15.165:0/3505019694 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f476001a8b0 con 0x55a4d93f26d0 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.747+0000 7f476fe0e800 1 -- 172.21.15.165:0/3505019694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d93f3fa0 con 0x55a4d93f26d0 2024-04-27T03:31:29.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.747+0000 7f46537fe640 1 -- 172.21.15.165:0/3505019694 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f476003f200 con 0x55a4d93f26d0 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.748+0000 7f46537fe640 1 --2- 172.21.15.165:0/3505019694 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4648036920 0x7f4648038de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.748+0000 7f46537fe640 1 -- 172.21.15.165:0/3505019694 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f476003bbd0 con 0x55a4d93f26d0 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.748+0000 7f476d10c640 1 --2- 172.21.15.165:0/3505019694 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4648036920 0x7f4648038de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.748+0000 7f476fe0e800 1 Processor -- start 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.748+0000 7f476fe0e800 1 -- start start 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.748+0000 7f476d10c640 1 --2- 172.21.15.165:0/3505019694 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4648036920 0x7f4648038de0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55a4d93d8400 tx=0x7f4764005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.748+0000 7f476fe0e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 0x55a4d93ffdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.748+0000 7f476fe0e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a4d92d83c0 con 0x55a4d93ff9d0 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.749+0000 7f476d90d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 0x55a4d93ffdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.749+0000 7f476d90d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 0x55a4d93ffdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49234/0 (socket says 172.21.15.165:49234) 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.749+0000 7f476d90d640 1 -- 172.21.15.165:0/2399905126 learned_addr learned my addr 172.21.15.165:0/2399905126 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.749+0000 7f476d90d640 1 -- 172.21.15.165:0/2399905126 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d94003c0 con 0x55a4d93ff9d0 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.749+0000 7f476d90d640 1 --2- 172.21.15.165:0/2399905126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 0x55a4d93ffdf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4760001610 tx=0x7f4760030f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f4986434af60dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.749+0000 7f46517fa640 1 -- 172.21.15.165:0/2399905126 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f476004f070 con 0x55a4d93ff9d0 2024-04-27T03:31:29.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.749+0000 7f46517fa640 1 -- 172.21.15.165:0/2399905126 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f476004b530 con 0x55a4d93ff9d0 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.749+0000 7f46517fa640 1 -- 172.21.15.165:0/2399905126 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f476003f000 con 0x55a4d93ff9d0 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.750+0000 7f476fe0e800 1 -- 172.21.15.165:0/2399905126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 msgr2=0x55a4d93ffdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.750+0000 7f476fe0e800 1 --2- 172.21.15.165:0/2399905126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 0x55a4d93ffdf0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f4760001610 tx=0x7f4760030f70 comp rx=0 tx=0).stop 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.750+0000 7f476fe0e800 1 -- 172.21.15.165:0/2399905126 shutdown_connections 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.750+0000 7f476fe0e800 1 --2- 172.21.15.165:0/2399905126 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 0x55a4d93ffdf0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.750+0000 7f476fe0e800 1 -- 172.21.15.165:0/2399905126 >> 172.21.15.165:0/2399905126 conn(0x55a4d93fbcf0 msgr2=0x55a4d93f9d50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.750+0000 7f476fe0e800 1 -- 172.21.15.165:0/2399905126 shutdown_connections 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.750+0000 7f476fe0e800 1 -- 172.21.15.165:0/2399905126 wait complete. 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.750+0000 7f476fe0e800 1 Processor -- start 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.750+0000 7f476fe0e800 1 -- start start 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.750+0000 7f476fe0e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 0x55a4d9406dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.750+0000 7f476fe0e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a4d9400ae0 con 0x55a4d93ff9d0 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.750+0000 7f476d90d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 0x55a4d9406dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.751+0000 7f476d90d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 0x55a4d9406dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49236/0 (socket says 172.21.15.165:49236) 2024-04-27T03:31:29.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.751+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 learned_addr learned my addr 172.21.15.165:0/3510409875 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:29.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.751+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a4d9407310 con 0x55a4d93ff9d0 2024-04-27T03:31:29.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.751+0000 7f476d90d640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 0x55a4d9406dd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f476004b6e0 tx=0x7f4760031260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.751+0000 7f463f7fe640 1 -- 172.21.15.165:0/3510409875 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f476003f000 con 0x55a4d93ff9d0 2024-04-27T03:31:29.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.751+0000 7f463f7fe640 1 -- 172.21.15.165:0/3510409875 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4760068030 con 0x55a4d93ff9d0 2024-04-27T03:31:29.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.751+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a4d9407310 con 0x55a4d93ff9d0 2024-04-27T03:31:29.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.751+0000 7f463f7fe640 1 -- 172.21.15.165:0/3510409875 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f476003f620 con 0x55a4d93ff9d0 2024-04-27T03:31:29.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.751+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a4d9407310 con 0x55a4d93ff9d0 2024-04-27T03:31:29.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.752+0000 7f463f7fe640 1 -- 172.21.15.165:0/3510409875 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f4760030610 con 0x55a4d93ff9d0 2024-04-27T03:31:29.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.752+0000 7f463f7fe640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4638036900 0x7f4638038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.752+0000 7f463f7fe640 1 -- 172.21.15.165:0/3510409875 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f476006ebc0 con 0x55a4d93ff9d0 2024-04-27T03:31:29.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.752+0000 7f476fe0e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.753+0000 7f476d10c640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4638036900 0x7f4638038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.753+0000 7f476fe0e800 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a4d9412340 0x55a4d9414800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.753+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9414f50 con 0x55a4d9412340 2024-04-27T03:31:29.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.753+0000 7f476e10e640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a4d9412340 0x55a4d9414800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.753+0000 7f476d10c640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4638036900 0x7f4638038dc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4764052fd0 tx=0x7f476404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.753+0000 7f476e10e640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a4d9412340 0x55a4d9414800 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.753+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.753+0000 7f476fe0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.753+0000 7f476fe0e800 20 realm 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.753+0000 7f476fe0e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.753+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415360 con 0x55a4d9412340 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476fe0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476fe0e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476fe0e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415c00 con 0x55a4d9412340 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476fe0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476fe0e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9412340 2024-04-27T03:31:29.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476fe0e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476fe0e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476fe0e800 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a4d9416d00 0x55a4d94190f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9419630 con 0x55a4d9416d00 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.754+0000 7f476d90d640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a4d9416d00 0x55a4d94190f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.755+0000 7f476d90d640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a4d9416d00 0x55a4d94190f0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.755+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.755+0000 7f476fe0e800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.755+0000 7f476fe0e800 20 searching for the correct realm 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.755+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941afe0 con 0x55a4d9416d00 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.755+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.755+0000 7f476d90d640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f476002f3e0 0x7f4760043620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.755+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941afe0 con 0x7f476002f3e0 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.756+0000 7f476d10c640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f476002f3e0 0x7f4760043620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.756+0000 7f476d10c640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f476002f3e0 0x7f4760043620 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.756+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.756+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941afe0 con 0x7f476002f3e0 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.756+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.756+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941afe0 con 0x55a4d9412340 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.757+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.757+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9412340 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.757+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.757+0000 7f476e10e640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4768016440 0x7f47680168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.757+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f4768016440 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.757+0000 7f476e10e640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4768016440 0x7f47680168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.758+0000 7f476e10e640 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4768016440 0x7f47680168c0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.758+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.758+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f4768016440 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.759+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.759+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f476002f3e0 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.759+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.759+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9416d00 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.759+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.759+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9412340 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.759+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.759+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f476002f3e0 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.760+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.760+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9412340 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.760+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.760+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f476002f3e0 2024-04-27T03:31:29.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.760+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.760+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9416d00 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.760+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.760+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9416d00 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.761+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.761+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f4768016440 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.761+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.761+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9416d00 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.761+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.761+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9416d00 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.762+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.762+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9412340 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.762+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.762+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9412340 2024-04-27T03:31:29.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.762+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.762+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f4768016440 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.762+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.762+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9416d00 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.763+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.763+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f476002f3e0 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.763+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.763+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9412340 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.763+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.763+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f4768016440 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.764+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.764+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9416d00 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.764+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.764+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f4768016440 2024-04-27T03:31:29.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.764+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.764+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f476002f3e0 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.765+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.765+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9416d00 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.765+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.765+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f4768016440 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.765+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.765+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x55a4d9412340 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9415fa0 con 0x7f476002f3e0 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:29.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9419fb0 con 0x55a4d9412340 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941a3a0 con 0x55a4d9412340 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.766+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941a3a0 con 0x7f476002f3e0 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 20 zone default found 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 4 Realm: () 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 20 zonegroup default 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941b4b0 con 0x7f4768016440 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941ce90 con 0x55a4d9416d00 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.767+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941d230 con 0x55a4d9416d00 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.768+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.768+0000 7f476fe0e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.768+0000 7f476fe0e800 20 started sync module instance, tier type = 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.768+0000 7f476fe0e800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.768+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941c330 con 0x7f476002f3e0 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.769+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.769+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94166507964240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941da80 con 0x7f476002f3e0 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.769+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941e2d0 con 0x55a4d9412340 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.770+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94166507964240] v27'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.770+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.770+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94166507967952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941e2d0 con 0x55a4d9412340 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.770+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941edc0 con 0x55a4d9416d00 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.772+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94166507967952] v27'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.772+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.772+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94166507970080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941e240 con 0x55a4d9416d00 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.772+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941fcc0 con 0x7f476002f3e0 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.773+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94166507970080] v27'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.773+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.773+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94166507975088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9420560 con 0x7f476002f3e0 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.773+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9420f20 con 0x55a4d9412340 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.774+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94166507975088] v27'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.775+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.775+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94166507981424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9421340 con 0x55a4d9412340 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.775+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9421d00 con 0x7f4768016440 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.776+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94166507981424] v27'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.776+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.776+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94166507982960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9421d00 con 0x7f4768016440 2024-04-27T03:31:29.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.776+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d94226c0 con 0x7f4768016440 2024-04-27T03:31:29.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.777+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94166507982960] v27'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.777+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.777+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94166507985456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d94226c0 con 0x7f4768016440 2024-04-27T03:31:29.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.777+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9423080 con 0x7f4768016440 2024-04-27T03:31:29.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.780+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94166507985456] v27'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.780+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'189 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.780+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94166507987952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9423080 con 0x7f4768016440 2024-04-27T03:31:29.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.780+0000 7f476fe0e800 20 add_watcher() i=0 2024-04-27T03:31:29.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.780+0000 7f476fe0e800 20 add_watcher() i=1 2024-04-27T03:31:29.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.780+0000 7f476fe0e800 20 add_watcher() i=2 2024-04-27T03:31:29.795 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.780+0000 7f476fe0e800 20 add_watcher() i=3 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.780+0000 7f476fe0e800 20 add_watcher() i=4 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.780+0000 7f476fe0e800 20 add_watcher() i=5 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.780+0000 7f476fe0e800 20 add_watcher() i=6 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.781+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94166507987952] v27'190 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.781+0000 7f476fe0e800 20 add_watcher() i=7 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.781+0000 7f476fe0e800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.781+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d94263a0 con 0x55a4d9412340 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.782+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.782+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94166507990336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9423610 con 0x55a4d9412340 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.783+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94166507990336] v27'202 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.783+0000 7f461f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.783+0000 7f476fe0e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.783+0000 7f476fe0e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:29.796 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.783+0000 7f476fe0e800 5 note: GC not initialized 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.783+0000 7f461e7fc640 20 reqs_thread_entry: start 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.783+0000 7f476fe0e800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.783+0000 7f476fe0e800 20 Filter name: none 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.783+0000 7f458ffff640 20 reqs_thread_entry: start 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.783+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9459cd0 con 0x55a4d9412340 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.784+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.784+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9459cd0 con 0x7f476002f3e0 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.784+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.784+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9459cd0 con 0x55a4d9412340 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.784+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.784+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9459c90 con 0x55a4d9416d00 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.784+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.785+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9459cd0 con 0x55a4d9416d00 2024-04-27T03:31:29.797 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.785+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.785+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9459c90 con 0x55a4d9416d00 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.785+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.785+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9459c90 con 0x55a4d9412340 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.785+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.785+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9459c90 con 0x55a4d9412340 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.785+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.785+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9459c90 con 0x55a4d9416d00 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.786+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.786+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9459c90 con 0x55a4d9416d00 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.786+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.786+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9459c90 con 0x7f4768016440 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.786+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(72 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) ==== 152+0+1268 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.798 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.786+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b7d0 con 0x55a4d9412340 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.787+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.787+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b7d0 con 0x7f476002f3e0 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.787+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.787+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b7d0 con 0x55a4d9416d00 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.787+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.787+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b7d0 con 0x7f4768016440 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.788+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.788+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b7d0 con 0x7f4768016440 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.788+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.788+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b7d0 con 0x55a4d9416d00 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.788+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.788+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b750 con 0x7f4768016440 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.789+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(79 usage.17 [call out=163b] v0'0 uv5 ondisk = 0) ==== 152+0+163 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.799 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.789+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b750 con 0x55a4d9412340 2024-04-27T03:31:29.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.789+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.800 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.789+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b750 con 0x55a4d9416d00 2024-04-27T03:31:29.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.789+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.789+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b750 con 0x7f4768016440 2024-04-27T03:31:29.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.789+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.789+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b750 con 0x7f476002f3e0 2024-04-27T03:31:29.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.790+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.790+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b750 con 0x7f4768016440 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.790+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.790+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b750 con 0x55a4d9412340 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.790+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.790+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b750 con 0x55a4d9412340 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.790+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.790+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b750 con 0x55a4d9416d00 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.791+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.791+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b730 con 0x55a4d9412340 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.791+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.791+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b730 con 0x55a4d9412340 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.791+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.791+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b730 con 0x55a4d9416d00 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.791+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.791+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b730 con 0x55a4d9416d00 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.792+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.792+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b730 con 0x55a4d9416d00 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.792+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.792+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a4d945b730 con 0x7f4768016440 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.792+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.793+0000 7f461f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.793+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94166507990336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9425320 con 0x55a4d9412340 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.794+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94166507990336] v27'203 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.794+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94166507964240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9422d40 con 0x7f476002f3e0 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.795+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94166507964240] v27'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.795+0000 7f476fe0e800 20 remove_watcher() i=0 2024-04-27T03:31:29.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.795+0000 7f476fe0e800 2 removed watcher, disabling cache 2024-04-27T03:31:29.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.795+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94166507967952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9422d40 con 0x55a4d9412340 2024-04-27T03:31:29.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.796+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94166507967952] v27'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.796+0000 7f476fe0e800 20 remove_watcher() i=1 2024-04-27T03:31:29.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.796+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94166507970080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941d5d0 con 0x55a4d9416d00 2024-04-27T03:31:29.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.798+0000 7f476d90d640 1 -- 172.21.15.165:0/3510409875 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94166507970080] v27'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4760074070 con 0x55a4d9416d00 2024-04-27T03:31:29.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.798+0000 7f476fe0e800 20 remove_watcher() i=2 2024-04-27T03:31:29.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.798+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94166507975088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9422d40 con 0x7f476002f3e0 2024-04-27T03:31:29.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.799+0000 7f476d10c640 1 -- 172.21.15.165:0/3510409875 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94166507975088] v27'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4764081070 con 0x7f476002f3e0 2024-04-27T03:31:29.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.799+0000 7f476fe0e800 20 remove_watcher() i=3 2024-04-27T03:31:29.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.799+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94166507981424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941de20 con 0x55a4d9412340 2024-04-27T03:31:29.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.800+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94166507981424] v27'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x55a4d9412340 2024-04-27T03:31:29.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.800+0000 7f476fe0e800 20 remove_watcher() i=4 2024-04-27T03:31:29.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.800+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94166507982960] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9422d40 con 0x7f4768016440 2024-04-27T03:31:29.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.801+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94166507982960] v27'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.801+0000 7f476fe0e800 20 remove_watcher() i=5 2024-04-27T03:31:29.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.801+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94166507985456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d941de20 con 0x7f4768016440 2024-04-27T03:31:29.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.802+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94166507985456] v27'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.802+0000 7f476fe0e800 20 remove_watcher() i=6 2024-04-27T03:31:29.809 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.802+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94166507987952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a4d9421070 con 0x7f4768016440 2024-04-27T03:31:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94166507987952] v27'191 uv189 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f476800a4a0 con 0x7f4768016440 2024-04-27T03:31:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 20 remove_watcher() i=7 2024-04-27T03:31:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4768016440 msgr2=0x7f47680168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4768016440 0x7f47680168c0 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a4d9412340 msgr2=0x55a4d9414800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a4d9412340 0x55a4d9414800 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f476002f3e0 msgr2=0x7f4760043620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f476002f3e0 0x7f4760043620 crc :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a4d9416d00 msgr2=0x55a4d94190f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a4d9416d00 0x55a4d94190f0 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4638036900 msgr2=0x7f4638038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4638036900 0x7f4638038dc0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f4764052fd0 tx=0x7f476404f000 comp rx=0 tx=0).stop 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 msgr2=0x55a4d9406dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.803+0000 7f476fe0e800 1 --2- 172.21.15.165:0/3510409875 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93ff9d0 0x55a4d9406dd0 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f476004b6e0 tx=0x7f4760031260 comp rx=0 tx=0).stop 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 reap_dead start 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476e10e640 1 -- 172.21.15.165:0/3510409875 reap_dead start 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 shutdown_connections 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 >> 172.21.15.165:0/3510409875 conn(0x55a4d93fbcf0 msgr2=0x55a4d93f9350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 shutdown_connections 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/3510409875 wait complete. 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/3505019694 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4648036920 msgr2=0x7f4648038de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 --2- 172.21.15.165:0/3505019694 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4648036920 0x7f4648038de0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x55a4d93d8400 tx=0x7f4764005a00 comp rx=0 tx=0).stop 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/3505019694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93f26d0 msgr2=0x55a4d93f2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 --2- 172.21.15.165:0/3505019694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93f26d0 0x55a4d93f2af0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f476004b4b0 tx=0x7f476000f840 comp rx=0 tx=0).stop 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/3505019694 shutdown_connections 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 --2- 172.21.15.165:0/3505019694 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4648036920 0x7f4648038de0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 --2- 172.21.15.165:0/3505019694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d93f26d0 0x55a4d93f2af0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/3505019694 >> 172.21.15.165:0/3505019694 conn(0x55a4d93e6770 msgr2=0x55a4d93e6b70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/3505019694 shutdown_connections 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/3505019694 wait complete. 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a4d92dd370 msgr2=0x55a4d93daf10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a4d92dd370 0x55a4d93daf10 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a4d93e1710 msgr2=0x55a4d93e3b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a4d93e1710 0x55a4d93e3b60 crc :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a4d93dd7d0 msgr2=0x55a4d93dfbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a4d93dd7d0 0x55a4d93dfbc0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4654036950 msgr2=0x7f4654038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4654036950 0x7f4654038e10 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f4764054000 tx=0x7f476404e000 comp rx=0 tx=0).stop 2024-04-27T03:31:29.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 msgr2=0x55a4d93d3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.804+0000 7f476fe0e800 1 --2- 172.21.15.165:0/1890037706 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a4d92d7100 0x55a4d93d3cd0 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x55a4d92d90b0 tx=0x7f4760000ed0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.805+0000 7f476e10e640 1 -- 172.21.15.165:0/1890037706 reap_dead start 2024-04-27T03:31:29.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.805+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 shutdown_connections 2024-04-27T03:31:29.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.805+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 >> 172.21.15.165:0/1890037706 conn(0x55a4d92d2b00 msgr2=0x55a4d92d8920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:29.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.805+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 shutdown_connections 2024-04-27T03:31:29.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.805+0000 7f476fe0e800 1 -- 172.21.15.165:0/1890037706 wait complete. 2024-04-27T03:31:29.819 INFO:teuthology.orchestra.run.smithi165.stdout:{"entries":[{"user":"anonymous","buckets":[{"bucket":"","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"anonymous","categories":[]}]},{"user":"foo","buckets":[{"bucket":"","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]},{"bucket":"myfoo","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}]},{"bucket":"myfoo2","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]},{"bucket":"myfoo3","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]},{"bucket":"myfoo4","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]}]}],"summary":[{"user":"anonymous","categories":[],"total":{"bytes_sent":0,"bytes_received":0,"ops":0,"successful_ops":0}},{"user":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"total":{"bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}}]} 2024-04-27T03:31:29.819 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'anonymous', 'categories': []}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}]} 2024-04-27T03:31:29.820 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show'] 2024-04-27T03:31:29.820 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show'] 2024-04-27T03:31:29.820 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show 2024-04-27T03:31:29.860 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:29.860 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:29.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.854+0000 7f9d8d3a2800 1 Processor -- start 2024-04-27T03:31:29.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.855+0000 7f9d8d3a2800 1 -- start start 2024-04-27T03:31:29.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.855+0000 7f9d8d3a2800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x56498f1cbd50 0x56498f1cc170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.855+0000 7f9d8d3a2800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56498f08bd40 con 0x56498f1ccfb0 2024-04-27T03:31:29.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.855+0000 7f9d8d3a2800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x56498f05e4d0 con 0x56498f1cbd50 2024-04-27T03:31:29.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.855+0000 7f9d8aea8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x56498f1cbd50 0x56498f1cc170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.855+0000 7f9d8a6a7640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x56498f1ccfb0 0x56498f1cd430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:54698/0 (socket says 172.21.15.165:54698) 2024-04-27T03:31:29.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.855+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1911218670 learned_addr learned my addr 172.21.15.165:0/1911218670 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:31:29.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.855+0000 7f9d89ea6640 1 -- 172.21.15.165:0/1911218670 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3407231879 0 0) 0x56498f08bd40 con 0x56498f1ccfb0 2024-04-27T03:31:29.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.855+0000 7f9d89ea6640 1 -- 172.21.15.165:0/1911218670 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f9d74001e10 con 0x56498f1ccfb0 2024-04-27T03:31:29.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.855+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1911218670 >> v1:172.21.15.89:6789/0 conn(0x56498f1ccfb0 legacy=0x56498f1cd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.855+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1911218670 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498f08a590 con 0x56498f1cbd50 2024-04-27T03:31:29.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.857+0000 7f9d8aea8640 1 --2- 172.21.15.165:0/1911218670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1cbd50 0x56498f1cc170 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00a470 tx=0x7f9d7c01cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9234f7925e01e95d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.858+0000 7f9d89ea6640 1 -- 172.21.15.165:0/1911218670 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d7c057070 con 0x56498f1cbd50 2024-04-27T03:31:29.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.858+0000 7f9d89ea6640 1 -- 172.21.15.165:0/1911218670 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c052570 con 0x56498f1cbd50 2024-04-27T03:31:29.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.858+0000 7f9d89ea6640 1 -- 172.21.15.165:0/1911218670 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d7c052870 con 0x56498f1cbd50 2024-04-27T03:31:29.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.858+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1911218670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1cbd50 msgr2=0x56498f1cc170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.858+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1911218670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1cbd50 0x56498f1cc170 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c00a470 tx=0x7f9d7c01cdb0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.858+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1911218670 shutdown_connections 2024-04-27T03:31:29.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.858+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1911218670 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1cbd50 0x56498f1cc170 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.858+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1911218670 >> 172.21.15.165:0/1911218670 conn(0x56498f1c7310 msgr2=0x56498f1c9730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:29.865 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.858+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1911218670 shutdown_connections 2024-04-27T03:31:29.866 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.858+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1911218670 wait complete. 2024-04-27T03:31:29.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.870+0000 7f9d8d3a2800 1 Processor -- start 2024-04-27T03:31:29.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.870+0000 7f9d8d3a2800 1 -- start start 2024-04-27T03:31:29.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.870+0000 7f9d8d3a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1cbd50 0x56498f2bc0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.870+0000 7f9d8d3a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56498f1183b0 con 0x56498f1cbd50 2024-04-27T03:31:29.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.870+0000 7f9d8aea8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1cbd50 0x56498f2bc0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.871+0000 7f9d8aea8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1cbd50 0x56498f2bc0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49264/0 (socket says 172.21.15.165:49264) 2024-04-27T03:31:29.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.871+0000 7f9d8aea8640 1 -- 172.21.15.165:0/380297107 learned_addr learned my addr 172.21.15.165:0/380297107 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:29.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.871+0000 7f9d8aea8640 1 -- 172.21.15.165:0/380297107 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498f2bc6b0 con 0x56498f1cbd50 2024-04-27T03:31:29.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.871+0000 7f9d8aea8640 1 --2- 172.21.15.165:0/380297107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1cbd50 0x56498f2bc0e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c055040 tx=0x7f9d7c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9be93b616af8fea server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.871+0000 7f9c78e09640 1 -- 172.21.15.165:0/380297107 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d7c057070 con 0x56498f1cbd50 2024-04-27T03:31:29.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.871+0000 7f9c78e09640 1 -- 172.21.15.165:0/380297107 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c063070 con 0x56498f1cbd50 2024-04-27T03:31:29.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.871+0000 7f9c78e09640 1 -- 172.21.15.165:0/380297107 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d7c052720 con 0x56498f1cbd50 2024-04-27T03:31:29.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.872+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/380297107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1cbd50 msgr2=0x56498f2bc0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.872+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/380297107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1cbd50 0x56498f2bc0e0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c055040 tx=0x7f9d7c04c000 comp rx=0 tx=0).stop 2024-04-27T03:31:29.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.872+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/380297107 shutdown_connections 2024-04-27T03:31:29.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.872+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/380297107 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1cbd50 0x56498f2bc0e0 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.872+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/380297107 >> 172.21.15.165:0/380297107 conn(0x56498f1c7310 msgr2=0x56498f2b7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:29.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.872+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/380297107 shutdown_connections 2024-04-27T03:31:29.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.872+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/380297107 wait complete. 2024-04-27T03:31:29.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.872+0000 7f9d8d3a2800 1 Processor -- start 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.872+0000 7f9d8d3a2800 1 -- start start 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.872+0000 7f9d8d3a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1ccfb0 0x56498f2c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.872+0000 7f9d8d3a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56498f13ff70 con 0x56498f1ccfb0 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.873+0000 7f9d8aea8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1ccfb0 0x56498f2c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.873+0000 7f9d8aea8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1ccfb0 0x56498f2c3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49274/0 (socket says 172.21.15.165:49274) 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.873+0000 7f9d8aea8640 1 -- 172.21.15.165:0/3287673123 learned_addr learned my addr 172.21.15.165:0/3287673123 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.873+0000 7f9d8aea8640 1 -- 172.21.15.165:0/3287673123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498f2c4300 con 0x56498f1ccfb0 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.873+0000 7f9d8aea8640 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1ccfb0 0x56498f2c3dc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c055040 tx=0x7f9d7c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.873+0000 7f9c77606640 1 -- 172.21.15.165:0/3287673123 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d7c052ae0 con 0x56498f1ccfb0 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.874+0000 7f9c77606640 1 -- 172.21.15.165:0/3287673123 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c06c070 con 0x56498f1ccfb0 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.874+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56498f2c4590 con 0x56498f1ccfb0 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.874+0000 7f9c77606640 1 -- 172.21.15.165:0/3287673123 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d7c052dd0 con 0x56498f1ccfb0 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.874+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498f2c7240 con 0x56498f1ccfb0 2024-04-27T03:31:29.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.874+0000 7f9c77606640 1 -- 172.21.15.165:0/3287673123 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9d7c077060 con 0x56498f1ccfb0 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.875+0000 7f9c77606640 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c70036900 0x7f9c70038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.875+0000 7f9c77606640 1 -- 172.21.15.165:0/3287673123 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f9d7c08dfc0 con 0x56498f1ccfb0 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.875+0000 7f9d8a6a7640 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c70036900 0x7f9c70038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.875+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56498f2c8ff0 0x56498f2cb3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.875+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f2cb960 con 0x56498f2c8ff0 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.875+0000 7f9d8b6a9640 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56498f2c8ff0 0x56498f2cb3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.875+0000 7f9d8a6a7640 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c70036900 0x7f9c70038dc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9d80000f60 tx=0x7f9d800010a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.875+0000 7f9d8b6a9640 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56498f2c8ff0 0x56498f2cb3e0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.875+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/3287673123 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f2c8ff0 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.875+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f2c85b0 con 0x56498f2c8ff0 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.876+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/3287673123 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f2c8ff0 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.876+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56498f2cdc90 0x56498f2d0080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.876+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f2d0720 con 0x56498f2cdc90 2024-04-27T03:31:29.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.876+0000 7f9d8aea8640 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56498f2cdc90 0x56498f2d0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.876+0000 7f9d8aea8640 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56498f2cdc90 0x56498f2d0080 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.876+0000 7f9d8aea8640 1 -- 172.21.15.165:0/3287673123 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f9d7c067b50 con 0x56498f2cdc90 2024-04-27T03:31:29.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.876+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f2ccc40 con 0x56498f2c8ff0 2024-04-27T03:31:29.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.877+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/3287673123 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f2c8ff0 2024-04-27T03:31:29.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.877+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56498f2d1bd0 0x56498f2d4020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.877+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f2d4780 con 0x56498f2d1bd0 2024-04-27T03:31:29.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.877+0000 7f9d8a6a7640 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56498f2d1bd0 0x56498f2d4020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.877+0000 7f9d8a6a7640 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56498f2d1bd0 0x56498f2d4020 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.877+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/3287673123 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f9d80009050 con 0x56498f2d1bd0 2024-04-27T03:31:29.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.878+0000 7f9d8d3a2800 1 Processor -- start 2024-04-27T03:31:29.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.878+0000 7f9d8d3a2800 1 -- start start 2024-04-27T03:31:29.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.878+0000 7f9d8d3a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2e2be0 0x56498f2e3000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.878+0000 7f9d8d3a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56498f0be6b0 con 0x56498f2e2be0 2024-04-27T03:31:29.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.878+0000 7f9d8aea8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2e2be0 0x56498f2e3000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.878+0000 7f9d8aea8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2e2be0 0x56498f2e3000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49290/0 (socket says 172.21.15.165:49290) 2024-04-27T03:31:29.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.878+0000 7f9d8aea8640 1 -- 172.21.15.165:0/3860942813 learned_addr learned my addr 172.21.15.165:0/3860942813 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.879+0000 7f9d8aea8640 1 -- 172.21.15.165:0/3860942813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498f2e3540 con 0x56498f2e2be0 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.879+0000 7f9d8aea8640 1 --2- 172.21.15.165:0/3860942813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2e2be0 0x56498f2e3000 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c08c410 tx=0x7f9d7c0678b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.879+0000 7f9c74e01640 1 -- 172.21.15.165:0/3860942813 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d7c089070 con 0x56498f2e2be0 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.879+0000 7f9c74e01640 1 -- 172.21.15.165:0/3860942813 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c058040 con 0x56498f2e2be0 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.879+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3860942813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56498f2e3fb0 con 0x56498f2e2be0 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.879+0000 7f9c74e01640 1 -- 172.21.15.165:0/3860942813 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d7c009bf0 con 0x56498f2e2be0 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.879+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3860942813 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498f2e3fb0 con 0x56498f2e2be0 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.880+0000 7f9c74e01640 1 -- 172.21.15.165:0/3860942813 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9d7c06e030 con 0x56498f2e2be0 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.880+0000 7f9c74e01640 1 --2- 172.21.15.165:0/3860942813 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c64036900 0x7f9c64038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.880+0000 7f9c74e01640 1 -- 172.21.15.165:0/3860942813 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f9d7c04b100 con 0x56498f2e2be0 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.880+0000 7f9d8a6a7640 1 --2- 172.21.15.165:0/3860942813 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c64036900 0x7f9c64038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.880+0000 7f9d8d3a2800 1 Processor -- start 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.880+0000 7f9d8d3a2800 1 -- start start 2024-04-27T03:31:29.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.880+0000 7f9d8a6a7640 1 --2- 172.21.15.165:0/3860942813 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c64036900 0x7f9c64038dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000bf90 tx=0x7f9d8000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.880+0000 7f9d8d3a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 0x56498f2f43a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.880+0000 7f9d8d3a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56498f2bc970 con 0x56498f2f3f80 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.881+0000 7f9d8aea8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 0x56498f2f43a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.881+0000 7f9d8aea8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 0x56498f2f43a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49304/0 (socket says 172.21.15.165:49304) 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.881+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1845511138 learned_addr learned my addr 172.21.15.165:0/1845511138 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.881+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1845511138 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498f2f4970 con 0x56498f2f3f80 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.881+0000 7f9d8aea8640 1 --2- 172.21.15.165:0/1845511138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 0x56498f2f43a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c05efd0 tx=0x7f9d7c092480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15908ed534e0e1fd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.881+0000 7f9c6e7fc640 1 -- 172.21.15.165:0/1845511138 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d7c0aa070 con 0x56498f2f3f80 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.881+0000 7f9c6e7fc640 1 -- 172.21.15.165:0/1845511138 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c05e9a0 con 0x56498f2f3f80 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.881+0000 7f9c6e7fc640 1 -- 172.21.15.165:0/1845511138 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d7c05e480 con 0x56498f2f3f80 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.882+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1845511138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 msgr2=0x56498f2f43a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.882+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1845511138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 0x56498f2f43a0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c05efd0 tx=0x7f9d7c092480 comp rx=0 tx=0).stop 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.882+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1845511138 shutdown_connections 2024-04-27T03:31:29.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.882+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1845511138 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 0x56498f2f43a0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.882+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1845511138 >> 172.21.15.165:0/1845511138 conn(0x56498f2ebe90 msgr2=0x56498f2e9ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.882+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1845511138 shutdown_connections 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.882+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1845511138 wait complete. 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.882+0000 7f9d8d3a2800 1 Processor -- start 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.882+0000 7f9d8d3a2800 1 -- start start 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.882+0000 7f9d8d3a2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 0x56498f2fb3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.882+0000 7f9d8d3a2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56498f2f5090 con 0x56498f2f3f80 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.883+0000 7f9d8aea8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 0x56498f2fb3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.883+0000 7f9d8aea8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 0x56498f2fb3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49312/0 (socket says 172.21.15.165:49312) 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.883+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 learned_addr learned my addr 172.21.15.165:0/1093886287 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.883+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56498f2fb910 con 0x56498f2f3f80 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.883+0000 7f9d8aea8640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 0x56498f2fb3d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c08cfd0 tx=0x7f9d7c07eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.883+0000 7f9c6cff9640 1 -- 172.21.15.165:0/1093886287 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d7c0aa050 con 0x56498f2f3f80 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.883+0000 7f9c6cff9640 1 -- 172.21.15.165:0/1093886287 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9d7c0ab040 con 0x56498f2f3f80 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.883+0000 7f9c6cff9640 1 -- 172.21.15.165:0/1093886287 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9d7c068a80 con 0x56498f2f3f80 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.883+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56498f2fb910 con 0x56498f2f3f80 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.883+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56498f2fb910 con 0x56498f2f3f80 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.884+0000 7f9c6cff9640 1 -- 172.21.15.165:0/1093886287 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9d7c0b8020 con 0x56498f2f3f80 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.884+0000 7f9c6cff9640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c54036900 0x7f9c54038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.884+0000 7f9d8d3a2800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.884+0000 7f9c6cff9640 1 -- 172.21.15.165:0/1093886287 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f9d7c0b6020 con 0x56498f2f3f80 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.884+0000 7f9d8a6a7640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c54036900 0x7f9c54038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56498f306a90 0x56498f308f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f309680 con 0x56498f306a90 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8b6a9640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56498f306a90 0x56498f308f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8a6a7640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c54036900 0x7f9c54038dc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x56498f2c84f0 tx=0x7f9d80010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8b6a9640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56498f306a90 0x56498f308f30 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8d3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8d3a2800 20 realm 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8d3a2800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f309a90 con 0x56498f306a90 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8d3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8d3a2800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8d3a2800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.885+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a330 con 0x56498f306a90 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.886+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.886+0000 7f9d8d3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.886+0000 7f9d8d3a2800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.886+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f306a90 2024-04-27T03:31:29.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.886+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.886+0000 7f9d8d3a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.886+0000 7f9d8d3a2800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.886+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56498f30b430 0x56498f30d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.886+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f30dd60 con 0x56498f30b430 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.886+0000 7f9d8aea8640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56498f30b430 0x56498f30d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.886+0000 7f9d8aea8640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56498f30b430 0x56498f30d820 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.887+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.887+0000 7f9d8d3a2800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.887+0000 7f9d8d3a2800 20 searching for the correct realm 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.887+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30f710 con 0x56498f30b430 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.887+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.887+0000 7f9d8aea8640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9d7c0b1070 0x7f9d7c0c4800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.887+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30f710 con 0x7f9d7c0b1070 2024-04-27T03:31:29.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.887+0000 7f9d8a6a7640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9d7c0b1070 0x7f9d7c0c4800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.888+0000 7f9d8a6a7640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9d7c0b1070 0x7f9d7c0c4800 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.888+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.888+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30f710 con 0x7f9d7c0b1070 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.888+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.888+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30f710 con 0x56498f306a90 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.888+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.888+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f306a90 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.888+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.888+0000 7f9d8b6a9640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9d84016440 0x7f9d840168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.888+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d84016440 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.889+0000 7f9d8b6a9640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9d84016440 0x7f9d840168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.889+0000 7f9d8b6a9640 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9d84016440 0x7f9d840168c0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:29.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.889+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.889+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d84016440 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.890+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.890+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d7c0b1070 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.890+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.890+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f30b430 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.890+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.890+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f306a90 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.890+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.890+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d7c0b1070 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.891+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.891+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f306a90 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.891+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.891+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d7c0b1070 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.891+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.891+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f30b430 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.891+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.891+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f30b430 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.892+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.892+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d84016440 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.892+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.892+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f30b430 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.892+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.892+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f30b430 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.892+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.892+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f306a90 2024-04-27T03:31:29.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.893+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.893+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f306a90 2024-04-27T03:31:29.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.893+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.893+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d84016440 2024-04-27T03:31:29.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.893+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.893+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f30b430 2024-04-27T03:31:29.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.894+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.894+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d7c0b1070 2024-04-27T03:31:29.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.894+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.894+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f306a90 2024-04-27T03:31:29.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.894+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.894+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d84016440 2024-04-27T03:31:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.894+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.894+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f30b430 2024-04-27T03:31:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.895+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.895+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d84016440 2024-04-27T03:31:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.895+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.895+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d7c0b1070 2024-04-27T03:31:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.895+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.895+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f30b430 2024-04-27T03:31:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.895+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.895+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d84016440 2024-04-27T03:31:29.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x56498f306a90 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56498f30a6d0 con 0x7f9d7c0b1070 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8d3a2800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8d3a2800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8d3a2800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8d3a2800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8d3a2800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8d3a2800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8d3a2800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f30e6e0 con 0x56498f306a90 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8d3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8d3a2800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.896+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f30ead0 con 0x56498f306a90 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f30ead0 con 0x7f9d7c0b1070 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 20 zone default found 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 4 Realm: () 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 20 zonegroup default 2024-04-27T03:31:29.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f30fbe0 con 0x7f9d84016440 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.897+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f3115c0 con 0x56498f30b430 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.898+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.898+0000 7f9d8d3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.898+0000 7f9d8d3a2800 20 rados->read ofs=0 len=0 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.898+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f311960 con 0x56498f30b430 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.898+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.898+0000 7f9d8d3a2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.898+0000 7f9d8d3a2800 20 started sync module instance, tier type = 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.898+0000 7f9d8d3a2800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:29.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.898+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f310a60 con 0x7f9d7c0b1070 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.899+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.899+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94873934958160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f312200 con 0x7f9d7c0b1070 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.899+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f312a50 con 0x56498f306a90 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.900+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94873934958160] v27'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.900+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.900+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94873934964128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f312e70 con 0x56498f306a90 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.900+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f3136c0 con 0x56498f30b430 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.901+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.901+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94873934966896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f313b40 con 0x56498f30b430 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.901+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f314660 con 0x7f9d7c0b1070 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.902+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94873934964128] v27'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.902+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.902+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94873934966896] v27'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.903+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94873934971184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f313590 con 0x7f9d7c0b1070 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.903+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f314f00 con 0x56498f306a90 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.904+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.904+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94873934974032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f315720 con 0x56498f306a90 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.904+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f316240 con 0x7f9d84016440 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.904+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94873934971184] v27'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.905+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94873934974032] v27'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.905+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.905+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94873934978320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f316200 con 0x7f9d84016440 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.905+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f316ae0 con 0x7f9d84016440 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.906+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94873934978320] v27'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.906+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.906+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94873934965184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f316200 con 0x7f9d84016440 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.906+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f3170f0 con 0x7f9d84016440 2024-04-27T03:31:29.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.907+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(58 notify.7 [create] v27'192 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.907+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94873934982112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f3170b0 con 0x7f9d84016440 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.907+0000 7f9d8d3a2800 20 add_watcher() i=0 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.907+0000 7f9d8d3a2800 20 add_watcher() i=1 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.907+0000 7f9d8d3a2800 20 add_watcher() i=2 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.907+0000 7f9d8d3a2800 20 add_watcher() i=3 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.907+0000 7f9d8d3a2800 20 add_watcher() i=4 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.907+0000 7f9d8d3a2800 20 add_watcher() i=5 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.907+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94873934965184] v27'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.907+0000 7f9d8d3a2800 20 add_watcher() i=6 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.908+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94873934982112] v27'193 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.908+0000 7f9d8d3a2800 20 add_watcher() i=7 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.908+0000 7f9d8d3a2800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.908+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f3165c0 con 0x56498f306a90 2024-04-27T03:31:29.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.909+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.909+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94873934984336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f3165c0 con 0x56498f306a90 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.910+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94873934984336] v27'204 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.910+0000 7f9c417e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.910+0000 7f9d8d3a2800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.910+0000 7f9d8d3a2800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.910+0000 7f9d8d3a2800 5 note: GC not initialized 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.910+0000 7f9bbbfff640 20 reqs_thread_entry: start 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.910+0000 7f9d8d3a2800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.910+0000 7f9d8d3a2800 20 Filter name: none 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.910+0000 7f9bb9ffb640 20 reqs_thread_entry: start 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.910+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.e 6:74cd889a:usage::usage.0:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34dde0 con 0x56498f306a90 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.911+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 usage.0 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.911+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 6.3 6:c0e78a64:usage::usage.1:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34dde0 con 0x7f9d7c0b1070 2024-04-27T03:31:29.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.911+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(63 usage.1 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.911+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 6.d 6:b564efa4:usage::usage.2:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34dda0 con 0x56498f306a90 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.911+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 usage.2 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.911+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:65 6.2 6:42d64401:usage::usage.3:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34dda0 con 0x56498f30b430 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.911+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(65 usage.3 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.911+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 6.7 6:e09b6f01:usage::usage.4:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34e2e0 con 0x56498f30b430 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.911+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(66 usage.4 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.912+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.6 6:672810f1:usage::usage.5:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34e2e0 con 0x56498f30b430 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.912+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(67 usage.5 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.912+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.4 6:25a86931:usage::usage.6:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34e2e0 con 0x56498f306a90 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.912+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 usage.6 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.912+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 6.17 6:ebbe7322:usage::usage.7:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34e2e0 con 0x56498f306a90 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.912+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(69 usage.7 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.912+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:70 6.6 6:607e1922:usage::usage.8:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34e260 con 0x56498f30b430 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.912+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(70 usage.8 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.912+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.b 6:d7809252:usage::usage.9:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34e260 con 0x56498f30b430 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.913+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(71 usage.9 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 151+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.913+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34e260 con 0x7f9d84016440 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.913+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(72 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) ==== 152+0+1268 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.913+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:73 6.4 6:21f845ae:usage::usage.11:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f720 con 0x56498f306a90 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.913+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(73 usage.11 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.913+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.f 6:f0386354:usage::usage.12:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f720 con 0x7f9d7c0b1070 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.914+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(74 usage.12 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.914+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:75 6.c 6:3735eec4:usage::usage.13:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f720 con 0x56498f30b430 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.914+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(75 usage.13 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.914+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 6.18 6:1f520d0a:usage::usage.14:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f720 con 0x7f9d84016440 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.914+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(76 usage.14 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.914+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 6.0 6:029cee12:usage::usage.15:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f720 con 0x7f9d84016440 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.914+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(77 usage.15 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.914+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:78 6.1e 6:7f3e030e:usage::usage.16:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f720 con 0x56498f30b430 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.915+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(78 usage.16 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.915+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:79 6.10 6:0fbdcb32:usage::usage.17:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f720 con 0x7f9d84016440 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.915+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(79 usage.17 [call out=163b] v0'0 uv5 ondisk = 0) ==== 152+0+163 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.915+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.19 6:993b03e9:usage::usage.18:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f720 con 0x56498f306a90 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.915+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(80 usage.18 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.915+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:81 6.1a 6:58fcd270:usage::usage.19:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f720 con 0x56498f30b430 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.915+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(81 usage.19 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.915+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:82 6.1f 6:fcd63d49:usage::usage.20:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f720 con 0x7f9d84016440 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.916+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(82 usage.20 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.916+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.5 6:a2c1a412:usage::usage.21:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f720 con 0x7f9d7c0b1070 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.916+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(83 usage.21 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.936 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.916+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:84 6.13 6:cdb71d68:usage::usage.22:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f6d0 con 0x7f9d84016440 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.916+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(84 usage.22 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.916+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:85 6.17 6:ef3bca03:usage::usage.23:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f6d0 con 0x56498f306a90 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.916+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(85 usage.23 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.916+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:86 6.e 6:7412960a:usage::usage.24:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f6d0 con 0x56498f306a90 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.917+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(86 usage.24 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.917+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.7 6:e62d1024:usage::usage.25:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f6d0 con 0x56498f30b430 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.917+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(87 usage.25 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.917+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:88 6.d 6:b172cd59:usage::usage.26:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f6d0 con 0x56498f306a90 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.917+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(88 usage.26 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.917+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:89 6.1 6:83912023:usage::usage.27:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f6d0 con 0x56498f306a90 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.917+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(89 usage.27 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.917+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:90 6.1e 6:7c73bdb3:usage::usage.28:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f6d0 con 0x56498f30b430 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.918+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(90 usage.28 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.918+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.16 6:6e59fa35:usage::usage.29:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f6d0 con 0x56498f30b430 2024-04-27T03:31:29.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.918+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(91 usage.29 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.918+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:92 6.1c 6:38fe6885:usage::usage.30:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f6d0 con 0x56498f30b430 2024-04-27T03:31:29.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.918+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(92 usage.30 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.918+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:93 6.1d 6:bb28dff0:usage::usage.31:head [call rgw.user_usage_log_read in=60b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56498f34f6d0 con 0x7f9d84016440 2024-04-27T03:31:29.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.918+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(93 usage.31 [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.919+0000 7f9c417e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:29.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.919+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:94 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94873934984336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f34bcd0 con 0x56498f306a90 2024-04-27T03:31:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.920+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(94 data_loggenerations_metadata [watch unwatch cookie 94873934984336] v27'205 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.920+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:95 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94873934958160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f34bcd0 con 0x7f9d7c0b1070 2024-04-27T03:31:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.921+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(95 notify.0 [watch unwatch cookie 94873934958160] v27'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.921+0000 7f9d8d3a2800 20 remove_watcher() i=0 2024-04-27T03:31:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.921+0000 7f9d8d3a2800 2 removed watcher, disabling cache 2024-04-27T03:31:29.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.921+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:96 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94873934964128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f310e50 con 0x56498f306a90 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.922+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(96 notify.1 [watch unwatch cookie 94873934964128] v27'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.922+0000 7f9d8d3a2800 20 remove_watcher() i=1 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.922+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:97 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94873934966896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f310e50 con 0x56498f30b430 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.923+0000 7f9d8aea8640 1 -- 172.21.15.165:0/1093886287 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(97 notify.2 [watch unwatch cookie 94873934966896] v27'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d7c0ad070 con 0x56498f30b430 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.923+0000 7f9d8d3a2800 20 remove_watcher() i=2 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.923+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:98 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94873934971184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f310e50 con 0x7f9d7c0b1070 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.924+0000 7f9d8a6a7640 1 -- 172.21.15.165:0/1093886287 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(98 notify.3 [watch unwatch cookie 94873934971184] v27'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d80009050 con 0x7f9d7c0b1070 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.924+0000 7f9d8d3a2800 20 remove_watcher() i=3 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.924+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:99 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94873934974032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f314130 con 0x56498f306a90 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.925+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(99 notify.4 [watch unwatch cookie 94873934974032] v27'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x56498f306a90 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.925+0000 7f9d8d3a2800 20 remove_watcher() i=4 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.926+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:100 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94873934978320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f310e50 con 0x7f9d84016440 2024-04-27T03:31:29.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.927+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(100 notify.5 [watch unwatch cookie 94873934978320] v27'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.927+0000 7f9d8d3a2800 20 remove_watcher() i=5 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.927+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94873934965184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f314130 con 0x7f9d84016440 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.928+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(101 notify.6 [watch unwatch cookie 94873934965184] v27'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.928+0000 7f9d8d3a2800 20 remove_watcher() i=6 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.928+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94873934982112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56498f314130 con 0x7f9d84016440 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(102 notify.7 [watch unwatch cookie 94873934982112] v27'194 uv192 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9d8400a4a0 con 0x7f9d84016440 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 20 remove_watcher() i=7 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9d84016440 msgr2=0x7f9d840168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9d84016440 0x7f9d840168c0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56498f306a90 msgr2=0x56498f308f30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56498f306a90 0x56498f308f30 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9d7c0b1070 msgr2=0x7f9d7c0c4800 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f9d7c0b1070 0x7f9d7c0c4800 crc :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56498f30b430 msgr2=0x56498f30d820 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56498f30b430 0x56498f30d820 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c54036900 msgr2=0x7f9c54038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c54036900 0x7f9c54038dc0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x56498f2c84f0 tx=0x7f9d80010000 comp rx=0 tx=0).stop 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 msgr2=0x56498f2fb3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/1093886287 reap_dead start 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 0x56498f2fb3d0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c08cfd0 tx=0x7f9d7c07eec0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 shutdown_connections 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/1093886287 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2f3f80 0x56498f2fb3d0 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 >> 172.21.15.165:0/1093886287 conn(0x56498f2ebe90 msgr2=0x56498f2e94f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 shutdown_connections 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/1093886287 wait complete. 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3860942813 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c64036900 msgr2=0x7f9c64038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/3860942813 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c64036900 0x7f9c64038dc0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9d8000bf90 tx=0x7f9d8000f000 comp rx=0 tx=0).stop 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3860942813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2e2be0 msgr2=0x56498f2e3000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/3860942813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2e2be0 0x56498f2e3000 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c08c410 tx=0x7f9d7c0678b0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3860942813 shutdown_connections 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/3860942813 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c64036900 0x7f9c64038dc0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/3860942813 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f2e2be0 0x56498f2e3000 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3860942813 >> 172.21.15.165:0/3860942813 conn(0x56498f2d6c80 msgr2=0x56498f2d7080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3860942813 shutdown_connections 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.929+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3860942813 wait complete. 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56498f2c8ff0 msgr2=0x56498f2cb3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56498f2c8ff0 0x56498f2cb3e0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56498f2d1bd0 msgr2=0x56498f2d4020 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56498f2d1bd0 0x56498f2d4020 crc :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56498f2cdc90 msgr2=0x56498f2d0080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56498f2cdc90 0x56498f2d0080 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c70036900 msgr2=0x7f9c70038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9c70036900 0x7f9c70038dc0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f9d80000f60 tx=0x7f9d800010a0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1ccfb0 msgr2=0x56498f2c3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:29.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 --2- 172.21.15.165:0/3287673123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56498f1ccfb0 0x56498f2c3dc0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f9d7c055040 tx=0x7f9d7c002fb0 comp rx=0 tx=0).stop 2024-04-27T03:31:29.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8b6a9640 1 -- 172.21.15.165:0/3287673123 reap_dead start 2024-04-27T03:31:29.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 shutdown_connections 2024-04-27T03:31:29.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 >> 172.21.15.165:0/3287673123 conn(0x56498f1c7310 msgr2=0x56498f2b6200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:29.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 shutdown_connections 2024-04-27T03:31:29.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:29.930+0000 7f9d8d3a2800 1 -- 172.21.15.165:0/3287673123 wait complete. 2024-04-27T03:31:29.949 INFO:teuthology.orchestra.run.smithi165.stdout:{"entries":[{"user":"anonymous","buckets":[{"bucket":"","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"anonymous","categories":[{"category":"list_buckets","bytes_sent":204,"bytes_received":0,"ops":1,"successful_ops":1}]}]},{"user":"foo","buckets":[{"bucket":"","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}]},{"bucket":"myfoo2","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo3","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo4","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]}]}],"summary":[{"user":"anonymous","categories":[{"category":"list_buckets","bytes_sent":204,"bytes_received":0,"ops":1,"successful_ops":1}],"total":{"bytes_sent":204,"bytes_received":0,"ops":1,"successful_ops":1}},{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":2103,"bytes_received":30000042,"ops":35,"successful_ops":34}}]} 2024-04-27T03:31:29.949 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34}}]} 2024-04-27T03:31:29.950 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-04-27T03:31:29.950 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-04-27T03:31:29.950 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-04-27T03:31:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:30.032 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.026+0000 7f68a7439800 1 Processor -- start 2024-04-27T03:31:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.026+0000 7f68a7439800 1 -- start start 2024-04-27T03:31:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.026+0000 7f68a7439800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560fefb012c0 0x560fefb09dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.026+0000 7f68a7439800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x560fef9bfd40 con 0x560fefaff780 2024-04-27T03:31:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.026+0000 7f68a7439800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x560fef9924d0 con 0x560fefb012c0 2024-04-27T03:31:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.027+0000 7f689bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560fefb012c0 0x560fefb09dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.027+0000 7f689bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560fefb012c0 0x560fefb09dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49322/0 (socket says 172.21.15.165:49322) 2024-04-27T03:31:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.027+0000 7f689bfff640 1 -- 172.21.15.165:0/4204173334 learned_addr learned my addr 172.21.15.165:0/4204173334 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.027+0000 7f68a206e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x560fefaff780 0x560fefaffba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:54706/0 (socket says 172.21.15.165:54706) 2024-04-27T03:31:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.027+0000 7f689bfff640 1 -- 172.21.15.165:0/4204173334 >> v1:172.21.15.89:6789/0 conn(0x560fefaff780 legacy=0x560fefaffba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.027+0000 7f689bfff640 1 -- 172.21.15.165:0/4204173334 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fef9be590 con 0x560fefb012c0 2024-04-27T03:31:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.027+0000 7f68a186d640 1 -- 172.21.15.165:0/4204173334 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 194588623 0 0) 0x560fef9bfd40 con 0x560fefaff780 2024-04-27T03:31:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.029+0000 7f689bfff640 1 --2- 172.21.15.165:0/4204173334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefb012c0 0x560fefb09dc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f688c00a520 tx=0x7f688c01d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c4ea68a6df668d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.029+0000 7f68a186d640 1 -- 172.21.15.165:0/4204173334 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f688c057070 con 0x560fefb012c0 2024-04-27T03:31:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.029+0000 7f68a186d640 1 -- 172.21.15.165:0/4204173334 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f688c052570 con 0x560fefb012c0 2024-04-27T03:31:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.029+0000 7f68a186d640 1 -- 172.21.15.165:0/4204173334 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f688c052870 con 0x560fefb012c0 2024-04-27T03:31:30.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.029+0000 7f68a7439800 1 -- 172.21.15.165:0/4204173334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefb012c0 msgr2=0x560fefb09dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.029+0000 7f68a7439800 1 --2- 172.21.15.165:0/4204173334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefb012c0 0x560fefb09dc0 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7f688c00a520 tx=0x7f688c01d010 comp rx=0 tx=0).stop 2024-04-27T03:31:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.029+0000 7f68a7439800 1 -- 172.21.15.165:0/4204173334 shutdown_connections 2024-04-27T03:31:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.029+0000 7f68a7439800 1 --2- 172.21.15.165:0/4204173334 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefb012c0 0x560fefb09dc0 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.029+0000 7f68a7439800 1 -- 172.21.15.165:0/4204173334 >> 172.21.15.165:0/4204173334 conn(0x560fefafb320 msgr2=0x560fefafd740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.030+0000 7f68a7439800 1 -- 172.21.15.165:0/4204173334 shutdown_connections 2024-04-27T03:31:30.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.030+0000 7f68a7439800 1 -- 172.21.15.165:0/4204173334 wait complete. 2024-04-27T03:31:30.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.041+0000 7f68a7439800 1 Processor -- start 2024-04-27T03:31:30.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.041+0000 7f68a7439800 1 -- start start 2024-04-27T03:31:30.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.041+0000 7f68a7439800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 0x560fefbf4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.042+0000 7f68a7439800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560fefa4c3c0 con 0x560fefaff780 2024-04-27T03:31:30.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.042+0000 7f68a206e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 0x560fefbf4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.042+0000 7f68a206e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 0x560fefbf4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49330/0 (socket says 172.21.15.165:49330) 2024-04-27T03:31:30.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.042+0000 7f68a206e640 1 -- 172.21.15.165:0/2816639094 learned_addr learned my addr 172.21.15.165:0/2816639094 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.042+0000 7f68a206e640 1 -- 172.21.15.165:0/2816639094 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fefbf4a30 con 0x560fefaff780 2024-04-27T03:31:30.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.042+0000 7f68a206e640 1 --2- 172.21.15.165:0/2816639094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 0x560fefbf4460 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f689400b210 tx=0x7f689400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63eabe520179d81 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.042+0000 7f6790608640 1 -- 172.21.15.165:0/2816639094 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6894008070 con 0x560fefaff780 2024-04-27T03:31:30.050 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.042+0000 7f6790608640 1 -- 172.21.15.165:0/2816639094 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f689400d500 con 0x560fefaff780 2024-04-27T03:31:30.051 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.042+0000 7f6790608640 1 -- 172.21.15.165:0/2816639094 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f689400d840 con 0x560fefaff780 2024-04-27T03:31:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.043+0000 7f68a7439800 1 -- 172.21.15.165:0/2816639094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 msgr2=0x560fefbf4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.043+0000 7f68a7439800 1 --2- 172.21.15.165:0/2816639094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 0x560fefbf4460 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f689400b210 tx=0x7f689400b6d0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.043+0000 7f68a7439800 1 -- 172.21.15.165:0/2816639094 shutdown_connections 2024-04-27T03:31:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.043+0000 7f68a7439800 1 --2- 172.21.15.165:0/2816639094 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 0x560fefbf4460 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.043+0000 7f68a7439800 1 -- 172.21.15.165:0/2816639094 >> 172.21.15.165:0/2816639094 conn(0x560fefafb320 msgr2=0x560fefbefd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.043+0000 7f68a7439800 1 -- 172.21.15.165:0/2816639094 shutdown_connections 2024-04-27T03:31:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.043+0000 7f68a7439800 1 -- 172.21.15.165:0/2816639094 wait complete. 2024-04-27T03:31:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.043+0000 7f68a7439800 1 Processor -- start 2024-04-27T03:31:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.044+0000 7f68a7439800 1 -- start start 2024-04-27T03:31:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.044+0000 7f68a7439800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 0x560fefbfc000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.044+0000 7f68a7439800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560fefb0c5e0 con 0x560fefaff780 2024-04-27T03:31:30.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.044+0000 7f68a206e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 0x560fefbfc000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.044+0000 7f68a206e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 0x560fefbfc000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49340/0 (socket says 172.21.15.165:49340) 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.044+0000 7f68a206e640 1 -- 172.21.15.165:0/1851518565 learned_addr learned my addr 172.21.15.165:0/1851518565 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.044+0000 7f68a206e640 1 -- 172.21.15.165:0/1851518565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fefbfc540 con 0x560fefaff780 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.044+0000 7f68a206e640 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 0x560fefbfc000 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6894001e60 tx=0x7f6894000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.045+0000 7f678ee05640 1 -- 172.21.15.165:0/1851518565 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6894019870 con 0x560fefaff780 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.045+0000 7f678ee05640 1 -- 172.21.15.165:0/1851518565 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6894019a10 con 0x560fefaff780 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.045+0000 7f678ee05640 1 -- 172.21.15.165:0/1851518565 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6894019cf0 con 0x560fefaff780 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.045+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560fefbfc7d0 con 0x560fefaff780 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.045+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fefbff480 con 0x560fefaff780 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.046+0000 7f678ee05640 1 -- 172.21.15.165:0/1851518565 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6894017040 con 0x560fefaff780 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.046+0000 7f678ee05640 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6788036950 0x7f6788038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.046+0000 7f678ee05640 1 -- 172.21.15.165:0/1851518565 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f6894015020 con 0x560fefaff780 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.046+0000 7f689bfff640 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6788036950 0x7f6788038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.046+0000 7f68a7439800 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560fefb012c0 0x560fefc03240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.046+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc03780 con 0x560fefb012c0 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.046+0000 7f68a286f640 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560fefb012c0 0x560fefc03240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.046+0000 7f689bfff640 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6788036950 0x7f6788038e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f688c054000 tx=0x7f688c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.046+0000 7f68a286f640 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560fefb012c0 0x560fefc03240 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.047+0000 7f68a286f640 1 -- 172.21.15.165:0/1851518565 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefb012c0 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.047+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc00810 con 0x560fefb012c0 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.047+0000 7f68a286f640 1 -- 172.21.15.165:0/1851518565 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefb012c0 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.047+0000 7f68a7439800 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560fefc05a90 0x560fefc07e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.047+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc08520 con 0x560fefc05a90 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.047+0000 7f68a206e640 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560fefc05a90 0x560fefc07e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.047+0000 7f68a206e640 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560fefc05a90 0x560fefc07e80 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.048+0000 7f68a206e640 1 -- 172.21.15.165:0/1851518565 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f689402d070 con 0x560fefc05a90 2024-04-27T03:31:30.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.048+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc04a40 con 0x560fefb012c0 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.048+0000 7f68a286f640 1 -- 172.21.15.165:0/1851518565 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefb012c0 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.048+0000 7f68a7439800 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x560fefc099d0 0x560fefc0be20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.048+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc0c580 con 0x560fefc099d0 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.048+0000 7f689bfff640 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x560fefc099d0 0x560fefc0be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.048+0000 7f689bfff640 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x560fefc099d0 0x560fefc0be20 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.049+0000 7f689bfff640 1 -- 172.21.15.165:0/1851518565 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f688c05a040 con 0x560fefc099d0 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.049+0000 7f68a7439800 1 Processor -- start 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.049+0000 7f68a7439800 1 -- start start 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.049+0000 7f68a7439800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc1a9e0 0x560fefc1ae00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.049+0000 7f68a7439800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560fef9f26b0 con 0x560fefc1a9e0 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.049+0000 7f68a206e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc1a9e0 0x560fefc1ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.049+0000 7f68a206e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc1a9e0 0x560fefc1ae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49356/0 (socket says 172.21.15.165:49356) 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.049+0000 7f68a206e640 1 -- 172.21.15.165:0/3318403461 learned_addr learned my addr 172.21.15.165:0/3318403461 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.050+0000 7f68a206e640 1 -- 172.21.15.165:0/3318403461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fefc1b340 con 0x560fefc1a9e0 2024-04-27T03:31:30.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.050+0000 7f68a206e640 1 --2- 172.21.15.165:0/3318403461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc1a9e0 0x560fefc1ae00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f689404b3a0 tx=0x7f689400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.050+0000 7f6787fff640 1 -- 172.21.15.165:0/3318403461 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f68940449d0 con 0x560fefc1a9e0 2024-04-27T03:31:30.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.050+0000 7f6787fff640 1 -- 172.21.15.165:0/3318403461 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6894044b70 con 0x560fefc1a9e0 2024-04-27T03:31:30.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.050+0000 7f68a7439800 1 -- 172.21.15.165:0/3318403461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560fefc1bdb0 con 0x560fefc1a9e0 2024-04-27T03:31:30.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.050+0000 7f6787fff640 1 -- 172.21.15.165:0/3318403461 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6894044e50 con 0x560fefc1a9e0 2024-04-27T03:31:30.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.050+0000 7f68a7439800 1 -- 172.21.15.165:0/3318403461 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fefc1c2b0 con 0x560fefc1a9e0 2024-04-27T03:31:30.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.051+0000 7f6787fff640 1 -- 172.21.15.165:0/3318403461 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f689403f1b0 con 0x560fefc1a9e0 2024-04-27T03:31:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.051+0000 7f6787fff640 1 --2- 172.21.15.165:0/3318403461 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f677c036900 0x7f677c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.051+0000 7f6787fff640 1 -- 172.21.15.165:0/3318403461 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f689403cb90 con 0x560fefc1a9e0 2024-04-27T03:31:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.051+0000 7f689bfff640 1 --2- 172.21.15.165:0/3318403461 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f677c036900 0x7f677c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.051+0000 7f68a7439800 1 Processor -- start 2024-04-27T03:31:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.051+0000 7f68a7439800 1 -- start start 2024-04-27T03:31:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.051+0000 7f689bfff640 1 --2- 172.21.15.165:0/3318403461 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f677c036900 0x7f677c038dc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x560fefc00730 tx=0x7f688c005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.052+0000 7f68a7439800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 0x560fefc280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.052+0000 7f68a7439800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560fefbf4db0 con 0x560fefc27ca0 2024-04-27T03:31:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.052+0000 7f68a206e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 0x560fefc280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.052+0000 7f68a206e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 0x560fefc280c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49360/0 (socket says 172.21.15.165:49360) 2024-04-27T03:31:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.052+0000 7f68a206e640 1 -- 172.21.15.165:0/4161070215 learned_addr learned my addr 172.21.15.165:0/4161070215 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.078 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.052+0000 7f68a206e640 1 -- 172.21.15.165:0/4161070215 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fefc286b0 con 0x560fefc27ca0 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.052+0000 7f68a206e640 1 --2- 172.21.15.165:0/4161070215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 0x560fefc280c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f689401aaa0 tx=0x7f68940442c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d547d4b868a15218 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.052+0000 7f6785ffb640 1 -- 172.21.15.165:0/4161070215 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f689405c020 con 0x560fefc27ca0 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.052+0000 7f6785ffb640 1 -- 172.21.15.165:0/4161070215 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6894026d00 con 0x560fefc27ca0 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.052+0000 7f6785ffb640 1 -- 172.21.15.165:0/4161070215 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6894040980 con 0x560fefc27ca0 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a7439800 1 -- 172.21.15.165:0/4161070215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 msgr2=0x560fefc280c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a7439800 1 --2- 172.21.15.165:0/4161070215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 0x560fefc280c0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f689401aaa0 tx=0x7f68940442c0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a7439800 1 -- 172.21.15.165:0/4161070215 shutdown_connections 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a7439800 1 --2- 172.21.15.165:0/4161070215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 0x560fefc280c0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a7439800 1 -- 172.21.15.165:0/4161070215 >> 172.21.15.165:0/4161070215 conn(0x560fefc23fe0 msgr2=0x560fefc22040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a7439800 1 -- 172.21.15.165:0/4161070215 shutdown_connections 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a7439800 1 -- 172.21.15.165:0/4161070215 wait complete. 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a7439800 1 Processor -- start 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a7439800 1 -- start start 2024-04-27T03:31:30.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a7439800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 0x560fefc2f090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a7439800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560fefc28dd0 con 0x560fefc27ca0 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a206e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 0x560fefc2f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a206e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 0x560fefc2f090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49368/0 (socket says 172.21.15.165:49368) 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.053+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 learned_addr learned my addr 172.21.15.165:0/4230586893 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.054+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560fefc2f5d0 con 0x560fefc27ca0 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.054+0000 7f68a206e640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 0x560fefc2f090 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6894044300 tx=0x7f689404ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.054+0000 7f6773fff640 1 -- 172.21.15.165:0/4230586893 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f689400fcb0 con 0x560fefc27ca0 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.054+0000 7f6773fff640 1 -- 172.21.15.165:0/4230586893 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f689400fe50 con 0x560fefc27ca0 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.054+0000 7f6773fff640 1 -- 172.21.15.165:0/4230586893 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f689400f740 con 0x560fefc27ca0 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.054+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560fefc2f5d0 con 0x560fefc27ca0 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.054+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560fefc2f5d0 con 0x560fefc27ca0 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.055+0000 7f6773fff640 1 -- 172.21.15.165:0/4230586893 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f689405f020 con 0x560fefc27ca0 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.055+0000 7f6773fff640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f676c036900 0x7f676c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.055+0000 7f68a7439800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.055+0000 7f6773fff640 1 -- 172.21.15.165:0/4230586893 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f6894052030 con 0x560fefc27ca0 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.055+0000 7f689bfff640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f676c036900 0x7f676c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.055+0000 7f68a7439800 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560fefc3a670 0x560fefc3cb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.055+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3d260 con 0x560fefc3a670 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a286f640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560fefc3a670 0x560fefc3cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f689bfff640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f676c036900 0x7f676c038dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f688c052fd0 tx=0x7f688c04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a286f640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560fefc3a670 0x560fefc3cb10 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a7439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a7439800 20 realm 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a7439800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3d670 con 0x560fefc3a670 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a7439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a7439800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a7439800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3df10 con 0x560fefc3a670 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a7439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.056+0000 7f68a7439800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.057+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3a670 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.057+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.057+0000 7f68a7439800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.057+0000 7f68a7439800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.057+0000 7f68a7439800 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560fefc3f010 0x560fefc41400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.057+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc41940 con 0x560fefc3f010 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.057+0000 7f68a206e640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560fefc3f010 0x560fefc41400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.057+0000 7f68a206e640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560fefc3f010 0x560fefc41400 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.057+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.057+0000 7f68a7439800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.057+0000 7f68a7439800 20 searching for the correct realm 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.057+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc432f0 con 0x560fefc3f010 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.058+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.058+0000 7f68a206e640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6894061050 0x7f6894030450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.058+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc432f0 con 0x7f6894061050 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.058+0000 7f689bfff640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6894061050 0x7f6894030450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.058+0000 7f689bfff640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6894061050 0x7f6894030450 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.059+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.059+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc432f0 con 0x7f6894061050 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.059+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.059+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc432f0 con 0x560fefc3a670 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.059+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.059+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3a670 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.059+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.059+0000 7f68a286f640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f689c016440 0x7f689c0168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.059+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f689c016440 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.060+0000 7f68a286f640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f689c016440 0x7f689c0168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.060+0000 7f68a286f640 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f689c016440 0x7f689c0168c0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.060+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.061+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f689c016440 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.061+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.061+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f6894061050 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.061+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.061+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3f010 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.061+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.061+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3a670 2024-04-27T03:31:30.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.062+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.062+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f6894061050 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.062+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.062+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3a670 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.062+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.062+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f6894061050 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.062+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.062+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3f010 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.063+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.063+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3f010 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.063+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.063+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f689c016440 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.063+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.063+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3f010 2024-04-27T03:31:30.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.063+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.063+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3f010 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.064+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.064+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3a670 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.064+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.064+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3a670 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.064+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.064+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f689c016440 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.064+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.064+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3f010 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.065+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.065+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f6894061050 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.065+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.065+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3a670 2024-04-27T03:31:30.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.065+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.065+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f689c016440 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.066+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.066+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3f010 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.066+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.066+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f689c016440 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.066+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.066+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f6894061050 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.066+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.066+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3f010 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.067+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.067+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f689c016440 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.067+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.067+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x560fefc3a670 2024-04-27T03:31:30.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.067+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.067+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560fefc3e2b0 con 0x7f6894061050 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc422c0 con 0x560fefc3a670 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc426b0 con 0x560fefc3a670 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc426b0 con 0x7f6894061050 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 zone default found 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 4 Realm: () 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 zonegroup default 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.068+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc437c0 con 0x7f689c016440 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.069+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.069+0000 7f68a7439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.069+0000 7f68a7439800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.069+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc451a0 con 0x560fefc3f010 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.069+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.069+0000 7f68a7439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.069+0000 7f68a7439800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.069+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc45540 con 0x560fefc3f010 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.069+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.070+0000 7f68a7439800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.070+0000 7f68a7439800 20 started sync module instance, tier type = 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.070+0000 7f68a7439800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.070+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc44640 con 0x7f6894061050 2024-04-27T03:31:30.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.071+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.071+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94626447116848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc442a0 con 0x7f6894061050 2024-04-27T03:31:30.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.071+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc462c0 con 0x560fefc3a670 2024-04-27T03:31:30.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.072+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94626447116848] v27'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.072+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.072+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94626447120608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc466b0 con 0x560fefc3a670 2024-04-27T03:31:30.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.072+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc47110 con 0x560fefc3f010 2024-04-27T03:31:30.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.073+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.073+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94626447125440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc479b0 con 0x560fefc3f010 2024-04-27T03:31:30.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.074+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc484b0 con 0x7f6894061050 2024-04-27T03:31:30.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.074+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94626447120608] v27'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.075+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.075+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94626447121664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc48d50 con 0x7f6894061050 2024-04-27T03:31:30.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.075+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc49710 con 0x560fefc3a670 2024-04-27T03:31:30.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.075+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94626447125440] v27'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.076+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94626447121664] v27'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.076+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.076+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94626447127808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc49710 con 0x560fefc3a670 2024-04-27T03:31:30.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.076+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc49f60 con 0x7f689c016440 2024-04-27T03:31:30.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.077+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.077+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94626447137008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc4a3c0 con 0x7f689c016440 2024-04-27T03:31:30.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.077+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc4aee0 con 0x7f689c016440 2024-04-27T03:31:30.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.077+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94626447127808] v27'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.078+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94626447137008] v27'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.078+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.078+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94626447141296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc49e10 con 0x7f689c016440 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.078+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc4b350 con 0x7f689c016440 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.079+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94626447141296] v27'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.080+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'195 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.080+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94626447135328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc4b350 con 0x7f689c016440 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.080+0000 7f68a7439800 20 add_watcher() i=0 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.080+0000 7f68a7439800 20 add_watcher() i=1 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.080+0000 7f68a7439800 20 add_watcher() i=2 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.080+0000 7f68a7439800 20 add_watcher() i=3 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.080+0000 7f68a7439800 20 add_watcher() i=4 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.080+0000 7f68a7439800 20 add_watcher() i=5 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.080+0000 7f68a7439800 20 add_watcher() i=6 2024-04-27T03:31:30.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.080+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94626447135328] v27'196 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.080+0000 7f68a7439800 20 add_watcher() i=7 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.080+0000 7f68a7439800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.081+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc4e750 con 0x560fefc3a670 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.081+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.081+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94626447143024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc4bca0 con 0x560fefc3a670 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.082+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94626447143024] v27'206 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.082+0000 7f6758fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.082+0000 7f68a7439800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.082+0000 7f68a7439800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.082+0000 7f68a7439800 5 note: GC not initialized 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.082+0000 7f66d37fe640 20 reqs_thread_entry: start 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.082+0000 7f68a7439800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.082+0000 7f68a7439800 20 Filter name: none 2024-04-27T03:31:30.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.082+0000 7f66d17fa640 20 reqs_thread_entry: start 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.083+0000 7f68a7439800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.083+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc81610 con 0x560fefc3a670 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.083+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.083+0000 7f68a7439800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.083+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560fefc82b60 con 0x7f689c016440 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.084+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) ==== 152+0+1268 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.084+0000 7f6758fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.084+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94626447143024] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc80a40 con 0x560fefc3a670 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.085+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94626447143024] v27'207 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.085+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94626447116848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc80a40 con 0x7f6894061050 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.086+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94626447116848] v27'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.086+0000 7f68a7439800 20 remove_watcher() i=0 2024-04-27T03:31:30.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.086+0000 7f68a7439800 2 removed watcher, disabling cache 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.086+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94626447120608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc80a40 con 0x560fefc3a670 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.088+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94626447120608] v27'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.088+0000 7f68a7439800 20 remove_watcher() i=1 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.088+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94626447125440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc458e0 con 0x560fefc3f010 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.089+0000 7f68a206e640 1 -- 172.21.15.165:0/4230586893 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94626447125440] v27'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6894055070 con 0x560fefc3f010 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.089+0000 7f68a7439800 20 remove_watcher() i=2 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.089+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94626447121664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc80a40 con 0x7f6894061050 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.090+0000 7f689bfff640 1 -- 172.21.15.165:0/4230586893 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94626447121664] v27'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f688c07f070 con 0x7f6894061050 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.090+0000 7f68a7439800 20 remove_watcher() i=3 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.090+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94626447127808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc80a40 con 0x560fefc3a670 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.091+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94626447127808] v27'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x560fefc3a670 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.091+0000 7f68a7439800 20 remove_watcher() i=4 2024-04-27T03:31:30.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.091+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94626447137008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc47500 con 0x7f689c016440 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.092+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94626447137008] v27'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.092+0000 7f68a7439800 20 remove_watcher() i=5 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.092+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94626447141296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc47500 con 0x7f689c016440 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.093+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94626447141296] v27'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.093+0000 7f68a7439800 20 remove_watcher() i=6 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.093+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94626447135328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560fefc47500 con 0x7f689c016440 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.094+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94626447135328] v27'197 uv195 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f689c00a4a0 con 0x7f689c016440 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.094+0000 7f68a7439800 20 remove_watcher() i=7 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f689c016440 msgr2=0x7f689c0168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f689c016440 0x7f689c0168c0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560fefc3a670 msgr2=0x560fefc3cb10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560fefc3a670 0x560fefc3cb10 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6894061050 msgr2=0x7f6894030450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6894061050 0x7f6894030450 crc :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560fefc3f010 msgr2=0x560fefc41400 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560fefc3f010 0x560fefc41400 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f676c036900 msgr2=0x7f676c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f676c036900 0x7f676c038dc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f688c052fd0 tx=0x7f688c04f000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 msgr2=0x560fefc2f090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a286f640 1 -- 172.21.15.165:0/4230586893 reap_dead start 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 0x560fefc2f090 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f6894044300 tx=0x7f689404ce20 comp rx=0 tx=0).stop 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 shutdown_connections 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 --2- 172.21.15.165:0/4230586893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc27ca0 0x560fefc2f090 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 >> 172.21.15.165:0/4230586893 conn(0x560fefc23fe0 msgr2=0x560fefc21640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 shutdown_connections 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/4230586893 wait complete. 2024-04-27T03:31:30.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/3318403461 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f677c036900 msgr2=0x7f677c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 --2- 172.21.15.165:0/3318403461 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f677c036900 0x7f677c038dc0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x560fefc00730 tx=0x7f688c005ac0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/3318403461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc1a9e0 msgr2=0x560fefc1ae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 --2- 172.21.15.165:0/3318403461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc1a9e0 0x560fefc1ae00 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f689404b3a0 tx=0x7f689400ed80 comp rx=0 tx=0).stop 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/3318403461 shutdown_connections 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 --2- 172.21.15.165:0/3318403461 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f677c036900 0x7f677c038dc0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 --2- 172.21.15.165:0/3318403461 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefc1a9e0 0x560fefc1ae00 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/3318403461 >> 172.21.15.165:0/3318403461 conn(0x560fefc0ea80 msgr2=0x560fefc0ee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/3318403461 shutdown_connections 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.095+0000 7f68a7439800 1 -- 172.21.15.165:0/3318403461 wait complete. 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560fefb012c0 msgr2=0x560fefc03240 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560fefb012c0 0x560fefc03240 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x560fefc099d0 msgr2=0x560fefc0be20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x560fefc099d0 0x560fefc0be20 crc :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560fefc05a90 msgr2=0x560fefc07e80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560fefc05a90 0x560fefc07e80 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6788036950 msgr2=0x7f6788038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6788036950 0x7f6788038e10 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f688c054000 tx=0x7f688c04e000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 msgr2=0x560fefbfc000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 --2- 172.21.15.165:0/1851518565 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560fefaff780 0x560fefbfc000 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f6894001e60 tx=0x7f6894000ed0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a286f640 1 -- 172.21.15.165:0/1851518565 reap_dead start 2024-04-27T03:31:30.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 shutdown_connections 2024-04-27T03:31:30.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 >> 172.21.15.165:0/1851518565 conn(0x560fefafb320 msgr2=0x560fefbee5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 shutdown_connections 2024-04-27T03:31:30.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.097+0000 7f68a7439800 1 -- 172.21.15.165:0/1851518565 wait complete. 2024-04-27T03:31:30.111 INFO:teuthology.orchestra.run.smithi165.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}]},{"bucket":"myfoo2","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo3","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo4","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]}]}],"summary":[{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8},{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4},{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4},{"category":"get_acls","bytes_sent":1026,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"get_obj","bytes_sent":4,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_bucket","bytes_sent":515,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"list_buckets","bytes_sent":322,"bytes_received":0,"ops":1,"successful_ops":1},{"category":"put_acls","bytes_sent":0,"bytes_received":0,"ops":2,"successful_ops":2},{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":2103,"bytes_received":30000042,"ops":35,"successful_ops":34}}]} 2024-04-27T03:31:30.112 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}, {'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}, {'category': 'get_acls', 'bytes_sent': 1026, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'get_obj', 'bytes_sent': 4, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_bucket', 'bytes_sent': 515, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'list_buckets', 'bytes_sent': 322, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_acls', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 2103, 'bytes_received': 30000042, 'ops': 35, 'successful_ops': 34}}]} 2024-04-27T03:31:30.112 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-04-27T03:31:30.112 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'create_bucket'] 2024-04-27T03:31:30.112 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories create_bucket 2024-04-27T03:31:30.198 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:30.199 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.193+0000 7f62bdf2f800 1 Processor -- start 2024-04-27T03:31:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.193+0000 7f62bdf2f800 1 -- start start 2024-04-27T03:31:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.193+0000 7f62bdf2f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x563129594300 0x56312959ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.193+0000 7f62bdf2f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x563129452d40 con 0x5631295927c0 2024-04-27T03:31:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.193+0000 7f62bdf2f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5631294254d0 con 0x563129594300 2024-04-27T03:31:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.193+0000 7f62b93b4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x563129594300 0x56312959ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.193+0000 7f62b9bb5640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5631295927c0 0x563129592be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:54720/0 (socket says 172.21.15.165:54720) 2024-04-27T03:31:30.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.193+0000 7f62b9bb5640 1 -- 172.21.15.165:0/2127234642 learned_addr learned my addr 172.21.15.165:0/2127234642 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:31:30.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.193+0000 7f62b93b4640 1 -- 172.21.15.165:0/2127234642 >> v1:172.21.15.89:6789/0 conn(0x5631295927c0 legacy=0x563129592be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.201 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.193+0000 7f62b93b4640 1 -- 172.21.15.165:0/2127234642 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x563129451590 con 0x563129594300 2024-04-27T03:31:30.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.195+0000 7f62b93b4640 1 --2- 172.21.15.165:0/2127234642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563129594300 0x56312959ce00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f62a400a470 tx=0x7f62a401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=877d0cd2a7b7c2de server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.195+0000 7f62b8bb3640 1 -- 172.21.15.165:0/2127234642 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62a4057070 con 0x563129594300 2024-04-27T03:31:30.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.195+0000 7f62b8bb3640 1 -- 172.21.15.165:0/2127234642 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62a4052440 con 0x563129594300 2024-04-27T03:31:30.202 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.195+0000 7f62b8bb3640 1 -- 172.21.15.165:0/2127234642 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62a4052740 con 0x563129594300 2024-04-27T03:31:30.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.196+0000 7f62bdf2f800 1 -- 172.21.15.165:0/2127234642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563129594300 msgr2=0x56312959ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.196+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/2127234642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563129594300 0x56312959ce00 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f62a400a470 tx=0x7f62a401ce90 comp rx=0 tx=0).stop 2024-04-27T03:31:30.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.196+0000 7f62bdf2f800 1 -- 172.21.15.165:0/2127234642 shutdown_connections 2024-04-27T03:31:30.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.196+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/2127234642 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563129594300 0x56312959ce00 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.196+0000 7f62bdf2f800 1 -- 172.21.15.165:0/2127234642 >> 172.21.15.165:0/2127234642 conn(0x56312958e340 msgr2=0x563129590760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.196+0000 7f62bdf2f800 1 -- 172.21.15.165:0/2127234642 shutdown_connections 2024-04-27T03:31:30.203 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.196+0000 7f62bdf2f800 1 -- 172.21.15.165:0/2127234642 wait complete. 2024-04-27T03:31:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.209+0000 7f62bdf2f800 1 Processor -- start 2024-04-27T03:31:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.209+0000 7f62bdf2f800 1 -- start start 2024-04-27T03:31:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.209+0000 7f62bdf2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 0x563129687580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.209+0000 7f62bdf2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5631294df3b0 con 0x5631295927c0 2024-04-27T03:31:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.209+0000 7f62b9bb5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 0x563129687580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.209+0000 7f62b9bb5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 0x563129687580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49382/0 (socket says 172.21.15.165:49382) 2024-04-27T03:31:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.209+0000 7f62b9bb5640 1 -- 172.21.15.165:0/3310637078 learned_addr learned my addr 172.21.15.165:0/3310637078 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.209+0000 7f62b9bb5640 1 -- 172.21.15.165:0/3310637078 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563129687b50 con 0x5631295927c0 2024-04-27T03:31:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.209+0000 7f62b9bb5640 1 --2- 172.21.15.165:0/3310637078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 0x563129687580 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f62b40017a0 tx=0x7f62b4001c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a5d06c8995de255 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.209+0000 7f61a7e07640 1 -- 172.21.15.165:0/3310637078 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62b4004520 con 0x5631295927c0 2024-04-27T03:31:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.209+0000 7f61a7e07640 1 -- 172.21.15.165:0/3310637078 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b40046c0 con 0x5631295927c0 2024-04-27T03:31:30.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.209+0000 7f61a7e07640 1 -- 172.21.15.165:0/3310637078 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62b4004a00 con 0x5631295927c0 2024-04-27T03:31:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3310637078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 msgr2=0x563129687580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/3310637078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 0x563129687580 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f62b40017a0 tx=0x7f62b4001c60 comp rx=0 tx=0).stop 2024-04-27T03:31:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3310637078 shutdown_connections 2024-04-27T03:31:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/3310637078 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 0x563129687580 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3310637078 >> 172.21.15.165:0/3310637078 conn(0x56312958e340 msgr2=0x563129682ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3310637078 shutdown_connections 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3310637078 wait complete. 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62bdf2f800 1 Processor -- start 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62bdf2f800 1 -- start start 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62bdf2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 0x56312968f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62bdf2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56312959f620 con 0x5631295927c0 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62b9bb5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 0x56312968f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62b9bb5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 0x56312968f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49398/0 (socket says 172.21.15.165:49398) 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.210+0000 7f62b9bb5640 1 -- 172.21.15.165:0/3427556556 learned_addr learned my addr 172.21.15.165:0/3427556556 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.211+0000 7f62b9bb5640 1 -- 172.21.15.165:0/3427556556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56312968f680 con 0x5631295927c0 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.211+0000 7f62b9bb5640 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 0x56312968f140 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f62b4000f80 tx=0x7f62b4001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.211+0000 7f61a6604640 1 -- 172.21.15.165:0/3427556556 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62b4022020 con 0x5631295927c0 2024-04-27T03:31:30.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.211+0000 7f61a6604640 1 -- 172.21.15.165:0/3427556556 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b4014870 con 0x5631295927c0 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.211+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56312968f910 con 0x5631295927c0 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.211+0000 7f61a6604640 1 -- 172.21.15.165:0/3427556556 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62b4014bb0 con 0x5631295927c0 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.211+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563129692520 con 0x5631295927c0 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.212+0000 7f61a6604640 1 -- 172.21.15.165:0/3427556556 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f62b4025020 con 0x5631295927c0 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.212+0000 7f61a6604640 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f61a0036950 0x7f61a0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.212+0000 7f61a6604640 1 -- 172.21.15.165:0/3427556556 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f62b4018030 con 0x5631295927c0 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.212+0000 7f62b93b4640 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f61a0036950 0x7f61a0038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.212+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563129594300 0x5631296962e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.212+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563129696870 con 0x563129594300 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.212+0000 7f62ba3b6640 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563129594300 0x5631296962e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.213+0000 7f62b93b4640 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f61a0036950 0x7f61a0038e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62a4054000 tx=0x7f62a404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.213+0000 7f62ba3b6640 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563129594300 0x5631296962e0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.213+0000 7f62ba3b6640 1 -- 172.21.15.165:0/3427556556 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x563129594300 2024-04-27T03:31:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.213+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563129693890 con 0x563129594300 2024-04-27T03:31:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.213+0000 7f62ba3b6640 1 -- 172.21.15.165:0/3427556556 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f62ac0083e0 con 0x563129594300 2024-04-27T03:31:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.213+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563129698ba0 0x56312969af90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.213+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56312969b680 con 0x563129698ba0 2024-04-27T03:31:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.213+0000 7f62b9bb5640 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563129698ba0 0x56312969af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.214+0000 7f62b9bb5640 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563129698ba0 0x56312969af90 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.214+0000 7f62b9bb5640 1 -- 172.21.15.165:0/3427556556 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f62b401b070 con 0x563129698ba0 2024-04-27T03:31:30.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.214+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563129697b50 con 0x563129594300 2024-04-27T03:31:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.214+0000 7f62ba3b6640 1 -- 172.21.15.165:0/3427556556 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f62ac0083e0 con 0x563129594300 2024-04-27T03:31:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.214+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56312969cb30 0x56312969ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.214+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56312969f6f0 con 0x56312969cb30 2024-04-27T03:31:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.214+0000 7f62b93b4640 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56312969cb30 0x56312969ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.215+0000 7f62b93b4640 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56312969cb30 0x56312969ef60 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.215+0000 7f62b93b4640 1 -- 172.21.15.165:0/3427556556 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f62a4052910 con 0x56312969cb30 2024-04-27T03:31:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.215+0000 7f62bdf2f800 1 Processor -- start 2024-04-27T03:31:30.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.215+0000 7f62bdf2f800 1 -- start start 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.215+0000 7f62bdf2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296adb50 0x5631296adf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.215+0000 7f62bdf2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5631294856b0 con 0x5631296adb50 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.216+0000 7f62b9bb5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296adb50 0x5631296adf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.216+0000 7f62b9bb5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296adb50 0x5631296adf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49404/0 (socket says 172.21.15.165:49404) 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.216+0000 7f62b9bb5640 1 -- 172.21.15.165:0/4058343134 learned_addr learned my addr 172.21.15.165:0/4058343134 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.216+0000 7f62b9bb5640 1 -- 172.21.15.165:0/4058343134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631296ae4b0 con 0x5631296adb50 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.216+0000 7f62b9bb5640 1 --2- 172.21.15.165:0/4058343134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296adb50 0x5631296adf70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f62b4006690 tx=0x7f62b40108f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.216+0000 7f619f7fe640 1 -- 172.21.15.165:0/4058343134 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62b4044d70 con 0x5631296adb50 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.216+0000 7f619f7fe640 1 -- 172.21.15.165:0/4058343134 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b404c0d0 con 0x5631296adb50 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.216+0000 7f619f7fe640 1 -- 172.21.15.165:0/4058343134 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62b404c3b0 con 0x5631296adb50 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.216+0000 7f62bdf2f800 1 -- 172.21.15.165:0/4058343134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5631296aef20 con 0x5631296adb50 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.216+0000 7f62bdf2f800 1 -- 172.21.15.165:0/4058343134 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631296af420 con 0x5631296adb50 2024-04-27T03:31:30.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.217+0000 7f619f7fe640 1 -- 172.21.15.165:0/4058343134 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f62b4010070 con 0x5631296adb50 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.217+0000 7f619f7fe640 1 --2- 172.21.15.165:0/4058343134 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6194036900 0x7f6194038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.217+0000 7f619f7fe640 1 -- 172.21.15.165:0/4058343134 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f62b4028be0 con 0x5631296adb50 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.217+0000 7f62b93b4640 1 --2- 172.21.15.165:0/4058343134 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6194036900 0x7f6194038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.218+0000 7f62bdf2f800 1 Processor -- start 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.218+0000 7f62bdf2f800 1 -- start start 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.218+0000 7f62bdf2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 0x5631296bb100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.218+0000 7f62b93b4640 1 --2- 172.21.15.165:0/4058343134 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6194036900 0x7f6194038dc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x563129696d40 tx=0x7f62a4005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.218+0000 7f62bdf2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563129687ed0 con 0x5631296bad00 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.218+0000 7f62b9bb5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 0x5631296bb100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.218+0000 7f62b9bb5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 0x5631296bb100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49408/0 (socket says 172.21.15.165:49408) 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.218+0000 7f62b9bb5640 1 -- 172.21.15.165:0/2313387369 learned_addr learned my addr 172.21.15.165:0/2313387369 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.218+0000 7f62b9bb5640 1 -- 172.21.15.165:0/2313387369 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631296bb6d0 con 0x5631296bad00 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.219+0000 7f62b9bb5640 1 --2- 172.21.15.165:0/2313387369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 0x5631296bb100 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f62b4010a40 tx=0x7f62b4049240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac1732799dad9ca server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.219+0000 7f619d7fa640 1 -- 172.21.15.165:0/2313387369 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62b405d070 con 0x5631296bad00 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.219+0000 7f619d7fa640 1 -- 172.21.15.165:0/2313387369 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b401e520 con 0x5631296bad00 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.219+0000 7f619d7fa640 1 -- 172.21.15.165:0/2313387369 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62b403c560 con 0x5631296bad00 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.219+0000 7f62bdf2f800 1 -- 172.21.15.165:0/2313387369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 msgr2=0x5631296bb100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.219+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/2313387369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 0x5631296bb100 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f62b4010a40 tx=0x7f62b4049240 comp rx=0 tx=0).stop 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.219+0000 7f62bdf2f800 1 -- 172.21.15.165:0/2313387369 shutdown_connections 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.219+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/2313387369 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 0x5631296bb100 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.219+0000 7f62bdf2f800 1 -- 172.21.15.165:0/2313387369 >> 172.21.15.165:0/2313387369 conn(0x5631296b6fd0 msgr2=0x5631296b5030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.219+0000 7f62bdf2f800 1 -- 172.21.15.165:0/2313387369 shutdown_connections 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.219+0000 7f62bdf2f800 1 -- 172.21.15.165:0/2313387369 wait complete. 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.219+0000 7f62bdf2f800 1 Processor -- start 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.220+0000 7f62bdf2f800 1 -- start start 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.220+0000 7f62bdf2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 0x5631296c2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.220+0000 7f62bdf2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5631296bbdc0 con 0x5631296bad00 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.220+0000 7f62b9bb5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 0x5631296c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.220+0000 7f62b9bb5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 0x5631296c2080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49416/0 (socket says 172.21.15.165:49416) 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.220+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 learned_addr learned my addr 172.21.15.165:0/1726479314 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.220+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631296c25c0 con 0x5631296bad00 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.220+0000 7f62b9bb5640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 0x5631296c2080 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f62b4049ac0 tx=0x7f62b40073e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.221+0000 7f618b7fe640 1 -- 172.21.15.165:0/1726479314 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62b4068020 con 0x5631296bad00 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.221+0000 7f618b7fe640 1 -- 172.21.15.165:0/1726479314 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f62b40123c0 con 0x5631296bad00 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.221+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5631296c25c0 con 0x5631296bad00 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.221+0000 7f618b7fe640 1 -- 172.21.15.165:0/1726479314 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f62b405d030 con 0x5631296bad00 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.221+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631296c25c0 con 0x5631296bad00 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.221+0000 7f618b7fe640 1 -- 172.21.15.165:0/1726479314 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f62b406b020 con 0x5631296bad00 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f618b7fe640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6184036950 0x7f6184038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f62bdf2f800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f618b7fe640 1 -- 172.21.15.165:0/1726479314 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f62b405e070 con 0x5631296bad00 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f62b93b4640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6184036950 0x7f6184038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631296cd6d0 0x5631296cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5631296d02e0 con 0x5631296cd6d0 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f62ba3b6640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631296cd6d0 0x5631296cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f62b93b4640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6184036950 0x7f6184038e10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f62a401ced0 tx=0x7f62a404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f62ba3b6640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631296cd6d0 0x5631296cfb90 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f62bdf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f62bdf2f800 20 realm 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f62bdf2f800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.222+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5631296d06f0 con 0x5631296cd6d0 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62bdf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62bdf2f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62bdf2f800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296cd6d0 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62bdf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62bdf2f800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5631296d1330 con 0x5631296cd6d0 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62bdf2f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62bdf2f800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631296d1cf0 0x5631296d40e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5631296d4660 con 0x5631296d1cf0 2024-04-27T03:31:30.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62b9bb5640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631296d1cf0 0x5631296d40e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.223+0000 7f62b9bb5640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631296d1cf0 0x5631296d40e0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.224+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.224+0000 7f62bdf2f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.224+0000 7f62bdf2f800 20 searching for the correct realm 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.224+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d6070 con 0x5631296d1cf0 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.224+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.224+0000 7f62b9bb5640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f62b4078040 0x7f62b4028070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.224+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d6070 con 0x7f62b4078040 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.225+0000 7f62b93b4640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f62b4078040 0x7f62b4028070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.225+0000 7f62b93b4640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f62b4078040 0x7f62b4028070 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.225+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.225+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d6070 con 0x7f62b4078040 2024-04-27T03:31:30.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.225+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.225+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d6070 con 0x5631296cd6d0 2024-04-27T03:31:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.226+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.226+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296cd6d0 2024-04-27T03:31:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.226+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.226+0000 7f62ba3b6640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f62ac018410 0x7f62ac018890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.226+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62ac018410 2024-04-27T03:31:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.226+0000 7f62ba3b6640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f62ac018410 0x7f62ac018890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.226+0000 7f62ba3b6640 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f62ac018410 0x7f62ac018890 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.227+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.227+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62ac018410 2024-04-27T03:31:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.227+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.227+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62b4078040 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.227+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.227+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296d1cf0 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.227+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.227+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296cd6d0 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.228+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.228+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62b4078040 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.228+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.228+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296cd6d0 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.228+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.228+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62b4078040 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.228+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.228+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296d1cf0 2024-04-27T03:31:30.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.229+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.229+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296d1cf0 2024-04-27T03:31:30.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.229+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.229+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62ac018410 2024-04-27T03:31:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.229+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.229+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296d1cf0 2024-04-27T03:31:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.230+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.230+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296d1cf0 2024-04-27T03:31:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.230+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.230+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296cd6d0 2024-04-27T03:31:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.230+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.230+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296cd6d0 2024-04-27T03:31:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.230+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.230+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62ac018410 2024-04-27T03:31:30.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.231+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.231+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296d1cf0 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.231+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.231+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62b4078040 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.231+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.231+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296cd6d0 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.231+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.231+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62ac018410 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.232+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.232+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296d1cf0 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.232+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.232+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62ac018410 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.233+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.233+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62b4078040 2024-04-27T03:31:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.233+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.233+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296d1cf0 2024-04-27T03:31:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.233+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.233+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62ac018410 2024-04-27T03:31:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.233+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.233+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x5631296cd6d0 2024-04-27T03:31:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5631296d0f90 con 0x7f62b4078040 2024-04-27T03:31:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:30.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5631296d5000 con 0x5631296cd6d0 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5631296d53f0 con 0x5631296cd6d0 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.234+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5631296d53f0 con 0x7f62b4078040 2024-04-27T03:31:30.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62bdf2f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62bdf2f800 20 zone default found 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62bdf2f800 4 Realm: () 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62bdf2f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62bdf2f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62bdf2f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62bdf2f800 20 zonegroup default 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62bdf2f800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5631296d6540 con 0x7f62ac018410 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62bdf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62bdf2f800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.235+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5631296d7f20 con 0x5631296d1cf0 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.236+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.236+0000 7f62bdf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.236+0000 7f62bdf2f800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.236+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5631296d82c0 con 0x5631296d1cf0 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.236+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.236+0000 7f62bdf2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.236+0000 7f62bdf2f800 20 started sync module instance, tier type = 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.236+0000 7f62bdf2f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.236+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296d73c0 con 0x7f62b4078040 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.237+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.237+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94769148426160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296d8b10 con 0x7f62b4078040 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.237+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296d9360 con 0x5631296cd6d0 2024-04-27T03:31:30.265 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.238+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94769148426160] v27'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.238+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.238+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94769148429920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296d93a0 con 0x5631296cd6d0 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.238+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296d9e90 con 0x5631296d1cf0 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.240+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.240+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94769148434784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296da730 con 0x5631296d1cf0 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.240+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296db210 con 0x7f62b4078040 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.240+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94769148429920] v27'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.241+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.241+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94769148432048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296dbab0 con 0x7f62b4078040 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.241+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296dc380 con 0x5631296cd6d0 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.241+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94769148434784] v27'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.242+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.266 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.242+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94769148444240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296dc770 con 0x5631296cd6d0 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.242+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94769148432048] v27'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.242+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296dd1b0 con 0x7f62ac018410 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.243+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.243+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94769148442192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296dd170 con 0x7f62ac018410 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.243+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296ddbf0 con 0x7f62ac018410 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.243+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94769148444240] v27'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.244+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94769148442192] v27'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.244+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.244+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94769148447872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296ddbb0 con 0x7f62ac018410 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.244+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296de4c0 con 0x7f62ac018410 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.245+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94769148447872] v27'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.245+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'198 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.245+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94769148450528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296ddbb0 con 0x7f62ac018410 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.245+0000 7f62bdf2f800 20 add_watcher() i=0 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.245+0000 7f62bdf2f800 20 add_watcher() i=1 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.245+0000 7f62bdf2f800 20 add_watcher() i=2 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.245+0000 7f62bdf2f800 20 add_watcher() i=3 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.245+0000 7f62bdf2f800 20 add_watcher() i=4 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.245+0000 7f62bdf2f800 20 add_watcher() i=5 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.245+0000 7f62bdf2f800 20 add_watcher() i=6 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.247+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94769148450528] v27'199 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.247+0000 7f62bdf2f800 20 add_watcher() i=7 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.247+0000 7f62bdf2f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.247+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5631296e14d0 con 0x5631296cd6d0 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.247+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.247+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94769148452768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296de9c0 con 0x5631296cd6d0 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.248+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94769148452768] v27'208 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.248+0000 7f616b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.248+0000 7f62bdf2f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.248+0000 7f62bdf2f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.248+0000 7f62bdf2f800 5 note: GC not initialized 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.248+0000 7f616a7fc640 20 reqs_thread_entry: start 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.248+0000 7f62bdf2f800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.249+0000 7f62bdf2f800 20 Filter name: none 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.249+0000 7f60dbfff640 20 reqs_thread_entry: start 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.249+0000 7f62bdf2f800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.249+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563129714340 con 0x5631296cd6d0 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.249+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.249+0000 7f62bdf2f800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:31:30.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.249+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563129715890 con 0x7f62ac018410 2024-04-27T03:31:30.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.250+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) ==== 152+0+1268 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.250+0000 7f616b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.250+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94769148452768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563129712aa0 con 0x5631296cd6d0 2024-04-27T03:31:30.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.251+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94769148452768] v27'209 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.251+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94769148426160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296ddfa0 con 0x7f62b4078040 2024-04-27T03:31:30.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.252+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94769148426160] v27'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.252+0000 7f62bdf2f800 20 remove_watcher() i=0 2024-04-27T03:31:30.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.252+0000 7f62bdf2f800 2 removed watcher, disabling cache 2024-04-27T03:31:30.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.252+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94769148429920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296ddfa0 con 0x5631296cd6d0 2024-04-27T03:31:30.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.254+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94769148429920] v27'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.254+0000 7f62bdf2f800 20 remove_watcher() i=1 2024-04-27T03:31:30.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.254+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94769148434784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296ddfa0 con 0x5631296d1cf0 2024-04-27T03:31:30.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.255+0000 7f62b9bb5640 1 -- 172.21.15.165:0/1726479314 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94769148434784] v27'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62b401b070 con 0x5631296d1cf0 2024-04-27T03:31:30.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.255+0000 7f62bdf2f800 20 remove_watcher() i=2 2024-04-27T03:31:30.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.255+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94769148432048] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296ddfa0 con 0x7f62b4078040 2024-04-27T03:31:30.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.256+0000 7f62b93b4640 1 -- 172.21.15.165:0/1726479314 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94769148432048] v27'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62a4081070 con 0x7f62b4078040 2024-04-27T03:31:30.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.256+0000 7f62bdf2f800 20 remove_watcher() i=3 2024-04-27T03:31:30.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.256+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94769148444240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296d8eb0 con 0x5631296cd6d0 2024-04-27T03:31:30.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.257+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94769148444240] v27'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x5631296cd6d0 2024-04-27T03:31:30.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.257+0000 7f62bdf2f800 20 remove_watcher() i=4 2024-04-27T03:31:30.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.257+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94769148442192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296ddfa0 con 0x7f62ac018410 2024-04-27T03:31:30.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.258+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94769148442192] v27'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.258+0000 7f62bdf2f800 20 remove_watcher() i=5 2024-04-27T03:31:30.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.258+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94769148447872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296db650 con 0x7f62ac018410 2024-04-27T03:31:30.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.259+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94769148447872] v27'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.259+0000 7f62bdf2f800 20 remove_watcher() i=6 2024-04-27T03:31:30.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.259+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94769148450528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5631296ddfa0 con 0x7f62ac018410 2024-04-27T03:31:30.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94769148450528] v27'200 uv198 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f62ac0083e0 con 0x7f62ac018410 2024-04-27T03:31:30.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 20 remove_watcher() i=7 2024-04-27T03:31:30.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f62ac018410 msgr2=0x7f62ac018890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f62ac018410 0x7f62ac018890 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631296cd6d0 msgr2=0x5631296cfb90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5631296cd6d0 0x5631296cfb90 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f62b4078040 msgr2=0x7f62b4028070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f62b4078040 0x7f62b4028070 crc :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631296d1cf0 msgr2=0x5631296d40e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5631296d1cf0 0x5631296d40e0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6184036950 msgr2=0x7f6184038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6184036950 0x7f6184038e10 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f62a401ced0 tx=0x7f62a404f000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 msgr2=0x5631296c2080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62ba3b6640 1 -- 172.21.15.165:0/1726479314 reap_dead start 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 0x5631296c2080 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f62b4049ac0 tx=0x7f62b40073e0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.260+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 shutdown_connections 2024-04-27T03:31:30.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/1726479314 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296bad00 0x5631296c2080 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 >> 172.21.15.165:0/1726479314 conn(0x5631296b6fd0 msgr2=0x5631296b4630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 shutdown_connections 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/1726479314 wait complete. 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/4058343134 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6194036900 msgr2=0x7f6194038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/4058343134 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6194036900 0x7f6194038dc0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x563129696d40 tx=0x7f62a4005a00 comp rx=0 tx=0).stop 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/4058343134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296adb50 msgr2=0x5631296adf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/4058343134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296adb50 0x5631296adf70 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f62b4006690 tx=0x7f62b40108f0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/4058343134 shutdown_connections 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/4058343134 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6194036900 0x7f6194038dc0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/4058343134 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631296adb50 0x5631296adf70 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/4058343134 >> 172.21.15.165:0/4058343134 conn(0x5631296a1bf0 msgr2=0x5631296a1ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/4058343134 shutdown_connections 2024-04-27T03:31:30.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/4058343134 wait complete. 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563129594300 msgr2=0x5631296962e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563129594300 0x5631296962e0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56312969cb30 msgr2=0x56312969ef60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56312969cb30 0x56312969ef60 crc :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563129698ba0 msgr2=0x56312969af90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563129698ba0 0x56312969af90 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f61a0036950 msgr2=0x7f61a0038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f61a0036950 0x7f61a0038e10 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f62a4054000 tx=0x7f62a404e000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 msgr2=0x56312968f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 --2- 172.21.15.165:0/3427556556 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5631295927c0 0x56312968f140 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f62b4000f80 tx=0x7f62b4001250 comp rx=0 tx=0).stop 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62ba3b6640 1 -- 172.21.15.165:0/3427556556 reap_dead start 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 shutdown_connections 2024-04-27T03:31:30.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 >> 172.21.15.165:0/3427556556 conn(0x56312958e340 msgr2=0x5631296816a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 shutdown_connections 2024-04-27T03:31:30.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.261+0000 7f62bdf2f800 1 -- 172.21.15.165:0/3427556556 wait complete. 2024-04-27T03:31:30.280 INFO:teuthology.orchestra.run.smithi165.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]},{"bucket":"myfoo","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":5,"successful_ops":5}]},{"bucket":"myfoo2","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo3","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo4","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]}]}],"summary":[{"user":"foo","categories":[{"category":"create_bucket","bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8}],"total":{"bytes_sent":0,"bytes_received":0,"ops":8,"successful_ops":8}}]} 2024-04-27T03:31:30.281 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 5, 'successful_ops': 5}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 8, 'successful_ops': 8}}]} 2024-04-27T03:31:30.281 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-04-27T03:31:30.281 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'put_obj'] 2024-04-27T03:31:30.281 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories put_obj 2024-04-27T03:31:30.326 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:30.326 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:30.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.320+0000 7feceeb19800 1 Processor -- start 2024-04-27T03:31:30.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.320+0000 7feceeb19800 1 -- start start 2024-04-27T03:31:30.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.320+0000 7feceeb19800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x556c2e5d5350 0x556c2e5d7740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.320+0000 7feceeb19800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x556c2e48fd40 con 0x556c2e5d0fb0 2024-04-27T03:31:30.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.320+0000 7feceeb19800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x556c2e4624d0 con 0x556c2e5d5350 2024-04-27T03:31:30.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.320+0000 7fecebe17640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x556c2e5d5350 0x556c2e5d7740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.320+0000 7fecec618640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x556c2e5d0fb0 0x556c2e5d13d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:54728/0 (socket says 172.21.15.165:54728) 2024-04-27T03:31:30.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.320+0000 7fecec618640 1 -- 172.21.15.165:0/11289926 learned_addr learned my addr 172.21.15.165:0/11289926 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:31:30.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.321+0000 7fecebe17640 1 -- 172.21.15.165:0/11289926 >> v1:172.21.15.89:6789/0 conn(0x556c2e5d0fb0 legacy=0x556c2e5d13d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.321+0000 7fecebe17640 1 -- 172.21.15.165:0/11289926 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c2e48e590 con 0x556c2e5d5350 2024-04-27T03:31:30.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.321+0000 7feceb616640 1 -- 172.21.15.165:0/11289926 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3986865247 0 0) 0x556c2e48fd40 con 0x556c2e5d0fb0 2024-04-27T03:31:30.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.323+0000 7fecebe17640 1 --2- 172.21.15.165:0/11289926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d5350 0x556c2e5d7740 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fece000a470 tx=0x7fece001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf59c92182109e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.323+0000 7feceb616640 1 -- 172.21.15.165:0/11289926 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece0057070 con 0x556c2e5d5350 2024-04-27T03:31:30.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.323+0000 7feceb616640 1 -- 172.21.15.165:0/11289926 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fece0052570 con 0x556c2e5d5350 2024-04-27T03:31:30.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.323+0000 7feceb616640 1 -- 172.21.15.165:0/11289926 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fece0052870 con 0x556c2e5d5350 2024-04-27T03:31:30.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.323+0000 7feceeb19800 1 -- 172.21.15.165:0/11289926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d5350 msgr2=0x556c2e5d7740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.323+0000 7feceeb19800 1 --2- 172.21.15.165:0/11289926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d5350 0x556c2e5d7740 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7fece000a470 tx=0x7fece001ce90 comp rx=0 tx=0).stop 2024-04-27T03:31:30.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.323+0000 7feceeb19800 1 -- 172.21.15.165:0/11289926 shutdown_connections 2024-04-27T03:31:30.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.323+0000 7feceeb19800 1 --2- 172.21.15.165:0/11289926 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d5350 0x556c2e5d7740 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.323+0000 7feceeb19800 1 -- 172.21.15.165:0/11289926 >> 172.21.15.165:0/11289926 conn(0x556c2e5cc550 msgr2=0x556c2e5ce970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.323+0000 7feceeb19800 1 -- 172.21.15.165:0/11289926 shutdown_connections 2024-04-27T03:31:30.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.323+0000 7feceeb19800 1 -- 172.21.15.165:0/11289926 wait complete. 2024-04-27T03:31:30.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.335+0000 7feceeb19800 1 Processor -- start 2024-04-27T03:31:30.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.335+0000 7feceeb19800 1 -- start start 2024-04-27T03:31:30.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.335+0000 7feceeb19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 0x556c2e5469c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.335+0000 7feceeb19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556c2e51c3b0 con 0x556c2e5d0fb0 2024-04-27T03:31:30.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.335+0000 7fecec618640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 0x556c2e5469c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.335+0000 7fecec618640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 0x556c2e5469c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49436/0 (socket says 172.21.15.165:49436) 2024-04-27T03:31:30.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.335+0000 7fecec618640 1 -- 172.21.15.165:0/1420726733 learned_addr learned my addr 172.21.15.165:0/1420726733 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.336+0000 7fecec618640 1 -- 172.21.15.165:0/1420726733 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c2e544f80 con 0x556c2e5d0fb0 2024-04-27T03:31:30.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.336+0000 7fecec618640 1 --2- 172.21.15.165:0/1420726733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 0x556c2e5469c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00b240 tx=0x7fecdc00b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4db0f84c0460017 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.336+0000 7febda60c640 1 -- 172.21.15.165:0/1420726733 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecdc008070 con 0x556c2e5d0fb0 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.336+0000 7febda60c640 1 -- 172.21.15.165:0/1420726733 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecdc00d500 con 0x556c2e5d0fb0 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.336+0000 7febda60c640 1 -- 172.21.15.165:0/1420726733 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecdc00d860 con 0x556c2e5d0fb0 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.337+0000 7feceeb19800 1 -- 172.21.15.165:0/1420726733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 msgr2=0x556c2e5469c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.337+0000 7feceeb19800 1 --2- 172.21.15.165:0/1420726733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 0x556c2e5469c0 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7fecdc00b240 tx=0x7fecdc00b700 comp rx=0 tx=0).stop 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.337+0000 7feceeb19800 1 -- 172.21.15.165:0/1420726733 shutdown_connections 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.337+0000 7feceeb19800 1 --2- 172.21.15.165:0/1420726733 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 0x556c2e5469c0 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.337+0000 7feceeb19800 1 -- 172.21.15.165:0/1420726733 >> 172.21.15.165:0/1420726733 conn(0x556c2e5cc550 msgr2=0x556c2e6bd8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.337+0000 7feceeb19800 1 -- 172.21.15.165:0/1420726733 shutdown_connections 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.337+0000 7feceeb19800 1 -- 172.21.15.165:0/1420726733 wait complete. 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.337+0000 7feceeb19800 1 Processor -- start 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.337+0000 7feceeb19800 1 -- start start 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.337+0000 7feceeb19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 0x556c2e6cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.337+0000 7feceeb19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556c2e5d8010 con 0x556c2e5d0fb0 2024-04-27T03:31:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.338+0000 7fecec618640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 0x556c2e6cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.338+0000 7fecec618640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 0x556c2e6cbda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49438/0 (socket says 172.21.15.165:49438) 2024-04-27T03:31:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.338+0000 7fecec618640 1 -- 172.21.15.165:0/1229962627 learned_addr learned my addr 172.21.15.165:0/1229962627 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.338+0000 7fecec618640 1 -- 172.21.15.165:0/1229962627 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c2e6cc2e0 con 0x556c2e5d0fb0 2024-04-27T03:31:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.338+0000 7fecec618640 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 0x556c2e6cbda0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x556c2e545ff0 tx=0x7fecdc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.338+0000 7febd8e09640 1 -- 172.21.15.165:0/1229962627 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecdc019870 con 0x556c2e5d0fb0 2024-04-27T03:31:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.338+0000 7febd8e09640 1 -- 172.21.15.165:0/1229962627 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecdc019a10 con 0x556c2e5d0fb0 2024-04-27T03:31:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.338+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556c2e6cc570 con 0x556c2e5d0fb0 2024-04-27T03:31:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.338+0000 7febd8e09640 1 -- 172.21.15.165:0/1229962627 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecdc019d30 con 0x556c2e5d0fb0 2024-04-27T03:31:30.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.338+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c2e6cf220 con 0x556c2e5d0fb0 2024-04-27T03:31:30.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.339+0000 7febd8e09640 1 -- 172.21.15.165:0/1229962627 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fecdc017040 con 0x556c2e5d0fb0 2024-04-27T03:31:30.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.339+0000 7febd8e09640 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febd4036950 0x7febd4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.339+0000 7febd8e09640 1 -- 172.21.15.165:0/1229962627 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fecdc015020 con 0x556c2e5d0fb0 2024-04-27T03:31:30.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.340+0000 7fecebe17640 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febd4036950 0x7febd4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.340+0000 7feceeb19800 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c2e5d5350 0x556c2e6d2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.340+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e6d3560 con 0x556c2e5d5350 2024-04-27T03:31:30.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.340+0000 7fecece19640 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c2e5d5350 0x556c2e6d2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.340+0000 7fecebe17640 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febd4036950 0x7febd4038e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fece0054000 tx=0x7fece004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.356 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.340+0000 7fecece19640 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c2e5d5350 0x556c2e6d2fe0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.340+0000 7fecece19640 1 -- 172.21.15.165:0/1229962627 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e5d5350 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.340+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e6d05b0 con 0x556c2e5d5350 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.341+0000 7fecece19640 1 -- 172.21.15.165:0/1229962627 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e5d5350 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.341+0000 7feceeb19800 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c2e6d5890 0x556c2e6d7c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.341+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e6d8320 con 0x556c2e6d5890 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.341+0000 7fecec618640 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c2e6d5890 0x556c2e6d7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.341+0000 7fecec618640 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c2e6d5890 0x556c2e6d7c80 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.341+0000 7fecec618640 1 -- 172.21.15.165:0/1229962627 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e6d5890 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.341+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e6d4840 con 0x556c2e5d5350 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.341+0000 7fecece19640 1 -- 172.21.15.165:0/1229962627 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e5d5350 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.342+0000 7feceeb19800 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x556c2e6d97d0 0x556c2e6dbc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.342+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e6dc380 con 0x556c2e6d97d0 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.342+0000 7fecebe17640 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x556c2e6d97d0 0x556c2e6dbc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.357 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.342+0000 7fecebe17640 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x556c2e6d97d0 0x556c2e6dbc20 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.342+0000 7fecebe17640 1 -- 172.21.15.165:0/1229962627 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fece0052940 con 0x556c2e6d97d0 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.343+0000 7feceeb19800 1 Processor -- start 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.343+0000 7feceeb19800 1 -- start start 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.343+0000 7feceeb19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6ea7e0 0x556c2e6eac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.343+0000 7feceeb19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556c2e4c26b0 con 0x556c2e6ea7e0 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.343+0000 7fecec618640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6ea7e0 0x556c2e6eac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.343+0000 7fecec618640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6ea7e0 0x556c2e6eac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49454/0 (socket says 172.21.15.165:49454) 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.343+0000 7fecec618640 1 -- 172.21.15.165:0/3855524514 learned_addr learned my addr 172.21.15.165:0/3855524514 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.343+0000 7fecec618640 1 -- 172.21.15.165:0/3855524514 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c2e6eb140 con 0x556c2e6ea7e0 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.344+0000 7fecec618640 1 --2- 172.21.15.165:0/3855524514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6ea7e0 0x556c2e6eac00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fecdc04b6a0 tx=0x7fecdc01a950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.344+0000 7febd1ffb640 1 -- 172.21.15.165:0/3855524514 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecdc046040 con 0x556c2e6ea7e0 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.344+0000 7febd1ffb640 1 -- 172.21.15.165:0/3855524514 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecdc00f870 con 0x556c2e6ea7e0 2024-04-27T03:31:30.358 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.344+0000 7feceeb19800 1 -- 172.21.15.165:0/3855524514 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556c2e6ebbb0 con 0x556c2e6ea7e0 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.344+0000 7febd1ffb640 1 -- 172.21.15.165:0/3855524514 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecdc044b80 con 0x556c2e6ea7e0 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.344+0000 7feceeb19800 1 -- 172.21.15.165:0/3855524514 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c2e6ec0b0 con 0x556c2e6ea7e0 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.344+0000 7febd1ffb640 1 -- 172.21.15.165:0/3855524514 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fecdc044d20 con 0x556c2e6ea7e0 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7febd1ffb640 1 --2- 172.21.15.165:0/3855524514 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febc4036900 0x7febc4038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7febd1ffb640 1 -- 172.21.15.165:0/3855524514 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fecdc04a320 con 0x556c2e6ea7e0 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7fecebe17640 1 --2- 172.21.15.165:0/3855524514 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febc4036900 0x7febc4038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7feceeb19800 1 Processor -- start 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7feceeb19800 1 -- start start 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7fecebe17640 1 --2- 172.21.15.165:0/3855524514 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febc4036900 0x7febc4038dc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x556c2e6d04d0 tx=0x7fece0005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7feceeb19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 0x556c2e6f7db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7feceeb19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556c2e545300 con 0x556c2e6f7990 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7fecec618640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 0x556c2e6f7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7fecec618640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 0x556c2e6f7db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49470/0 (socket says 172.21.15.165:49470) 2024-04-27T03:31:30.359 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7fecec618640 1 -- 172.21.15.165:0/2931321473 learned_addr learned my addr 172.21.15.165:0/2931321473 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7fecec618640 1 -- 172.21.15.165:0/2931321473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c2e6f8380 con 0x556c2e6f7990 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7fecec618640 1 --2- 172.21.15.165:0/2931321473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 0x556c2e6f7db0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fecdc017fd0 tx=0x7fecdc018f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24f68883571f2814 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7febcb7fe640 1 -- 172.21.15.165:0/2931321473 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecdc032e10 con 0x556c2e6f7990 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7febcb7fe640 1 -- 172.21.15.165:0/2931321473 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecdc05d020 con 0x556c2e6f7990 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7febcb7fe640 1 -- 172.21.15.165:0/2931321473 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecdc03b440 con 0x556c2e6f7990 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7feceeb19800 1 -- 172.21.15.165:0/2931321473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 msgr2=0x556c2e6f7db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7feceeb19800 1 --2- 172.21.15.165:0/2931321473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 0x556c2e6f7db0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7fecdc017fd0 tx=0x7fecdc018f40 comp rx=0 tx=0).stop 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7feceeb19800 1 -- 172.21.15.165:0/2931321473 shutdown_connections 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7feceeb19800 1 --2- 172.21.15.165:0/2931321473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 0x556c2e6f7db0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7feceeb19800 1 -- 172.21.15.165:0/2931321473 >> 172.21.15.165:0/2931321473 conn(0x556c2e6f3c60 msgr2=0x556c2e6f1cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7feceeb19800 1 -- 172.21.15.165:0/2931321473 shutdown_connections 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.346+0000 7feceeb19800 1 -- 172.21.15.165:0/2931321473 wait complete. 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.347+0000 7feceeb19800 1 Processor -- start 2024-04-27T03:31:30.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.347+0000 7feceeb19800 1 -- start start 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.347+0000 7feceeb19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 0x556c2e6fec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.347+0000 7feceeb19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556c2e6f89e0 con 0x556c2e6f7990 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.347+0000 7fecec618640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 0x556c2e6fec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.347+0000 7fecec618640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 0x556c2e6fec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49482/0 (socket says 172.21.15.165:49482) 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.347+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 learned_addr learned my addr 172.21.15.165:0/3858101330 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.348+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c2e6ff1d0 con 0x556c2e6f7990 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.348+0000 7fecec618640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 0x556c2e6fec90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0265f0 tx=0x7fecdc012ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.348+0000 7febc9ffb640 1 -- 172.21.15.165:0/3858101330 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecdc05c070 con 0x556c2e6f7990 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.348+0000 7febc9ffb640 1 -- 172.21.15.165:0/3858101330 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fecdc051060 con 0x556c2e6f7990 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.348+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556c2e6ff1d0 con 0x556c2e6f7990 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.348+0000 7febc9ffb640 1 -- 172.21.15.165:0/3858101330 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fecdc01c940 con 0x556c2e6f7990 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.348+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c2e6ff1d0 con 0x556c2e6f7990 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.349+0000 7febc9ffb640 1 -- 172.21.15.165:0/3858101330 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fecdc021900 con 0x556c2e6f7990 2024-04-27T03:31:30.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.349+0000 7febc9ffb640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febbc036900 0x7febbc038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.349+0000 7febc9ffb640 1 -- 172.21.15.165:0/3858101330 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fecdc06b030 con 0x556c2e6f7990 2024-04-27T03:31:30.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.349+0000 7fecebe17640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febbc036900 0x7febbc038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.349+0000 7feceeb19800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.349+0000 7feceeb19800 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c2e70a350 0x556c2e70c810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.349+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70cf60 con 0x556c2e70a350 2024-04-27T03:31:30.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.349+0000 7fecece19640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c2e70a350 0x556c2e70c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.349+0000 7fecebe17640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febbc036900 0x7febbc038dc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fece0052fd0 tx=0x7fece004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7fecece19640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c2e70a350 0x556c2e70c810 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 20 realm 2024-04-27T03:31:30.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70d370 con 0x556c2e70a350 2024-04-27T03:31:30.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dc10 con 0x556c2e70a350 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70a350 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c2e70ed10 0x556c2e711100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.350+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e711640 con 0x556c2e70ed10 2024-04-27T03:31:30.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.351+0000 7fecec618640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c2e70ed10 0x556c2e711100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.351+0000 7fecec618640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c2e70ed10 0x556c2e711100 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.351+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.351+0000 7feceeb19800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.351+0000 7feceeb19800 20 searching for the correct realm 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.351+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e712ff0 con 0x556c2e70ed10 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.353+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.353+0000 7fecec618640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fecdc030450 0x7fecdc063840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.353+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e712ff0 con 0x7fecdc030450 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.353+0000 7fecebe17640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fecdc030450 0x7fecdc063840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.353+0000 7fecebe17640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fecdc030450 0x7fecdc063840 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.353+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.353+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e712ff0 con 0x7fecdc030450 2024-04-27T03:31:30.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.353+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.353+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e712ff0 con 0x556c2e70a350 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70a350 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecece19640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fece4016440 0x7fece40168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fece4016440 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecece19640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fece4016440 0x7fece40168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecece19640 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fece4016440 0x7fece40168c0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fece4016440 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fecdc030450 2024-04-27T03:31:30.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.354+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70ed10 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.356+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.356+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70a350 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.356+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.356+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fecdc030450 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.356+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.356+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70a350 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.356+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.356+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fecdc030450 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.356+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.356+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70ed10 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70ed10 2024-04-27T03:31:30.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fece4016440 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70ed10 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70ed10 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70a350 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70a350 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.357+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fece4016440 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70ed10 2024-04-27T03:31:30.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fecdc030450 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70a350 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fece4016440 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70ed10 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fece4016440 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.359+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fecdc030450 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70ed10 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fece4016440 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x556c2e70a350 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c2e70dfb0 con 0x7fecdc030450 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:30.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e711fc0 con 0x556c2e70a350 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e7123b0 con 0x556c2e70a350 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.361+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e7123b0 con 0x7fecdc030450 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:30.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 20 zone default found 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 4 Realm: () 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 20 zonegroup default 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e7134c0 con 0x7fece4016440 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e714ea0 con 0x556c2e70ed10 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e715240 con 0x556c2e70ed10 2024-04-27T03:31:30.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 20 started sync module instance, tier type = 2024-04-27T03:31:30.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:30.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.362+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e714340 con 0x7fecdc030450 2024-04-27T03:31:30.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.365+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.365+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93923124004704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e715a90 con 0x7fecdc030450 2024-04-27T03:31:30.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.365+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e7162e0 con 0x556c2e70a350 2024-04-27T03:31:30.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.365+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 93923124004704] v27'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.365+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.365+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93923124008416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e716340 con 0x556c2e70a350 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.365+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e716e30 con 0x556c2e70ed10 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.367+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93923124008416] v27'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.367+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.367+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93923124013312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e717250 con 0x556c2e70ed10 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.367+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e717c50 con 0x7fecdc030450 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.368+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.368+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93923124019184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e7188c0 con 0x7fecdc030450 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.368+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e719330 con 0x556c2e70a350 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.368+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93923124013312] v27'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.369+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 93923124019184] v27'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.369+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.369+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93923124020240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e715e30 con 0x556c2e70a350 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.369+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e719d40 con 0x7fece4016440 2024-04-27T03:31:30.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.370+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93923124020240] v27'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.370+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.370+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93923124022784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e719d40 con 0x7fece4016440 2024-04-27T03:31:30.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.370+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e71a700 con 0x7fece4016440 2024-04-27T03:31:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.371+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(56 notify.6 [create] v27'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.372+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93923124027984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e71ab20 con 0x7fece4016440 2024-04-27T03:31:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.372+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e71b580 con 0x7fece4016440 2024-04-27T03:31:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.372+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(55 notify.5 [watch watch cookie 93923124022784] v27'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.372+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 93923124027984] v27'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.373+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'201 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.373+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93923124025488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e71a670 con 0x7fece4016440 2024-04-27T03:31:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.373+0000 7feceeb19800 20 add_watcher() i=0 2024-04-27T03:31:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.373+0000 7feceeb19800 20 add_watcher() i=1 2024-04-27T03:31:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.373+0000 7feceeb19800 20 add_watcher() i=2 2024-04-27T03:31:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.373+0000 7feceeb19800 20 add_watcher() i=3 2024-04-27T03:31:30.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.373+0000 7feceeb19800 20 add_watcher() i=4 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.373+0000 7feceeb19800 20 add_watcher() i=5 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.373+0000 7feceeb19800 20 add_watcher() i=6 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.374+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 93923124025488] v27'202 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.374+0000 7feceeb19800 20 add_watcher() i=7 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.374+0000 7feceeb19800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.374+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e71e680 con 0x556c2e70a350 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.374+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.374+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93923124031632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e71bb00 con 0x556c2e70a350 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.375+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93923124031632] v27'210 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.375+0000 7feba27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.375+0000 7feceeb19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.375+0000 7feceeb19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.375+0000 7feceeb19800 5 note: GC not initialized 2024-04-27T03:31:30.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.375+0000 7feba17e2640 20 reqs_thread_entry: start 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.375+0000 7feceeb19800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.375+0000 7feceeb19800 20 Filter name: none 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.376+0000 7feb12ffd640 20 reqs_thread_entry: start 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.376+0000 7feceeb19800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.376+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e751310 con 0x556c2e70a350 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.376+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.376+0000 7feceeb19800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.376+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c2e752860 con 0x7fece4016440 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.376+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) ==== 152+0+1268 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.377+0000 7feba27e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.377+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93923124031632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e7077f0 con 0x556c2e70a350 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.378+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 93923124031632] v27'211 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.378+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93923124004704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e71b090 con 0x7fecdc030450 2024-04-27T03:31:30.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.379+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 93923124004704] v27'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.379+0000 7feceeb19800 20 remove_watcher() i=0 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.379+0000 7feceeb19800 2 removed watcher, disabling cache 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.379+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93923124008416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e71b090 con 0x556c2e70a350 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.380+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 93923124008416] v27'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.380+0000 7feceeb19800 20 remove_watcher() i=1 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.380+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93923124013312] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e71b090 con 0x556c2e70ed10 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.381+0000 7fecec618640 1 -- 172.21.15.165:0/3858101330 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 93923124013312] v27'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fecdc02d070 con 0x556c2e70ed10 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.381+0000 7feceeb19800 20 remove_watcher() i=2 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.381+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93923124019184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e716900 con 0x7fecdc030450 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.382+0000 7fecebe17640 1 -- 172.21.15.165:0/3858101330 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 93923124019184] v27'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece0081070 con 0x7fecdc030450 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.382+0000 7feceeb19800 20 remove_watcher() i=3 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.382+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93923124020240] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e717ff0 con 0x556c2e70a350 2024-04-27T03:31:30.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.383+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 93923124020240] v27'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x556c2e70a350 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.383+0000 7feceeb19800 20 remove_watcher() i=4 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.383+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93923124022784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e7155e0 con 0x7fece4016440 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.384+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 93923124022784] v27'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.384+0000 7feceeb19800 20 remove_watcher() i=5 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.384+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93923124027984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e7155e0 con 0x7fece4016440 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.385+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 93923124027984] v27'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.385+0000 7feceeb19800 20 remove_watcher() i=6 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.385+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93923124025488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c2e7155e0 con 0x7fece4016440 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.386+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 93923124025488] v27'203 uv201 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fece400a4a0 con 0x7fece4016440 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.386+0000 7feceeb19800 20 remove_watcher() i=7 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.386+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fece4016440 msgr2=0x7fece40168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.386+0000 7feceeb19800 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fece4016440 0x7fece40168c0 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.386+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c2e70a350 msgr2=0x556c2e70c810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.386+0000 7feceeb19800 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c2e70a350 0x556c2e70c810 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.386+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fecdc030450 msgr2=0x7fecdc063840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.386+0000 7feceeb19800 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fecdc030450 0x7fecdc063840 crc :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.386+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c2e70ed10 msgr2=0x556c2e711100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.386+0000 7feceeb19800 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c2e70ed10 0x556c2e711100 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febbc036900 msgr2=0x7febbc038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febbc036900 0x7febbc038dc0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fece0052fd0 tx=0x7fece004f000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 msgr2=0x556c2e6fec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 --2- 172.21.15.165:0/3858101330 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6f7990 0x556c2e6fec90 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7fecdc0265f0 tx=0x7fecdc012ef0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 reap_dead start 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7fecece19640 1 -- 172.21.15.165:0/3858101330 reap_dead start 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 shutdown_connections 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 >> 172.21.15.165:0/3858101330 conn(0x556c2e6f3c60 msgr2=0x556c2e6f12c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 shutdown_connections 2024-04-27T03:31:30.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/3858101330 wait complete. 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/3855524514 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febc4036900 msgr2=0x7febc4038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 --2- 172.21.15.165:0/3855524514 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febc4036900 0x7febc4038dc0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x556c2e6d04d0 tx=0x7fece0005a00 comp rx=0 tx=0).stop 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/3855524514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6ea7e0 msgr2=0x556c2e6eac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 --2- 172.21.15.165:0/3855524514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6ea7e0 0x556c2e6eac00 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fecdc04b6a0 tx=0x7fecdc01a950 comp rx=0 tx=0).stop 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/3855524514 shutdown_connections 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 --2- 172.21.15.165:0/3855524514 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febc4036900 0x7febc4038dc0 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 --2- 172.21.15.165:0/3855524514 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e6ea7e0 0x556c2e6eac00 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/3855524514 >> 172.21.15.165:0/3855524514 conn(0x556c2e6de880 msgr2=0x556c2e6dec80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/3855524514 shutdown_connections 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/3855524514 wait complete. 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c2e5d5350 msgr2=0x556c2e6d2fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c2e5d5350 0x556c2e6d2fe0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x556c2e6d97d0 msgr2=0x556c2e6dbc20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x556c2e6d97d0 0x556c2e6dbc20 crc :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c2e6d5890 msgr2=0x556c2e6d7c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.387+0000 7feceeb19800 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c2e6d5890 0x556c2e6d7c80 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.388+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febd4036950 msgr2=0x7febd4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.388+0000 7feceeb19800 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7febd4036950 0x7febd4038e10 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fece0054000 tx=0x7fece004e000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.388+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 msgr2=0x556c2e6cbda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.388+0000 7feceeb19800 1 --2- 172.21.15.165:0/1229962627 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c2e5d0fb0 0x556c2e6cbda0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x556c2e545ff0 tx=0x7fecdc000ed0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.388+0000 7fecece19640 1 -- 172.21.15.165:0/1229962627 reap_dead start 2024-04-27T03:31:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.388+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 shutdown_connections 2024-04-27T03:31:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.388+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 >> 172.21.15.165:0/1229962627 conn(0x556c2e5cc550 msgr2=0x556c2e545860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.388+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 shutdown_connections 2024-04-27T03:31:30.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.388+0000 7feceeb19800 1 -- 172.21.15.165:0/1229962627 wait complete. 2024-04-27T03:31:30.404 INFO:teuthology.orchestra.run.smithi165.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]},{"bucket":"myfoo","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}]},{"bucket":"myfoo2","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]},{"bucket":"myfoo3","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]},{"bucket":"myfoo4","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]}]}],"summary":[{"user":"foo","categories":[{"category":"put_obj","bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}],"total":{"bytes_sent":0,"bytes_received":30000042,"ops":11,"successful_ops":11}}]} 2024-04-27T03:31:30.405 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}], 'total': {'bytes_sent': 0, 'bytes_received': 30000042, 'ops': 11, 'successful_ops': 11}}]} 2024-04-27T03:31:30.405 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-04-27T03:31:30.405 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_obj'] 2024-04-27T03:31:30.405 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_obj 2024-04-27T03:31:30.450 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:30.450 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:30.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.445+0000 7f26d983c800 1 Processor -- start 2024-04-27T03:31:30.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.445+0000 7f26d983c800 1 -- start start 2024-04-27T03:31:30.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.445+0000 7f26d983c800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55df963df4a0 0x55df963e1890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.445+0000 7f26d983c800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55df96299d40 con 0x55df963db990 2024-04-27T03:31:30.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.445+0000 7f26d983c800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55df9626c4d0 con 0x55df963df4a0 2024-04-27T03:31:30.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.445+0000 7f26d3cd4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55df963df4a0 0x55df963e1890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.445+0000 7f26d44d5640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55df963db990 0x55df963ddd80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:54732/0 (socket says 172.21.15.165:54732) 2024-04-27T03:31:30.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.445+0000 7f26d44d5640 1 -- 172.21.15.165:0/379485908 learned_addr learned my addr 172.21.15.165:0/379485908 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:31:30.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.446+0000 7f26d3cd4640 1 -- 172.21.15.165:0/379485908 >> v1:172.21.15.89:6789/0 conn(0x55df963db990 legacy=0x55df963ddd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.446+0000 7f26d3cd4640 1 -- 172.21.15.165:0/379485908 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df96298590 con 0x55df963df4a0 2024-04-27T03:31:30.454 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.446+0000 7f26d34d3640 1 -- 172.21.15.165:0/379485908 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2972144584 0 0) 0x55df96299d40 con 0x55df963db990 2024-04-27T03:31:30.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.448+0000 7f26d3cd4640 1 --2- 172.21.15.165:0/379485908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963df4a0 0x55df963e1890 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f26c800a470 tx=0x7f26c801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b7fdf6be24b3714 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.448+0000 7f26d34d3640 1 -- 172.21.15.165:0/379485908 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c8057070 con 0x55df963df4a0 2024-04-27T03:31:30.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.448+0000 7f26d34d3640 1 -- 172.21.15.165:0/379485908 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c8052570 con 0x55df963df4a0 2024-04-27T03:31:30.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.448+0000 7f26d34d3640 1 -- 172.21.15.165:0/379485908 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c8052870 con 0x55df963df4a0 2024-04-27T03:31:30.455 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.449+0000 7f26d983c800 1 -- 172.21.15.165:0/379485908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963df4a0 msgr2=0x55df963e1890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.449+0000 7f26d983c800 1 --2- 172.21.15.165:0/379485908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963df4a0 0x55df963e1890 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7f26c800a470 tx=0x7f26c801ce90 comp rx=0 tx=0).stop 2024-04-27T03:31:30.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.449+0000 7f26d983c800 1 -- 172.21.15.165:0/379485908 shutdown_connections 2024-04-27T03:31:30.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.449+0000 7f26d983c800 1 --2- 172.21.15.165:0/379485908 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963df4a0 0x55df963e1890 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.449+0000 7f26d983c800 1 -- 172.21.15.165:0/379485908 >> 172.21.15.165:0/379485908 conn(0x55df963d5340 msgr2=0x55df963d7760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.449+0000 7f26d983c800 1 -- 172.21.15.165:0/379485908 shutdown_connections 2024-04-27T03:31:30.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.449+0000 7f26d983c800 1 -- 172.21.15.165:0/379485908 wait complete. 2024-04-27T03:31:30.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.460+0000 7f26d983c800 1 Processor -- start 2024-04-27T03:31:30.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.461+0000 7f26d983c800 1 -- start start 2024-04-27T03:31:30.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.461+0000 7f26d983c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 0x55df963db2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.461+0000 7f26d983c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55df963263b0 con 0x55df963db990 2024-04-27T03:31:30.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.461+0000 7f26d44d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 0x55df963db2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.461+0000 7f26d44d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 0x55df963db2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49492/0 (socket says 172.21.15.165:49492) 2024-04-27T03:31:30.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.461+0000 7f26d44d5640 1 -- 172.21.15.165:0/1967169159 learned_addr learned my addr 172.21.15.165:0/1967169159 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.461+0000 7f26d44d5640 1 -- 172.21.15.165:0/1967169159 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df963d9890 con 0x55df963db990 2024-04-27T03:31:30.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.462+0000 7f26d44d5640 1 --2- 172.21.15.165:0/1967169159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 0x55df963db2d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f26c400b210 tx=0x7f26c400b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8157ef3363016e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.462+0000 7f25c260c640 1 -- 172.21.15.165:0/1967169159 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c4008070 con 0x55df963db990 2024-04-27T03:31:30.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.462+0000 7f25c260c640 1 -- 172.21.15.165:0/1967169159 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c400d500 con 0x55df963db990 2024-04-27T03:31:30.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.462+0000 7f25c260c640 1 -- 172.21.15.165:0/1967169159 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c400d840 con 0x55df963db990 2024-04-27T03:31:30.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.462+0000 7f26d983c800 1 -- 172.21.15.165:0/1967169159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 msgr2=0x55df963db2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.462+0000 7f26d983c800 1 --2- 172.21.15.165:0/1967169159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 0x55df963db2d0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f26c400b210 tx=0x7f26c400b6d0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.462+0000 7f26d983c800 1 -- 172.21.15.165:0/1967169159 shutdown_connections 2024-04-27T03:31:30.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.462+0000 7f26d983c800 1 --2- 172.21.15.165:0/1967169159 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 0x55df963db2d0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.462+0000 7f26d983c800 1 -- 172.21.15.165:0/1967169159 >> 172.21.15.165:0/1967169159 conn(0x55df963d5340 msgr2=0x55df964c78d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.470 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.462+0000 7f26d983c800 1 -- 172.21.15.165:0/1967169159 shutdown_connections 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.463+0000 7f26d983c800 1 -- 172.21.15.165:0/1967169159 wait complete. 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.463+0000 7f26d983c800 1 Processor -- start 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.463+0000 7f26d983c800 1 -- start start 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.463+0000 7f26d983c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 0x55df964d5d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.463+0000 7f26d983c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55df963e40e0 con 0x55df963db990 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.463+0000 7f26d44d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 0x55df964d5d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.463+0000 7f26d44d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 0x55df964d5d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49508/0 (socket says 172.21.15.165:49508) 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.463+0000 7f26d44d5640 1 -- 172.21.15.165:0/3574906617 learned_addr learned my addr 172.21.15.165:0/3574906617 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.463+0000 7f26d44d5640 1 -- 172.21.15.165:0/3574906617 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df964d6280 con 0x55df963db990 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.464+0000 7f26d44d5640 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 0x55df964d5d40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f26c400b210 tx=0x7f26c4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.464+0000 7f25c0e09640 1 -- 172.21.15.165:0/3574906617 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c400f950 con 0x55df963db990 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.464+0000 7f25c0e09640 1 -- 172.21.15.165:0/3574906617 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c400faf0 con 0x55df963db990 2024-04-27T03:31:30.471 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.464+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55df964d6510 con 0x55df963db990 2024-04-27T03:31:30.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.464+0000 7f25c0e09640 1 -- 172.21.15.165:0/3574906617 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c400fdd0 con 0x55df963db990 2024-04-27T03:31:30.472 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.464+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df964d91c0 con 0x55df963db990 2024-04-27T03:31:30.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.465+0000 7f25c0e09640 1 -- 172.21.15.165:0/3574906617 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f26c4017040 con 0x55df963db990 2024-04-27T03:31:30.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.465+0000 7f25c0e09640 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25bc036900 0x7f25bc038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.465+0000 7f25c0e09640 1 -- 172.21.15.165:0/3574906617 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f26c4015020 con 0x55df963db990 2024-04-27T03:31:30.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.465+0000 7f26d3cd4640 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25bc036900 0x7f25bc038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.465+0000 7f26d983c800 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55df963df4a0 0x55df964dcf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.465+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df964dd4c0 con 0x55df963df4a0 2024-04-27T03:31:30.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.465+0000 7f26d4cd6640 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55df963df4a0 0x55df964dcf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.465+0000 7f26d3cd4640 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25bc036900 0x7f25bc038dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f26c8054000 tx=0x7f26c804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.466+0000 7f26d4cd6640 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55df963df4a0 0x55df964dcf80 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.479 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.466+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3574906617 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df963df4a0 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.466+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df964da530 con 0x55df963df4a0 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.466+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3574906617 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df963df4a0 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.466+0000 7f26d983c800 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55df964df7e0 0x55df964e1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.466+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df964e2270 con 0x55df964df7e0 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.466+0000 7f26d44d5640 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55df964df7e0 0x55df964e1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.467+0000 7f26d44d5640 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55df964df7e0 0x55df964e1bd0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.467+0000 7f26d44d5640 1 -- 172.21.15.165:0/3574906617 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f26c402d070 con 0x55df964df7e0 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.467+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df964de790 con 0x55df963df4a0 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.467+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3574906617 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df963df4a0 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.467+0000 7f26d983c800 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55df964e3720 0x55df964e5b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.467+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df964e62d0 con 0x55df964e3720 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.467+0000 7f26d3cd4640 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55df964e3720 0x55df964e5b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.480 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.468+0000 7f26d3cd4640 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55df964e3720 0x55df964e5b70 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.468+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3574906617 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f26c8052940 con 0x55df964e3720 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.468+0000 7f26d983c800 1 Processor -- start 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.468+0000 7f26d983c800 1 -- start start 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.468+0000 7f26d983c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df964f46e0 0x55df964f4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.468+0000 7f26d983c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55df962cc6b0 con 0x55df964f46e0 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.469+0000 7f26d44d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df964f46e0 0x55df964f4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.469+0000 7f26d44d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df964f46e0 0x55df964f4b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49524/0 (socket says 172.21.15.165:49524) 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.469+0000 7f26d44d5640 1 -- 172.21.15.165:0/2326660674 learned_addr learned my addr 172.21.15.165:0/2326660674 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.469+0000 7f26d44d5640 1 -- 172.21.15.165:0/2326660674 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df964f5040 con 0x55df964f46e0 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.469+0000 7f26d44d5640 1 --2- 172.21.15.165:0/2326660674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df964f46e0 0x55df964f4b00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f26c404b4b0 tx=0x7f26c401aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.469+0000 7f25b9ffb640 1 -- 172.21.15.165:0/2326660674 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c401ade0 con 0x55df964f46e0 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.469+0000 7f25b9ffb640 1 -- 172.21.15.165:0/2326660674 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c4019470 con 0x55df964f46e0 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.469+0000 7f26d983c800 1 -- 172.21.15.165:0/2326660674 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55df964f5ab0 con 0x55df964f46e0 2024-04-27T03:31:30.481 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.469+0000 7f25b9ffb640 1 -- 172.21.15.165:0/2326660674 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c4019750 con 0x55df964f46e0 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.469+0000 7f26d983c800 1 -- 172.21.15.165:0/2326660674 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df964f5fb0 con 0x55df964f46e0 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.470+0000 7f25b9ffb640 1 -- 172.21.15.165:0/2326660674 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f26c403f1b0 con 0x55df964f46e0 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.470+0000 7f25b9ffb640 1 --2- 172.21.15.165:0/2326660674 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25ac036970 0x7f25ac038e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.470+0000 7f25b9ffb640 1 -- 172.21.15.165:0/2326660674 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f26c4050020 con 0x55df964f46e0 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.470+0000 7f26d3cd4640 1 --2- 172.21.15.165:0/2326660674 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25ac036970 0x7f25ac038e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f26d983c800 1 Processor -- start 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f26d983c800 1 -- start start 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f26d983c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 0x55df96501ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f26d3cd4640 1 --2- 172.21.15.165:0/2326660674 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25ac036970 0x7f25ac038e30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55df964da470 tx=0x7f26c8005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f26d983c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55df963d9c10 con 0x55df96501af0 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f26d44d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 0x55df96501ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f26d44d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 0x55df96501ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49526/0 (socket says 172.21.15.165:49526) 2024-04-27T03:31:30.482 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f26d44d5640 1 -- 172.21.15.165:0/180848077 learned_addr learned my addr 172.21.15.165:0/180848077 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f26d44d5640 1 -- 172.21.15.165:0/180848077 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df965024a0 con 0x55df96501af0 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f26d44d5640 1 --2- 172.21.15.165:0/180848077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 0x55df96501ed0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f26c401ac40 tx=0x7f26c4032e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9764eb38678c4eab server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f25b37fe640 1 -- 172.21.15.165:0/180848077 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c4030d50 con 0x55df96501af0 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f25b37fe640 1 -- 172.21.15.165:0/180848077 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c403cd30 con 0x55df96501af0 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.471+0000 7f25b37fe640 1 -- 172.21.15.165:0/180848077 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c4040a30 con 0x55df96501af0 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.472+0000 7f26d983c800 1 -- 172.21.15.165:0/180848077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 msgr2=0x55df96501ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.472+0000 7f26d983c800 1 --2- 172.21.15.165:0/180848077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 0x55df96501ed0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f26c401ac40 tx=0x7f26c4032e30 comp rx=0 tx=0).stop 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.472+0000 7f26d983c800 1 -- 172.21.15.165:0/180848077 shutdown_connections 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.472+0000 7f26d983c800 1 --2- 172.21.15.165:0/180848077 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 0x55df96501ed0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.472+0000 7f26d983c800 1 -- 172.21.15.165:0/180848077 >> 172.21.15.165:0/180848077 conn(0x55df964fddc0 msgr2=0x55df964fbe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.472+0000 7f26d983c800 1 -- 172.21.15.165:0/180848077 shutdown_connections 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.472+0000 7f26d983c800 1 -- 172.21.15.165:0/180848077 wait complete. 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.472+0000 7f26d983c800 1 Processor -- start 2024-04-27T03:31:30.483 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.472+0000 7f26d983c800 1 -- start start 2024-04-27T03:31:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.472+0000 7f26d983c800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 0x55df96508f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.472+0000 7f26d983c800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55df96502bb0 con 0x55df96501af0 2024-04-27T03:31:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.473+0000 7f26d44d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 0x55df96508f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.473+0000 7f26d44d5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 0x55df96508f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49532/0 (socket says 172.21.15.165:49532) 2024-04-27T03:31:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.473+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 learned_addr learned my addr 172.21.15.165:0/3580001250 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.473+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55df96509450 con 0x55df96501af0 2024-04-27T03:31:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.473+0000 7f26d44d5640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 0x55df96508f10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f26c4019b00 tx=0x7f26c4027120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.473+0000 7f25b1ffb640 1 -- 172.21.15.165:0/3580001250 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c4040e20 con 0x55df96501af0 2024-04-27T03:31:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.473+0000 7f25b1ffb640 1 -- 172.21.15.165:0/3580001250 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f26c400c280 con 0x55df96501af0 2024-04-27T03:31:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.473+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55df96509450 con 0x55df96501af0 2024-04-27T03:31:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.473+0000 7f25b1ffb640 1 -- 172.21.15.165:0/3580001250 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f26c401c5e0 con 0x55df96501af0 2024-04-27T03:31:30.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.473+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55df96509450 con 0x55df96501af0 2024-04-27T03:31:30.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.474+0000 7f25b1ffb640 1 -- 172.21.15.165:0/3580001250 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f26c4053040 con 0x55df96501af0 2024-04-27T03:31:30.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.474+0000 7f25b1ffb640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25a4036950 0x7f25a4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.474+0000 7f25b1ffb640 1 -- 172.21.15.165:0/3580001250 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f26c406a070 con 0x55df96501af0 2024-04-27T03:31:30.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.474+0000 7f26d3cd4640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25a4036950 0x7f25a4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.474+0000 7f26d983c800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.498 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.474+0000 7f26d983c800 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55df96514590 0x55df96516a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.474+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df965171a0 con 0x55df96514590 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.474+0000 7f26d4cd6640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55df96514590 0x55df96516a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d3cd4640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25a4036950 0x7f25a4038e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f26c8052fd0 tx=0x7f26c804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d4cd6640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55df96514590 0x55df96516a50 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d983c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d983c800 20 realm 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d983c800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df965175b0 con 0x55df96514590 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d983c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d983c800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d983c800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.499 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df96517e80 con 0x55df96514590 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d983c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d983c800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.475+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96514590 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.476+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.476+0000 7f26d983c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.476+0000 7f26d983c800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.476+0000 7f26d983c800 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55df96518d10 0x55df9651b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.476+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df9651b860 con 0x55df96518d10 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.476+0000 7f26d44d5640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55df96518d10 0x55df9651b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.476+0000 7f26d44d5640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55df96518d10 0x55df9651b100 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.476+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.500 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.476+0000 7f26d983c800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.476+0000 7f26d983c800 20 searching for the correct realm 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.476+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df9651d240 con 0x55df96518d10 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.477+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.477+0000 7f26d44d5640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f26c403c790 0x7f26c40351a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.477+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df9651d240 con 0x7f26c403c790 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.477+0000 7f26d3cd4640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f26c403c790 0x7f26c40351a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.477+0000 7f26d3cd4640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f26c403c790 0x7f26c40351a0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.478+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.478+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df9651d240 con 0x7f26c403c790 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.478+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.478+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df9651d240 con 0x55df96514590 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.478+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.501 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.478+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96514590 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.478+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.478+0000 7f26d4cd6640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f26cc016440 0x7f26cc0168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.478+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26cc016440 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.479+0000 7f26d4cd6640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f26cc016440 0x7f26cc0168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.479+0000 7f26d4cd6640 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f26cc016440 0x7f26cc0168c0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.480+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.480+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26cc016440 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.480+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.480+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26c403c790 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.480+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.480+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96518d10 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.480+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.480+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96514590 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.481+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.481+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26c403c790 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.481+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.481+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96514590 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.481+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.481+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26c403c790 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.481+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.481+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96518d10 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.482+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.482+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96518d10 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.482+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.482+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26cc016440 2024-04-27T03:31:30.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.482+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.482+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96518d10 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.482+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.482+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96518d10 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.483+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.483+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96514590 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.483+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.483+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96514590 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.483+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.483+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26cc016440 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.484+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.484+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96518d10 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.484+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.484+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26c403c790 2024-04-27T03:31:30.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.484+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.484+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96514590 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.484+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.484+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26cc016440 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.485+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.485+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96518d10 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.485+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.485+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26cc016440 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.485+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.485+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26c403c790 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.485+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.485+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96518d10 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.486+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.486+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26cc016440 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.486+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.486+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x55df96514590 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.486+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.486+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55df96518220 con 0x7f26c403c790 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df9651c210 con 0x55df96514590 2024-04-27T03:31:30.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df9651c600 con 0x55df96514590 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df9651c600 con 0x7f26c403c790 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 zone default found 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 4 Realm: () 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:30.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 zonegroup default 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.487+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df9651d710 con 0x7f26cc016440 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d983c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d983c800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df9651f0f0 con 0x55df96518d10 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d983c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d983c800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df9651f490 con 0x55df96518d10 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d983c800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d983c800 20 started sync module instance, tier type = 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d983c800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.488+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df9651e590 con 0x7f26c403c790 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.489+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.489+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94418788018480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df9651ed50 con 0x7f26c403c790 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.489+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df965201d0 con 0x55df96514590 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.490+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94418788018480] v27'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.491+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.491+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94418788022320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df965205f0 con 0x55df96514590 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.491+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df96521010 con 0x55df96518d10 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.492+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.492+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94418788027136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df965218e0 con 0x55df96518d10 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.492+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df965223c0 con 0x7f26c403c790 2024-04-27T03:31:30.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.492+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94418788022320] v27'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.493+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94418788027136] v27'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.493+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.493+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94418788032208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df965227b0 con 0x7f26c403c790 2024-04-27T03:31:30.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.493+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df96523040 con 0x55df96514590 2024-04-27T03:31:30.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.494+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94418788032208] v27'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.494+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.494+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94418788023376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df96523430 con 0x55df96514590 2024-04-27T03:31:30.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.494+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df96523e50 con 0x7f26cc016440 2024-04-27T03:31:30.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.495+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94418788023376] v27'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.496+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.496+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94418788038976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df96523e10 con 0x7f26cc016440 2024-04-27T03:31:30.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.496+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df96524890 con 0x7f26cc016440 2024-04-27T03:31:30.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.496+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94418788038976] v27'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.497+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.497+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94418788035408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df96524890 con 0x7f26cc016440 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.497+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df96525250 con 0x7f26cc016440 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.498+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94418788035408] v27'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.498+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'204 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.498+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94418788041696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df96525250 con 0x7f26cc016440 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.498+0000 7f26d983c800 20 add_watcher() i=0 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.498+0000 7f26d983c800 20 add_watcher() i=1 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.498+0000 7f26d983c800 20 add_watcher() i=2 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.498+0000 7f26d983c800 20 add_watcher() i=3 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.498+0000 7f26d983c800 20 add_watcher() i=4 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.498+0000 7f26d983c800 20 add_watcher() i=5 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.498+0000 7f26d983c800 20 add_watcher() i=6 2024-04-27T03:31:30.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.499+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94418788041696] v27'205 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.499+0000 7f26d983c800 20 add_watcher() i=7 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.499+0000 7f26d983c800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.499+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df965286b0 con 0x55df96514590 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.499+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.499+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94418788044656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df96525b70 con 0x55df96514590 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.500+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94418788044656] v27'212 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.500+0000 7f258a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.500+0000 7f26d983c800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.500+0000 7f26d983c800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.500+0000 7f26d983c800 5 note: GC not initialized 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.500+0000 7f25897e2640 20 reqs_thread_entry: start 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.501+0000 7f26d983c800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:30.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.501+0000 7f26d983c800 20 Filter name: none 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.501+0000 7f24faffd640 20 reqs_thread_entry: start 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.501+0000 7f26d983c800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.501+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df9655b550 con 0x55df96514590 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.501+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.501+0000 7f26d983c800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.501+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55df9655caa0 con 0x7f26cc016440 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.502+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) ==== 152+0+1268 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.503+0000 7f258a7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.503+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94418788044656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df96527b80 con 0x55df96514590 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.503+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94418788044656] v27'213 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.503+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94418788018480] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df96524f40 con 0x7f26c403c790 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.505+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94418788018480] v27'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.505+0000 7f26d983c800 20 remove_watcher() i=0 2024-04-27T03:31:30.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.505+0000 7f26d983c800 2 removed watcher, disabling cache 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.505+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94418788022320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df9651e930 con 0x55df96514590 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.506+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94418788022320] v27'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.506+0000 7f26d983c800 20 remove_watcher() i=1 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.506+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94418788027136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df9651e930 con 0x55df96518d10 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.507+0000 7f26d44d5640 1 -- 172.21.15.165:0/3580001250 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94418788027136] v27'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26c4074020 con 0x55df96518d10 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.507+0000 7f26d983c800 20 remove_watcher() i=2 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.507+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94418788032208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df9651e930 con 0x7f26c403c790 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.508+0000 7f26d3cd4640 1 -- 172.21.15.165:0/3580001250 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94418788032208] v27'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26c8081070 con 0x7f26c403c790 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.508+0000 7f26d983c800 20 remove_watcher() i=3 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.508+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94418788023376] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df9651e930 con 0x55df96514590 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.509+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94418788023376] v27'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x55df96514590 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.509+0000 7f26d983c800 20 remove_watcher() i=4 2024-04-27T03:31:30.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.509+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94418788038976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df9651e930 con 0x7f26cc016440 2024-04-27T03:31:30.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.510+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94418788038976] v27'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.510+0000 7f26d983c800 20 remove_watcher() i=5 2024-04-27T03:31:30.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.510+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94418788035408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df9651e930 con 0x7f26cc016440 2024-04-27T03:31:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.511+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94418788035408] v27'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.511+0000 7f26d983c800 20 remove_watcher() i=6 2024-04-27T03:31:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.511+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94418788041696] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55df9651e930 con 0x7f26cc016440 2024-04-27T03:31:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94418788041696] v27'206 uv204 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f26cc00a4a0 con 0x7f26cc016440 2024-04-27T03:31:30.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 20 remove_watcher() i=7 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f26cc016440 msgr2=0x7f26cc0168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f26cc016440 0x7f26cc0168c0 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55df96514590 msgr2=0x55df96516a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55df96514590 0x55df96516a50 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f26c403c790 msgr2=0x7f26c40351a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f26c403c790 0x7f26c40351a0 crc :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55df96518d10 msgr2=0x55df9651b100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55df96518d10 0x55df9651b100 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25a4036950 msgr2=0x7f25a4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25a4036950 0x7f25a4038e10 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f26c8052fd0 tx=0x7f26c804f000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 msgr2=0x55df96508f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3580001250 reap_dead start 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 0x55df96508f10 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f26c4019b00 tx=0x7f26c4027120 comp rx=0 tx=0).stop 2024-04-27T03:31:30.525 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 shutdown_connections 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 --2- 172.21.15.165:0/3580001250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df96501af0 0x55df96508f10 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 >> 172.21.15.165:0/3580001250 conn(0x55df964fddc0 msgr2=0x55df964fb420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 shutdown_connections 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.512+0000 7f26d983c800 1 -- 172.21.15.165:0/3580001250 wait complete. 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/2326660674 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25ac036970 msgr2=0x7f25ac038e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 --2- 172.21.15.165:0/2326660674 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25ac036970 0x7f25ac038e30 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x55df964da470 tx=0x7f26c8005a00 comp rx=0 tx=0).stop 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/2326660674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df964f46e0 msgr2=0x55df964f4b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 --2- 172.21.15.165:0/2326660674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df964f46e0 0x55df964f4b00 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7f26c404b4b0 tx=0x7f26c401aaf0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/2326660674 shutdown_connections 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 --2- 172.21.15.165:0/2326660674 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25ac036970 0x7f25ac038e30 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 --2- 172.21.15.165:0/2326660674 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df964f46e0 0x55df964f4b00 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/2326660674 >> 172.21.15.165:0/2326660674 conn(0x55df964e8780 msgr2=0x55df964e8b80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/2326660674 shutdown_connections 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/2326660674 wait complete. 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55df963df4a0 msgr2=0x55df964dcf80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55df963df4a0 0x55df964dcf80 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55df964e3720 msgr2=0x55df964e5b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55df964e3720 0x55df964e5b70 crc :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55df964df7e0 msgr2=0x55df964e1bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55df964df7e0 0x55df964e1bd0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25bc036900 msgr2=0x7f25bc038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f25bc036900 0x7f25bc038dc0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f26c8054000 tx=0x7f26c804e000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 msgr2=0x55df964d5d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 --2- 172.21.15.165:0/3574906617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55df963db990 0x55df964d5d40 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f26c400b210 tx=0x7f26c4000ed0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d4cd6640 1 -- 172.21.15.165:0/3574906617 reap_dead start 2024-04-27T03:31:30.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 shutdown_connections 2024-04-27T03:31:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 >> 172.21.15.165:0/3574906617 conn(0x55df963d5340 msgr2=0x55df963da170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 shutdown_connections 2024-04-27T03:31:30.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.513+0000 7f26d983c800 1 -- 172.21.15.165:0/3574906617 wait complete. 2024-04-27T03:31:30.529 INFO:teuthology.orchestra.run.smithi165.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]},{"bucket":"myfoo","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}]},{"bucket":"myfoo2","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]},{"bucket":"myfoo3","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]},{"bucket":"myfoo4","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]}]}],"summary":[{"user":"foo","categories":[{"category":"delete_obj","bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}],"total":{"bytes_sent":0,"bytes_received":0,"ops":4,"successful_ops":4}}]} 2024-04-27T03:31:30.529 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo3', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo4', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}]} 2024-04-27T03:31:30.529 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-04-27T03:31:30.529 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo', '--categories', 'delete_bucket'] 2024-04-27T03:31:30.529 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo --categories delete_bucket 2024-04-27T03:31:30.573 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:30.574 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:30.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.569+0000 7f6ac8894800 1 Processor -- start 2024-04-27T03:31:30.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.569+0000 7f6ac8894800 1 -- start start 2024-04-27T03:31:30.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.569+0000 7f6ac8894800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55bd95fc4350 0x55bd95fc6740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.569+0000 7f6ac8894800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55bd95e7ed40 con 0x55bd95fbffb0 2024-04-27T03:31:30.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.569+0000 7f6ac8894800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55bd95e514d0 con 0x55bd95fc4350 2024-04-27T03:31:30.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.569+0000 7f6ac5b92640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55bd95fc4350 0x55bd95fc6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.569+0000 7f6ac6393640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55bd95fbffb0 0x55bd95fc03d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:54738/0 (socket says 172.21.15.165:54738) 2024-04-27T03:31:30.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.569+0000 7f6ac6393640 1 -- 172.21.15.165:0/2353855151 learned_addr learned my addr 172.21.15.165:0/2353855151 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:31:30.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.569+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2353855151 >> v1:172.21.15.89:6789/0 conn(0x55bd95fbffb0 legacy=0x55bd95fc03d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.569+0000 7f6ac5391640 1 -- 172.21.15.165:0/2353855151 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 33287687 0 0) 0x55bd95e7ed40 con 0x55bd95fbffb0 2024-04-27T03:31:30.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.569+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2353855151 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd95e7d590 con 0x55bd95fc4350 2024-04-27T03:31:30.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.571+0000 7f6ac5b92640 1 --2- 172.21.15.165:0/2353855151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fc4350 0x55bd95fc6740 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00a470 tx=0x7f6abc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=677b315af3df87b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.571+0000 7f6ac5391640 1 -- 172.21.15.165:0/2353855151 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6abc057070 con 0x55bd95fc4350 2024-04-27T03:31:30.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.571+0000 7f6ac5391640 1 -- 172.21.15.165:0/2353855151 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6abc052570 con 0x55bd95fc4350 2024-04-27T03:31:30.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.571+0000 7f6ac5391640 1 -- 172.21.15.165:0/2353855151 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6abc052870 con 0x55bd95fc4350 2024-04-27T03:31:30.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.572+0000 7f6ac8894800 1 -- 172.21.15.165:0/2353855151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fc4350 msgr2=0x55bd95fc6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.572+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2353855151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fc4350 0x55bd95fc6740 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f6abc00a470 tx=0x7f6abc01ce90 comp rx=0 tx=0).stop 2024-04-27T03:31:30.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.572+0000 7f6ac8894800 1 -- 172.21.15.165:0/2353855151 shutdown_connections 2024-04-27T03:31:30.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.572+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2353855151 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fc4350 0x55bd95fc6740 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.572+0000 7f6ac8894800 1 -- 172.21.15.165:0/2353855151 >> 172.21.15.165:0/2353855151 conn(0x55bd95fbb550 msgr2=0x55bd95fbd970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.572+0000 7f6ac8894800 1 -- 172.21.15.165:0/2353855151 shutdown_connections 2024-04-27T03:31:30.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.572+0000 7f6ac8894800 1 -- 172.21.15.165:0/2353855151 wait complete. 2024-04-27T03:31:30.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.584+0000 7f6ac8894800 1 Processor -- start 2024-04-27T03:31:30.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.584+0000 7f6ac8894800 1 -- start start 2024-04-27T03:31:30.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.584+0000 7f6ac8894800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 0x55bd95f359c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.584+0000 7f6ac8894800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bd95f0b3b0 con 0x55bd95fbffb0 2024-04-27T03:31:30.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.584+0000 7f6ac6393640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 0x55bd95f359c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.584+0000 7f6ac6393640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 0x55bd95f359c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49556/0 (socket says 172.21.15.165:49556) 2024-04-27T03:31:30.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.584+0000 7f6ac6393640 1 -- 172.21.15.165:0/2562633215 learned_addr learned my addr 172.21.15.165:0/2562633215 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.584+0000 7f6ac6393640 1 -- 172.21.15.165:0/2562633215 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd95f33f80 con 0x55bd95fbffb0 2024-04-27T03:31:30.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.585+0000 7f6ac6393640 1 --2- 172.21.15.165:0/2562633215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 0x55bd95f359c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800b240 tx=0x7f6ab800b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13358e45f5494797 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.585+0000 7f69b4608640 1 -- 172.21.15.165:0/2562633215 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab8008070 con 0x55bd95fbffb0 2024-04-27T03:31:30.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.585+0000 7f69b4608640 1 -- 172.21.15.165:0/2562633215 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab800d500 con 0x55bd95fbffb0 2024-04-27T03:31:30.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.585+0000 7f69b4608640 1 -- 172.21.15.165:0/2562633215 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab800d860 con 0x55bd95fbffb0 2024-04-27T03:31:30.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac8894800 1 -- 172.21.15.165:0/2562633215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 msgr2=0x55bd95f359c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2562633215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 0x55bd95f359c0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f6ab800b240 tx=0x7f6ab800b700 comp rx=0 tx=0).stop 2024-04-27T03:31:30.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac8894800 1 -- 172.21.15.165:0/2562633215 shutdown_connections 2024-04-27T03:31:30.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2562633215 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 0x55bd95f359c0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac8894800 1 -- 172.21.15.165:0/2562633215 >> 172.21.15.165:0/2562633215 conn(0x55bd95fbb550 msgr2=0x55bd960ac8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac8894800 1 -- 172.21.15.165:0/2562633215 shutdown_connections 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac8894800 1 -- 172.21.15.165:0/2562633215 wait complete. 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac8894800 1 Processor -- start 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac8894800 1 -- start start 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac8894800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 0x55bd960bada0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac8894800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bd95fc7010 con 0x55bd95fbffb0 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac6393640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 0x55bd960bada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac6393640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 0x55bd960bada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49558/0 (socket says 172.21.15.165:49558) 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.586+0000 7f6ac6393640 1 -- 172.21.15.165:0/3289334280 learned_addr learned my addr 172.21.15.165:0/3289334280 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.587+0000 7f6ac6393640 1 -- 172.21.15.165:0/3289334280 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd960bb2e0 con 0x55bd95fbffb0 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.587+0000 7f6ac6393640 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 0x55bd960bada0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55bd95f34ff0 tx=0x7f6ab8000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.587+0000 7f69b2e05640 1 -- 172.21.15.165:0/3289334280 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab8019870 con 0x55bd95fbffb0 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.587+0000 7f69b2e05640 1 -- 172.21.15.165:0/3289334280 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab8019a10 con 0x55bd95fbffb0 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.587+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bd960bb570 con 0x55bd95fbffb0 2024-04-27T03:31:30.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.587+0000 7f69b2e05640 1 -- 172.21.15.165:0/3289334280 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab8019d30 con 0x55bd95fbffb0 2024-04-27T03:31:30.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.587+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd960be220 con 0x55bd95fbffb0 2024-04-27T03:31:30.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.588+0000 7f69b2e05640 1 -- 172.21.15.165:0/3289334280 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6ab8017040 con 0x55bd95fbffb0 2024-04-27T03:31:30.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.588+0000 7f69b2e05640 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f69ac036950 0x7f69ac038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.588+0000 7f69b2e05640 1 -- 172.21.15.165:0/3289334280 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f6ab8015020 con 0x55bd95fbffb0 2024-04-27T03:31:30.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.588+0000 7f6ac5b92640 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f69ac036950 0x7f69ac038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.588+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bd95fc4350 0x55bd960c1fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.588+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd960c2560 con 0x55bd95fc4350 2024-04-27T03:31:30.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.588+0000 7f6ac6b94640 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bd95fc4350 0x55bd960c1fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.588+0000 7f6ac5b92640 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f69ac036950 0x7f69ac038e10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6abc054000 tx=0x7f6abc04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.589+0000 7f6ac6b94640 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bd95fc4350 0x55bd960c1fe0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.589+0000 7f6ac6b94640 1 -- 172.21.15.165:0/3289334280 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd95fc4350 2024-04-27T03:31:30.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.589+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd960bf5b0 con 0x55bd95fc4350 2024-04-27T03:31:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.589+0000 7f6ac6b94640 1 -- 172.21.15.165:0/3289334280 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd95fc4350 2024-04-27T03:31:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.589+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bd960c4890 0x55bd960c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.589+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd960c7370 con 0x55bd960c4890 2024-04-27T03:31:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.589+0000 7f6ac6393640 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bd960c4890 0x55bd960c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.589+0000 7f6ac6393640 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bd960c4890 0x55bd960c6c80 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.590+0000 7f6ac6393640 1 -- 172.21.15.165:0/3289334280 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f6ab802d070 con 0x55bd960c4890 2024-04-27T03:31:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.590+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd960c3840 con 0x55bd95fc4350 2024-04-27T03:31:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.590+0000 7f6ac6b94640 1 -- 172.21.15.165:0/3289334280 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd95fc4350 2024-04-27T03:31:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.590+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55bd960c8820 0x55bd960cac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.590+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd960cb3d0 con 0x55bd960c8820 2024-04-27T03:31:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.590+0000 7f6ac5b92640 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55bd960c8820 0x55bd960cac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.591+0000 7f6ac5b92640 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55bd960c8820 0x55bd960cac70 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.591+0000 7f6ac5b92640 1 -- 172.21.15.165:0/3289334280 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f6abc052940 con 0x55bd960c8820 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.591+0000 7f6ac8894800 1 Processor -- start 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.591+0000 7f6ac8894800 1 -- start start 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.591+0000 7f6ac8894800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960d9830 0x55bd960d9c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.591+0000 7f6ac8894800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bd95eb16b0 con 0x55bd960d9830 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.592+0000 7f6ac6393640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960d9830 0x55bd960d9c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.592+0000 7f6ac6393640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960d9830 0x55bd960d9c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49566/0 (socket says 172.21.15.165:49566) 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.592+0000 7f6ac6393640 1 -- 172.21.15.165:0/3058331003 learned_addr learned my addr 172.21.15.165:0/3058331003 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.592+0000 7f6ac6393640 1 -- 172.21.15.165:0/3058331003 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd960da190 con 0x55bd960d9830 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.592+0000 7f6ac6393640 1 --2- 172.21.15.165:0/3058331003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960d9830 0x55bd960d9c50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6ab804b480 tx=0x7f6ab800f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.592+0000 7f69abfff640 1 -- 172.21.15.165:0/3058331003 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab801a8f0 con 0x55bd960d9830 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.592+0000 7f69abfff640 1 -- 172.21.15.165:0/3058331003 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab8041e10 con 0x55bd960d9830 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.592+0000 7f6ac8894800 1 -- 172.21.15.165:0/3058331003 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bd960dac00 con 0x55bd960d9830 2024-04-27T03:31:30.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.592+0000 7f69abfff640 1 -- 172.21.15.165:0/3058331003 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab8044ba0 con 0x55bd960d9830 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.592+0000 7f6ac8894800 1 -- 172.21.15.165:0/3058331003 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd960db100 con 0x55bd960d9830 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.593+0000 7f69abfff640 1 -- 172.21.15.165:0/3058331003 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6ab8044d40 con 0x55bd960d9830 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.593+0000 7f69abfff640 1 --2- 172.21.15.165:0/3058331003 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f69a0036950 0x7f69a0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.593+0000 7f69abfff640 1 -- 172.21.15.165:0/3058331003 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f6ab803bc10 con 0x55bd960d9830 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.593+0000 7f6ac5b92640 1 --2- 172.21.15.165:0/3058331003 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f69a0036950 0x7f69a0038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.593+0000 7f6ac8894800 1 Processor -- start 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.593+0000 7f6ac8894800 1 -- start start 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.593+0000 7f6ac5b92640 1 --2- 172.21.15.165:0/3058331003 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f69a0036950 0x7f69a0038e10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55bd960c2e40 tx=0x7f6abc005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.594+0000 7f6ac8894800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 0x55bd960e6f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.594+0000 7f6ac8894800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bd95f34300 con 0x55bd960e6b10 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.594+0000 7f6ac6393640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 0x55bd960e6f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.594+0000 7f6ac6393640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 0x55bd960e6f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49568/0 (socket says 172.21.15.165:49568) 2024-04-27T03:31:30.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.594+0000 7f6ac6393640 1 -- 172.21.15.165:0/3509113194 learned_addr learned my addr 172.21.15.165:0/3509113194 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.594+0000 7f6ac6393640 1 -- 172.21.15.165:0/3509113194 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd960e7520 con 0x55bd960e6b10 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.594+0000 7f6ac6393640 1 --2- 172.21.15.165:0/3509113194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 0x55bd960e6f30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8018600 tx=0x7f6ab8026610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c18cef9191580b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.594+0000 7f69a9ffb640 1 -- 172.21.15.165:0/3509113194 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab805c070 con 0x55bd960e6b10 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.594+0000 7f69a9ffb640 1 -- 172.21.15.165:0/3509113194 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab8018810 con 0x55bd960e6b10 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.594+0000 7f69a9ffb640 1 -- 172.21.15.165:0/3509113194 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab803f4e0 con 0x55bd960e6b10 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac8894800 1 -- 172.21.15.165:0/3509113194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 msgr2=0x55bd960e6f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3509113194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 0x55bd960e6f30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6ab8018600 tx=0x7f6ab8026610 comp rx=0 tx=0).stop 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac8894800 1 -- 172.21.15.165:0/3509113194 shutdown_connections 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3509113194 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 0x55bd960e6f30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac8894800 1 -- 172.21.15.165:0/3509113194 >> 172.21.15.165:0/3509113194 conn(0x55bd960e2e50 msgr2=0x55bd960e0eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac8894800 1 -- 172.21.15.165:0/3509113194 shutdown_connections 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac8894800 1 -- 172.21.15.165:0/3509113194 wait complete. 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac8894800 1 Processor -- start 2024-04-27T03:31:30.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac8894800 1 -- start start 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac8894800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 0x55bd960edeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac8894800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55bd960e7c40 con 0x55bd960e6b10 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac6393640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 0x55bd960edeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac6393640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 0x55bd960edeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49578/0 (socket says 172.21.15.165:49578) 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.595+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 learned_addr learned my addr 172.21.15.165:0/2753020110 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.596+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bd960ee3f0 con 0x55bd960e6b10 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.596+0000 7f6ac6393640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 0x55bd960edeb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6ab804b8f0 tx=0x7f6ab804d120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.596+0000 7f6997fff640 1 -- 172.21.15.165:0/2753020110 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab803de20 con 0x55bd960e6b10 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.596+0000 7f6997fff640 1 -- 172.21.15.165:0/2753020110 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f6ab806b070 con 0x55bd960e6b10 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.596+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55bd960ee3f0 con 0x55bd960e6b10 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.596+0000 7f6997fff640 1 -- 172.21.15.165:0/2753020110 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ab804b960 con 0x55bd960e6b10 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.596+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bd960ee3f0 con 0x55bd960e6b10 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.597+0000 7f6997fff640 1 -- 172.21.15.165:0/2753020110 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6ab805f070 con 0x55bd960e6b10 2024-04-27T03:31:30.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.597+0000 7f6997fff640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6990036950 0x7f6990038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.597+0000 7f6997fff640 1 -- 172.21.15.165:0/2753020110 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f6ab8004150 con 0x55bd960e6b10 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.597+0000 7f6ac5b92640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6990036950 0x7f6990038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.597+0000 7f6ac8894800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.597+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bd960f9420 0x55bd960fb8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.597+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fc030 con 0x55bd960f9420 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.597+0000 7f6ac6b94640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bd960f9420 0x55bd960fb8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.597+0000 7f6ac5b92640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6990036950 0x7f6990038e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6abc052fd0 tx=0x7f6abc04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac6b94640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bd960f9420 0x55bd960fb8e0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac8894800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac8894800 20 realm 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac8894800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fc440 con 0x55bd960f9420 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac8894800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac8894800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac8894800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fcd10 con 0x55bd960f9420 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac8894800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac8894800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.598+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960f9420 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.599+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.599+0000 7f6ac8894800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.599+0000 7f6ac8894800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.599+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bd960fdba0 0x55bd960fff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.599+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd961006f0 con 0x55bd960fdba0 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.599+0000 7f6ac6393640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bd960fdba0 0x55bd960fff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.599+0000 7f6ac6393640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bd960fdba0 0x55bd960fff90 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.599+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.599+0000 7f6ac8894800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.599+0000 7f6ac8894800 20 searching for the correct realm 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.599+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd961020d0 con 0x55bd960fdba0 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.600+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.600+0000 7f6ac6393640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6ab8026c20 0x7f6ab80083b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.600+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd961020d0 con 0x7f6ab8026c20 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.600+0000 7f6ac5b92640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6ab8026c20 0x7f6ab80083b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.600+0000 7f6ac5b92640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6ab8026c20 0x7f6ab80083b0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.600+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.600+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd961020d0 con 0x7f6ab8026c20 2024-04-27T03:31:30.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.601+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.601+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd961020d0 con 0x55bd960f9420 2024-04-27T03:31:30.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.601+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.601+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960f9420 2024-04-27T03:31:30.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.601+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.601+0000 7f6ac6b94640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6ac0016440 0x7f6ac00168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.601+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ac0016440 2024-04-27T03:31:30.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.601+0000 7f6ac6b94640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6ac0016440 0x7f6ac00168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.602+0000 7f6ac6b94640 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6ac0016440 0x7f6ac00168c0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.602+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.602+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ac0016440 2024-04-27T03:31:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.602+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.602+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ab8026c20 2024-04-27T03:31:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.602+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.602+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960fdba0 2024-04-27T03:31:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.603+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.603+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960f9420 2024-04-27T03:31:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.603+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.603+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ab8026c20 2024-04-27T03:31:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.603+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.603+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960f9420 2024-04-27T03:31:30.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.603+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.603+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ab8026c20 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.604+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.604+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960fdba0 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.604+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.604+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960fdba0 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.604+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.604+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ac0016440 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.604+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.604+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960fdba0 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.605+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.605+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960fdba0 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.605+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.605+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960f9420 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.605+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.605+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960f9420 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.605+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.605+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ac0016440 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.606+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.606+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960fdba0 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.606+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.606+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ab8026c20 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.606+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.606+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960f9420 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.606+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.606+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ac0016440 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.607+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.607+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960fdba0 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.607+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.607+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ac0016440 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.607+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.607+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ab8026c20 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.607+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.607+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960fdba0 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.608+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.608+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ac0016440 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.608+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.608+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x55bd960f9420 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.608+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.608+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55bd960fd0b0 con 0x7f6ab8026c20 2024-04-27T03:31:30.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.608+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd961010a0 con 0x55bd960f9420 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd96101490 con 0x55bd960f9420 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd96101490 con 0x7f6ab8026c20 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 zone default found 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 4 Realm: () 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 zonegroup default 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.609+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd961025a0 con 0x7f6ac0016440 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac8894800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac8894800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd96103f80 con 0x55bd960fdba0 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac8894800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac8894800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd96104320 con 0x55bd960fdba0 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac8894800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac8894800 20 started sync module instance, tier type = 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac8894800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.610+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96103420 con 0x7f6ab8026c20 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.611+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.611+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94272754825280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96104b70 con 0x7f6ab8026c20 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.611+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd961053c0 con 0x55bd960f9420 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.612+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94272754825280] v27'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.613+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.613+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94272754828992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd961053c0 con 0x55bd960f9420 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.613+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96105e90 con 0x55bd960fdba0 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.614+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.614+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94272754833792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96106730 con 0x55bd960fdba0 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.614+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96107230 con 0x7f6ab8026c20 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.614+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94272754828992] v27'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.615+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94272754833792] v27'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.615+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.615+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94272754838816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96107620 con 0x7f6ab8026c20 2024-04-27T03:31:30.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.615+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96107e70 con 0x55bd960f9420 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.616+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.616+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94272754842528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96108710 con 0x55bd960f9420 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.616+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96109190 con 0x7f6ac0016440 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.616+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94272754838816] v27'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.617+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.617+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94272754846880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96109190 con 0x7f6ac0016440 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.617+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96109c30 con 0x7f6ac0016440 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.617+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94272754842528] v27'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.618+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94272754846880] v27'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.618+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.618+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94272754831120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96108260 con 0x7f6ac0016440 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.618+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd9610a0a0 con 0x7f6ac0016440 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.619+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94272754831120] v27'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.619+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'207 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.619+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94272754849568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd9610a0a0 con 0x7f6ac0016440 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.619+0000 7f6ac8894800 20 add_watcher() i=0 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.619+0000 7f6ac8894800 20 add_watcher() i=1 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.619+0000 7f6ac8894800 20 add_watcher() i=2 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.619+0000 7f6ac8894800 20 add_watcher() i=3 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.619+0000 7f6ac8894800 20 add_watcher() i=4 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.619+0000 7f6ac8894800 20 add_watcher() i=5 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.619+0000 7f6ac8894800 20 add_watcher() i=6 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.620+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94272754849568] v27'208 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.620+0000 7f6ac8894800 20 add_watcher() i=7 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.620+0000 7f6ac8894800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.620+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd9610d4d0 con 0x55bd960f9420 2024-04-27T03:31:30.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.620+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.620+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94272754851264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd9610aa20 con 0x55bd960f9420 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.621+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94272754851264] v27'214 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f697cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f6ac8894800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f6ac8894800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f6ac8894800 5 note: GC not initialized 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f68f77fe640 20 reqs_thread_entry: start 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f6ac8894800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f6ac8894800 20 Filter name: none 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f68f57fa640 20 reqs_thread_entry: start 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f6ac8894800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd96140350 con 0x55bd960f9420 2024-04-27T03:31:30.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f6ac8894800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:31:30.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.622+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55bd961418a0 con 0x7f6ac0016440 2024-04-27T03:31:30.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.623+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 usage.10 [call out=1268b] v0'0 uv29 ondisk = 0) ==== 152+0+1268 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.624+0000 7f697cfe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.624+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94272754851264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd960f6a70 con 0x55bd960f9420 2024-04-27T03:31:30.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.625+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94272754851264] v27'215 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.625+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94272754825280] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96109d90 con 0x7f6ab8026c20 2024-04-27T03:31:30.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.626+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94272754825280] v27'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.626+0000 7f6ac8894800 20 remove_watcher() i=0 2024-04-27T03:31:30.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.626+0000 7f6ac8894800 2 removed watcher, disabling cache 2024-04-27T03:31:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.626+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94272754828992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96103840 con 0x55bd960f9420 2024-04-27T03:31:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.627+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94272754828992] v27'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.627+0000 7f6ac8894800 20 remove_watcher() i=1 2024-04-27T03:31:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.627+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94272754833792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96103840 con 0x55bd960fdba0 2024-04-27T03:31:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.628+0000 7f6ac6393640 1 -- 172.21.15.165:0/2753020110 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94272754833792] v27'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ab805e030 con 0x55bd960fdba0 2024-04-27T03:31:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.628+0000 7f6ac8894800 20 remove_watcher() i=2 2024-04-27T03:31:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.628+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94272754838816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96103840 con 0x7f6ab8026c20 2024-04-27T03:31:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.629+0000 7f6ac5b92640 1 -- 172.21.15.165:0/2753020110 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94272754838816] v27'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6abc081070 con 0x7f6ab8026c20 2024-04-27T03:31:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.629+0000 7f6ac8894800 20 remove_watcher() i=3 2024-04-27T03:31:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.629+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94272754842528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96106d20 con 0x55bd960f9420 2024-04-27T03:31:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.630+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94272754842528] v27'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x55bd960f9420 2024-04-27T03:31:30.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.630+0000 7f6ac8894800 20 remove_watcher() i=4 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.630+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94272754846880] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96103840 con 0x7f6ac0016440 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.631+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94272754846880] v27'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.631+0000 7f6ac8894800 20 remove_watcher() i=5 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.631+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94272754831120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96103840 con 0x7f6ac0016440 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.632+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94272754831120] v27'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.632+0000 7f6ac8894800 20 remove_watcher() i=6 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.632+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94272754849568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55bd96103840 con 0x7f6ac0016440 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94272754849568] v27'209 uv207 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f6ac000a4a0 con 0x7f6ac0016440 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 20 remove_watcher() i=7 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6ac0016440 msgr2=0x7f6ac00168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f6ac0016440 0x7f6ac00168c0 crc :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bd960f9420 msgr2=0x55bd960fb8e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bd960f9420 0x55bd960fb8e0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6ab8026c20 msgr2=0x7f6ab80083b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f6ab8026c20 0x7f6ab80083b0 crc :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bd960fdba0 msgr2=0x55bd960fff90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bd960fdba0 0x55bd960fff90 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6990036950 msgr2=0x7f6990038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f6990036950 0x7f6990038e10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6abc052fd0 tx=0x7f6abc04f000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 msgr2=0x55bd960edeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac6b94640 1 -- 172.21.15.165:0/2753020110 reap_dead start 2024-04-27T03:31:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 0x55bd960edeb0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6ab804b8f0 tx=0x7f6ab804d120 comp rx=0 tx=0).stop 2024-04-27T03:31:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 shutdown_connections 2024-04-27T03:31:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 --2- 172.21.15.165:0/2753020110 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960e6b10 0x55bd960edeb0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 >> 172.21.15.165:0/2753020110 conn(0x55bd960e2e50 msgr2=0x55bd960e04b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 shutdown_connections 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.633+0000 7f6ac8894800 1 -- 172.21.15.165:0/2753020110 wait complete. 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3058331003 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f69a0036950 msgr2=0x7f69a0038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3058331003 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f69a0036950 0x7f69a0038e10 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x55bd960c2e40 tx=0x7f6abc005a00 comp rx=0 tx=0).stop 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3058331003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960d9830 msgr2=0x55bd960d9c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3058331003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960d9830 0x55bd960d9c50 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6ab804b480 tx=0x7f6ab800f840 comp rx=0 tx=0).stop 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3058331003 shutdown_connections 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3058331003 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f69a0036950 0x7f69a0038e10 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3058331003 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd960d9830 0x55bd960d9c50 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3058331003 >> 172.21.15.165:0/3058331003 conn(0x55bd960cd8d0 msgr2=0x55bd960cdcd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3058331003 shutdown_connections 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3058331003 wait complete. 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bd95fc4350 msgr2=0x55bd960c1fe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55bd95fc4350 0x55bd960c1fe0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55bd960c8820 msgr2=0x55bd960cac70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55bd960c8820 0x55bd960cac70 crc :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bd960c4890 msgr2=0x55bd960c6c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55bd960c4890 0x55bd960c6c80 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f69ac036950 msgr2=0x7f69ac038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f69ac036950 0x7f69ac038e10 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6abc054000 tx=0x7f6abc04e000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 msgr2=0x55bd960bada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 --2- 172.21.15.165:0/3289334280 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55bd95fbffb0 0x55bd960bada0 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x55bd95f34ff0 tx=0x7f6ab8000ed0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac6b94640 1 -- 172.21.15.165:0/3289334280 reap_dead start 2024-04-27T03:31:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 shutdown_connections 2024-04-27T03:31:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 >> 172.21.15.165:0/3289334280 conn(0x55bd95fbb550 msgr2=0x55bd95f34860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 shutdown_connections 2024-04-27T03:31:30.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.634+0000 7f6ac8894800 1 -- 172.21.15.165:0/3289334280 wait complete. 2024-04-27T03:31:30.651 INFO:teuthology.orchestra.run.smithi165.stdout:{"entries":[{"user":"foo","buckets":[{"bucket":"","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[]},{"bucket":"myfoo","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":2,"successful_ops":1}]},{"bucket":"myfoo2","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo3","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]},{"bucket":"myfoo4","time":"2024-04-27T03:00:00.000000Z","epoch":1714186800,"owner":"foo","categories":[{"category":"delete_bucket","bytes_sent":0,"bytes_received":0,"ops":1,"successful_ops":1}]}]}],"summary":[{"user":"foo","categories":[{"category":"delete_bucket","bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4}],"total":{"bytes_sent":236,"bytes_received":0,"ops":5,"successful_ops":4}}]} 2024-04-27T03:31:30.652 DEBUG:tasks.util.rgw: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': '', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}, {'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 2, 'successful_ops': 1}]}, {'bucket': 'myfoo2', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo3', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'myfoo4', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}], 'total': {'bytes_sent': 236, 'bytes_received': 0, 'ops': 5, 'successful_ops': 4}}]} 2024-04-27T03:31:30.652 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-04-27T03:31:30.652 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'bar', '--display-name', 'Bar', '--access-key', 'NX5QOQKC6BH2IDN8HC7A', '--secret', 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', '--max-buckets', '4'] 2024-04-27T03:31:30.652 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid bar --display-name Bar --access-key NX5QOQKC6BH2IDN8HC7A --secret LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn --max-buckets 4 2024-04-27T03:31:30.704 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:30.704 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:30.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.698+0000 7f4c2a698800 1 Processor -- start 2024-04-27T03:31:30.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.698+0000 7f4c2a698800 1 -- start start 2024-04-27T03:31:30.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.698+0000 7f4c2a698800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5603468bbd70 0x5603468bc190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.698+0000 7f4c2a698800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56034677bd40 con 0x5603468bcfd0 2024-04-27T03:31:30.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.698+0000 7f4c2a698800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x56034674e4d0 con 0x5603468bbd70 2024-04-27T03:31:30.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.698+0000 7f4c2819e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5603468bbd70 0x5603468bc190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.698+0000 7f4c1f99d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5603468bcfd0 0x5603468bd450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:54752/0 (socket says 172.21.15.165:54752) 2024-04-27T03:31:30.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.698+0000 7f4c1f99d640 1 -- 172.21.15.165:0/1565513636 learned_addr learned my addr 172.21.15.165:0/1565513636 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:31:30.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.698+0000 7f4c2819e640 1 -- 172.21.15.165:0/1565513636 >> v1:172.21.15.89:6789/0 conn(0x5603468bcfd0 legacy=0x5603468bd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.698+0000 7f4c2819e640 1 -- 172.21.15.165:0/1565513636 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56034677a590 con 0x5603468bbd70 2024-04-27T03:31:30.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.700+0000 7f4c2819e640 1 --2- 172.21.15.165:0/1565513636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x5603468bc190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800a470 tx=0x7f4c1801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=504d9625ea4431f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.700+0000 7f4c2799d640 1 -- 172.21.15.165:0/1565513636 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c18057070 con 0x5603468bbd70 2024-04-27T03:31:30.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.700+0000 7f4c2799d640 1 -- 172.21.15.165:0/1565513636 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c18052570 con 0x5603468bbd70 2024-04-27T03:31:30.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.700+0000 7f4c2799d640 1 -- 172.21.15.165:0/1565513636 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c18052870 con 0x5603468bbd70 2024-04-27T03:31:30.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.701+0000 7f4c2a698800 1 -- 172.21.15.165:0/1565513636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 msgr2=0x5603468bc190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.701+0000 7f4c2a698800 1 --2- 172.21.15.165:0/1565513636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x5603468bc190 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f4c1800a470 tx=0x7f4c1801ce90 comp rx=0 tx=0).stop 2024-04-27T03:31:30.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.701+0000 7f4c2a698800 1 -- 172.21.15.165:0/1565513636 shutdown_connections 2024-04-27T03:31:30.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.701+0000 7f4c2a698800 1 --2- 172.21.15.165:0/1565513636 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x5603468bc190 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.701+0000 7f4c2a698800 1 -- 172.21.15.165:0/1565513636 >> 172.21.15.165:0/1565513636 conn(0x5603468b73d0 msgr2=0x5603468b97f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.701+0000 7f4c2a698800 1 -- 172.21.15.165:0/1565513636 shutdown_connections 2024-04-27T03:31:30.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.701+0000 7f4c2a698800 1 -- 172.21.15.165:0/1565513636 wait complete. 2024-04-27T03:31:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.713+0000 7f4c2a698800 1 Processor -- start 2024-04-27T03:31:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.713+0000 7f4c2a698800 1 -- start start 2024-04-27T03:31:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.713+0000 7f4c2a698800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x560346830040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.713+0000 7f4c2a698800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5603468083e0 con 0x5603468bbd70 2024-04-27T03:31:30.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.713+0000 7f4c2819e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x560346830040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.713+0000 7f4c2819e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x560346830040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49594/0 (socket says 172.21.15.165:49594) 2024-04-27T03:31:30.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.713+0000 7f4c2819e640 1 -- 172.21.15.165:0/136756071 learned_addr learned my addr 172.21.15.165:0/136756071 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.713+0000 7f4c2819e640 1 -- 172.21.15.165:0/136756071 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603469b1390 con 0x5603468bbd70 2024-04-27T03:31:30.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.714+0000 7f4c2819e640 1 --2- 172.21.15.165:0/136756071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x560346830040 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4c18055040 tx=0x7f4c1804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8e359fe48fd60d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.714+0000 7f4b1660c640 1 -- 172.21.15.165:0/136756071 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c18057070 con 0x5603468bbd70 2024-04-27T03:31:30.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.714+0000 7f4b1660c640 1 -- 172.21.15.165:0/136756071 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c18063070 con 0x5603468bbd70 2024-04-27T03:31:30.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.714+0000 7f4b1660c640 1 -- 172.21.15.165:0/136756071 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c18052720 con 0x5603468bbd70 2024-04-27T03:31:30.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.714+0000 7f4c2a698800 1 -- 172.21.15.165:0/136756071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 msgr2=0x560346830040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.739 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.714+0000 7f4c2a698800 1 --2- 172.21.15.165:0/136756071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x560346830040 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f4c18055040 tx=0x7f4c1804c000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.714+0000 7f4c2a698800 1 -- 172.21.15.165:0/136756071 shutdown_connections 2024-04-27T03:31:30.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.714+0000 7f4c2a698800 1 --2- 172.21.15.165:0/136756071 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x560346830040 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.714+0000 7f4c2a698800 1 -- 172.21.15.165:0/136756071 >> 172.21.15.165:0/136756071 conn(0x5603468b73d0 msgr2=0x5603469a8e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.715+0000 7f4c2a698800 1 -- 172.21.15.165:0/136756071 shutdown_connections 2024-04-27T03:31:30.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.715+0000 7f4c2a698800 1 -- 172.21.15.165:0/136756071 wait complete. 2024-04-27T03:31:30.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.715+0000 7f4c2a698800 1 Processor -- start 2024-04-27T03:31:30.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.715+0000 7f4c2a698800 1 -- start start 2024-04-27T03:31:30.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.715+0000 7f4c2a698800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x5603469b6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.715+0000 7f4c2a698800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5603468c5440 con 0x5603468bbd70 2024-04-27T03:31:30.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.715+0000 7f4c2819e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x5603469b6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.715+0000 7f4c2819e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x5603469b6170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49602/0 (socket says 172.21.15.165:49602) 2024-04-27T03:31:30.740 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.715+0000 7f4c2819e640 1 -- 172.21.15.165:0/2721318449 learned_addr learned my addr 172.21.15.165:0/2721318449 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.716+0000 7f4c2819e640 1 -- 172.21.15.165:0/2721318449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603469b66b0 con 0x5603468bbd70 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.716+0000 7f4c2819e640 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x5603469b6170 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4c18060040 tx=0x7f4c18002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.716+0000 7f4b14e09640 1 -- 172.21.15.165:0/2721318449 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c18052c70 con 0x5603468bbd70 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.716+0000 7f4b14e09640 1 -- 172.21.15.165:0/2721318449 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c1806c070 con 0x5603468bbd70 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.716+0000 7f4b14e09640 1 -- 172.21.15.165:0/2721318449 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c1805e480 con 0x5603468bbd70 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.716+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5603469b6940 con 0x5603468bbd70 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.716+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603469b95f0 con 0x5603468bbd70 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.717+0000 7f4b14e09640 1 -- 172.21.15.165:0/2721318449 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f4c1805e620 con 0x5603468bbd70 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.717+0000 7f4b14e09640 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4b10036900 0x7f4b10038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.717+0000 7f4b14e09640 1 -- 172.21.15.165:0/2721318449 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f4c1808e5f0 con 0x5603468bbd70 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.717+0000 7f4c1f99d640 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4b10036900 0x7f4b10038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.717+0000 7f4c2a698800 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5603468bcfd0 0x5603469bd3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.717+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469bd980 con 0x5603468bcfd0 2024-04-27T03:31:30.741 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.717+0000 7f4c2899f640 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5603468bcfd0 0x5603469bd3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.717+0000 7f4c1f99d640 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4b10036900 0x7f4b10038dc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4c10000f60 tx=0x7f4c100012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.718+0000 7f4c2899f640 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5603468bcfd0 0x5603469bd3b0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.718+0000 7f4c2899f640 1 -- 172.21.15.165:0/2721318449 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603468bcfd0 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.718+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469ba980 con 0x5603468bcfd0 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.718+0000 7f4c2899f640 1 -- 172.21.15.165:0/2721318449 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603468bcfd0 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.718+0000 7f4c2a698800 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5603469bfcb0 0x5603469c20a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.718+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469c2790 con 0x5603469bfcb0 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.718+0000 7f4c2819e640 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5603469bfcb0 0x5603469c20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.718+0000 7f4c2819e640 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5603469bfcb0 0x5603469c20a0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.719+0000 7f4c2819e640 1 -- 172.21.15.165:0/2721318449 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469bfcb0 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.719+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469bec60 con 0x5603468bcfd0 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.719+0000 7f4c2899f640 1 -- 172.21.15.165:0/2721318449 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603468bcfd0 2024-04-27T03:31:30.742 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.719+0000 7f4c2a698800 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5603469c3c40 0x5603469c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.719+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469c67f0 con 0x5603469c3c40 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.719+0000 7f4c1f99d640 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5603469c3c40 0x5603469c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.719+0000 7f4c1f99d640 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5603469c3c40 0x5603469c6090 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.720+0000 7f4c1f99d640 1 -- 172.21.15.165:0/2721318449 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4c10009050 con 0x5603469c3c40 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.720+0000 7f4c2a698800 1 Processor -- start 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.720+0000 7f4c2a698800 1 -- start start 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.720+0000 7f4c2a698800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469d4be0 0x5603469d5000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.720+0000 7f4c2a698800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5603467ae6b0 con 0x5603469d4be0 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.721+0000 7f4c2819e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469d4be0 0x5603469d5000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.721+0000 7f4c2819e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469d4be0 0x5603469d5000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49604/0 (socket says 172.21.15.165:49604) 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.721+0000 7f4c2819e640 1 -- 172.21.15.165:0/2545019668 learned_addr learned my addr 172.21.15.165:0/2545019668 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.721+0000 7f4c2819e640 1 -- 172.21.15.165:0/2545019668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603469d5540 con 0x5603469d4be0 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.721+0000 7f4c2819e640 1 --2- 172.21.15.165:0/2545019668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469d4be0 0x5603469d5000 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4c18001eb0 tx=0x7f4c18067e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.721+0000 7f4b0dffb640 1 -- 172.21.15.165:0/2545019668 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c18081040 con 0x5603469d4be0 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.721+0000 7f4b0dffb640 1 -- 172.21.15.165:0/2545019668 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c18058040 con 0x5603469d4be0 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.721+0000 7f4c2a698800 1 -- 172.21.15.165:0/2545019668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5603469d5fb0 con 0x5603469d4be0 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.721+0000 7f4b0dffb640 1 -- 172.21.15.165:0/2545019668 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c18079850 con 0x5603469d4be0 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.721+0000 7f4c2a698800 1 -- 172.21.15.165:0/2545019668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603469d64b0 con 0x5603469d4be0 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.722+0000 7f4b0dffb640 1 -- 172.21.15.165:0/2545019668 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f4c18095060 con 0x5603469d4be0 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.722+0000 7f4b0dffb640 1 --2- 172.21.15.165:0/2545019668 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4b00036950 0x7f4b00038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.722+0000 7f4b0dffb640 1 -- 172.21.15.165:0/2545019668 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f4c1804b100 con 0x5603469d4be0 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.722+0000 7f4c1f99d640 1 --2- 172.21.15.165:0/2545019668 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4b00036950 0x7f4b00038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.722+0000 7f4c2a698800 1 Processor -- start 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.722+0000 7f4c2a698800 1 -- start start 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.723+0000 7f4c1f99d640 1 --2- 172.21.15.165:0/2545019668 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4b00036950 0x7f4b00038e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000bf50 tx=0x7f4c1000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.723+0000 7f4c2a698800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 0x5603469ad170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.723+0000 7f4c2a698800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5603469b1620 con 0x5603469aea40 2024-04-27T03:31:30.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.723+0000 7f4c2819e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 0x5603469ad170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.723+0000 7f4c2819e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 0x5603469ad170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49606/0 (socket says 172.21.15.165:49606) 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.723+0000 7f4c2819e640 1 -- 172.21.15.165:0/1451233533 learned_addr learned my addr 172.21.15.165:0/1451233533 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.723+0000 7f4c2819e640 1 -- 172.21.15.165:0/1451233533 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603469aeef0 con 0x5603469aea40 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.723+0000 7f4c2819e640 1 --2- 172.21.15.165:0/1451233533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 0x5603469ad170 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4c18074000 tx=0x7f4c180094d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28832e2456886f42 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.723+0000 7f4b077fe640 1 -- 172.21.15.165:0/1451233533 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c180a7070 con 0x5603469aea40 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.723+0000 7f4b077fe640 1 -- 172.21.15.165:0/1451233533 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c180799f0 con 0x5603469aea40 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.723+0000 7f4b077fe640 1 -- 172.21.15.165:0/1451233533 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c1807f100 con 0x5603469aea40 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.724+0000 7f4c2a698800 1 -- 172.21.15.165:0/1451233533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 msgr2=0x5603469ad170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.724+0000 7f4c2a698800 1 --2- 172.21.15.165:0/1451233533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 0x5603469ad170 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f4c18074000 tx=0x7f4c180094d0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.724+0000 7f4c2a698800 1 -- 172.21.15.165:0/1451233533 shutdown_connections 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.724+0000 7f4c2a698800 1 --2- 172.21.15.165:0/1451233533 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 0x5603469ad170 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.724+0000 7f4c2a698800 1 -- 172.21.15.165:0/1451233533 >> 172.21.15.165:0/1451233533 conn(0x5603469de060 msgr2=0x5603469dc0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.724+0000 7f4c2a698800 1 -- 172.21.15.165:0/1451233533 shutdown_connections 2024-04-27T03:31:30.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.724+0000 7f4c2a698800 1 -- 172.21.15.165:0/1451233533 wait complete. 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.724+0000 7f4c2a698800 1 Processor -- start 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.724+0000 7f4c2a698800 1 -- start start 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.724+0000 7f4c2a698800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 0x5603469eb300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.724+0000 7f4c2a698800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5603469ad6b0 con 0x5603469aea40 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.725+0000 7f4c2819e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 0x5603469eb300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.725+0000 7f4c2819e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 0x5603469eb300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49610/0 (socket says 172.21.15.165:49610) 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.725+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 learned_addr learned my addr 172.21.15.165:0/3231264483 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.725+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603469eb840 con 0x5603469aea40 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.725+0000 7f4c2819e640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 0x5603469eb300 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4c18079bc0 tx=0x7f4c1807ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.725+0000 7f4b05ffb640 1 -- 172.21.15.165:0/3231264483 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c180b2020 con 0x5603469aea40 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.725+0000 7f4b05ffb640 1 -- 172.21.15.165:0/3231264483 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c180095f0 con 0x5603469aea40 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.725+0000 7f4b05ffb640 1 -- 172.21.15.165:0/3231264483 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c180a7030 con 0x5603469aea40 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.725+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5603469eb840 con 0x5603469aea40 2024-04-27T03:31:30.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.725+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603469eb840 con 0x5603469aea40 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.726+0000 7f4b05ffb640 1 -- 172.21.15.165:0/3231264483 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f4c180b5020 con 0x5603469aea40 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.726+0000 7f4b05ffb640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4af8036950 0x7f4af8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.726+0000 7f4b05ffb640 1 -- 172.21.15.165:0/3231264483 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f4c180a8070 con 0x5603469aea40 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.726+0000 7f4c1f99d640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4af8036950 0x7f4af8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c1f99d640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4af8036950 0x7f4af8038e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c10001b60 tx=0x7f4c10010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2a698800 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5603469f6a10 0x5603469f8ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469f9620 con 0x5603469f6a10 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2899f640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5603469f6a10 0x5603469f8ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2899f640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5603469f6a10 0x5603469f8ed0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2a698800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2a698800 20 realm 2024-04-27T03:31:30.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469f9a30 con 0x5603469f6a10 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2a698800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.727+0000 7f4c2a698800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa300 con 0x5603469f6a10 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2a698800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469f6a10 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2a698800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2a698800 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5603469fb400 0x5603469fd7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469fde00 con 0x5603469fb400 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2819e640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5603469fb400 0x5603469fd7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.728+0000 7f4c2819e640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5603469fb400 0x5603469fd7f0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.729+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.729+0000 7f4c2a698800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.729+0000 7f4c2a698800 20 searching for the correct realm 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.729+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469ff730 con 0x5603469fb400 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.729+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.729+0000 7f4c2819e640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f4c180c2040 0x7f4c1807d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.729+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469ff730 con 0x7f4c180c2040 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.729+0000 7f4c1f99d640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f4c180c2040 0x7f4c1807d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.730+0000 7f4c1f99d640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f4c180c2040 0x7f4c1807d700 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.730+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.730+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469ff730 con 0x7f4c180c2040 2024-04-27T03:31:30.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.730+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.730+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469ff730 con 0x5603469f6a10 2024-04-27T03:31:30.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.730+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.730+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469f6a10 2024-04-27T03:31:30.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.730+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.731+0000 7f4c2899f640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4c20016440 0x7f4c200168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.731+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c20016440 2024-04-27T03:31:30.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.731+0000 7f4c2899f640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4c20016440 0x7f4c200168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.731+0000 7f4c2899f640 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4c20016440 0x7f4c200168c0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.732+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.732+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c20016440 2024-04-27T03:31:30.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.732+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.732+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c180c2040 2024-04-27T03:31:30.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.732+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.732+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469fb400 2024-04-27T03:31:30.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.732+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.732+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469f6a10 2024-04-27T03:31:30.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.733+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.733+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c180c2040 2024-04-27T03:31:30.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.733+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.733+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469f6a10 2024-04-27T03:31:30.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.733+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.733+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c180c2040 2024-04-27T03:31:30.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.733+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.733+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469fb400 2024-04-27T03:31:30.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.733+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.734+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469fb400 2024-04-27T03:31:30.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.734+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.734+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c20016440 2024-04-27T03:31:30.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.734+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.734+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469fb400 2024-04-27T03:31:30.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.735+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.735+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469fb400 2024-04-27T03:31:30.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.735+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.767 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.735+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469f6a10 2024-04-27T03:31:30.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.735+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.735+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469f6a10 2024-04-27T03:31:30.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.735+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.735+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c20016440 2024-04-27T03:31:30.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.735+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.735+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469fb400 2024-04-27T03:31:30.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.736+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.736+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c180c2040 2024-04-27T03:31:30.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.736+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.736+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469f6a10 2024-04-27T03:31:30.768 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.736+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.736+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c20016440 2024-04-27T03:31:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.737+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.737+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469fb400 2024-04-27T03:31:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.737+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.737+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c20016440 2024-04-27T03:31:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.738+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.738+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c180c2040 2024-04-27T03:31:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.738+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.738+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469fb400 2024-04-27T03:31:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.738+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.738+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c20016440 2024-04-27T03:31:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.738+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.738+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x5603469f6a10 2024-04-27T03:31:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5603469fa6a0 con 0x7f4c180c2040 2024-04-27T03:31:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:30.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469fe6c0 con 0x5603469f6a10 2024-04-27T03:31:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469feab0 con 0x5603469f6a10 2024-04-27T03:31:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.739+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469feab0 con 0x7f4c180c2040 2024-04-27T03:31:30.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2a698800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2a698800 20 zone default found 2024-04-27T03:31:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2a698800 4 Realm: () 2024-04-27T03:31:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2a698800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2a698800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2a698800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2a698800 20 zonegroup default 2024-04-27T03:31:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5603469ffbb0 con 0x7f4c20016440 2024-04-27T03:31:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2a698800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.740+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560346a015f0 con 0x5603469fb400 2024-04-27T03:31:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.741+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.741+0000 7f4c2a698800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.741+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.741+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560346a01aa0 con 0x5603469fb400 2024-04-27T03:31:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.741+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.741+0000 7f4c2a698800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.741+0000 7f4c2a698800 20 started sync module instance, tier type = 2024-04-27T03:31:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.741+0000 7f4c2a698800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:30.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.741+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a00a30 con 0x7f4c180c2040 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.742+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.742+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94572069785120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a023b0 con 0x7f4c180c2040 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.742+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a02c00 con 0x5603469f6a10 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.743+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94572069785120] v27'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.743+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.743+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94572069789440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a02c00 con 0x5603469f6a10 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.743+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a03700 con 0x5603469fb400 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.744+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.744+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94572069794288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a03fd0 con 0x5603469fb400 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.744+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a04a80 con 0x7f4c180c2040 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.744+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94572069789440] v27'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.745+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94572069794288] v27'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.745+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.745+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94572069791568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a04ea0 con 0x7f4c180c2040 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.745+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a056f0 con 0x5603469f6a10 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.746+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94572069791568] v27'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.747+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.747+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94572069802560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a05660 con 0x5603469f6a10 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.747+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a065b0 con 0x7f4c20016440 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.748+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94572069802560] v27'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.748+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.748+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94572069804624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a065b0 con 0x7f4c20016440 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.748+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a06f70 con 0x7f4c20016440 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.749+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94572069804624] v27'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.749+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.749+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94572069806816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a06f70 con 0x7f4c20016440 2024-04-27T03:31:30.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.749+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a07930 con 0x7f4c20016440 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.750+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(58 notify.7 [create] v27'210 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.750+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94572069806816] v27'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.750+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94572069811328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a07d50 con 0x7f4c20016440 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.750+0000 7f4c2a698800 20 add_watcher() i=0 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.750+0000 7f4c2a698800 20 add_watcher() i=1 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.750+0000 7f4c2a698800 20 add_watcher() i=2 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.750+0000 7f4c2a698800 20 add_watcher() i=3 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.750+0000 7f4c2a698800 20 add_watcher() i=4 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.750+0000 7f4c2a698800 20 add_watcher() i=5 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.750+0000 7f4c2a698800 20 add_watcher() i=6 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.751+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94572069811328] v27'211 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.751+0000 7f4c2a698800 20 add_watcher() i=7 2024-04-27T03:31:30.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.751+0000 7f4c2a698800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.751+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560346a07d50 con 0x5603469f6a10 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.751+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.751+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94572069812384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a07cc0 con 0x5603469f6a10 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.752+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94572069812384] v27'216 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.752+0000 7f4ade7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.752+0000 7f4c2a698800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.752+0000 7f4c2a698800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.752+0000 7f4c2a698800 5 note: GC not initialized 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.752+0000 7f4add7e2640 20 reqs_thread_entry: start 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 20 Filter name: none 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4a4effd640 20 reqs_thread_entry: start 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-04-27T03:31:30.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560346a3e080 con 0x7f4c20016440 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 bar [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x0 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : miss 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560346a3c820 con 0x5603469f6a10 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(63 NX5QOQKC6BH2IDN8HC7A [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x0 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.753+0000 7f4c2a698800 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.754+0000 7f4c2a698800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-04-27T03:31:30.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.754+0000 7f4c2a698800 10 cache get: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A : hit (negative entry) 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.754+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:64 8.e 8:718f6a54:users.uid::bar:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a3d0f0 con 0x7f4c20016440 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.755+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(64 bar [delete,create,call,writefull 0~380] v27'3 uv3 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.755+0000 7f4c2a698800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.755+0000 7f4c2a698800 10 moving default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.755+0000 7f4c2a698800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 0, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.755+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:65 7.9 7:93e5b521:::notify.7:head [notify cookie 94572070030400 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560346a41b20 con 0x7f4c20016440 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.755+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== watch-notify(notify (1) cookie 94572069811328 notify 115964117006 ret 0) ==== 598+0+0 (crc 0 0 0) 0x560346a3d490 con 0x7f4c20016440 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.755+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(65 notify.7 [notify cookie 94572070030400 out=8b] v0'0 uv210 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.755+0000 7f4b077fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117006 cookie 94572069811328 notifier 4714 bl.length()=556 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.755+0000 7f4b077fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bar info.flags=0x17 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.755+0000 7f4b077fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.755+0000 7f4b077fe640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:66 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f4aec003ea0 con 0x7f4c20016440 2024-04-27T03:31:30.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.756+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(66 notify.7 [notify-ack] v0'0 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.756+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== watch-notify(notify_complete (2) cookie 94572070030400 notify 115964117006 ret 0) ==== 42+0+48 (crc 0 0 0) 0x560346a3dad0 con 0x7f4c20016440 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.756+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a3cbf0 con 0x5603469f6a10 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(67 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v27'3 uv3 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4c2a698800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4c2a698800 10 moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4c2a698800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 94572070030320 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560346a41b20 con 0x7f4c20016440 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== watch-notify(notify (1) cookie 94572069804624 notify 115964117007 ret 0) ==== 219+0+0 (crc 0 0 0) 0x560346a3dad0 con 0x7f4c20016440 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4b077fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117007 cookie 94572069804624 notifier 4714 bl.length()=177 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4b077fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4b077fe640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(68 notify.5 [notify cookie 94572070030320 out=8b] v0'0 uv210 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4b077fe640 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f4aec003ea0 con 0x7f4c20016440 2024-04-27T03:31:30.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.757+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== watch-notify(notify_complete (2) cookie 94572070030320 notify 115964117007 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f4c20030040 con 0x7f4c20016440 2024-04-27T03:31:30.781 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"bar","display_name":"Bar","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"bar","access_key":"NX5QOQKC6BH2IDN8HC7A","secret_key":"LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn","active":true,"create_date":"2024-04-27T03:31:30.754980Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:31:30.754965Z","tags":[],"group_ids":[]} 2024-04-27T03:31:30.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.758+0000 7f4ade7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.758+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94572069812384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5603469f3130 con 0x5603469f6a10 2024-04-27T03:31:30.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.759+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94572069812384] v27'217 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.759+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94572069785120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a078a0 con 0x7f4c180c2040 2024-04-27T03:31:30.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.760+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94572069785120] v27'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.760+0000 7f4c2a698800 20 remove_watcher() i=0 2024-04-27T03:31:30.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.760+0000 7f4c2a698800 2 removed watcher, disabling cache 2024-04-27T03:31:30.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.760+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94572069789440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a078a0 con 0x5603469f6a10 2024-04-27T03:31:30.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.761+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94572069789440] v27'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.761+0000 7f4c2a698800 20 remove_watcher() i=1 2024-04-27T03:31:30.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.761+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94572069794288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a01f00 con 0x5603469fb400 2024-04-27T03:31:30.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.762+0000 7f4c2819e640 1 -- 172.21.15.165:0/3231264483 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94572069794288] v27'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c18009d10 con 0x5603469fb400 2024-04-27T03:31:30.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.762+0000 7f4c2a698800 20 remove_watcher() i=2 2024-04-27T03:31:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.762+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94572069791568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a078a0 con 0x7f4c180c2040 2024-04-27T03:31:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.763+0000 7f4c1f99d640 1 -- 172.21.15.165:0/3231264483 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94572069791568] v27'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c10009050 con 0x7f4c180c2040 2024-04-27T03:31:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.763+0000 7f4c2a698800 20 remove_watcher() i=3 2024-04-27T03:31:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.763+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94572069802560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a01f00 con 0x5603469f6a10 2024-04-27T03:31:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.764+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94572069802560] v27'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x5603469f6a10 2024-04-27T03:31:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.764+0000 7f4c2a698800 20 remove_watcher() i=4 2024-04-27T03:31:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.764+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94572069804624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a02750 con 0x7f4c20016440 2024-04-27T03:31:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.765+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94572069804624] v27'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.765+0000 7f4c2a698800 20 remove_watcher() i=5 2024-04-27T03:31:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.765+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94572069806816] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a078a0 con 0x7f4c20016440 2024-04-27T03:31:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.766+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94572069806816] v27'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.766+0000 7f4c2a698800 20 remove_watcher() i=6 2024-04-27T03:31:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.766+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94572069811328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560346a05240 con 0x7f4c20016440 2024-04-27T03:31:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.767+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94572069811328] v27'212 uv210 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4c2000a4a0 con 0x7f4c20016440 2024-04-27T03:31:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.767+0000 7f4c2a698800 20 remove_watcher() i=7 2024-04-27T03:31:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.767+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4c20016440 msgr2=0x7f4c200168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.767+0000 7f4c2a698800 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4c20016440 0x7f4c200168c0 crc :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.767+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5603469f6a10 msgr2=0x5603469f8ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.767+0000 7f4c2a698800 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5603469f6a10 0x5603469f8ed0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.767+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f4c180c2040 msgr2=0x7f4c1807d700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.767+0000 7f4c2a698800 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f4c180c2040 0x7f4c1807d700 crc :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.767+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5603469fb400 msgr2=0x5603469fd7f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.767+0000 7f4c2a698800 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5603469fb400 0x5603469fd7f0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4af8036950 msgr2=0x7f4af8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4af8036950 0x7f4af8038e10 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c10001b60 tx=0x7f4c10010000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 msgr2=0x5603469eb300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2899f640 1 -- 172.21.15.165:0/3231264483 reap_dead start 2024-04-27T03:31:30.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 0x5603469eb300 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f4c18079bc0 tx=0x7f4c1807ff80 comp rx=0 tx=0).stop 2024-04-27T03:31:30.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 shutdown_connections 2024-04-27T03:31:30.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 --2- 172.21.15.165:0/3231264483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469aea40 0x5603469eb300 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 >> 172.21.15.165:0/3231264483 conn(0x5603469de060 msgr2=0x5603469db6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 shutdown_connections 2024-04-27T03:31:30.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/3231264483 wait complete. 2024-04-27T03:31:30.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/2545019668 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4b00036950 msgr2=0x7f4b00038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 --2- 172.21.15.165:0/2545019668 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4b00036950 0x7f4b00038e10 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4c1000bf50 tx=0x7f4c1000f000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/2545019668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469d4be0 msgr2=0x5603469d5000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 --2- 172.21.15.165:0/2545019668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469d4be0 0x5603469d5000 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f4c18001eb0 tx=0x7f4c18067e60 comp rx=0 tx=0).stop 2024-04-27T03:31:30.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/2545019668 shutdown_connections 2024-04-27T03:31:30.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 --2- 172.21.15.165:0/2545019668 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4b00036950 0x7f4b00038e10 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 --2- 172.21.15.165:0/2545019668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603469d4be0 0x5603469d5000 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/2545019668 >> 172.21.15.165:0/2545019668 conn(0x5603469c8c80 msgr2=0x5603469c9080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/2545019668 shutdown_connections 2024-04-27T03:31:30.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/2545019668 wait complete. 2024-04-27T03:31:30.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5603468bcfd0 msgr2=0x5603469bd3b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5603468bcfd0 0x5603469bd3b0 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5603469c3c40 msgr2=0x5603469c6090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5603469c3c40 0x5603469c6090 crc :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5603469bfcb0 msgr2=0x5603469c20a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.768+0000 7f4c2a698800 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5603469bfcb0 0x5603469c20a0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.769+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4b10036900 msgr2=0x7f4b10038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.769+0000 7f4c2a698800 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4b10036900 0x7f4b10038dc0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f4c10000f60 tx=0x7f4c100012b0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.769+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 msgr2=0x5603469b6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.769+0000 7f4c2a698800 1 --2- 172.21.15.165:0/2721318449 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5603468bbd70 0x5603469b6170 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f4c18060040 tx=0x7f4c18002fb0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.769+0000 7f4c2899f640 1 -- 172.21.15.165:0/2721318449 reap_dead start 2024-04-27T03:31:30.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.769+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 shutdown_connections 2024-04-27T03:31:30.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.769+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 >> 172.21.15.165:0/2721318449 conn(0x5603468b73d0 msgr2=0x5603469a7700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.769+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 shutdown_connections 2024-04-27T03:31:30.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.769+0000 7f4c2a698800 1 -- 172.21.15.165:0/2721318449 wait complete. 2024-04-27T03:31:30.794 DEBUG:tasks.util.rgw: json result: {'user_id': 'bar', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bar', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', 'active': True, 'create_date': '2024-04-27T03:31:30.754980Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:31:30.754965Z', 'tags': [], 'group_ids': []} 2024-04-27T03:31:30.808 INFO:tasks.radosgw_admin:RL: create_bucket myfoo6 bar bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:31:30.813 INFO:tasks.radosgw_admin:RL: put_obj myfoo6 bar bytes_out=8 bytes_in=0 failed=False 2024-04-27T03:31:30.813 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-04-27T03:31:30.813 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bar', '--new-uid', 'bud'] 2024-04-27T03:31:30.813 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bar --new-uid bud 2024-04-27T03:31:30.859 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:30.859 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:30.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.854+0000 7fede4f36800 1 Processor -- start 2024-04-27T03:31:30.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.854+0000 7fede4f36800 1 -- start start 2024-04-27T03:31:30.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.854+0000 7fede4f36800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x56534a6a7d40 0x56534a6a8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.854+0000 7fede4f36800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x56534a567d40 con 0x56534a6a8fa0 2024-04-27T03:31:30.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.854+0000 7fede4f36800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x56534a53a4d0 con 0x56534a6a7d40 2024-04-27T03:31:30.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.854+0000 7fede0ccc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x56534a6a7d40 0x56534a6a8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.854+0000 7fede04cb640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x56534a6a8fa0 0x56534a6a9420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:54760/0 (socket says 172.21.15.165:54760) 2024-04-27T03:31:30.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.854+0000 7fede0ccc640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x56534a6a7d40 0x56534a6a8160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49614/0 (socket says 172.21.15.165:49614) 2024-04-27T03:31:30.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.854+0000 7fede04cb640 1 -- 172.21.15.165:0/743152054 learned_addr learned my addr 172.21.15.165:0/743152054 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:31:30.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.855+0000 7fede0ccc640 1 -- 172.21.15.165:0/743152054 >> v1:172.21.15.89:6789/0 conn(0x56534a6a8fa0 legacy=0x56534a6a9420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.862 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.855+0000 7fede0ccc640 1 -- 172.21.15.165:0/743152054 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534a566590 con 0x56534a6a7d40 2024-04-27T03:31:30.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.856+0000 7fede0ccc640 1 --2- 172.21.15.165:0/743152054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a6a8160 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fedd800a470 tx=0x7fedd801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c67902074b212bf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.857+0000 7feddfcca640 1 -- 172.21.15.165:0/743152054 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedd8057070 con 0x56534a6a7d40 2024-04-27T03:31:30.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.857+0000 7feddfcca640 1 -- 172.21.15.165:0/743152054 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd8052570 con 0x56534a6a7d40 2024-04-27T03:31:30.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.857+0000 7feddfcca640 1 -- 172.21.15.165:0/743152054 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedd8052870 con 0x56534a6a7d40 2024-04-27T03:31:30.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.857+0000 7fede4f36800 1 -- 172.21.15.165:0/743152054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 msgr2=0x56534a6a8160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.857+0000 7fede4f36800 1 --2- 172.21.15.165:0/743152054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a6a8160 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7fedd800a470 tx=0x7fedd801ce90 comp rx=0 tx=0).stop 2024-04-27T03:31:30.863 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.857+0000 7fede4f36800 1 -- 172.21.15.165:0/743152054 shutdown_connections 2024-04-27T03:31:30.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.857+0000 7fede4f36800 1 --2- 172.21.15.165:0/743152054 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a6a8160 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.857+0000 7fede4f36800 1 -- 172.21.15.165:0/743152054 >> 172.21.15.165:0/743152054 conn(0x56534a6a33a0 msgr2=0x56534a6a57c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.857+0000 7fede4f36800 1 -- 172.21.15.165:0/743152054 shutdown_connections 2024-04-27T03:31:30.864 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.857+0000 7fede4f36800 1 -- 172.21.15.165:0/743152054 wait complete. 2024-04-27T03:31:30.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.869+0000 7fede4f36800 1 Processor -- start 2024-04-27T03:31:30.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.869+0000 7fede4f36800 1 -- start start 2024-04-27T03:31:30.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.869+0000 7fede4f36800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a79c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.869+0000 7fede4f36800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56534a5f43b0 con 0x56534a6a7d40 2024-04-27T03:31:30.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.869+0000 7fede0ccc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a79c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.869+0000 7fede0ccc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a79c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49618/0 (socket says 172.21.15.165:49618) 2024-04-27T03:31:30.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.869+0000 7fede0ccc640 1 -- 172.21.15.165:0/3584018781 learned_addr learned my addr 172.21.15.165:0/3584018781 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.870+0000 7fede0ccc640 1 -- 172.21.15.165:0/3584018781 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534a79cd50 con 0x56534a6a7d40 2024-04-27T03:31:30.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.870+0000 7fede0ccc640 1 --2- 172.21.15.165:0/3584018781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a79c780 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fedd8055040 tx=0x7fedd804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1b5ba57dce5a8ea server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.870+0000 7feccee0d640 1 -- 172.21.15.165:0/3584018781 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedd8057070 con 0x56534a6a7d40 2024-04-27T03:31:30.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.870+0000 7feccee0d640 1 -- 172.21.15.165:0/3584018781 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd8063070 con 0x56534a6a7d40 2024-04-27T03:31:30.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.870+0000 7feccee0d640 1 -- 172.21.15.165:0/3584018781 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedd8052720 con 0x56534a6a7d40 2024-04-27T03:31:30.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.871+0000 7fede4f36800 1 -- 172.21.15.165:0/3584018781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 msgr2=0x56534a79c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.871+0000 7fede4f36800 1 --2- 172.21.15.165:0/3584018781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a79c780 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7fedd8055040 tx=0x7fedd804c000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.871+0000 7fede4f36800 1 -- 172.21.15.165:0/3584018781 shutdown_connections 2024-04-27T03:31:30.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.871+0000 7fede4f36800 1 --2- 172.21.15.165:0/3584018781 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a79c780 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.871+0000 7fede4f36800 1 -- 172.21.15.165:0/3584018781 >> 172.21.15.165:0/3584018781 conn(0x56534a6a33a0 msgr2=0x56534a793c90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.871+0000 7fede4f36800 1 -- 172.21.15.165:0/3584018781 shutdown_connections 2024-04-27T03:31:30.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.871+0000 7fede4f36800 1 -- 172.21.15.165:0/3584018781 wait complete. 2024-04-27T03:31:30.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.871+0000 7fede4f36800 1 Processor -- start 2024-04-27T03:31:30.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.871+0000 7fede4f36800 1 -- start start 2024-04-27T03:31:30.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.871+0000 7fede4f36800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a7a42c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.871+0000 7fede4f36800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56534a61b710 con 0x56534a6a7d40 2024-04-27T03:31:30.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.872+0000 7fede0ccc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a7a42c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.872+0000 7fede0ccc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a7a42c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49632/0 (socket says 172.21.15.165:49632) 2024-04-27T03:31:30.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.872+0000 7fede0ccc640 1 -- 172.21.15.165:0/3754089057 learned_addr learned my addr 172.21.15.165:0/3754089057 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.872+0000 7fede0ccc640 1 -- 172.21.15.165:0/3754089057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534a7a4800 con 0x56534a6a7d40 2024-04-27T03:31:30.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.872+0000 7fede0ccc640 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a7a42c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fedd8060040 tx=0x7fedd8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.872+0000 7feccd60a640 1 -- 172.21.15.165:0/3754089057 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedd8052c70 con 0x56534a6a7d40 2024-04-27T03:31:30.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.872+0000 7feccd60a640 1 -- 172.21.15.165:0/3754089057 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd806c070 con 0x56534a6a7d40 2024-04-27T03:31:30.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.872+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56534a7a4a90 con 0x56534a6a7d40 2024-04-27T03:31:30.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.872+0000 7feccd60a640 1 -- 172.21.15.165:0/3754089057 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedd805e480 con 0x56534a6a7d40 2024-04-27T03:31:30.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.872+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534a7a76f0 con 0x56534a6a7d40 2024-04-27T03:31:30.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.873+0000 7feccd60a640 1 -- 172.21.15.165:0/3754089057 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fedd805e620 con 0x56534a6a7d40 2024-04-27T03:31:30.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.873+0000 7feccd60a640 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecc80369a0 0x7fecc8038e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.873+0000 7feccd60a640 1 -- 172.21.15.165:0/3754089057 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fedd808e660 con 0x56534a6a7d40 2024-04-27T03:31:30.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.873+0000 7fede04cb640 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecc80369a0 0x7fecc8038e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.873+0000 7fede4f36800 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56534a6a8fa0 0x56534a7ab4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.873+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7aba40 con 0x56534a6a8fa0 2024-04-27T03:31:30.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.873+0000 7fede14cd640 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56534a6a8fa0 0x56534a7ab4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.874+0000 7fede04cb640 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecc80369a0 0x7fecc8038e60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fedd4000f60 tx=0x7fedd40012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.874+0000 7fede14cd640 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56534a6a8fa0 0x56534a7ab4b0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.874+0000 7fede14cd640 1 -- 172.21.15.165:0/3754089057 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a6a8fa0 2024-04-27T03:31:30.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.874+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7a8a80 con 0x56534a6a8fa0 2024-04-27T03:31:30.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.874+0000 7fede14cd640 1 -- 172.21.15.165:0/3754089057 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a6a8fa0 2024-04-27T03:31:30.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.874+0000 7fede4f36800 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56534a7add50 0x56534a7b0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.874+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7b0830 con 0x56534a7add50 2024-04-27T03:31:30.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.874+0000 7fede0ccc640 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56534a7add50 0x56534a7b0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.875+0000 7fede0ccc640 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56534a7add50 0x56534a7b0140 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.875+0000 7fede0ccc640 1 -- 172.21.15.165:0/3754089057 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fedd8009d10 con 0x56534a7add50 2024-04-27T03:31:30.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.875+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7acd00 con 0x56534a6a8fa0 2024-04-27T03:31:30.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.875+0000 7fede14cd640 1 -- 172.21.15.165:0/3754089057 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a6a8fa0 2024-04-27T03:31:30.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.875+0000 7fede4f36800 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56534a7b1ce0 0x56534a7b4130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.875+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7b4890 con 0x56534a7b1ce0 2024-04-27T03:31:30.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.875+0000 7fede04cb640 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56534a7b1ce0 0x56534a7b4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.876+0000 7fede04cb640 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56534a7b1ce0 0x56534a7b4130 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.876+0000 7fede04cb640 1 -- 172.21.15.165:0/3754089057 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fedd4009050 con 0x56534a7b1ce0 2024-04-27T03:31:30.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.876+0000 7fede4f36800 1 Processor -- start 2024-04-27T03:31:30.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.876+0000 7fede4f36800 1 -- start start 2024-04-27T03:31:30.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.876+0000 7fede4f36800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7c2c80 0x56534a7c30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.876+0000 7fede4f36800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56534a59a6b0 con 0x56534a7c2c80 2024-04-27T03:31:30.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.877+0000 7fede0ccc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7c2c80 0x56534a7c30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.877+0000 7fede0ccc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7c2c80 0x56534a7c30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49638/0 (socket says 172.21.15.165:49638) 2024-04-27T03:31:30.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.877+0000 7fede0ccc640 1 -- 172.21.15.165:0/1230008809 learned_addr learned my addr 172.21.15.165:0/1230008809 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.877+0000 7fede0ccc640 1 -- 172.21.15.165:0/1230008809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534a7c35e0 con 0x56534a7c2c80 2024-04-27T03:31:30.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.877+0000 7fede0ccc640 1 --2- 172.21.15.165:0/1230008809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7c2c80 0x56534a7c30a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fedd8052520 tx=0x7fedd808ce80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.877+0000 7fecc67fc640 1 -- 172.21.15.165:0/1230008809 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedd8081040 con 0x56534a7c2c80 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.877+0000 7fecc67fc640 1 -- 172.21.15.165:0/1230008809 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd8058040 con 0x56534a7c2c80 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.877+0000 7fecc67fc640 1 -- 172.21.15.165:0/1230008809 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedd8079890 con 0x56534a7c2c80 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.877+0000 7fede4f36800 1 -- 172.21.15.165:0/1230008809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56534a7c4050 con 0x56534a7c2c80 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.877+0000 7fede4f36800 1 -- 172.21.15.165:0/1230008809 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534a7c4550 con 0x56534a7c2c80 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.878+0000 7fecc67fc640 1 -- 172.21.15.165:0/1230008809 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fedd8095060 con 0x56534a7c2c80 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.878+0000 7fecc67fc640 1 --2- 172.21.15.165:0/1230008809 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecb8036950 0x7fecb8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.878+0000 7fecc67fc640 1 -- 172.21.15.165:0/1230008809 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fedd804c0e0 con 0x56534a7c2c80 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.878+0000 7fede04cb640 1 --2- 172.21.15.165:0/1230008809 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecb8036950 0x7fecb8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.878+0000 7fede4f36800 1 Processor -- start 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.879+0000 7fede4f36800 1 -- start start 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.879+0000 7fede04cb640 1 --2- 172.21.15.165:0/1230008809 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecb8036950 0x7fecb8038e10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fedd400bf50 tx=0x7fedd400f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.879+0000 7fede4f36800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 0x56534a7d01c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.879+0000 7fede4f36800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56534a79d010 con 0x56534a7cfda0 2024-04-27T03:31:30.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.879+0000 7fede0ccc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 0x56534a7d01c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.879+0000 7fede0ccc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 0x56534a7d01c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49640/0 (socket says 172.21.15.165:49640) 2024-04-27T03:31:30.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.879+0000 7fede0ccc640 1 -- 172.21.15.165:0/1822568468 learned_addr learned my addr 172.21.15.165:0/1822568468 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.879+0000 7fede0ccc640 1 -- 172.21.15.165:0/1822568468 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534a7d0790 con 0x56534a7cfda0 2024-04-27T03:31:30.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.879+0000 7fede0ccc640 1 --2- 172.21.15.165:0/1822568468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 0x56534a7d01c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fedd808b880 tx=0x7fedd8097510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e2a070639daf15 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.879+0000 7fecb7fff640 1 -- 172.21.15.165:0/1822568468 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedd80a7070 con 0x56534a7cfda0 2024-04-27T03:31:30.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.879+0000 7fecb7fff640 1 -- 172.21.15.165:0/1822568468 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd806bad0 con 0x56534a7cfda0 2024-04-27T03:31:30.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.879+0000 7fecb7fff640 1 -- 172.21.15.165:0/1822568468 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedd805abf0 con 0x56534a7cfda0 2024-04-27T03:31:30.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.880+0000 7fede4f36800 1 -- 172.21.15.165:0/1822568468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 msgr2=0x56534a7d01c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.880+0000 7fede4f36800 1 --2- 172.21.15.165:0/1822568468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 0x56534a7d01c0 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7fedd808b880 tx=0x7fedd8097510 comp rx=0 tx=0).stop 2024-04-27T03:31:30.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.880+0000 7fede4f36800 1 -- 172.21.15.165:0/1822568468 shutdown_connections 2024-04-27T03:31:30.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.880+0000 7fede4f36800 1 --2- 172.21.15.165:0/1822568468 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 0x56534a7d01c0 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.880+0000 7fede4f36800 1 -- 172.21.15.165:0/1822568468 >> 172.21.15.165:0/1822568468 conn(0x56534a7cc070 msgr2=0x56534a7ca0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.880+0000 7fede4f36800 1 -- 172.21.15.165:0/1822568468 shutdown_connections 2024-04-27T03:31:30.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.880+0000 7fede4f36800 1 -- 172.21.15.165:0/1822568468 wait complete. 2024-04-27T03:31:30.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.880+0000 7fede4f36800 1 Processor -- start 2024-04-27T03:31:30.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.880+0000 7fede4f36800 1 -- start start 2024-04-27T03:31:30.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.880+0000 7fede4f36800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 0x56534a7d70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.880+0000 7fede4f36800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56534a7d0df0 con 0x56534a7cfda0 2024-04-27T03:31:30.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.881+0000 7fede0ccc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 0x56534a7d70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.881+0000 7fede0ccc640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 0x56534a7d70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:49644/0 (socket says 172.21.15.165:49644) 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.881+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 learned_addr learned my addr 172.21.15.165:0/1996289815 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.881+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56534a7d7600 con 0x56534a7cfda0 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.881+0000 7fede0ccc640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 0x56534a7d70c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fedd8073d50 tx=0x7fedd806eed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.881+0000 7fecb67fc640 1 -- 172.21.15.165:0/1996289815 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedd80b1030 con 0x56534a7cfda0 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.881+0000 7fecb67fc640 1 -- 172.21.15.165:0/1996289815 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fedd808ee40 con 0x56534a7cfda0 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.881+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56534a7d7600 con 0x56534a7cfda0 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.881+0000 7fecb67fc640 1 -- 172.21.15.165:0/1996289815 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fedd80a7040 con 0x56534a7cfda0 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.881+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56534a7d7600 con 0x56534a7cfda0 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.882+0000 7fecb67fc640 1 -- 172.21.15.165:0/1996289815 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fedd80b4030 con 0x56534a7cfda0 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.882+0000 7fecb67fc640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecac036950 0x7fecac038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.882+0000 7fecb67fc640 1 -- 172.21.15.165:0/1996289815 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fedd80b2070 con 0x56534a7cfda0 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.882+0000 7fede04cb640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecac036950 0x7fecac038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.883+0000 7fede04cb640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecac036950 0x7fecac038e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fedd4001b60 tx=0x7fedd4010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.883+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.883+0000 7fede4f36800 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56534a7e2870 0x56534a7e4d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.883+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e5480 con 0x56534a7e2870 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.883+0000 7fede14cd640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56534a7e2870 0x56534a7e4d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.883+0000 7fede14cd640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56534a7e2870 0x56534a7e4d30 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.883+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.883+0000 7fede4f36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.883+0000 7fede4f36800 20 realm 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.883+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.883+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e5890 con 0x56534a7e2870 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede4f36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede4f36800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6160 con 0x56534a7e2870 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede4f36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e2870 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede4f36800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede4f36800 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56534a7e7260 0x56534a7e9650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e9c60 con 0x56534a7e7260 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede0ccc640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56534a7e7260 0x56534a7e9650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.916 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.884+0000 7fede0ccc640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56534a7e7260 0x56534a7e9650 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.885+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.885+0000 7fede4f36800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.885+0000 7fede4f36800 20 searching for the correct realm 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.885+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7eb590 con 0x56534a7e7260 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.885+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.885+0000 7fede0ccc640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fedd8067420 0x7fedd80a7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.885+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7eb590 con 0x7fedd8067420 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.885+0000 7fede04cb640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fedd8067420 0x7fedd80a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.886+0000 7fede04cb640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fedd8067420 0x7fedd80a7940 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.886+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.886+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7eb590 con 0x7fedd8067420 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.886+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.886+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7eb590 con 0x56534a7e2870 2024-04-27T03:31:30.917 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.886+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.886+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e2870 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.887+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.887+0000 7fede14cd640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fedd0016440 0x7fedd00168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.887+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd0016440 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.887+0000 7fede14cd640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fedd0016440 0x7fedd00168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.887+0000 7fede14cd640 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fedd0016440 0x7fedd00168c0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.887+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.887+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd0016440 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.888+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.888+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd8067420 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.888+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.888+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e7260 2024-04-27T03:31:30.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.888+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.888+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e2870 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.888+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.888+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd8067420 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.888+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.889+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e2870 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.889+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.889+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd8067420 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.889+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.889+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e7260 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.889+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.889+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e7260 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.889+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.889+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd0016440 2024-04-27T03:31:30.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.890+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.890+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e7260 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.890+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.890+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e7260 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.890+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.890+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e2870 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.891+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.891+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e2870 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.891+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.891+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd0016440 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.891+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.891+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e7260 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.891+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.891+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd8067420 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.892+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.892+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e2870 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.892+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.892+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd0016440 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.892+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.892+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e7260 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.892+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.892+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd0016440 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.893+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.893+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd8067420 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.893+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.893+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e7260 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.893+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.893+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd0016440 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.893+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.893+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x56534a7e2870 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x56534a7e6500 con 0x7fedd8067420 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7ea520 con 0x56534a7e2870 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7ea910 con 0x56534a7e2870 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.894+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7ea910 con 0x7fedd8067420 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 20 zone default found 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 4 Realm: () 2024-04-27T03:31:30.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 20 zonegroup default 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7eba10 con 0x7fedd0016440 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7ed420 con 0x56534a7e7260 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.895+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7ed8b0 con 0x56534a7e7260 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.896+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.896+0000 7fede4f36800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.896+0000 7fede4f36800 20 started sync module instance, tier type = 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.896+0000 7fede4f36800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.896+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7ec890 con 0x7fedd8067420 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.897+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.897+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94915732098176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7ec4f0 con 0x7fedd8067420 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.897+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7ee7e0 con 0x56534a7e2870 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.898+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94915732098176] v27'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.898+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.898+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94915732103984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7eec00 con 0x56534a7e2870 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.898+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7ef5c0 con 0x56534a7e7260 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.899+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94915732103984] v27'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.899+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.899+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94915732105040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7eeb70 con 0x56534a7e7260 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.899+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7f0480 con 0x7fedd8067420 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.900+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94915732105040] v27'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.900+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.900+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94915732109744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7f0cd0 con 0x7fedd8067420 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.900+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7f15a0 con 0x56534a7e2870 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.901+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.901+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94915732109744] v27'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.901+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94915732115568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7f1990 con 0x56534a7e2870 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.901+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7f23b0 con 0x7fedd0016440 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.902+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.902+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94915732115568] v27'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.902+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94915732119200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7f2370 con 0x7fedd0016440 2024-04-27T03:31:30.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.902+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7f0860 con 0x7fedd0016440 2024-04-27T03:31:30.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.903+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94915732119200] v27'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.903+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.903+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94915732113504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7f2ea0 con 0x7fedd0016440 2024-04-27T03:31:30.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.903+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7f3860 con 0x7fedd0016440 2024-04-27T03:31:30.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.904+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(58 notify.7 [create] v27'213 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.904+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94915732113504] v27'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.904+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94915732124592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7f3c80 con 0x7fedd0016440 2024-04-27T03:31:30.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.904+0000 7fede4f36800 20 add_watcher() i=0 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.904+0000 7fede4f36800 20 add_watcher() i=1 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.904+0000 7fede4f36800 20 add_watcher() i=2 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.904+0000 7fede4f36800 20 add_watcher() i=3 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.904+0000 7fede4f36800 20 add_watcher() i=4 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.904+0000 7fede4f36800 20 add_watcher() i=5 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.904+0000 7fede4f36800 20 add_watcher() i=6 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.905+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94915732124592] v27'214 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.905+0000 7fede4f36800 20 add_watcher() i=7 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.905+0000 7fede4f36800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.906+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a7f6cb0 con 0x56534a7e2870 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.906+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.906+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94915732125648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a7f4170 con 0x56534a7e2870 2024-04-27T03:31:30.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.907+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94915732125648] v27'218 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.907+0000 7fec96fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.907+0000 7fede4f36800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.907+0000 7fede4f36800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.907+0000 7fede4f36800 5 note: GC not initialized 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.907+0000 7fec95fe3640 20 reqs_thread_entry: start 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.907+0000 7fede4f36800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.907+0000 7fede4f36800 20 Filter name: none 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.908+0000 7fec077fe640 20 reqs_thread_entry: start 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.908+0000 7fede4f36800 10 cache get: name=default.rgw.meta+users.uid+bar : miss 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.908+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.908+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.e 8:718f6a54:users.uid::bar:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a829f60 con 0x7fedd0016440 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.908+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 bar [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.908+0000 7fede4f36800 20 rados_obj.operate() r=0 bl.length=380 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.908+0000 7fede4f36800 10 cache put: name=default.rgw.meta+users.uid+bar info.flags=0x13 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.908+0000 7fede4f36800 10 adding default.rgw.meta+users.uid+bar to cache LRU end 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.908+0000 7fede4f36800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.909+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:63 8.4 8:267425a8:users.uid::bud:head [create,call version.set in=58b,writefull 0~206 in=206b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a82e2f0 con 0x7fedd8067420 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(63 bud [create,call,writefull 0~206] v27'1 uv1 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fede4f36800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fede4f36800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fede4f36800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:64 7.3 7:c609908c:::notify.5:head [notify cookie 94915732364464 in=394b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a82efe0 con 0x7fedd0016440 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== watch-notify(notify (1) cookie 94915732119200 notify 115964117008 ret 0) ==== 424+0+0 (crc 0 0 0) 0x7fedd0030040 con 0x7fedd0016440 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fecb7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117008 cookie 94915732119200 notifier 4721 bl.length()=382 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fecb7fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fecb7fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T03:31:30.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(64 notify.5 [notify cookie 94915732364464 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fecb7fff640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7feca4003ea0 con 0x7fedd0016440 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(65 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.910+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== watch-notify(notify_complete (2) cookie 94915732364464 notify 115964117008 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fedd002a440 con 0x7fedd0016440 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.911+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 8.5 8:a00000ee:users.uid::bar.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a82dcb0 con 0x56534a7e2870 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.911+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(66 bar.buckets [call out=184b] v0'0 uv3 ondisk = 0) ==== 155+0+184 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.911+0000 7fede4f36800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 : miss 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.911+0000 7fede4f36800 20 rados->read ofs=0 len=0 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.911+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:67 8.1a 8:5ae7e5ec:root::.bucket.meta.myfoo6%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a82c100 con 0x7fedd0016440 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.912+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(67 .bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 [call out=48b,stat out=16b,read 0~380 out=380b,getxattrs out=218b] v0'0 uv3 ondisk = 0) ==== 333+0+662 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.912+0000 7fede4f36800 20 rados_obj.operate() r=0 bl.length=380 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.912+0000 7fede4f36800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 info.flags=0x17 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.912+0000 7fede4f36800 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 to cache LRU end 2024-04-27T03:31:30.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.912+0000 7fede4f36800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.912+0000 7fede4f36800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.912+0000 7fede4f36800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.912+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:68 8.1a 8:5ae7e5ec:root::.bucket.meta.myfoo6%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~380 in=380b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a8361f0 con 0x7fedd0016440 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(68 .bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 [delete,create,call,call,writefull 0~380,setxattr (117)] v27'4 uv4 ondisk = 0) ==== 417+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fede4f36800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 info.flags=0x17 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fede4f36800 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 to cache LRU end 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fede4f36800 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fede4f36800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9, ofs0, ns] 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [notify cookie 94915732352064 in=760b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a834e60 con 0x56534a7e2870 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== watch-notify(notify (1) cookie 94915732115568 notify 115964117032 ret 0) ==== 790+0+0 (crc 0 0 0) 0x56534a82c4f0 con 0x56534a7e2870 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fecb7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117032 cookie 94915732115568 notifier 4721 bl.length()=748 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fecb7fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 info.flags=0x17 2024-04-27T03:31:30.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fecb7fff640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 to cache LRU end 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fecb7fff640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fecb7fff640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7feca4003ea0 con 0x56534a7e2870 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.913+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(69 notify.4 [notify cookie 94915732352064 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.914+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify_complete (2) cookie 94915732352064 notify 115964117032 ret 0) ==== 42+0+48 (crc 0 0 0) 0x56534a82f1d0 con 0x56534a7e2870 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.914+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(70 notify.4 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.914+0000 7fede4f36800 20 handle_bi_update(): bucket=:myfoo6[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9]): orig_sources= new_sources= 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.914+0000 7fede4f36800 20 handle_bi_update(): bucket=:myfoo6[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9]): potential sources added=[] removed=[] 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.914+0000 7fede4f36800 20 handle_bi_update(): bucket=:myfoo6[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9]): orig_dests= new_dests= 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.914+0000 7fede4f36800 20 handle_bi_update(): bucket=:myfoo6[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9]): potential dests added=[] removed=[] 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.914+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=145b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a8349b0 con 0x56534a7e2870 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.915+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(71 bud.buckets [call] v27'5 uv0 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.915+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a8349b0 con 0x56534a7e2870 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.916+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(72 bud.buckets [call] v27'6 uv6 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.916+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:73 8.18 8:1944a257:root::myfoo6:head [delete,create,call version.inc in=38b,writefull 0~166 in=166b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a8359c0 con 0x56534a7e2870 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.917+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(73 myfoo6 [delete,create,call,writefull 0~166] v27'17 uv17 ondisk = 0) ==== 276+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.917+0000 7fede4f36800 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.917+0000 7fede4f36800 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.917+0000 7fede4f36800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 0, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.917+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [notify cookie 94915732352064 in=328b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a8359c0 con 0x7fedd8067420 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.917+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== watch-notify(notify (1) cookie 94915732109744 notify 115964117009 ret 0) ==== 358+0+0 (crc 0 0 0) 0x7fedd402cd00 con 0x7fedd8067420 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.917+0000 7fecb77fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117009 cookie 94915732109744 notifier 4721 bl.length()=316 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.917+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(74 notify.3 [notify cookie 94915732352064 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.917+0000 7fecb77fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x7 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.917+0000 7fecb77fe640 10 rgw watcher librados: moving default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.917+0000 7fecb77fe640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:75 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fec14003400 con 0x7fedd8067420 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.918+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== watch-notify(notify_complete (2) cookie 94915732352064 notify 115964117009 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fedd4046530 con 0x7fedd8067420 2024-04-27T03:31:30.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.918+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(75 notify.3 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.918+0000 7fede4f36800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 : hit (requested=0x17, cached=0x17) 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.918+0000 7fede4f36800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.918+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 8.1a 8:5ae7e5ec:root::.bucket.meta.myfoo6%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~380 in=380b,setxattr user.rgw.acl (117) in=129b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a835420 con 0x7fedd0016440 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(76 .bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 [delete,create,call,call,writefull 0~380,setxattr (117)] v27'5 uv5 ondisk = 0) ==== 417+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fede4f36800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 info.flags=0x17 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fede4f36800 10 moving default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 to cache LRU end 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fede4f36800 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fede4f36800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:root:.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9, ofs0, ns] 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:77 7.d 7:b4812045:::notify.4:head [notify cookie 94915732392368 in=760b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a8353c0 con 0x56534a7e2870 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== watch-notify(notify (1) cookie 94915732115568 notify 115964117033 ret 0) ==== 790+0+0 (crc 0 0 0) 0x56534a82f1d0 con 0x56534a7e2870 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fecb77fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117033 cookie 94915732115568 notifier 4721 bl.length()=748 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(77 notify.4 [notify cookie 94915732392368 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fecb77fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 info.flags=0x17 2024-04-27T03:31:30.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fecb77fe640 10 rgw watcher librados: moving default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 to cache LRU end 2024-04-27T03:31:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fecb77fe640 10 rgw watcher librados: updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T03:31:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fecb77fe640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:78 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fec140034d0 con 0x56534a7e2870 2024-04-27T03:31:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.919+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(78 notify.4 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.920+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== watch-notify(notify_complete (2) cookie 94915732392368 notify 115964117033 ret 0) ==== 42+0+48 (crc 0 0 0) 0x56534a833450 con 0x56534a7e2870 2024-04-27T03:31:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.920+0000 7fede4f36800 20 handle_bi_update(): bucket=:myfoo6[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9]): orig_sources= new_sources= 2024-04-27T03:31:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.920+0000 7fede4f36800 20 handle_bi_update(): bucket=:myfoo6[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9]): potential sources added=[] removed=[] 2024-04-27T03:31:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.920+0000 7fede4f36800 20 handle_bi_update(): bucket=:myfoo6[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9]): orig_dests= new_dests= 2024-04-27T03:31:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.920+0000 7fede4f36800 20 handle_bi_update(): bucket=:myfoo6[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9]): potential dests added=[] removed=[] 2024-04-27T03:31:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.920+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:79 4.2d 4:b6d005ec:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a8349b0 con 0x7fedd0016440 2024-04-27T03:31:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.920+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(79 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.0 [call out=85b] v0'0 uv2 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.920+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 4.2 4:42eb879e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a8349b0 con 0x56534a7e2870 2024-04-27T03:31:30.950 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.920+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(80 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.1 [call out=85b] v0'0 uv9 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.920+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:81 4.11 4:8b17784a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a8349b0 con 0x7fedd0016440 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.921+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(81 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.921+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 4.39 4:9e6af1e5:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a8349b0 con 0x56534a7e7260 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.921+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(82 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.3 [call out=85b] v0'0 uv7 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.921+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:83 4.36 4:6c26a203:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a8349b0 con 0x56534a7e2870 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.921+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(83 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.4 [call out=85b] v0'0 uv8 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.921+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 4.30 4:0f8f4ebf:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a8349b0 con 0x56534a7e7260 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.922+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(84 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.5 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.922+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:85 4.31 4:8e6b3321:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a8349b0 con 0x56534a7e2870 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.922+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(85 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.6 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.922+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:86 4.2 4:4019d6bb:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a8349b0 con 0x56534a7e2870 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.922+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 36 ==== osd_op_reply(86 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.7 [call out=85b] v0'0 uv10 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.922+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:87 4.1 4:809e6ae6:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a8349b0 con 0x7fedd8067420 2024-04-27T03:31:30.951 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.922+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(87 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.923+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:88 4.15 4:aa655c17:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a8349b0 con 0x56534a7e7260 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.923+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(88 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9 [call out=376b] v0'0 uv7 ondisk = 0) ==== 194+0+376 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.923+0000 7fede4f36800 10 cls_bucket_list_unordered: got thirteen 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.923+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:89 4.2c 4:35f4e7f3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a835320 con 0x56534a7e2870 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.923+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 37 ==== osd_op_reply(89 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.923+0000 7fede4f36800 20 get_obj_state: octx=0x7fedd401f8b0 obj=myfoo6:thirteen state=0x56534a82ae28 s->prefetch_data=0 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.923+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:90 3.22 3:4798bd04:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a835320 con 0x56534a7e7260 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(90 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9_thirteen [getxattrs out=852b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 238+0+868 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 10 manifest: total_size = 8 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 20 get_obj_state: setting s->obj_tag to 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.341348598665991022 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 20 Read xattr rgw_rados: user.rgw.acl 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 20 Read xattr rgw_rados: user.rgw.content_type 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 20 Read xattr rgw_rados: user.rgw.etag 2024-04-27T03:31:30.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 20 Read xattr rgw_rados: user.rgw.idtag 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 20 Read xattr rgw_rados: user.rgw.manifest 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 20 Read xattr rgw_rados: user.rgw.pg_ver 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 20 Read xattr rgw_rados: user.rgw.source_zone 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 20 Read xattr rgw_rados: user.rgw.tail_tag 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 20 get_obj_state: octx=0x7fedd401f8b0 obj=myfoo6:thirteen state=0x56534a82ae28 s->prefetch_data=0 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 : hit (requested=0x17, cached=0x17) 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 20 bucket index object: :.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.924+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 4.15 4:aa655c17:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a834ed0 con 0x56534a7e7260 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.925+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(91 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9 [stat,call,call] v27'8 uv8 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.925+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:92 3.22 3:4798bd04:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9_thirteen:head [cmpxattr user.rgw.idtag (61) op 1 mode 1 in=75b,setxattr user.rgw.acl (117) in=129b,setxattr user.rgw.idtag (33) in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a837940 con 0x56534a7e7260 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.926+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(92 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9_thirteen [cmpxattr (61) op 1 mode 1,setxattr (117),setxattr (33)] v27'2 uv2 ondisk = 0) ==== 280+0+0 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.926+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:93 4.15 4:aa655c17:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=267b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a837940 con 0x56534a7e7260 2024-04-27T03:31:30.954 INFO:teuthology.orchestra.run.smithi165.stderr:1 objects processed in :myfoo6[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9]). Next marker thirteen 2024-04-27T03:31:30.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.926+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a837940 con 0x7fedd8067420 2024-04-27T03:31:30.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.927+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(93 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9 [stat,call,call] v27'9 uv9 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.927+0000 7fecb77fe640 20 handle_completion(): completion ok for obj=thirteen 2024-04-27T03:31:30.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.927+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(94 bud [delete,create,call,writefull 0~380] v27'2 uv2 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.927+0000 7fede4f36800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-27T03:31:30.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.927+0000 7fede4f36800 10 moving default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T03:31:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.927+0000 7fede4f36800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-04-27T03:31:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.927+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:95 7.3 7:c609908c:::notify.5:head [notify cookie 94915732358992 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a837940 con 0x7fedd0016440 2024-04-27T03:31:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.928+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== watch-notify(notify (1) cookie 94915732119200 notify 115964117009 ret 0) ==== 598+0+0 (crc 0 0 0) 0x56534a833450 con 0x7fedd0016440 2024-04-27T03:31:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.928+0000 7fecb7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117009 cookie 94915732119200 notifier 4721 bl.length()=556 2024-04-27T03:31:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.928+0000 7fecb7fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+bud info.flags=0x17 2024-04-27T03:31:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.928+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(95 notify.5 [notify cookie 94915732358992 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.928+0000 7fecb7fff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T03:31:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.928+0000 7fecb7fff640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:96 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7feca4003ea0 con 0x7fedd0016440 2024-04-27T03:31:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.928+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(96 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.928+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== watch-notify(notify_complete (2) cookie 94915732358992 notify 115964117009 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fedd002a440 con 0x7fedd0016440 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.928+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:97 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a82e930 con 0x56534a7e2870 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.929+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 38 ==== osd_op_reply(97 NX5QOQKC6BH2IDN8HC7A [delete,create,writefull 0~7] v27'4 uv4 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.929+0000 7fede4f36800 10 cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.929+0000 7fede4f36800 10 adding default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.929+0000 7fede4f36800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.929+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:98 7.3 7:c609908c:::notify.5:head [notify cookie 94915732358992 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a82e930 con 0x7fedd0016440 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.929+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== watch-notify(notify (1) cookie 94915732119200 notify 115964117010 ret 0) ==== 219+0+0 (crc 0 0 0) 0x7fedd002a440 con 0x7fedd0016440 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.930+0000 7fecb7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117010 cookie 94915732119200 notifier 4721 bl.length()=177 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.930+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 30 ==== osd_op_reply(98 notify.5 [notify cookie 94915732358992 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.930+0000 7fecb7fff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A info.flags=0x7 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.930+0000 7fecb7fff640 10 rgw watcher librados: moving default.rgw.meta+users.keys+NX5QOQKC6BH2IDN8HC7A to cache LRU end 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.930+0000 7fecb7fff640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:99 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7feca4003ea0 con 0x7fedd0016440 2024-04-27T03:31:30.956 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.930+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 31 ==== osd_op_reply(99 notify.5 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.930+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 32 ==== watch-notify(notify_complete (2) cookie 94915732358992 notify 115964117010 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fedd002acf0 con 0x7fedd0016440 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.930+0000 7fede4f36800 10 removing user index: bar 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.930+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:100 8.e 8:718f6a54:users.uid::bar:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a837940 con 0x7fedd0016440 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.931+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 33 ==== osd_op_reply(100 bar [delete] v27'4 uv4 ondisk = 0) ==== 147+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.931+0000 7fede4f36800 10 removing default.rgw.meta+users.uid+bar from cache 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.931+0000 7fede4f36800 10 distributing notification oid=default.rgw.control:notify.7 cni=[op: 1, obj: default.rgw.meta:users.uid:bar, ofs0, ns] 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.931+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 7.9 7:93e5b521:::notify.7:head [notify cookie 94915732358992 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56534a837940 con 0x7fedd0016440 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.931+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 34 ==== watch-notify(notify (1) cookie 94915732124592 notify 115964117011 ret 0) ==== 194+0+0 (crc 0 0 0) 0x7fedd002acf0 con 0x7fedd0016440 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.931+0000 7fecb7fff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117011 cookie 94915732124592 notifier 4721 bl.length()=152 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.931+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 35 ==== osd_op_reply(101 notify.7 [notify cookie 94915732358992 out=8b] v0'0 uv213 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.931+0000 7fecb7fff640 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:102 7.9 7:93e5b521:::notify.7:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7feca4003ea0 con 0x7fedd0016440 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.931+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 36 ==== osd_op_reply(102 notify.7 [notify-ack] v0'0 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.957 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.931+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 37 ==== watch-notify(notify_complete (2) cookie 94915732358992 notify 115964117011 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fedd004f650 con 0x7fedd0016440 2024-04-27T03:31:30.958 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"bud","display_name":"Bar","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"bud","access_key":"NX5QOQKC6BH2IDN8HC7A","secret_key":"LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn","active":true,"create_date":"2024-04-27T03:31:30.754980Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:31:30.754965Z","tags":[],"group_ids":[]} 2024-04-27T03:31:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.932+0000 7fec96fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.932+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:103 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94915732125648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a812a80 con 0x56534a7e2870 2024-04-27T03:31:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.933+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 39 ==== osd_op_reply(103 data_loggenerations_metadata [watch unwatch cookie 94915732125648] v27'219 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.933+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94915732098176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a812a80 con 0x7fedd8067420 2024-04-27T03:31:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.934+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94915732098176] v27'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.934+0000 7fede4f36800 20 remove_watcher() i=0 2024-04-27T03:31:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.934+0000 7fede4f36800 2 removed watcher, disabling cache 2024-04-27T03:31:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.934+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94915732103984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a812a80 con 0x56534a7e2870 2024-04-27T03:31:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.935+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 40 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94915732103984] v27'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.964 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.935+0000 7fede4f36800 20 remove_watcher() i=1 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.935+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94915732105040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a812a80 con 0x56534a7e7260 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.936+0000 7fede0ccc640 1 -- 172.21.15.165:0/1996289815 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94915732105040] v27'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd80bd070 con 0x56534a7e7260 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.936+0000 7fede4f36800 20 remove_watcher() i=2 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.936+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94915732109744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a812a80 con 0x7fedd8067420 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.937+0000 7fede04cb640 1 -- 172.21.15.165:0/1996289815 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94915732109744] v27'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd4009050 con 0x7fedd8067420 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.937+0000 7fede4f36800 20 remove_watcher() i=3 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.937+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94915732115568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a812a80 con 0x56534a7e2870 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.938+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.1 v2:172.21.15.165:6800/79879041 41 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94915732115568] v27'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x56534a7e2870 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.938+0000 7fede4f36800 20 remove_watcher() i=4 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.938+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94915732119200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a812a80 con 0x7fedd0016440 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.939+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 38 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94915732119200] v27'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd000a4a0 con 0x7fedd0016440 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.939+0000 7fede4f36800 20 remove_watcher() i=5 2024-04-27T03:31:30.965 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.939+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94915732113504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a812a80 con 0x7fedd0016440 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.941+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 39 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94915732113504] v27'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd0068040 con 0x7fedd0016440 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.941+0000 7fede4f36800 20 remove_watcher() i=6 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.941+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94915732124592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x56534a812a80 con 0x7fedd0016440 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 <== osd.0 v2:172.21.15.89:6800/2353668432 40 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94915732124592] v27'215 uv213 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fedd0068040 con 0x7fedd0016440 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 20 remove_watcher() i=7 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fedd0016440 msgr2=0x7fedd00168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fedd0016440 0x7fedd00168c0 crc :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56534a7e2870 msgr2=0x56534a7e4d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56534a7e2870 0x56534a7e4d30 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fedd8067420 msgr2=0x7fedd80a7940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fedd8067420 0x7fedd80a7940 crc :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56534a7e7260 msgr2=0x56534a7e9650 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56534a7e7260 0x56534a7e9650 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.966 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecac036950 msgr2=0x7fecac038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecac036950 0x7fecac038e10 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fedd4001b60 tx=0x7fedd4010000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 msgr2=0x56534a7d70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 --2- 172.21.15.165:0/1996289815 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7cfda0 0x56534a7d70c0 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fedd8073d50 tx=0x7fedd806eed0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede14cd640 1 -- 172.21.15.165:0/1996289815 reap_dead start 2024-04-27T03:31:30.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 shutdown_connections 2024-04-27T03:31:30.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 >> 172.21.15.165:0/1996289815 conn(0x56534a7cc070 msgr2=0x56534a7c96d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 shutdown_connections 2024-04-27T03:31:30.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.942+0000 7fede4f36800 1 -- 172.21.15.165:0/1996289815 wait complete. 2024-04-27T03:31:30.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/1230008809 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecb8036950 msgr2=0x7fecb8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 --2- 172.21.15.165:0/1230008809 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecb8036950 0x7fecb8038e10 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fedd400bf50 tx=0x7fedd400f000 comp rx=0 tx=0).stop 2024-04-27T03:31:30.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/1230008809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7c2c80 msgr2=0x56534a7c30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.967 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 --2- 172.21.15.165:0/1230008809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7c2c80 0x56534a7c30a0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fedd8052520 tx=0x7fedd808ce80 comp rx=0 tx=0).stop 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/1230008809 shutdown_connections 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 --2- 172.21.15.165:0/1230008809 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecb8036950 0x7fecb8038e10 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 --2- 172.21.15.165:0/1230008809 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a7c2c80 0x56534a7c30a0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/1230008809 >> 172.21.15.165:0/1230008809 conn(0x56534a7b6d20 msgr2=0x56534a7b7120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/1230008809 shutdown_connections 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/1230008809 wait complete. 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56534a6a8fa0 msgr2=0x56534a7ab4b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56534a6a8fa0 0x56534a7ab4b0 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56534a7b1ce0 msgr2=0x56534a7b4130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x56534a7b1ce0 0x56534a7b4130 crc :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56534a7add50 msgr2=0x56534a7b0140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x56534a7add50 0x56534a7b0140 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecc80369a0 msgr2=0x7fecc8038e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.968 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fecc80369a0 0x7fecc8038e60 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fedd4000f60 tx=0x7fedd40012b0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 msgr2=0x56534a7a42c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 --2- 172.21.15.165:0/3754089057 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56534a6a7d40 0x56534a7a42c0 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fedd8060040 tx=0x7fedd8002fb0 comp rx=0 tx=0).stop 2024-04-27T03:31:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede14cd640 1 -- 172.21.15.165:0/3754089057 reap_dead start 2024-04-27T03:31:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 shutdown_connections 2024-04-27T03:31:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 >> 172.21.15.165:0/3754089057 conn(0x56534a6a33a0 msgr2=0x56534a7924d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 shutdown_connections 2024-04-27T03:31:30.969 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:30.943+0000 7fede4f36800 1 -- 172.21.15.165:0/3754089057 wait complete. 2024-04-27T03:31:30.969 DEBUG:tasks.util.rgw: json result: {'user_id': 'bud', 'display_name': 'Bar', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'bud', 'access_key': 'NX5QOQKC6BH2IDN8HC7A', 'secret_key': 'LnEsqNNqZIpkzauboDcLXLcYaWwLQ3Kop0zAnKIn', 'active': True, 'create_date': '2024-04-27T03:31:30.754980Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:31:30.754965Z', 'tags': [], 'group_ids': []} 2024-04-27T03:31:35.976 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:31:35.977 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-04-27T03:31:35.977 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-04-27T03:31:35.977 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'fud', '--display-name', 'Fud', '--access-key', 'p5YnriCv1nAtykxBrupQ', '--secret', 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', '--max-buckets', '4'] 2024-04-27T03:31:35.977 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid fud --display-name Fud --access-key p5YnriCv1nAtykxBrupQ --secret Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh --max-buckets 4 2024-04-27T03:31:36.036 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:36.036 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:36.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.029+0000 7f3571b2a800 1 Processor -- start 2024-04-27T03:31:36.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.030+0000 7f3571b2a800 1 -- start start 2024-04-27T03:31:36.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.030+0000 7f3571b2a800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a04c9b12f0 0x55a04c9b9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.030+0000 7f3571b2a800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a04c86fd40 con 0x55a04c9af7b0 2024-04-27T03:31:36.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.030+0000 7f3571b2a800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55a04c8424d0 con 0x55a04c9b12f0 2024-04-27T03:31:36.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.030+0000 7f356d063640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a04c9b12f0 0x55a04c9b9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.030+0000 7f356d864640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55a04c9af7b0 0x55a04c9afbd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:41184/0 (socket says 172.21.15.165:41184) 2024-04-27T03:31:36.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.030+0000 7f356d864640 1 -- 172.21.15.165:0/740091466 learned_addr learned my addr 172.21.15.165:0/740091466 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:31:36.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.031+0000 7f356d063640 1 -- 172.21.15.165:0/740091466 >> v1:172.21.15.89:6789/0 conn(0x55a04c9af7b0 legacy=0x55a04c9afbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.031+0000 7f356d063640 1 -- 172.21.15.165:0/740091466 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04c86e590 con 0x55a04c9b12f0 2024-04-27T03:31:36.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.033+0000 7f356d063640 1 --2- 172.21.15.165:0/740091466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9b12f0 0x55a04c9b9df0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f356400a470 tx=0x7f356401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd0e69d446811ec server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.033+0000 7f356c862640 1 -- 172.21.15.165:0/740091466 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3564057070 con 0x55a04c9b12f0 2024-04-27T03:31:36.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.033+0000 7f356c862640 1 -- 172.21.15.165:0/740091466 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3564052440 con 0x55a04c9b12f0 2024-04-27T03:31:36.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.033+0000 7f356c862640 1 -- 172.21.15.165:0/740091466 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3564052740 con 0x55a04c9b12f0 2024-04-27T03:31:36.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.033+0000 7f3571b2a800 1 -- 172.21.15.165:0/740091466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9b12f0 msgr2=0x55a04c9b9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.033+0000 7f3571b2a800 1 --2- 172.21.15.165:0/740091466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9b12f0 0x55a04c9b9df0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f356400a470 tx=0x7f356401ce90 comp rx=0 tx=0).stop 2024-04-27T03:31:36.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.033+0000 7f3571b2a800 1 -- 172.21.15.165:0/740091466 shutdown_connections 2024-04-27T03:31:36.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.033+0000 7f3571b2a800 1 --2- 172.21.15.165:0/740091466 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9b12f0 0x55a04c9b9df0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.033+0000 7f3571b2a800 1 -- 172.21.15.165:0/740091466 >> 172.21.15.165:0/740091466 conn(0x55a04c9ab3d0 msgr2=0x55a04c9ad7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.033+0000 7f3571b2a800 1 -- 172.21.15.165:0/740091466 shutdown_connections 2024-04-27T03:31:36.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.033+0000 7f3571b2a800 1 -- 172.21.15.165:0/740091466 wait complete. 2024-04-27T03:31:36.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.047+0000 7f3571b2a800 1 Processor -- start 2024-04-27T03:31:36.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.047+0000 7f3571b2a800 1 -- start start 2024-04-27T03:31:36.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.047+0000 7f3571b2a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 0x55a04caa4740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.047+0000 7f3571b2a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a04c8fc3e0 con 0x55a04c9af7b0 2024-04-27T03:31:36.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.048+0000 7f356d864640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 0x55a04caa4740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.048+0000 7f356d864640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 0x55a04caa4740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58386/0 (socket says 172.21.15.165:58386) 2024-04-27T03:31:36.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.048+0000 7f356d864640 1 -- 172.21.15.165:0/2763048343 learned_addr learned my addr 172.21.15.165:0/2763048343 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.048+0000 7f356d864640 1 -- 172.21.15.165:0/2763048343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04caa4d10 con 0x55a04c9af7b0 2024-04-27T03:31:36.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.048+0000 7f356d864640 1 --2- 172.21.15.165:0/2763048343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 0x55a04caa4740 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3560001dd0 tx=0x7f356000b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c2b8bf4ed35325e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.048+0000 7f345be07640 1 -- 172.21.15.165:0/2763048343 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3560012020 con 0x55a04c9af7b0 2024-04-27T03:31:36.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.048+0000 7f345be07640 1 -- 172.21.15.165:0/2763048343 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f356000e5a0 con 0x55a04c9af7b0 2024-04-27T03:31:36.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.048+0000 7f345be07640 1 -- 172.21.15.165:0/2763048343 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f356000e900 con 0x55a04c9af7b0 2024-04-27T03:31:36.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.049+0000 7f3571b2a800 1 -- 172.21.15.165:0/2763048343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 msgr2=0x55a04caa4740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.049+0000 7f3571b2a800 1 --2- 172.21.15.165:0/2763048343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 0x55a04caa4740 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3560001dd0 tx=0x7f356000b6c0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.049+0000 7f3571b2a800 1 -- 172.21.15.165:0/2763048343 shutdown_connections 2024-04-27T03:31:36.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.049+0000 7f3571b2a800 1 --2- 172.21.15.165:0/2763048343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 0x55a04caa4740 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.049+0000 7f3571b2a800 1 -- 172.21.15.165:0/2763048343 >> 172.21.15.165:0/2763048343 conn(0x55a04c9ab3d0 msgr2=0x55a04ca9ffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.049+0000 7f3571b2a800 1 -- 172.21.15.165:0/2763048343 shutdown_connections 2024-04-27T03:31:36.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.049+0000 7f3571b2a800 1 -- 172.21.15.165:0/2763048343 wait complete. 2024-04-27T03:31:36.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.049+0000 7f3571b2a800 1 Processor -- start 2024-04-27T03:31:36.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.049+0000 7f3571b2a800 1 -- start start 2024-04-27T03:31:36.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.050+0000 7f3571b2a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 0x55a04caac2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.050+0000 7f3571b2a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a04c9bc610 con 0x55a04c9af7b0 2024-04-27T03:31:36.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.050+0000 7f356d864640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 0x55a04caac2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.050+0000 7f356d864640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 0x55a04caac2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58400/0 (socket says 172.21.15.165:58400) 2024-04-27T03:31:36.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.050+0000 7f356d864640 1 -- 172.21.15.165:0/1974258491 learned_addr learned my addr 172.21.15.165:0/1974258491 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.050+0000 7f356d864640 1 -- 172.21.15.165:0/1974258491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04caac7e0 con 0x55a04c9af7b0 2024-04-27T03:31:36.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.050+0000 7f356d864640 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 0x55a04caac2a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3560001dd0 tx=0x7f3560000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.051+0000 7f345a604640 1 -- 172.21.15.165:0/1974258491 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3560012030 con 0x55a04c9af7b0 2024-04-27T03:31:36.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.051+0000 7f345a604640 1 -- 172.21.15.165:0/1974258491 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f356001a7a0 con 0x55a04c9af7b0 2024-04-27T03:31:36.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.051+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a04caaca70 con 0x55a04c9af7b0 2024-04-27T03:31:36.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.051+0000 7f345a604640 1 -- 172.21.15.165:0/1974258491 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f356001a990 con 0x55a04c9af7b0 2024-04-27T03:31:36.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.051+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04caaf720 con 0x55a04c9af7b0 2024-04-27T03:31:36.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.052+0000 7f345a604640 1 -- 172.21.15.165:0/1974258491 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f3560015020 con 0x55a04c9af7b0 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.052+0000 7f345a604640 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3454036900 0x7f3454038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.052+0000 7f345a604640 1 -- 172.21.15.165:0/1974258491 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f3560020030 con 0x55a04c9af7b0 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.052+0000 7f356d063640 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3454036900 0x7f3454038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.052+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a04cab14d0 0x55a04cab38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.052+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04cab3e40 con 0x55a04cab14d0 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.052+0000 7f356e065640 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a04cab14d0 0x55a04cab38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.052+0000 7f356d063640 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3454036900 0x7f3454038dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3564054000 tx=0x7f356404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.052+0000 7f356e065640 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a04cab14d0 0x55a04cab38c0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.053+0000 7f356e065640 1 -- 172.21.15.165:0/1974258491 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04cab14d0 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.053+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04cab0ab0 con 0x55a04cab14d0 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.053+0000 7f356e065640 1 -- 172.21.15.165:0/1974258491 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04cab14d0 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.053+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a04cab6170 0x55a04cab8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.053+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04cab8c00 con 0x55a04cab6170 2024-04-27T03:31:36.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.053+0000 7f356d864640 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a04cab6170 0x55a04cab8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.053+0000 7f356d864640 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a04cab6170 0x55a04cab8560 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.054+0000 7f356d864640 1 -- 172.21.15.165:0/1974258491 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3560018490 con 0x55a04cab6170 2024-04-27T03:31:36.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.054+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04cab5120 con 0x55a04cab14d0 2024-04-27T03:31:36.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.054+0000 7f356e065640 1 -- 172.21.15.165:0/1974258491 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04cab14d0 2024-04-27T03:31:36.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.054+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a04caba0b0 0x55a04cabc500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.054+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04cabcc60 con 0x55a04caba0b0 2024-04-27T03:31:36.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.054+0000 7f356d063640 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a04caba0b0 0x55a04cabc500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.055+0000 7f356d063640 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a04caba0b0 0x55a04cabc500 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.055+0000 7f356d063640 1 -- 172.21.15.165:0/1974258491 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3564052850 con 0x55a04caba0b0 2024-04-27T03:31:36.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.055+0000 7f3571b2a800 1 Processor -- start 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.055+0000 7f3571b2a800 1 -- start start 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.055+0000 7f3571b2a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cacb0c0 0x55a04cacb4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.055+0000 7f3571b2a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a04c8a26b0 con 0x55a04cacb0c0 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.056+0000 7f356d864640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cacb0c0 0x55a04cacb4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.056+0000 7f356d864640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cacb0c0 0x55a04cacb4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58410/0 (socket says 172.21.15.165:58410) 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.056+0000 7f356d864640 1 -- 172.21.15.165:0/2128123079 learned_addr learned my addr 172.21.15.165:0/2128123079 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.056+0000 7f356d864640 1 -- 172.21.15.165:0/2128123079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04cacba20 con 0x55a04cacb0c0 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.056+0000 7f356d864640 1 --2- 172.21.15.165:0/2128123079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cacb0c0 0x55a04cacb4e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f356004ad30 tx=0x7f356000db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.057+0000 7f34537fe640 1 -- 172.21.15.165:0/2128123079 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f356003e070 con 0x55a04cacb0c0 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.057+0000 7f34537fe640 1 -- 172.21.15.165:0/2128123079 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f35600446f0 con 0x55a04cacb0c0 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.057+0000 7f3571b2a800 1 -- 172.21.15.165:0/2128123079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a04cacc490 con 0x55a04cacb0c0 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.057+0000 7f34537fe640 1 -- 172.21.15.165:0/2128123079 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f356004c150 con 0x55a04cacb0c0 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.057+0000 7f3571b2a800 1 -- 172.21.15.165:0/2128123079 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04cacc990 con 0x55a04cacb0c0 2024-04-27T03:31:36.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.057+0000 7f34537fe640 1 -- 172.21.15.165:0/2128123079 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f356004c2f0 con 0x55a04cacb0c0 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.058+0000 7f34537fe640 1 --2- 172.21.15.165:0/2128123079 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3448036900 0x7f3448038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.058+0000 7f34537fe640 1 -- 172.21.15.165:0/2128123079 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f3560040040 con 0x55a04cacb0c0 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.058+0000 7f356d063640 1 --2- 172.21.15.165:0/2128123079 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3448036900 0x7f3448038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.058+0000 7f3571b2a800 1 Processor -- start 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.058+0000 7f3571b2a800 1 -- start start 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.058+0000 7f356d063640 1 --2- 172.21.15.165:0/2128123079 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3448036900 0x7f3448038dc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55a04cab09d0 tx=0x7f3564005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.058+0000 7f3571b2a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 0x55a04cad8660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.058+0000 7f3571b2a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a04caa5090 con 0x55a04cad8260 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.059+0000 7f356d864640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 0x55a04cad8660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.059+0000 7f356d864640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 0x55a04cad8660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58412/0 (socket says 172.21.15.165:58412) 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.059+0000 7f356d864640 1 -- 172.21.15.165:0/954344566 learned_addr learned my addr 172.21.15.165:0/954344566 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.059+0000 7f356d864640 1 -- 172.21.15.165:0/954344566 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04cad8c30 con 0x55a04cad8260 2024-04-27T03:31:36.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.059+0000 7f356d864640 1 --2- 172.21.15.165:0/954344566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 0x55a04cad8660 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3560014fd0 tx=0x7f356004c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b7e4ba9934d76b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.059+0000 7f34517fa640 1 -- 172.21.15.165:0/954344566 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3560051030 con 0x55a04cad8260 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.059+0000 7f34517fa640 1 -- 172.21.15.165:0/954344566 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3560049210 con 0x55a04cad8260 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.059+0000 7f34517fa640 1 -- 172.21.15.165:0/954344566 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3560038500 con 0x55a04cad8260 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.060+0000 7f3571b2a800 1 -- 172.21.15.165:0/954344566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 msgr2=0x55a04cad8660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.060+0000 7f3571b2a800 1 --2- 172.21.15.165:0/954344566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 0x55a04cad8660 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f3560014fd0 tx=0x7f356004c6c0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.060+0000 7f3571b2a800 1 -- 172.21.15.165:0/954344566 shutdown_connections 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.060+0000 7f3571b2a800 1 --2- 172.21.15.165:0/954344566 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 0x55a04cad8660 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.060+0000 7f3571b2a800 1 -- 172.21.15.165:0/954344566 >> 172.21.15.165:0/954344566 conn(0x55a04cad4530 msgr2=0x55a04cad2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.060+0000 7f3571b2a800 1 -- 172.21.15.165:0/954344566 shutdown_connections 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.060+0000 7f3571b2a800 1 -- 172.21.15.165:0/954344566 wait complete. 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.060+0000 7f3571b2a800 1 Processor -- start 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.060+0000 7f3571b2a800 1 -- start start 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.060+0000 7f3571b2a800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 0x55a04cadf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.060+0000 7f3571b2a800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a04cad9320 con 0x55a04cad8260 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.061+0000 7f356d864640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 0x55a04cadf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.061+0000 7f356d864640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 0x55a04cadf5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58422/0 (socket says 172.21.15.165:58422) 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.061+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 learned_addr learned my addr 172.21.15.165:0/1072630603 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.061+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a04cadfb30 con 0x55a04cad8260 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.061+0000 7f356d864640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 0x55a04cadf5f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3560040010 tx=0x7f3560004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.061+0000 7f343f7fe640 1 -- 172.21.15.165:0/1072630603 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3560070070 con 0x55a04cad8260 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.061+0000 7f343f7fe640 1 -- 172.21.15.165:0/1072630603 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3560038be0 con 0x55a04cad8260 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.061+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a04cadfb30 con 0x55a04cad8260 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.061+0000 7f343f7fe640 1 -- 172.21.15.165:0/1072630603 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3560038ec0 con 0x55a04cad8260 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.061+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a04cadfb30 con 0x55a04cad8260 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.062+0000 7f343f7fe640 1 -- 172.21.15.165:0/1072630603 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f3560073070 con 0x55a04cad8260 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.062+0000 7f343f7fe640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3438036a50 0x7f3438038f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.062+0000 7f343f7fe640 1 -- 172.21.15.165:0/1072630603 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f3560067030 con 0x55a04cad8260 2024-04-27T03:31:36.072 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.063+0000 7f356d063640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3438036a50 0x7f3438038f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.063+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.063+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a04caead50 0x55a04caed210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.063+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04caed960 con 0x55a04caead50 2024-04-27T03:31:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.063+0000 7f356d063640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3438036a50 0x7f3438038f10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55a04cabd060 tx=0x7f356404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.063+0000 7f356e065640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a04caead50 0x55a04caed210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.063+0000 7f356e065640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a04caead50 0x55a04caed210 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 20 realm 2024-04-27T03:31:36.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04caedd70 con 0x55a04caead50 2024-04-27T03:31:36.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:36.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee640 con 0x55a04caead50 2024-04-27T03:31:36.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caead50 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a04caef740 0x55a04caf1b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.064+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf2140 con 0x55a04caef740 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.065+0000 7f356d864640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a04caef740 0x55a04caf1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.065+0000 7f356d864640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a04caef740 0x55a04caf1b30 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.065+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.065+0000 7f3571b2a800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:36.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.065+0000 7f3571b2a800 20 searching for the correct realm 2024-04-27T03:31:36.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.065+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf3a70 con 0x55a04caef740 2024-04-27T03:31:36.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.066+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.066+0000 7f356d864640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3560075060 0x7f3560056900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.066+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf3a70 con 0x7f3560075060 2024-04-27T03:31:36.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.066+0000 7f356d063640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3560075060 0x7f3560056900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.066+0000 7f356d063640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3560075060 0x7f3560056900 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.067+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.067+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf3a70 con 0x7f3560075060 2024-04-27T03:31:36.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.067+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.067+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf3a70 con 0x55a04caead50 2024-04-27T03:31:36.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.067+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.067+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caead50 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.067+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.067+0000 7f356e065640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3568016440 0x7f35680168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.067+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3568016440 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.068+0000 7f356e065640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3568016440 0x7f35680168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.068+0000 7f356e065640 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3568016440 0x7f35680168c0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.068+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.068+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3568016440 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.069+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.069+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3560075060 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.069+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.069+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caef740 2024-04-27T03:31:36.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.069+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.069+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caead50 2024-04-27T03:31:36.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.070+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.070+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3560075060 2024-04-27T03:31:36.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.070+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.070+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caead50 2024-04-27T03:31:36.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.070+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.070+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3560075060 2024-04-27T03:31:36.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.070+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.070+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caef740 2024-04-27T03:31:36.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.071+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.071+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caef740 2024-04-27T03:31:36.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.071+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.071+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3568016440 2024-04-27T03:31:36.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.071+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.071+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caef740 2024-04-27T03:31:36.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.071+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.071+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caef740 2024-04-27T03:31:36.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.072+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.072+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caead50 2024-04-27T03:31:36.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.072+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.072+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caead50 2024-04-27T03:31:36.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.072+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.072+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3568016440 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.073+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.073+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caef740 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.073+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.073+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3560075060 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.073+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.073+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caead50 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.073+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.073+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3568016440 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.074+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.074+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caef740 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.074+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.074+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3568016440 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.074+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.074+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3560075060 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.075+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.075+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caef740 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.075+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.075+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3568016440 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.075+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.075+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x55a04caead50 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a04caee9e0 con 0x7f3560075060 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:36.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf2a00 con 0x55a04caead50 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf2df0 con 0x55a04caead50 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.076+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf2df0 con 0x7f3560075060 2024-04-27T03:31:36.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f3571b2a800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f3571b2a800 20 zone default found 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f3571b2a800 4 Realm: () 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f3571b2a800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f3571b2a800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f3571b2a800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f3571b2a800 20 zonegroup default 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf3ef0 con 0x7f3568016440 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f3571b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf58d0 con 0x55a04caef740 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.077+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.078+0000 7f3571b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.078+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.078+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf5df0 con 0x55a04caef740 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.078+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.078+0000 7f3571b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.078+0000 7f3571b2a800 20 started sync module instance, tier type = 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.078+0000 7f3571b2a800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.078+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf4d70 con 0x7f3560075060 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.079+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.079+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94146969686368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf49d0 con 0x7f3560075060 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.079+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf6bb0 con 0x55a04caead50 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.080+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94146969686368] v27'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.080+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.080+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94146969691184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf6fa0 con 0x55a04caead50 2024-04-27T03:31:36.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.080+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf79e0 con 0x55a04caef740 2024-04-27T03:31:36.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.082+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94146969691184] v27'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.082+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.082+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94146969695408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf7dd0 con 0x55a04caef740 2024-04-27T03:31:36.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.082+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf8810 con 0x7f3560075060 2024-04-27T03:31:36.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.083+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.083+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94146969699088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf90e0 con 0x7f3560075060 2024-04-27T03:31:36.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.083+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf9b60 con 0x55a04caead50 2024-04-27T03:31:36.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.083+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94146969695408] v27'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.084+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94146969699088] v27'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.084+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.084+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94146969701424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf68b0 con 0x55a04caead50 2024-04-27T03:31:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.084+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04cafa440 con 0x7f3568016440 2024-04-27T03:31:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.086+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.086+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94146969706320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04cafa410 con 0x7f3568016440 2024-04-27T03:31:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.086+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04cafb430 con 0x7f3568016440 2024-04-27T03:31:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.087+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94146969701424] v27'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.087+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94146969706320] v27'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.087+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.087+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94146969708592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf9650 con 0x7f3568016440 2024-04-27T03:31:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.087+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04cafb8a0 con 0x7f3568016440 2024-04-27T03:31:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.088+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(58 notify.7 [create] v27'216 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.088+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94146969708592] v27'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.088+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94146969711600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04cafbcc0 con 0x7f3568016440 2024-04-27T03:31:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.088+0000 7f3571b2a800 20 add_watcher() i=0 2024-04-27T03:31:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.088+0000 7f3571b2a800 20 add_watcher() i=1 2024-04-27T03:31:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.088+0000 7f3571b2a800 20 add_watcher() i=2 2024-04-27T03:31:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.088+0000 7f3571b2a800 20 add_watcher() i=3 2024-04-27T03:31:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.088+0000 7f3571b2a800 20 add_watcher() i=4 2024-04-27T03:31:36.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.088+0000 7f3571b2a800 20 add_watcher() i=5 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.088+0000 7f3571b2a800 20 add_watcher() i=6 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.089+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94146969711600] v27'217 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.089+0000 7f3571b2a800 20 add_watcher() i=7 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.089+0000 7f3571b2a800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.089+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04cafbcc0 con 0x55a04caead50 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.089+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.090+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94146969712656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04cafbc30 con 0x55a04caead50 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.091+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94146969712656] v27'220 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.091+0000 7f341f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.091+0000 7f3571b2a800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.091+0000 7f3571b2a800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.091+0000 7f3571b2a800 5 note: GC not initialized 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.091+0000 7f341e7fc640 20 reqs_thread_entry: start 2024-04-27T03:31:36.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.091+0000 7f3571b2a800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.091+0000 7f3571b2a800 20 Filter name: none 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.091+0000 7f338ffff640 20 reqs_thread_entry: start 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.091+0000 7f3571b2a800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.091+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.091+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04cb320a0 con 0x7f3568016440 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 fud [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f3571b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f3571b2a800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x0 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f3571b2a800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f3571b2a800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : miss 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f3571b2a800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04cb30840 con 0x7f3568016440 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(63 p5YnriCv1nAtykxBrupQ [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f3571b2a800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f3571b2a800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x0 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f3571b2a800 10 adding default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f3571b2a800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f3571b2a800 10 cache get: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ : hit (negative entry) 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.092+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:64 8.e 8:75f48285:users.uid::fud:head [delete,create,call version.set in=58b,writefull 0~380 in=380b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04cb31110 con 0x7f3568016440 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.093+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(64 fud [delete,create,call,writefull 0~380] v27'5 uv5 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.093+0000 7f3571b2a800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.094+0000 7f3571b2a800 10 moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.094+0000 7f3571b2a800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.094+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:65 7.3 7:c609908c:::notify.5:head [notify cookie 94146969930848 in=568b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04cb35b40 con 0x7f3568016440 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.094+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== watch-notify(notify (1) cookie 94146969706320 notify 115964117012 ret 0) ==== 598+0+0 (crc 0 0 0) 0x55a04cb314b0 con 0x7f3568016440 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.094+0000 7f34517fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117012 cookie 94146969706320 notifier 4728 bl.length()=556 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.094+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(65 notify.5 [notify cookie 94146969930848 out=8b] v0'0 uv216 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.094+0000 7f34517fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+fud info.flags=0x17 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.094+0000 7f34517fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.094+0000 7f34517fa640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:66 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f3430003ea0 con 0x7f3568016440 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.094+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(66 notify.5 [notify-ack] v0'0 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.094+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== watch-notify(notify_complete (2) cookie 94146969930848 notify 115964117012 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a04cb31af0 con 0x7f3568016440 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.094+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:67 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04cb33670 con 0x7f3568016440 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.095+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(67 p5YnriCv1nAtykxBrupQ [delete,create,writefull 0~7] v27'10 uv10 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.095+0000 7f3571b2a800 10 cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.095+0000 7f3571b2a800 10 moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.095+0000 7f3571b2a800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.095+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94146969930720 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a04cb35b40 con 0x55a04caef740 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.096+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== watch-notify(notify (1) cookie 94146969695408 notify 115964116998 ret 0) ==== 219+0+0 (crc 0 0 0) 0x55a04cacc490 con 0x55a04caef740 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.096+0000 7f34517fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116998 cookie 94146969695408 notifier 4728 bl.length()=177 2024-04-27T03:31:36.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.096+0000 7f34517fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ info.flags=0x7 2024-04-27T03:31:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.096+0000 7f34517fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+p5YnriCv1nAtykxBrupQ to cache LRU end 2024-04-27T03:31:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.096+0000 7f34517fa640 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f3430003ea0 con 0x55a04caef740 2024-04-27T03:31:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.096+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(68 notify.2 [notify cookie 94146969930720 out=8b] v0'0 uv216 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.096+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.096+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== watch-notify(notify_complete (2) cookie 94146969930720 notify 115964116998 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a04cacba20 con 0x55a04caef740 2024-04-27T03:31:36.115 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"fud","display_name":"Fud","email":"","suspended":0,"max_buckets":4,"subusers":[],"keys":[{"user":"fud","access_key":"p5YnriCv1nAtykxBrupQ","secret_key":"Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh","active":true,"create_date":"2024-04-27T03:31:36.093683Z"}],"swift_keys":[],"caps":[],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:31:36.093669Z","tags":[],"group_ids":[]} 2024-04-27T03:31:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.097+0000 7f341f7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.097+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94146969712656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04cae7700 con 0x55a04caead50 2024-04-27T03:31:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.098+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94146969712656] v27'221 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.098+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94146969686368] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04cafb810 con 0x7f3560075060 2024-04-27T03:31:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.099+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94146969686368] v27'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.099+0000 7f3571b2a800 20 remove_watcher() i=0 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.099+0000 7f3571b2a800 2 removed watcher, disabling cache 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.099+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94146969691184] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04cafb810 con 0x55a04caead50 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.100+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94146969691184] v27'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.100+0000 7f3571b2a800 20 remove_watcher() i=1 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.100+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94146969695408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf6430 con 0x55a04caef740 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.102+0000 7f356d864640 1 -- 172.21.15.165:0/1072630603 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94146969695408] v27'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3560072030 con 0x55a04caef740 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.102+0000 7f3571b2a800 20 remove_watcher() i=2 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.102+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94146969699088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04cafb810 con 0x7f3560075060 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.102+0000 7f356d063640 1 -- 172.21.15.165:0/1072630603 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94146969699088] v27'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3564081070 con 0x7f3560075060 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.103+0000 7f3571b2a800 20 remove_watcher() i=3 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.103+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94146969701424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf74b0 con 0x55a04caead50 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.104+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94146969701424] v27'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x55a04caead50 2024-04-27T03:31:36.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.104+0000 7f3571b2a800 20 remove_watcher() i=4 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.104+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94146969706320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf6430 con 0x7f3568016440 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.105+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94146969706320] v27'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.105+0000 7f3571b2a800 20 remove_watcher() i=5 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.105+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94146969708592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf8c30 con 0x7f3568016440 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.106+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94146969708592] v27'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.106+0000 7f3571b2a800 20 remove_watcher() i=6 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.106+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94146969711600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a04caf74b0 con 0x7f3568016440 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.107+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94146969711600] v27'218 uv216 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f356800a4a0 con 0x7f3568016440 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.107+0000 7f3571b2a800 20 remove_watcher() i=7 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.107+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3568016440 msgr2=0x7f35680168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.107+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3568016440 0x7f35680168c0 crc :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.107+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a04caead50 msgr2=0x55a04caed210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.107+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a04caead50 0x55a04caed210 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.107+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3560075060 msgr2=0x7f3560056900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.107+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3560075060 0x7f3560056900 crc :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.107+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a04caef740 msgr2=0x55a04caf1b30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.107+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a04caef740 0x55a04caf1b30 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3438036a50 msgr2=0x7f3438038f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3438036a50 0x7f3438038f10 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x55a04cabd060 tx=0x7f356404f000 comp rx=0 tx=0).stop 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 msgr2=0x55a04cadf5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f356e065640 1 -- 172.21.15.165:0/1072630603 reap_dead start 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 0x55a04cadf5f0 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f3560040010 tx=0x7f3560004d50 comp rx=0 tx=0).stop 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 shutdown_connections 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1072630603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cad8260 0x55a04cadf5f0 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 >> 172.21.15.165:0/1072630603 conn(0x55a04cad4530 msgr2=0x55a04cad1b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 shutdown_connections 2024-04-27T03:31:36.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/1072630603 wait complete. 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/2128123079 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3448036900 msgr2=0x7f3448038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 --2- 172.21.15.165:0/2128123079 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3448036900 0x7f3448038dc0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x55a04cab09d0 tx=0x7f3564005a00 comp rx=0 tx=0).stop 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/2128123079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cacb0c0 msgr2=0x55a04cacb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 --2- 172.21.15.165:0/2128123079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cacb0c0 0x55a04cacb4e0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f356004ad30 tx=0x7f356000db20 comp rx=0 tx=0).stop 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/2128123079 shutdown_connections 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 --2- 172.21.15.165:0/2128123079 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3448036900 0x7f3448038dc0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 --2- 172.21.15.165:0/2128123079 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04cacb0c0 0x55a04cacb4e0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/2128123079 >> 172.21.15.165:0/2128123079 conn(0x55a04cabf160 msgr2=0x55a04cabf560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/2128123079 shutdown_connections 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/2128123079 wait complete. 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a04cab14d0 msgr2=0x55a04cab38c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a04cab14d0 0x55a04cab38c0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a04caba0b0 msgr2=0x55a04cabc500 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a04caba0b0 0x55a04cabc500 crc :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a04cab6170 msgr2=0x55a04cab8560 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a04cab6170 0x55a04cab8560 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.108+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3454036900 msgr2=0x7f3454038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.109+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3454036900 0x7f3454038dc0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f3564054000 tx=0x7f356404e000 comp rx=0 tx=0).stop 2024-04-27T03:31:36.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.109+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 msgr2=0x55a04caac2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.109+0000 7f3571b2a800 1 --2- 172.21.15.165:0/1974258491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a04c9af7b0 0x55a04caac2a0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3560001dd0 tx=0x7f3560000ed0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.109+0000 7f356e065640 1 -- 172.21.15.165:0/1974258491 reap_dead start 2024-04-27T03:31:36.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.109+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 shutdown_connections 2024-04-27T03:31:36.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.109+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 >> 172.21.15.165:0/1974258491 conn(0x55a04c9ab3d0 msgr2=0x55a04ca9e880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.109+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 shutdown_connections 2024-04-27T03:31:36.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.109+0000 7f3571b2a800 1 -- 172.21.15.165:0/1974258491 wait complete. 2024-04-27T03:31:36.126 DEBUG:tasks.util.rgw: json result: {'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True, 'create_date': '2024-04-27T03:31:36.093683Z'}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:31:36.093669Z', 'tags': [], 'group_ids': []} 2024-04-27T03:31:36.143 INFO:tasks.radosgw_admin:RL: create_bucket myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:31:36.148 INFO:tasks.radosgw_admin:RL: put_obj myfoo7 fud bytes_out=8 bytes_in=0 failed=False 2024-04-27T03:31:36.148 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-04-27T03:31:36.148 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rename', '--uid', 'bud', '--new-uid', 'fud'] 2024-04-27T03:31:36.148 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rename --uid bud --new-uid fud 2024-04-27T03:31:36.193 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:36.194 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:36.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.189+0000 7f9292533800 1 Processor -- start 2024-04-27T03:31:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.189+0000 7f9292533800 1 -- start start 2024-04-27T03:31:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.189+0000 7f9292533800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5622dbe111d0 0x5622dbe135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.189+0000 7f9292533800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5622dbcccd40 con 0x5622dbe0cd40 2024-04-27T03:31:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.189+0000 7f9292533800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5622dbc9f4d0 con 0x5622dbe111d0 2024-04-27T03:31:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.189+0000 7f9287fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5622dbe111d0 0x5622dbe135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.189+0000 7f9287fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5622dbe111d0 0x5622dbe135c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58426/0 (socket says 172.21.15.165:58426) 2024-04-27T03:31:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.189+0000 7f928e1f5640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5622dbe0cd40 0x5622dbe0d160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:41188/0 (socket says 172.21.15.165:41188) 2024-04-27T03:31:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.189+0000 7f9287fff640 1 -- 172.21.15.165:0/3743366233 learned_addr learned my addr 172.21.15.165:0/3743366233 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.189+0000 7f9287fff640 1 -- 172.21.15.165:0/3743366233 >> v1:172.21.15.89:6789/0 conn(0x5622dbe0cd40 legacy=0x5622dbe0d160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.189+0000 7f9287fff640 1 -- 172.21.15.165:0/3743366233 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622dbccb590 con 0x5622dbe111d0 2024-04-27T03:31:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.191+0000 7f9287fff640 1 --2- 172.21.15.165:0/3743366233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe111d0 0x5622dbe135c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f927800a520 tx=0x7f927801d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec55a1fa22e44b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.191+0000 7f928d9f4640 1 -- 172.21.15.165:0/3743366233 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9278057070 con 0x5622dbe111d0 2024-04-27T03:31:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.191+0000 7f928d9f4640 1 -- 172.21.15.165:0/3743366233 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9278052570 con 0x5622dbe111d0 2024-04-27T03:31:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.191+0000 7f928d9f4640 1 -- 172.21.15.165:0/3743366233 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f927804ae60 con 0x5622dbe111d0 2024-04-27T03:31:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.192+0000 7f9292533800 1 -- 172.21.15.165:0/3743366233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe111d0 msgr2=0x5622dbe135c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.192+0000 7f9292533800 1 --2- 172.21.15.165:0/3743366233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe111d0 0x5622dbe135c0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f927800a520 tx=0x7f927801d010 comp rx=0 tx=0).stop 2024-04-27T03:31:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.192+0000 7f9292533800 1 -- 172.21.15.165:0/3743366233 shutdown_connections 2024-04-27T03:31:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.192+0000 7f9292533800 1 --2- 172.21.15.165:0/3743366233 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe111d0 0x5622dbe135c0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.192+0000 7f9292533800 1 -- 172.21.15.165:0/3743366233 >> 172.21.15.165:0/3743366233 conn(0x5622dbe083a0 msgr2=0x5622dbe0a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.192+0000 7f9292533800 1 -- 172.21.15.165:0/3743366233 shutdown_connections 2024-04-27T03:31:36.200 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.192+0000 7f9292533800 1 -- 172.21.15.165:0/3743366233 wait complete. 2024-04-27T03:31:36.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.204+0000 7f9292533800 1 Processor -- start 2024-04-27T03:31:36.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.204+0000 7f9292533800 1 -- start start 2024-04-27T03:31:36.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.204+0000 7f9292533800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 0x5622dbd81010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.204+0000 7f9292533800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5622dbd593b0 con 0x5622dbe0cd40 2024-04-27T03:31:36.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.204+0000 7f928e1f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 0x5622dbd81010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.204+0000 7f928e1f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 0x5622dbd81010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58430/0 (socket says 172.21.15.165:58430) 2024-04-27T03:31:36.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.204+0000 7f928e1f5640 1 -- 172.21.15.165:0/1197332816 learned_addr learned my addr 172.21.15.165:0/1197332816 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.210 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.205+0000 7f928e1f5640 1 -- 172.21.15.165:0/1197332816 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622dbf01e20 con 0x5622dbe0cd40 2024-04-27T03:31:36.212 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.205+0000 7f928e1f5640 1 --2- 172.21.15.165:0/1197332816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 0x5622dbd81010 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f928000b230 tx=0x7f928000b6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41276b7c816710eb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.205+0000 7f917c608640 1 -- 172.21.15.165:0/1197332816 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f928000e5c0 con 0x5622dbe0cd40 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.205+0000 7f917c608640 1 -- 172.21.15.165:0/1197332816 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9280006070 con 0x5622dbe0cd40 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.205+0000 7f917c608640 1 -- 172.21.15.165:0/1197332816 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f928000e890 con 0x5622dbe0cd40 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.206+0000 7f9292533800 1 -- 172.21.15.165:0/1197332816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 msgr2=0x5622dbd81010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.206+0000 7f9292533800 1 --2- 172.21.15.165:0/1197332816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 0x5622dbd81010 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f928000b230 tx=0x7f928000b6f0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.206+0000 7f9292533800 1 -- 172.21.15.165:0/1197332816 shutdown_connections 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.206+0000 7f9292533800 1 --2- 172.21.15.165:0/1197332816 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 0x5622dbd81010 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.206+0000 7f9292533800 1 -- 172.21.15.165:0/1197332816 >> 172.21.15.165:0/1197332816 conn(0x5622dbe083a0 msgr2=0x5622dbef9910 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.206+0000 7f9292533800 1 -- 172.21.15.165:0/1197332816 shutdown_connections 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.206+0000 7f9292533800 1 -- 172.21.15.165:0/1197332816 wait complete. 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.206+0000 7f9292533800 1 Processor -- start 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.206+0000 7f9292533800 1 -- start start 2024-04-27T03:31:36.213 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.206+0000 7f9292533800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 0x5622dbeff750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.206+0000 7f9292533800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5622dbe13e90 con 0x5622dbe0cd40 2024-04-27T03:31:36.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.207+0000 7f928e1f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 0x5622dbeff750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.207+0000 7f928e1f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 0x5622dbeff750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58454/0 (socket says 172.21.15.165:58454) 2024-04-27T03:31:36.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.207+0000 7f928e1f5640 1 -- 172.21.15.165:0/2543719730 learned_addr learned my addr 172.21.15.165:0/2543719730 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.207+0000 7f928e1f5640 1 -- 172.21.15.165:0/2543719730 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622dbefde40 con 0x5622dbe0cd40 2024-04-27T03:31:36.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.207+0000 7f928e1f5640 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 0x5622dbeff750 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9280000fd0 tx=0x7f9280000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.207+0000 7f917ae05640 1 -- 172.21.15.165:0/2543719730 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f928000f8d0 con 0x5622dbe0cd40 2024-04-27T03:31:36.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.207+0000 7f917ae05640 1 -- 172.21.15.165:0/2543719730 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928000fa90 con 0x5622dbe0cd40 2024-04-27T03:31:36.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.207+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5622dbefe0d0 con 0x5622dbe0cd40 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.208+0000 7f917ae05640 1 -- 172.21.15.165:0/2543719730 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f928000fdb0 con 0x5622dbe0cd40 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.208+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622dbefe5f0 con 0x5622dbe0cd40 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.208+0000 7f917ae05640 1 -- 172.21.15.165:0/2543719730 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9280017070 con 0x5622dbe0cd40 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.209+0000 7f917ae05640 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9174036950 0x7f9174038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.209+0000 7f917ae05640 1 -- 172.21.15.165:0/2543719730 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f9280015020 con 0x5622dbe0cd40 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.209+0000 7f9287fff640 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9174036950 0x7f9174038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.209+0000 7f9292533800 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5622dbe111d0 0x5622dbf100e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.209+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf10670 con 0x5622dbe111d0 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.209+0000 7f928e9f6640 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5622dbe111d0 0x5622dbf100e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.209+0000 7f9287fff640 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9174036950 0x7f9174038e10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9278054000 tx=0x7f927804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.209+0000 7f928e9f6640 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5622dbe111d0 0x5622dbf100e0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.209+0000 7f928e9f6640 1 -- 172.21.15.165:0/2543719730 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbe111d0 2024-04-27T03:31:36.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.210+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf10fa0 con 0x5622dbe111d0 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.210+0000 7f928e9f6640 1 -- 172.21.15.165:0/2543719730 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbe111d0 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.210+0000 7f9292533800 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5622dbf129d0 0x5622dbf14e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.210+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf155f0 con 0x5622dbf129d0 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.210+0000 7f928e1f5640 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5622dbf129d0 0x5622dbf14e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.210+0000 7f928e1f5640 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5622dbf129d0 0x5622dbf14e60 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.210+0000 7f928e1f5640 1 -- 172.21.15.165:0/2543719730 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f928002e070 con 0x5622dbf129d0 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.211+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf11820 con 0x5622dbe111d0 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.211+0000 7f928e9f6640 1 -- 172.21.15.165:0/2543719730 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbe111d0 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.211+0000 7f9292533800 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5622dbf16aa0 0x5622dbf18f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.211+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf19670 con 0x5622dbf16aa0 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.211+0000 7f9287fff640 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5622dbf16aa0 0x5622dbf18f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.211+0000 7f9287fff640 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5622dbf16aa0 0x5622dbf18f10 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.211+0000 7f9287fff640 1 -- 172.21.15.165:0/2543719730 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f927805a040 con 0x5622dbf16aa0 2024-04-27T03:31:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.212+0000 7f9292533800 1 Processor -- start 2024-04-27T03:31:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.212+0000 7f9292533800 1 -- start start 2024-04-27T03:31:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.212+0000 7f9292533800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf27a00 0x5622dbf27e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.212+0000 7f9292533800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5622dbeffc90 con 0x5622dbf27a00 2024-04-27T03:31:36.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.212+0000 7f928e1f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf27a00 0x5622dbf27e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.212+0000 7f928e1f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf27a00 0x5622dbf27e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58458/0 (socket says 172.21.15.165:58458) 2024-04-27T03:31:36.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.212+0000 7f928e1f5640 1 -- 172.21.15.165:0/3493700340 learned_addr learned my addr 172.21.15.165:0/3493700340 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.213+0000 7f928e1f5640 1 -- 172.21.15.165:0/3493700340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622dbf28360 con 0x5622dbf27a00 2024-04-27T03:31:36.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.213+0000 7f928e1f5640 1 --2- 172.21.15.165:0/3493700340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf27a00 0x5622dbf27e20 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f928004b470 tx=0x7f9280017d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.213+0000 7f9173fff640 1 -- 172.21.15.165:0/3493700340 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9280044a40 con 0x5622dbf27a00 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.213+0000 7f9173fff640 1 -- 172.21.15.165:0/3493700340 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9280044be0 con 0x5622dbf27a00 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.213+0000 7f9173fff640 1 -- 172.21.15.165:0/3493700340 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9280044ec0 con 0x5622dbf27a00 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.213+0000 7f9292533800 1 -- 172.21.15.165:0/3493700340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5622dbf28dd0 con 0x5622dbf27a00 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.213+0000 7f9292533800 1 -- 172.21.15.165:0/3493700340 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622dbf28dd0 con 0x5622dbf27a00 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.214+0000 7f9173fff640 1 -- 172.21.15.165:0/3493700340 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f9280019860 con 0x5622dbf27a00 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.214+0000 7f9173fff640 1 --2- 172.21.15.165:0/3493700340 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9168036900 0x7f9168038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.214+0000 7f9173fff640 1 -- 172.21.15.165:0/3493700340 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f9280038020 con 0x5622dbf27a00 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.214+0000 7f9287fff640 1 --2- 172.21.15.165:0/3493700340 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9168036900 0x7f9168038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.215+0000 7f9292533800 1 Processor -- start 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.215+0000 7f9292533800 1 -- start start 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.215+0000 7f9287fff640 1 --2- 172.21.15.165:0/3493700340 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9168036900 0x7f9168038dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5622dbefea10 tx=0x7f9278005ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.215+0000 7f9292533800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 0x5622dbf34da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.215+0000 7f9292533800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5622dbcff6b0 con 0x5622dbf34980 2024-04-27T03:31:36.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.215+0000 7f928e1f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 0x5622dbf34da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.215+0000 7f928e1f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 0x5622dbf34da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58472/0 (socket says 172.21.15.165:58472) 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.215+0000 7f928e1f5640 1 -- 172.21.15.165:0/2177021343 learned_addr learned my addr 172.21.15.165:0/2177021343 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.215+0000 7f928e1f5640 1 -- 172.21.15.165:0/2177021343 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622dbf35370 con 0x5622dbf34980 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.216+0000 7f928e1f5640 1 --2- 172.21.15.165:0/2177021343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 0x5622dbf34da0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f928004a190 tx=0x7f9280026800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af65e74e58d4c94c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.216+0000 7f9171ffb640 1 -- 172.21.15.165:0/2177021343 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f928004de10 con 0x5622dbf34980 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.216+0000 7f9171ffb640 1 -- 172.21.15.165:0/2177021343 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f9280050060 con 0x5622dbf34980 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.216+0000 7f9171ffb640 1 -- 172.21.15.165:0/2177021343 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f928003fdf0 con 0x5622dbf34980 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.216+0000 7f9292533800 1 -- 172.21.15.165:0/2177021343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 msgr2=0x5622dbf34da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.216+0000 7f9292533800 1 --2- 172.21.15.165:0/2177021343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 0x5622dbf34da0 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7f928004a190 tx=0x7f9280026800 comp rx=0 tx=0).stop 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.216+0000 7f9292533800 1 -- 172.21.15.165:0/2177021343 shutdown_connections 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.216+0000 7f9292533800 1 --2- 172.21.15.165:0/2177021343 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 0x5622dbf34da0 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.216+0000 7f9292533800 1 -- 172.21.15.165:0/2177021343 >> 172.21.15.165:0/2177021343 conn(0x5622dbf30c50 msgr2=0x5622dbf2ecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.216+0000 7f9292533800 1 -- 172.21.15.165:0/2177021343 shutdown_connections 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.216+0000 7f9292533800 1 -- 172.21.15.165:0/2177021343 wait complete. 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.217+0000 7f9292533800 1 Processor -- start 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.217+0000 7f9292533800 1 -- start start 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.217+0000 7f9292533800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 0x5622dbf3bbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.217+0000 7f9292533800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5622dbf35850 con 0x5622dbf34980 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.217+0000 7f928e1f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 0x5622dbf3bbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.217+0000 7f928e1f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 0x5622dbf3bbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58488/0 (socket says 172.21.15.165:58488) 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.217+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 learned_addr learned my addr 172.21.15.165:0/2080695012 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.217+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5622dbf3c0f0 con 0x5622dbf34980 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.217+0000 7f928e1f5640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 0x5622dbf3bbb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9280026840 tx=0x7f92800325e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.218+0000 7f915ffff640 1 -- 172.21.15.165:0/2080695012 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9280028e60 con 0x5622dbf34980 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.218+0000 7f915ffff640 1 -- 172.21.15.165:0/2080695012 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f928003f8d0 con 0x5622dbf34980 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.218+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5622dbf3c0f0 con 0x5622dbf34980 2024-04-27T03:31:36.232 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.218+0000 7f915ffff640 1 -- 172.21.15.165:0/2080695012 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f928001cc90 con 0x5622dbf34980 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.218+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5622dbf3c0f0 con 0x5622dbf34980 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.218+0000 7f915ffff640 1 -- 172.21.15.165:0/2080695012 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f928006d070 con 0x5622dbf34980 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.219+0000 7f915ffff640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9158036900 0x7f9158038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.219+0000 7f915ffff640 1 -- 172.21.15.165:0/2080695012 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f9280071bc0 con 0x5622dbf34980 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.219+0000 7f9287fff640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9158036900 0x7f9158038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.219+0000 7f9292533800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.219+0000 7f9292533800 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5622dbf47290 0x5622dbf49750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.219+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf49ea0 con 0x5622dbf47290 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.219+0000 7f928e9f6640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5622dbf47290 0x5622dbf49750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.219+0000 7f9287fff640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9158036900 0x7f9158038dc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x5622dbf19a70 tx=0x7f927804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.219+0000 7f928e9f6640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5622dbf47290 0x5622dbf49750 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.233 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 20 realm 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4a2b0 con 0x5622dbf47290 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4ab80 con 0x5622dbf47290 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf47290 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.234 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:36.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5622dbf4bc80 0x5622dbf4e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.220+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4e680 con 0x5622dbf4bc80 2024-04-27T03:31:36.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.221+0000 7f928e1f5640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5622dbf4bc80 0x5622dbf4e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.221+0000 7f928e1f5640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5622dbf4bc80 0x5622dbf4e070 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.221+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.221+0000 7f9292533800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:36.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.221+0000 7f9292533800 20 searching for the correct realm 2024-04-27T03:31:36.235 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.221+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4ffd0 con 0x5622dbf4bc80 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.222+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.222+0000 7f928e1f5640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f928000f070 0x7f928000d770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.222+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4ffd0 con 0x7f928000f070 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.222+0000 7f9287fff640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f928000f070 0x7f928000d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.222+0000 7f9287fff640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f928000f070 0x7f928000d770 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.222+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.222+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4ffd0 con 0x7f928000f070 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.223+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.223+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4ffd0 con 0x5622dbf47290 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.223+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.223+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf47290 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.223+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.223+0000 7f928e9f6640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9288016440 0x7f92880168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.223+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f9288016440 2024-04-27T03:31:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.223+0000 7f928e9f6640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9288016440 0x7f92880168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.224+0000 7f928e9f6640 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9288016440 0x7f92880168c0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.224+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.224+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f9288016440 2024-04-27T03:31:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.224+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.224+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f928000f070 2024-04-27T03:31:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.225+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.247 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.225+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf4bc80 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.225+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.225+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf47290 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.225+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.225+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f928000f070 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.225+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.225+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf47290 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.226+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.226+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f928000f070 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.226+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.226+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf4bc80 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.226+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.226+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf4bc80 2024-04-27T03:31:36.248 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.226+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.226+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f9288016440 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.227+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.227+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf4bc80 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.227+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.227+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf4bc80 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.227+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.227+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf47290 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.227+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.227+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf47290 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.227+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.228+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f9288016440 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.228+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.228+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf4bc80 2024-04-27T03:31:36.249 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.228+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.228+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f928000f070 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.228+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.228+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf47290 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.229+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.229+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f9288016440 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.229+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.229+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf4bc80 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.229+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.229+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f9288016440 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.229+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.230+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f928000f070 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.230+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.250 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.230+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf4bc80 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.230+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.230+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f9288016440 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.230+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.230+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x5622dbf47290 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4af20 con 0x7f928000f070 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:36.251 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4ef40 con 0x5622dbf47290 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4f330 con 0x5622dbf47290 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.231+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf4f330 con 0x7f928000f070 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 20 zone default found 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 4 Realm: () 2024-04-27T03:31:36.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 20 zonegroup default 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf50450 con 0x7f9288016440 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf51e30 con 0x5622dbf4bc80 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.232+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf52380 con 0x5622dbf4bc80 2024-04-27T03:31:36.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.233+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.233+0000 7f9292533800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.233+0000 7f9292533800 20 started sync module instance, tier type = 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.233+0000 7f9292533800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.233+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf512d0 con 0x7f928000f070 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.234+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.234+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94707719149248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf50f30 con 0x7f928000f070 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.234+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf53140 con 0x5622dbf47290 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.235+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94707719149248] v27'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.235+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.235+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94707719154112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf53560 con 0x5622dbf47290 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.235+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf53f80 con 0x5622dbf4bc80 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.236+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94707719154112] v27'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.237+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.237+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94707719158384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf54370 con 0x5622dbf4bc80 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.237+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf54dd0 con 0x7f928000f070 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.238+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.238+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94707719162032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf55670 con 0x7f928000f070 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.238+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf560f0 con 0x5622dbf47290 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.238+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94707719158384] v27'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.239+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.239+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94707719166944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf564e0 con 0x5622dbf47290 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.239+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf56f20 con 0x7f9288016440 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.239+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94707719162032] v27'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.240+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94707719166944] v27'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.240+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.240+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94707719155296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf56ee0 con 0x7f9288016440 2024-04-27T03:31:36.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.240+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf577c0 con 0x7f9288016440 2024-04-27T03:31:36.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.241+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94707719155296] v27'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.241+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.241+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94707719170544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf56ee0 con 0x7f9288016440 2024-04-27T03:31:36.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.242+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf57e00 con 0x7f9288016440 2024-04-27T03:31:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.243+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(58 notify.7 [create] v27'219 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.243+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94707719170544] v27'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.243+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94707719174448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf582b0 con 0x7f9288016440 2024-04-27T03:31:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.243+0000 7f9292533800 20 add_watcher() i=0 2024-04-27T03:31:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.243+0000 7f9292533800 20 add_watcher() i=1 2024-04-27T03:31:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.243+0000 7f9292533800 20 add_watcher() i=2 2024-04-27T03:31:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.243+0000 7f9292533800 20 add_watcher() i=3 2024-04-27T03:31:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.243+0000 7f9292533800 20 add_watcher() i=4 2024-04-27T03:31:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.243+0000 7f9292533800 20 add_watcher() i=5 2024-04-27T03:31:36.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.243+0000 7f9292533800 20 add_watcher() i=6 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.243+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94707719174448] v27'220 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.244+0000 7f9292533800 20 add_watcher() i=7 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.244+0000 7f9292533800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.244+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf582b0 con 0x5622dbf47290 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.244+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.244+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94707719175680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf58b50 con 0x5622dbf47290 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.245+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94707719175680] v27'222 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.245+0000 7f9144fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.245+0000 7f9292533800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.245+0000 7f9292533800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.245+0000 7f9292533800 5 note: GC not initialized 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.245+0000 7f90bf7fe640 20 reqs_thread_entry: start 2024-04-27T03:31:36.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.245+0000 7f9292533800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.246+0000 7f9292533800 20 Filter name: none 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.246+0000 7f90bd7fa640 20 reqs_thread_entry: start 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.246+0000 7f9292533800 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.246+0000 7f9292533800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.246+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:62 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf8e490 con 0x7f928000f070 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.246+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(62 bud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.246+0000 7f9292533800 20 rados_obj.operate() r=0 bl.length=380 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.246+0000 7f9292533800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.246+0000 7f9292533800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.246+0000 7f9292533800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.246+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.e 8:75f48285:users.uid::fud:head [create,call version.set in=58b,writefull 0~206 in=206b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf928b0 con 0x7f9288016440 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.248+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 fud [create,call,writefull 0~206] v27'6 uv5 ondisk = -17 ((17) File exists)) ==== 231+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:could not rename user: unable to rename user, user name given by --new-uid already exists. to overwrite this user, add --yes-i-really-mean-it 2024-04-27T03:31:36.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.249+0000 7f9144fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.249+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94707719175680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf43f40 con 0x5622dbf47290 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.250+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94707719175680] v27'223 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.250+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94707719149248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf57e00 con 0x7f928000f070 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.251+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94707719149248] v27'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.251+0000 7f9292533800 20 remove_watcher() i=0 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.251+0000 7f9292533800 2 removed watcher, disabling cache 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.251+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94707719154112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf57e00 con 0x5622dbf47290 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.252+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94707719154112] v27'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.252+0000 7f9292533800 20 remove_watcher() i=1 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.252+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94707719158384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf529c0 con 0x5622dbf4bc80 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.253+0000 7f928e1f5640 1 -- 172.21.15.165:0/2080695012 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94707719158384] v27'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f9280082040 con 0x5622dbf4bc80 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.253+0000 7f9292533800 20 remove_watcher() i=2 2024-04-27T03:31:36.263 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.253+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94707719162032] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf57e00 con 0x7f928000f070 2024-04-27T03:31:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.254+0000 7f9287fff640 1 -- 172.21.15.165:0/2080695012 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94707719162032] v27'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f927807f070 con 0x7f928000f070 2024-04-27T03:31:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.254+0000 7f9292533800 20 remove_watcher() i=3 2024-04-27T03:31:36.264 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.254+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94707719166944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf53a70 con 0x5622dbf47290 2024-04-27T03:31:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.255+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94707719166944] v27'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x5622dbf47290 2024-04-27T03:31:36.267 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.255+0000 7f9292533800 20 remove_watcher() i=4 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.255+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94707719155296] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf529c0 con 0x7f9288016440 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.256+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94707719155296] v27'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.256+0000 7f9292533800 20 remove_watcher() i=5 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.256+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94707719170544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf57e00 con 0x7f9288016440 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.258+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94707719170544] v27'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.258+0000 7f9292533800 20 remove_watcher() i=6 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.258+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94707719174448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5622dbf53a70 con 0x7f9288016440 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94707719174448] v27'221 uv219 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f928800a4a0 con 0x7f9288016440 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 20 remove_watcher() i=7 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9288016440 msgr2=0x7f92880168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f9288016440 0x7f92880168c0 crc :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5622dbf47290 msgr2=0x5622dbf49750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.268 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5622dbf47290 0x5622dbf49750 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f928000f070 msgr2=0x7f928000d770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f928000f070 0x7f928000d770 crc :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5622dbf4bc80 msgr2=0x5622dbf4e070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5622dbf4bc80 0x5622dbf4e070 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9158036900 msgr2=0x7f9158038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9158036900 0x7f9158038dc0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x5622dbf19a70 tx=0x7f927804f000 comp rx=0 tx=0).stop 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 msgr2=0x5622dbf3bbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 --2- 172.21.15.165:0/2080695012 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf34980 0x5622dbf3bbb0 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7f9280026840 tx=0x7f92800325e0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 reap_dead start 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f928e9f6640 1 -- 172.21.15.165:0/2080695012 reap_dead start 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 shutdown_connections 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 >> 172.21.15.165:0/2080695012 conn(0x5622dbf30c50 msgr2=0x5622dbf2e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.269 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 shutdown_connections 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/2080695012 wait complete. 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/3493700340 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9168036900 msgr2=0x7f9168038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 --2- 172.21.15.165:0/3493700340 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9168036900 0x7f9168038dc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x5622dbefea10 tx=0x7f9278005ac0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/3493700340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf27a00 msgr2=0x5622dbf27e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 --2- 172.21.15.165:0/3493700340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf27a00 0x5622dbf27e20 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7f928004b470 tx=0x7f9280017d50 comp rx=0 tx=0).stop 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/3493700340 shutdown_connections 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 --2- 172.21.15.165:0/3493700340 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9168036900 0x7f9168038dc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 --2- 172.21.15.165:0/3493700340 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbf27a00 0x5622dbf27e20 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/3493700340 >> 172.21.15.165:0/3493700340 conn(0x5622dbf1baa0 msgr2=0x5622dbf1bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/3493700340 shutdown_connections 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.259+0000 7f9292533800 1 -- 172.21.15.165:0/3493700340 wait complete. 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5622dbe111d0 msgr2=0x5622dbf100e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5622dbe111d0 0x5622dbf100e0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.270 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5622dbf16aa0 msgr2=0x5622dbf18f10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5622dbf16aa0 0x5622dbf18f10 crc :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5622dbf129d0 msgr2=0x5622dbf14e60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5622dbf129d0 0x5622dbf14e60 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9174036950 msgr2=0x7f9174038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9174036950 0x7f9174038e10 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f9278054000 tx=0x7f927804e000 comp rx=0 tx=0).stop 2024-04-27T03:31:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 msgr2=0x5622dbeff750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 --2- 172.21.15.165:0/2543719730 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5622dbe0cd40 0x5622dbeff750 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f9280000fd0 tx=0x7f9280000ed0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f928e9f6640 1 -- 172.21.15.165:0/2543719730 reap_dead start 2024-04-27T03:31:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 shutdown_connections 2024-04-27T03:31:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 >> 172.21.15.165:0/2543719730 conn(0x5622dbe083a0 msgr2=0x5622dbef81d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 shutdown_connections 2024-04-27T03:31:36.271 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.260+0000 7f9292533800 1 -- 172.21.15.165:0/2543719730 wait complete. 2024-04-27T03:31:36.273 DEBUG:teuthology.orchestra.run:got remote process result: 17 2024-04-27T03:31:36.278 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:31:36.278 INFO:tasks.radosgw_admin:RL: get_obj myfoo6 bud bytes_out=0 bytes_in=8 failed=False 2024-04-27T03:31:36.282 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=0 failed=False 2024-04-27T03:31:36.282 INFO:tasks.radosgw_admin:RL: get_obj myfoo7 fud bytes_out=0 bytes_in=8 failed=False 2024-04-27T03:31:36.282 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'bud', '--purge-data'] 2024-04-27T03:31:36.282 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'bud', '--purge-data'] 2024-04-27T03:31:36.283 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid bud --purge-data 2024-04-27T03:31:36.325 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:36.326 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:36.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.321+0000 7fbb9c53e800 1 Processor -- start 2024-04-27T03:31:36.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.321+0000 7fbb9c53e800 1 -- start start 2024-04-27T03:31:36.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.321+0000 7fbb9c53e800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x556c00ba54a0 0x556c00ba7890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.321+0000 7fbb9c53e800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x556c00a5fd40 con 0x556c00ba1990 2024-04-27T03:31:36.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.321+0000 7fbb9c53e800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x556c00a324d0 con 0x556c00ba54a0 2024-04-27T03:31:36.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.321+0000 7fbb8ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x556c00ba54a0 0x556c00ba7890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.321+0000 7fbb9823f640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x556c00ba1990 0x556c00ba3d80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:41198/0 (socket says 172.21.15.165:41198) 2024-04-27T03:31:36.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.321+0000 7fbb9823f640 1 -- 172.21.15.165:0/196022188 learned_addr learned my addr 172.21.15.165:0/196022188 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:31:36.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.321+0000 7fbb8ffff640 1 -- 172.21.15.165:0/196022188 >> v1:172.21.15.89:6789/0 conn(0x556c00ba1990 legacy=0x556c00ba3d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.321+0000 7fbb97a3e640 1 -- 172.21.15.165:0/196022188 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 475990027 0 0) 0x556c00a5fd40 con 0x556c00ba1990 2024-04-27T03:31:36.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.322+0000 7fbb8ffff640 1 -- 172.21.15.165:0/196022188 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c00a5e590 con 0x556c00ba54a0 2024-04-27T03:31:36.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.323+0000 7fbb8ffff640 1 --2- 172.21.15.165:0/196022188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba54a0 0x556c00ba7890 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000a470 tx=0x7fbb8001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81725c817067109c server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.324+0000 7fbb97a3e640 1 -- 172.21.15.165:0/196022188 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb80057070 con 0x556c00ba54a0 2024-04-27T03:31:36.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.324+0000 7fbb97a3e640 1 -- 172.21.15.165:0/196022188 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb80052570 con 0x556c00ba54a0 2024-04-27T03:31:36.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.324+0000 7fbb97a3e640 1 -- 172.21.15.165:0/196022188 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb80052870 con 0x556c00ba54a0 2024-04-27T03:31:36.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.324+0000 7fbb9c53e800 1 -- 172.21.15.165:0/196022188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba54a0 msgr2=0x556c00ba7890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.324+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/196022188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba54a0 0x556c00ba7890 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fbb8000a470 tx=0x7fbb8001ce90 comp rx=0 tx=0).stop 2024-04-27T03:31:36.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.324+0000 7fbb9c53e800 1 -- 172.21.15.165:0/196022188 shutdown_connections 2024-04-27T03:31:36.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.324+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/196022188 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba54a0 0x556c00ba7890 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.324+0000 7fbb9c53e800 1 -- 172.21.15.165:0/196022188 >> 172.21.15.165:0/196022188 conn(0x556c00b9b340 msgr2=0x556c00b9d760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.324+0000 7fbb9c53e800 1 -- 172.21.15.165:0/196022188 shutdown_connections 2024-04-27T03:31:36.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.324+0000 7fbb9c53e800 1 -- 172.21.15.165:0/196022188 wait complete. 2024-04-27T03:31:36.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.336+0000 7fbb9c53e800 1 Processor -- start 2024-04-27T03:31:36.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.336+0000 7fbb9c53e800 1 -- start start 2024-04-27T03:31:36.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.336+0000 7fbb9c53e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 0x556c00ba12d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.336+0000 7fbb9c53e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556c00aec3b0 con 0x556c00ba1990 2024-04-27T03:31:36.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.336+0000 7fbb9823f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 0x556c00ba12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.342 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.336+0000 7fbb9823f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 0x556c00ba12d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58508/0 (socket says 172.21.15.165:58508) 2024-04-27T03:31:36.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.336+0000 7fbb9823f640 1 -- 172.21.15.165:0/2131171017 learned_addr learned my addr 172.21.15.165:0/2131171017 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.343 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.337+0000 7fbb9823f640 1 -- 172.21.15.165:0/2131171017 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c00b9f890 con 0x556c00ba1990 2024-04-27T03:31:36.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.337+0000 7fbb9823f640 1 --2- 172.21.15.165:0/2131171017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 0x556c00ba12d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800b210 tx=0x7fbb8800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2e7e5cca1a37b70 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.337+0000 7fba8660c640 1 -- 172.21.15.165:0/2131171017 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb88008070 con 0x556c00ba1990 2024-04-27T03:31:36.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.337+0000 7fba8660c640 1 -- 172.21.15.165:0/2131171017 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb8800d500 con 0x556c00ba1990 2024-04-27T03:31:36.344 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.337+0000 7fba8660c640 1 -- 172.21.15.165:0/2131171017 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb8800d840 con 0x556c00ba1990 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.338+0000 7fbb9c53e800 1 -- 172.21.15.165:0/2131171017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 msgr2=0x556c00ba12d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.338+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/2131171017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 0x556c00ba12d0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800b210 tx=0x7fbb8800b6d0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.338+0000 7fbb9c53e800 1 -- 172.21.15.165:0/2131171017 shutdown_connections 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.338+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/2131171017 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 0x556c00ba12d0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.338+0000 7fbb9c53e800 1 -- 172.21.15.165:0/2131171017 >> 172.21.15.165:0/2131171017 conn(0x556c00b9b340 msgr2=0x556c00c8d890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.338+0000 7fbb9c53e800 1 -- 172.21.15.165:0/2131171017 shutdown_connections 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.338+0000 7fbb9c53e800 1 -- 172.21.15.165:0/2131171017 wait complete. 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.338+0000 7fbb9c53e800 1 Processor -- start 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.338+0000 7fbb9c53e800 1 -- start start 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.338+0000 7fbb9c53e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 0x556c00c9bd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.338+0000 7fbb9c53e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556c00baa0e0 con 0x556c00ba1990 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.339+0000 7fbb9823f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 0x556c00c9bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.339+0000 7fbb9823f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 0x556c00c9bd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58520/0 (socket says 172.21.15.165:58520) 2024-04-27T03:31:36.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.339+0000 7fbb9823f640 1 -- 172.21.15.165:0/900589583 learned_addr learned my addr 172.21.15.165:0/900589583 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.339+0000 7fbb9823f640 1 -- 172.21.15.165:0/900589583 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c00c9c240 con 0x556c00ba1990 2024-04-27T03:31:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.340+0000 7fbb9823f640 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 0x556c00c9bd00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800b210 tx=0x7fbb88000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.340+0000 7fba84e09640 1 -- 172.21.15.165:0/900589583 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb8800f950 con 0x556c00ba1990 2024-04-27T03:31:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.340+0000 7fba84e09640 1 -- 172.21.15.165:0/900589583 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb8800faf0 con 0x556c00ba1990 2024-04-27T03:31:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.340+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556c00c9c4d0 con 0x556c00ba1990 2024-04-27T03:31:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.340+0000 7fba84e09640 1 -- 172.21.15.165:0/900589583 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb8800fdd0 con 0x556c00ba1990 2024-04-27T03:31:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.340+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c00c9f180 con 0x556c00ba1990 2024-04-27T03:31:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.341+0000 7fba84e09640 1 -- 172.21.15.165:0/900589583 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fbb88017040 con 0x556c00ba1990 2024-04-27T03:31:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.341+0000 7fba84e09640 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba80036900 0x7fba80038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.341+0000 7fba84e09640 1 -- 172.21.15.165:0/900589583 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fbb88015020 con 0x556c00ba1990 2024-04-27T03:31:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.341+0000 7fbb8ffff640 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba80036900 0x7fba80038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.341+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c00ba54a0 0x556c00ca2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.341+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00ca3480 con 0x556c00ba54a0 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.341+0000 7fbb98a40640 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c00ba54a0 0x556c00ca2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.341+0000 7fbb8ffff640 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba80036900 0x7fba80038dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbb80054000 tx=0x7fbb8004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.341+0000 7fbb98a40640 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c00ba54a0 0x556c00ca2f40 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.342+0000 7fbb98a40640 1 -- 172.21.15.165:0/900589583 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00ba54a0 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.342+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00ca04f0 con 0x556c00ba54a0 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.342+0000 7fbb98a40640 1 -- 172.21.15.165:0/900589583 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00ba54a0 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.342+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c00ca57a0 0x556c00ca7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.342+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00ca8230 con 0x556c00ca57a0 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.342+0000 7fbb9823f640 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c00ca57a0 0x556c00ca7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.342+0000 7fbb9823f640 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c00ca57a0 0x556c00ca7b90 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.343+0000 7fbb9823f640 1 -- 172.21.15.165:0/900589583 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fbb8802d070 con 0x556c00ca57a0 2024-04-27T03:31:36.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.343+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00ca4750 con 0x556c00ba54a0 2024-04-27T03:31:36.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.343+0000 7fbb98a40640 1 -- 172.21.15.165:0/900589583 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00ba54a0 2024-04-27T03:31:36.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.343+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x556c00ca96e0 0x556c00cabb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.343+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00cac290 con 0x556c00ca96e0 2024-04-27T03:31:36.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.343+0000 7fbb8ffff640 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x556c00ca96e0 0x556c00cabb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.343+0000 7fbb8ffff640 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x556c00ca96e0 0x556c00cabb30 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.344+0000 7fbb8ffff640 1 -- 172.21.15.165:0/900589583 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fbb80052940 con 0x556c00ca96e0 2024-04-27T03:31:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.344+0000 7fbb9c53e800 1 Processor -- start 2024-04-27T03:31:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.344+0000 7fbb9c53e800 1 -- start start 2024-04-27T03:31:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.344+0000 7fbb9c53e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cba6a0 0x556c00cbaac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.344+0000 7fbb9c53e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556c00a926b0 con 0x556c00cba6a0 2024-04-27T03:31:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.344+0000 7fbb9823f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cba6a0 0x556c00cbaac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.344+0000 7fbb9823f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cba6a0 0x556c00cbaac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58528/0 (socket says 172.21.15.165:58528) 2024-04-27T03:31:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.344+0000 7fbb9823f640 1 -- 172.21.15.165:0/3209572564 learned_addr learned my addr 172.21.15.165:0/3209572564 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.345+0000 7fbb9823f640 1 -- 172.21.15.165:0/3209572564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c00cbb000 con 0x556c00cba6a0 2024-04-27T03:31:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.345+0000 7fbb9823f640 1 --2- 172.21.15.165:0/3209572564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cba6a0 0x556c00cbaac0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbb8804b4b0 tx=0x7fbb8801aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.360 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.345+0000 7fba7dffb640 1 -- 172.21.15.165:0/3209572564 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb88019470 con 0x556c00cba6a0 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.345+0000 7fba7dffb640 1 -- 172.21.15.165:0/3209572564 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb88019610 con 0x556c00cba6a0 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.345+0000 7fbb9c53e800 1 -- 172.21.15.165:0/3209572564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556c00cbba70 con 0x556c00cba6a0 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.345+0000 7fba7dffb640 1 -- 172.21.15.165:0/3209572564 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb880198f0 con 0x556c00cba6a0 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.345+0000 7fbb9c53e800 1 -- 172.21.15.165:0/3209572564 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c00cbbf70 con 0x556c00cba6a0 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.346+0000 7fba7dffb640 1 -- 172.21.15.165:0/3209572564 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fbb8803f1b0 con 0x556c00cba6a0 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.346+0000 7fba7dffb640 1 --2- 172.21.15.165:0/3209572564 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba70036970 0x7fba70038e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.346+0000 7fba7dffb640 1 -- 172.21.15.165:0/3209572564 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fbb8803cbb0 con 0x556c00cba6a0 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.346+0000 7fbb8ffff640 1 --2- 172.21.15.165:0/3209572564 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba70036970 0x7fba70038e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.346+0000 7fbb9c53e800 1 Processor -- start 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.346+0000 7fbb9c53e800 1 -- start start 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.346+0000 7fbb8ffff640 1 --2- 172.21.15.165:0/3209572564 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba70036970 0x7fba70038e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x556c00ca0430 tx=0x7fbb80005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.346+0000 7fbb9c53e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 0x556c00cc7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.346+0000 7fbb9c53e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556c00b9fc10 con 0x556c00cc7820 2024-04-27T03:31:36.361 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.347+0000 7fbb9823f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 0x556c00cc7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.347+0000 7fbb9823f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 0x556c00cc7c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58530/0 (socket says 172.21.15.165:58530) 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.347+0000 7fbb9823f640 1 -- 172.21.15.165:0/2137775034 learned_addr learned my addr 172.21.15.165:0/2137775034 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.347+0000 7fbb9823f640 1 -- 172.21.15.165:0/2137775034 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c00cc81f0 con 0x556c00cc7820 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.347+0000 7fbb9823f640 1 --2- 172.21.15.165:0/2137775034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 0x556c00cc7c20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbb88026780 tx=0x7fbb88030f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62e2af1c9523e16a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.347+0000 7fba777fe640 1 -- 172.21.15.165:0/2137775034 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb88026430 con 0x556c00cc7820 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fba777fe640 1 -- 172.21.15.165:0/2137775034 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb8800c280 con 0x556c00cc7820 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fba777fe640 1 -- 172.21.15.165:0/2137775034 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb88041dd0 con 0x556c00cc7820 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fbb9c53e800 1 -- 172.21.15.165:0/2137775034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 msgr2=0x556c00cc7c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/2137775034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 0x556c00cc7c20 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fbb88026780 tx=0x7fbb88030f90 comp rx=0 tx=0).stop 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fbb9c53e800 1 -- 172.21.15.165:0/2137775034 shutdown_connections 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/2137775034 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 0x556c00cc7c20 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fbb9c53e800 1 -- 172.21.15.165:0/2137775034 >> 172.21.15.165:0/2137775034 conn(0x556c00cc3af0 msgr2=0x556c00cc1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.362 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fbb9c53e800 1 -- 172.21.15.165:0/2137775034 shutdown_connections 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fbb9c53e800 1 -- 172.21.15.165:0/2137775034 wait complete. 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fbb9c53e800 1 Processor -- start 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fbb9c53e800 1 -- start start 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fbb9c53e800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 0x556c00ccec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.348+0000 7fbb9c53e800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x556c00cc88e0 con 0x556c00cc7820 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.349+0000 7fbb9823f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 0x556c00ccec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.349+0000 7fbb9823f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 0x556c00ccec20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58536/0 (socket says 172.21.15.165:58536) 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.349+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 learned_addr learned my addr 172.21.15.165:0/1696544059 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.349+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c00ccf160 con 0x556c00cc7820 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.349+0000 7fbb9823f640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 0x556c00ccec20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbb88026780 tx=0x7fbb880272b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.349+0000 7fba75ffb640 1 -- 172.21.15.165:0/1696544059 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb88040e20 con 0x556c00cc7820 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.349+0000 7fba75ffb640 1 -- 172.21.15.165:0/1696544059 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbb88041dd0 con 0x556c00cc7820 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.349+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x556c00ccf160 con 0x556c00cc7820 2024-04-27T03:31:36.363 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.349+0000 7fba75ffb640 1 -- 172.21.15.165:0/1696544059 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbb88040a50 con 0x556c00cc7820 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.349+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c00ccf160 con 0x556c00cc7820 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.350+0000 7fba75ffb640 1 -- 172.21.15.165:0/1696544059 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fbb88030450 con 0x556c00cc7820 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.350+0000 7fba75ffb640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba68036900 0x7fba68038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.350+0000 7fba75ffb640 1 -- 172.21.15.165:0/1696544059 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fbb8806a030 con 0x556c00cc7820 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.350+0000 7fbb8ffff640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba68036900 0x7fba68038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.351+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.351+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c00cda320 0x556c00cdc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.351+0000 7fbb8ffff640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba68036900 0x7fba68038dc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbb80052fd0 tx=0x7fbb8004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.351+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00cdcf30 con 0x556c00cda320 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.351+0000 7fbb98a40640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c00cda320 0x556c00cdc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.351+0000 7fbb98a40640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c00cda320 0x556c00cdc7e0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.351+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.351+0000 7fbb9c53e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.351+0000 7fbb9c53e800 20 realm 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00cdd340 con 0x556c00cda320 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddc10 con 0x556c00cda320 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cda320 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:36.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c00cded10 0x556c00ce1100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce1710 con 0x556c00cded10 2024-04-27T03:31:36.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.352+0000 7fbb9823f640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c00cded10 0x556c00ce1100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.353+0000 7fbb9823f640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c00cded10 0x556c00ce1100 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.353+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.353+0000 7fbb9c53e800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:36.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.353+0000 7fbb9c53e800 20 searching for the correct realm 2024-04-27T03:31:36.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.353+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce3040 con 0x556c00cded10 2024-04-27T03:31:36.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.353+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.353+0000 7fbb9823f640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fbb8801ca60 0x7fbb88013a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.353+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce3040 con 0x7fbb8801ca60 2024-04-27T03:31:36.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.354+0000 7fbb8ffff640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fbb8801ca60 0x7fbb88013a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.354+0000 7fbb8ffff640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fbb8801ca60 0x7fbb88013a60 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.354+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.354+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce3040 con 0x7fbb8801ca60 2024-04-27T03:31:36.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.354+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.354+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce3040 con 0x556c00cda320 2024-04-27T03:31:36.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.354+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.354+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cda320 2024-04-27T03:31:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.355+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.355+0000 7fbb98a40640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fbb90016440 0x7fbb900168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.355+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb90016440 2024-04-27T03:31:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.355+0000 7fbb98a40640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fbb90016440 0x7fbb900168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.355+0000 7fbb98a40640 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fbb90016440 0x7fbb900168c0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.356+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.356+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb90016440 2024-04-27T03:31:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.356+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.356+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb8801ca60 2024-04-27T03:31:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.356+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.356+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cded10 2024-04-27T03:31:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.356+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.356+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cda320 2024-04-27T03:31:36.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.357+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.357+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb8801ca60 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.357+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.357+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cda320 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.357+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.357+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb8801ca60 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.357+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.357+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cded10 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.358+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.358+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cded10 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.358+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.358+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb90016440 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.358+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.358+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cded10 2024-04-27T03:31:36.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.358+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.358+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cded10 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.359+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.359+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cda320 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.359+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.359+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cda320 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.359+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.359+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb90016440 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.359+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.359+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cded10 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.360+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.360+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb8801ca60 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.360+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.360+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cda320 2024-04-27T03:31:36.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.360+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.360+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb90016440 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.360+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.360+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cded10 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.361+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.361+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb90016440 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.361+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.361+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb8801ca60 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.361+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.361+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cded10 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb90016440 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.381 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x556c00cda320 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x556c00cddfb0 con 0x7fbb8801ca60 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb9c53e800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb9c53e800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb9c53e800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb9c53e800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb9c53e800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb9c53e800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.362+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce1fd0 con 0x556c00cda320 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.382 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce23c0 con 0x556c00cda320 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce23c0 con 0x7fbb8801ca60 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 20 zone default found 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 4 Realm: () 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:36.383 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 20 zonegroup default 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.363+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce34c0 con 0x7fbb90016440 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb9c53e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce4ea0 con 0x556c00cded10 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb9c53e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce5320 con 0x556c00cded10 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb9c53e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb9c53e800 20 started sync module instance, tier type = 2024-04-27T03:31:36.384 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb9c53e800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.364+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce4340 con 0x7fbb8801ca60 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.365+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.365+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 93922358347568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce3fa0 con 0x7fbb8801ca60 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.365+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce6140 con 0x556c00cda320 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.366+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 93922358347568] v27'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.367+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.367+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 93922358351744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce6530 con 0x556c00cda320 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.367+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce6f90 con 0x556c00cded10 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.368+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 93922358351744] v27'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.368+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.368+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 93922358356544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce7380 con 0x556c00cded10 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.368+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce7de0 con 0x7fbb8801ca60 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.369+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.369+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 93922358362576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce8aa0 con 0x7fbb8801ca60 2024-04-27T03:31:36.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.369+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce94e0 con 0x556c00cda320 2024-04-27T03:31:36.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.369+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 93922358356544] v27'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.370+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 93922358362576] v27'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.370+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.370+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 93922358363632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce5ba0 con 0x556c00cda320 2024-04-27T03:31:36.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.370+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce9dc0 con 0x7fbb90016440 2024-04-27T03:31:36.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.371+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.371+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 93922358363632] v27'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.371+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 93922358368464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce9d70 con 0x7fbb90016440 2024-04-27T03:31:36.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.371+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cea830 con 0x7fbb90016440 2024-04-27T03:31:36.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.372+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 93922358368464] v27'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.373+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.373+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 93922358366128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cea830 con 0x7fbb90016440 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.373+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ceb1f0 con 0x7fbb90016440 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.374+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 93922358366128] v27'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.374+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'222 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.374+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 93922358370736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ceb1f0 con 0x7fbb90016440 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.374+0000 7fbb9c53e800 20 add_watcher() i=0 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.374+0000 7fbb9c53e800 20 add_watcher() i=1 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.374+0000 7fbb9c53e800 20 add_watcher() i=2 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.374+0000 7fbb9c53e800 20 add_watcher() i=3 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.374+0000 7fbb9c53e800 20 add_watcher() i=4 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.374+0000 7fbb9c53e800 20 add_watcher() i=5 2024-04-27T03:31:36.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.374+0000 7fbb9c53e800 20 add_watcher() i=6 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.375+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 93922358370736] v27'223 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.375+0000 7fbb9c53e800 20 add_watcher() i=7 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.375+0000 7fbb9c53e800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.375+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00cee5f0 con 0x556c00cda320 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.375+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.375+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 93922358374160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cebab0 con 0x556c00cda320 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.376+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 93922358374160] v27'224 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.376+0000 7fba4e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.376+0000 7fbb9c53e800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.376+0000 7fbb9c53e800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.376+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cee2c0 con 0x556c00cda320 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.377+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'56 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.377+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf7620 con 0x7fbb90016440 2024-04-27T03:31:36.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.379+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.379+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf7620 con 0x556c00cded10 2024-04-27T03:31:36.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.380+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'33 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.380+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf6c90 con 0x556c00cda320 2024-04-27T03:31:36.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.381+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'225 uv172 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.381+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf6c90 con 0x7fbb8801ca60 2024-04-27T03:31:36.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.382+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'74 uv59 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.382+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf8ba0 con 0x556c00cded10 2024-04-27T03:31:36.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.383+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'59 uv49 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.383+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf8ba0 con 0x556c00cda320 2024-04-27T03:31:36.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.384+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'34 uv29 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.384+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf8ba0 con 0x7fbb90016440 2024-04-27T03:31:36.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.385+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'51 uv42 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.385+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf8c30 con 0x7fbb8801ca60 2024-04-27T03:31:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.386+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'26 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.386+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf91a0 con 0x556c00cded10 2024-04-27T03:31:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.387+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.387+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf91a0 con 0x7fbb8801ca60 2024-04-27T03:31:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.388+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'75 uv61 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.388+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x7fbb90016440 2024-04-27T03:31:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.389+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'30 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.395 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.389+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x7fbb8801ca60 2024-04-27T03:31:36.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.390+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.396 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.390+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x7fbb90016440 2024-04-27T03:31:36.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.392+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'63 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.397 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.392+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x556c00cda320 2024-04-27T03:31:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.393+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'226 uv168 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.393+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x556c00cded10 2024-04-27T03:31:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.394+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'69 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.394+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x7fbb8801ca60 2024-04-27T03:31:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.395+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'76 uv57 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.395+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x556c00cded10 2024-04-27T03:31:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.396+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'38 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.396+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x556c00cda320 2024-04-27T03:31:36.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.397+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'34 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.405 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.397+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x556c00cda320 2024-04-27T03:31:36.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.398+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'227 uv170 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.398+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x556c00cded10 2024-04-27T03:31:36.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.399+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'70 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.399+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x7fbb8801ca60 2024-04-27T03:31:36.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.400+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'42 uv37 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.400+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x556c00cded10 2024-04-27T03:31:36.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.401+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'34 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.407 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.401+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x556c00cded10 2024-04-27T03:31:36.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.402+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'45 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.402+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x556c00cded10 2024-04-27T03:31:36.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.403+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'28 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.408 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.403+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x556c00cded10 2024-04-27T03:31:36.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.404+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'60 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.409 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.404+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x7fbb90016440 2024-04-27T03:31:36.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.405+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'36 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.410 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.405+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x556c00cded10 2024-04-27T03:31:36.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.406+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'38 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.411 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.406+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x7fbb90016440 2024-04-27T03:31:36.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.407+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'52 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.407+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x556c00cded10 2024-04-27T03:31:36.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.408+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'71 uv56 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.408+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x7fbb90016440 2024-04-27T03:31:36.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.409+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'64 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.409+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf9650 con 0x7fbb8801ca60 2024-04-27T03:31:36.416 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.410+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'32 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fba4d7e2640 20 reqs_thread_entry: start 2024-04-27T03:31:36.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fbb9c53e800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fbb9c53e800 20 Filter name: none 2024-04-27T03:31:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fb9beffd640 20 reqs_thread_entry: start 2024-04-27T03:31:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fbb9c53e800 10 cache get: name=default.rgw.meta+users.uid+bud : miss 2024-04-27T03:31:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:94 8.4 8:267425a8:users.uid::bud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d24630 con 0x7fbb8801ca60 2024-04-27T03:31:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== osd_op_reply(94 bud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv2 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fbb9c53e800 20 rados_obj.operate() r=0 bl.length=380 2024-04-27T03:31:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fbb9c53e800 10 cache put: name=default.rgw.meta+users.uid+bud info.flags=0x13 2024-04-27T03:31:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fbb9c53e800 10 adding default.rgw.meta+users.uid+bud to cache LRU end 2024-04-27T03:31:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fbb9c53e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:31:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.411+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:95 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d27000 con 0x556c00cda320 2024-04-27T03:31:36.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.412+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(95 bud.buckets [call out=184b] v0'0 uv6 ondisk = 0) ==== 155+0+184 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.412+0000 7fbb9c53e800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 : miss 2024-04-27T03:31:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.412+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.412+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:96 8.1a 8:5ae7e5ec:root::.bucket.meta.myfoo6%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d26e60 con 0x7fbb90016440 2024-04-27T03:31:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.413+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(96 .bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 [call out=48b,stat out=16b,read 0~380 out=380b,getxattrs out=212b] v0'0 uv5 ondisk = 0) ==== 333+0+656 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.413+0000 7fbb9c53e800 20 rados_obj.operate() r=0 bl.length=380 2024-04-27T03:31:36.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.413+0000 7fbb9c53e800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 info.flags=0x17 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.413+0000 7fbb9c53e800 10 adding default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 to cache LRU end 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.413+0000 7fbb9c53e800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.413+0000 7fbb9c53e800 10 updating xattr: name=user.rgw.acl bl.length()=117 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.413+0000 7fbb9c53e800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.413+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:97 4.2d 4:b6d005ec:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da4340 con 0x7fbb90016440 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.413+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(97 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.0 [call out=85b] v0'0 uv2 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.413+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:98 4.2 4:42eb879e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da4380 con 0x556c00cda320 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.413+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(98 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.1 [call out=85b] v0'0 uv9 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb9000a4a0 con 0x556c00cda320 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.414+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:99 4.11 4:8b17784a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da4380 con 0x7fbb90016440 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.414+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(99 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb9000a4a0 con 0x7fbb90016440 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.414+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:100 4.39 4:9e6af1e5:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da4380 con 0x556c00cded10 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.414+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(100 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.3 [call out=85b] v0'0 uv7 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.414+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:101 4.36 4:6c26a203:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da4380 con 0x556c00cda320 2024-04-27T03:31:36.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.414+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(101 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.4 [call out=85b] v0'0 uv8 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.415+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:102 4.30 4:0f8f4ebf:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da43c0 con 0x556c00cded10 2024-04-27T03:31:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.415+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(102 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.5 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.415+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:103 4.31 4:8e6b3321:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da43c0 con 0x556c00cda320 2024-04-27T03:31:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.415+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(103 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.6 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.415+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:104 4.2 4:4019d6bb:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da43c0 con 0x556c00cda320 2024-04-27T03:31:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.415+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(104 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.7 [call out=85b] v0'0 uv10 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.415+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:105 4.1 4:809e6ae6:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da43c0 con 0x7fbb8801ca60 2024-04-27T03:31:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.416+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(105 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.416+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:106 4.15 4:aa655c17:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da43c0 con 0x556c00cded10 2024-04-27T03:31:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.416+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 29 ==== osd_op_reply(106 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9 [call out=289b] v0'0 uv9 ondisk = 0) ==== 194+0+289 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.416+0000 7fbb9c53e800 10 cls_bucket_list_unordered: got thirteen 2024-04-27T03:31:36.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.416+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:107 4.2c 4:35f4e7f3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da43c0 con 0x556c00cda320 2024-04-27T03:31:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.416+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(107 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.416+0000 7fbb9c53e800 20 get_obj_state: octx=0x7fbb8007b040 obj=myfoo6:thirteen state=0x556c00da23c8 s->prefetch_data=0 2024-04-27T03:31:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.416+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:108 3.22 3:4798bd04:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9_thirteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da31f0 con 0x556c00cded10 2024-04-27T03:31:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.417+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 30 ==== osd_op_reply(108 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9_thirteen [getxattrs out=818b,stat out=16b] v0'0 uv2 ondisk = 0) ==== 238+0+834 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.417+0000 7fbb9c53e800 10 manifest: total_size = 8 2024-04-27T03:31:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.417+0000 7fbb9c53e800 20 get_obj_state: setting s->obj_tag to _OonFstrqicnbRj44wCdI0h1EiQki5k0 2024-04-27T03:31:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.417+0000 7fbb9c53e800 20 get_obj_state: octx=0x7fbb8007b040 obj=myfoo6:thirteen state=0x556c00da23c8 s->prefetch_data=0 2024-04-27T03:31:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.417+0000 7fbb9c53e800 20 prepare_atomic_modification: state is not atomic. state=0x556c00da23c8 2024-04-27T03:31:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.417+0000 7fbb9c53e800 20 bucket index object: :.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9 2024-04-27T03:31:36.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.417+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:109 4.15 4:aa655c17:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=187b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00da3ab0 con 0x556c00cded10 2024-04-27T03:31:36.434 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.418+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 31 ==== osd_op_reply(109 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9 [stat,call,call] v27'10 uv10 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.418+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:110 3.22 3:4798bd04:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9_thirteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00da3ab0 con 0x556c00cded10 2024-04-27T03:31:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 32 ==== osd_op_reply(110 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9_thirteen [call] v27'3 uv3 ondisk = 0) ==== 196+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:111 4.15 4:aa655c17:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=264b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00da3a20 con 0x556c00cded10 2024-04-27T03:31:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9c53e800 20 list_objects_ordered: starting attempt 1 2024-04-27T03:31:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9c53e800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-27T03:31:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:112 4.2d 4:b6d005ec:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da3a20 con 0x7fbb90016440 2024-04-27T03:31:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:113 4.2 4:42eb879e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d306c0 con 0x556c00cda320 2024-04-27T03:31:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:114 4.11 4:8b17784a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00da3a20 con 0x7fbb90016440 2024-04-27T03:31:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:115 4.39 4:9e6af1e5:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d35890 con 0x556c00cded10 2024-04-27T03:31:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:116 4.36 4:6c26a203:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d35890 con 0x556c00cda320 2024-04-27T03:31:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:117 4.30 4:0f8f4ebf:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d3a950 con 0x556c00cded10 2024-04-27T03:31:36.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:118 4.31 4:8e6b3321:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d3d7d0 con 0x556c00cda320 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:119 4.2 4:4019d6bb:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d40650 con 0x556c00cda320 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.419+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:120 4.1 4:809e6ae6:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d434d0 con 0x7fbb8801ca60 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.420+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:121 4.15 4:aa655c17:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d45e10 con 0x556c00cded10 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.420+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:122 4.2c 4:35f4e7f3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d48c90 con 0x556c00cda320 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.420+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(114 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.420+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(112 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.0 [call out=85b] v0'0 uv2 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.420+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(113 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.1 [call out=85b] v0'0 uv9 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.420+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(116 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.4 [call out=85b] v0'0 uv8 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.420+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 33 ==== osd_op_reply(115 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.3 [call out=85b] v0'0 uv7 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.420+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(118 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.6 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.420+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 34 ==== osd_op_reply(117 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.5 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.420+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 36 ==== osd_op_reply(119 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.7 [call out=85b] v0'0 uv10 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.420+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 37 ==== osd_op_reply(122 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.420+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 23 ==== osd_op_reply(120 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 35 ==== osd_op_reply(111 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9 [stat,call,call] v27'11 uv11 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fba777fe640 20 handle_completion(): completion ok for obj=thirteen 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 36 ==== osd_op_reply(121 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9 [call out=124b] v0'0 uv11 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo6[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9])"; uploads.size()=0; is_truncated=0 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:123 4.2d 4:b6d005ec:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d43530 con 0x7fbb90016440 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:124 4.2 4:42eb879e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d48c90 con 0x556c00cda320 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:125 4.11 4:8b17784a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d48500 con 0x7fbb90016440 2024-04-27T03:31:36.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:126 4.39 4:9e6af1e5:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d35890 con 0x556c00cded10 2024-04-27T03:31:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:127 4.36 4:6c26a203:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d3a300 con 0x556c00cda320 2024-04-27T03:31:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:128 4.30 4:0f8f4ebf:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d48500 con 0x556c00cded10 2024-04-27T03:31:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:129 4.31 4:8e6b3321:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d45df0 con 0x556c00cda320 2024-04-27T03:31:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:130 4.2 4:4019d6bb:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d35890 con 0x556c00cda320 2024-04-27T03:31:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:131 4.1 4:809e6ae6:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d48c90 con 0x7fbb8801ca60 2024-04-27T03:31:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:132 4.15 4:aa655c17:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d3a300 con 0x556c00cded10 2024-04-27T03:31:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:133 4.2c 4:35f4e7f3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d35890 con 0x556c00cda320 2024-04-27T03:31:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(125 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(123 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.0 [call out=85b] v0'0 uv2 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 38 ==== osd_op_reply(124 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.1 [call out=85b] v0'0 uv9 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 39 ==== osd_op_reply(129 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.6 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.421+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 40 ==== osd_op_reply(133 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.422+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 37 ==== osd_op_reply(126 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.3 [call out=85b] v0'0 uv7 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.422+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 38 ==== osd_op_reply(128 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.5 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.422+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 24 ==== osd_op_reply(131 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.422+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 41 ==== osd_op_reply(130 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.7 [call out=85b] v0'0 uv10 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.422+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 42 ==== osd_op_reply(127 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.4 [call out=85b] v0'0 uv8 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.422+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 39 ==== osd_op_reply(132 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9 [call out=124b] v0'0 uv11 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.422+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:134 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.set_buckets_info in=208b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d42e50 con 0x556c00cda320 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.423+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 43 ==== osd_op_reply(134 bud.buckets [call] v27'7 uv7 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.423+0000 7fbb9c53e800 10 cache get: name=default.rgw.meta+root+myfoo6 : miss 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.423+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.423+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:135 8.18 8:1944a257:root::myfoo6:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d42e50 con 0x556c00cda320 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.423+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 44 ==== osd_op_reply(135 myfoo6 [call out=48b,read 0~166 out=166b] v0'0 uv17 ondisk = 0) ==== 192+0+214 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.423+0000 7fbb9c53e800 20 rados_obj.operate() r=0 bl.length=166 2024-04-27T03:31:36.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.423+0000 7fbb9c53e800 10 cache put: name=default.rgw.meta+root+myfoo6 info.flags=0x11 2024-04-27T03:31:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.423+0000 7fbb9c53e800 10 adding default.rgw.meta+root+myfoo6 to cache LRU end 2024-04-27T03:31:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.423+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:136 8.18 8:1944a257:root::myfoo6:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d42e50 con 0x556c00cda320 2024-04-27T03:31:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.424+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 45 ==== osd_op_reply(136 myfoo6 [call,call,delete] v27'18 uv18 ondisk = 0) ==== 234+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.424+0000 7fbb9c53e800 10 removing default.rgw.meta+root+myfoo6 from cache 2024-04-27T03:31:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.424+0000 7fbb9c53e800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo6, ofs0, ns] 2024-04-27T03:31:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.424+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [notify cookie 93922358712272 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d42e50 con 0x7fbb8801ca60 2024-04-27T03:31:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.424+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 25 ==== watch-notify(notify (1) cookie 93922358362576 notify 115964117011 ret 0) ==== 192+0+0 (crc 0 0 0) 0x556c00cf67e0 con 0x7fbb8801ca60 2024-04-27T03:31:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.424+0000 7fba777fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117011 cookie 93922358362576 notifier 4742 bl.length()=150 2024-04-27T03:31:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.424+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 26 ==== osd_op_reply(137 notify.3 [notify cookie 93922358712272 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.424+0000 7fba777fe640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:138 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fba5c003d60 con 0x7fbb8801ca60 2024-04-27T03:31:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.425+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 27 ==== osd_op_reply(138 notify.3 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.425+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 28 ==== watch-notify(notify_complete (2) cookie 93922358712272 notify 115964117011 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556c00cf8fd0 con 0x7fbb8801ca60 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.425+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:139 8.1a 8:5ae7e5ec:root::.bucket.meta.myfoo6%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d42e50 con 0x7fbb90016440 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.426+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== osd_op_reply(139 .bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 [call,call,delete] v27'6 uv6 ondisk = 0) ==== 291+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.426+0000 7fbb9c53e800 10 removing default.rgw.meta+root+.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 from cache 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.426+0000 7fbb9c53e800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo6:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9, ofs0, ns] 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.426+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:140 7.d 7:b4812045:::notify.4:head [notify cookie 93922358712272 in=219b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d42e50 con 0x556c00cda320 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.426+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 46 ==== watch-notify(notify (1) cookie 93922358363632 notify 115964117034 ret 0) ==== 249+0+0 (crc 0 0 0) 0x556c00d263b0 con 0x556c00cda320 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.426+0000 7fba76ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117034 cookie 93922358363632 notifier 4742 bl.length()=207 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.426+0000 7fba76ffd640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:141 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fb9cc0032c0 con 0x556c00cda320 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.426+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 47 ==== osd_op_reply(140 notify.4 [notify cookie 93922358712272 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.426+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 48 ==== osd_op_reply(141 notify.4 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.426+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 49 ==== watch-notify(notify_complete (2) cookie 93922358712272 notify 115964117034 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556c00cf6610 con 0x556c00cda320 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.426+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:142 4.2d 4:b6d005ec:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d42e50 con 0x7fbb90016440 2024-04-27T03:31:36.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.427+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:143 4.2 4:42eb879e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d251e0 con 0x556c00cda320 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.427+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:144 4.11 4:8b17784a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d25a30 con 0x7fbb90016440 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.427+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:145 4.39 4:9e6af1e5:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d25e50 con 0x556c00cded10 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.427+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:146 4.36 4:6c26a203:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d42e50 con 0x556c00cda320 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.427+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:147 4.30 4:0f8f4ebf:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d34270 con 0x556c00cded10 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.427+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:148 4.31 4:8e6b3321:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d25a30 con 0x556c00cda320 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.427+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:149 4.2 4:4019d6bb:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d251e0 con 0x556c00cda320 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.427+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:150 4.1 4:809e6ae6:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d41c90 con 0x7fbb8801ca60 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.427+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:151 4.15 4:aa655c17:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d25e50 con 0x556c00cded10 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.427+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:152 4.2c 4:35f4e7f3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d25a30 con 0x556c00cda320 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.428+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 40 ==== osd_op_reply(147 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.5 [delete] v27'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.429+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 41 ==== osd_op_reply(151 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.9 [delete] v27'12 uv12 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.429+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 30 ==== osd_op_reply(144 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.2 [delete] v27'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.429+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 29 ==== osd_op_reply(150 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.8 [delete] v27'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.429+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 31 ==== osd_op_reply(142 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.0 [delete] v27'5 uv5 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.429+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 50 ==== osd_op_reply(152 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.10 [delete] v27'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.429+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 51 ==== osd_op_reply(148 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.6 [delete] v27'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.429+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 42 ==== osd_op_reply(145 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.3 [delete] v27'9 uv9 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.429+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 52 ==== osd_op_reply(143 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.1 [delete] v27'11 uv11 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.443 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.429+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 53 ==== osd_op_reply(146 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.4 [delete] v27'9 uv9 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.430+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 54 ==== osd_op_reply(149 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9.7 [delete] v27'12 uv12 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.430+0000 7fbb9c53e800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo6/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 : miss 2024-04-27T03:31:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.430+0000 7fbb9c53e800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.430+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:153 6.10 6:0fbfabd9:::pubsub..bucket.myfoo6%2f708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d25e10 con 0x7fbb90016440 2024-04-27T03:31:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.430+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 32 ==== osd_op_reply(153 pubsub..bucket.myfoo6/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 209+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.430+0000 7fbb9c53e800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.430+0000 7fbb9c53e800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo6/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 info.flags=0x0 2024-04-27T03:31:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.430+0000 7fbb9c53e800 10 adding default.rgw.log++pubsub..bucket.myfoo6/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 to cache LRU end 2024-04-27T03:31:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.430+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:154 6.10 6:0fbfabd9:::pubsub..bucket.myfoo6%2f708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d25e10 con 0x7fbb90016440 2024-04-27T03:31:36.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.432+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 33 ==== osd_op_reply(154 pubsub..bucket.myfoo6/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.9 [delete] v27'6 uv5 ondisk = -2 ((2) No such file or directory)) ==== 209+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.432+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:155 8.19 8:99adfd5d:users.uid::bud.buckets:head [call user.remove_bucket in=145b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d25e10 con 0x556c00cda320 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.433+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 55 ==== osd_op_reply(155 bud.buckets [call] v27'8 uv8 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.433+0000 7fbb9c53e800 10 removing key index: NX5QOQKC6BH2IDN8HC7A 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.433+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:156 8.a 8:50c8eddb:users.keys::NX5QOQKC6BH2IDN8HC7A:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d3d7d0 con 0x556c00cda320 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.433+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 56 ==== osd_op_reply(156 NX5QOQKC6BH2IDN8HC7A [delete] v27'5 uv5 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.434+0000 7fbb9c53e800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:NX5QOQKC6BH2IDN8HC7A, ofs0, ns] 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.434+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:157 7.3 7:c609908c:::notify.5:head [notify cookie 93922358619600 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d42e50 con 0x7fbb90016440 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.434+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 34 ==== watch-notify(notify (1) cookie 93922358368464 notify 115964117014 ret 0) ==== 212+0+0 (crc 0 0 0) 0x556c00cf6610 con 0x7fbb90016440 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.434+0000 7fba76ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117014 cookie 93922358368464 notifier 4742 bl.length()=170 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.434+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 35 ==== osd_op_reply(157 notify.5 [notify cookie 93922358619600 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.434+0000 7fba76ffd640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:158 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fb9cc0032a0 con 0x7fbb90016440 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.434+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 36 ==== osd_op_reply(158 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.434+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 37 ==== watch-notify(notify_complete (2) cookie 93922358619600 notify 115964117014 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556c00cee430 con 0x7fbb90016440 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.434+0000 7fbb9c53e800 10 removing email index: 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.434+0000 7fbb9c53e800 10 removing user buckets index 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.434+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:159 8.19 8:99adfd5d:users.uid::bud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d3d7d0 con 0x556c00cda320 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.435+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 57 ==== osd_op_reply(159 bud.buckets [delete] v27'9 uv9 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.435+0000 7fbb9c53e800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:bud.buckets, ofs0, ns] 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.435+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:160 7.14 7:2b04a3e9:::notify.6:head [notify cookie 93922358619600 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d42e50 con 0x7fbb90016440 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.436+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 38 ==== watch-notify(notify (1) cookie 93922358366128 notify 115964117015 ret 0) ==== 202+0+0 (crc 0 0 0) 0x556c00cee430 con 0x7fbb90016440 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.436+0000 7fba777fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117015 cookie 93922358366128 notifier 4742 bl.length()=160 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.436+0000 7fba777fe640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:161 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fba5c003d60 con 0x7fbb90016440 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.436+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 39 ==== osd_op_reply(160 notify.6 [notify cookie 93922358619600 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.436+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 40 ==== watch-notify(notify_complete (2) cookie 93922358619600 notify 115964117015 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556c00cedd70 con 0x7fbb90016440 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.436+0000 7fbb9c53e800 10 removing user index: bud 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.436+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 41 ==== osd_op_reply(161 notify.6 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.436+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:162 8.4 8:267425a8:users.uid::bud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00d3d7d0 con 0x7fbb8801ca60 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.437+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 30 ==== osd_op_reply(162 bud [call,call,delete] v27'3 uv3 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.437+0000 7fbb9c53e800 10 removing default.rgw.meta+users.uid+bud from cache 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.437+0000 7fbb9c53e800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:bud, ofs0, ns] 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.437+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:163 7.3 7:c609908c:::notify.5:head [notify cookie 93922358619600 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x556c00d42e50 con 0x7fbb90016440 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.437+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 42 ==== watch-notify(notify (1) cookie 93922358368464 notify 115964117016 ret 0) ==== 194+0+0 (crc 0 0 0) 0x556c00cedd70 con 0x7fbb90016440 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.437+0000 7fba76ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117016 cookie 93922358368464 notifier 4742 bl.length()=152 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.437+0000 7fba76ffd640 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fb9cc0032a0 con 0x7fbb90016440 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.437+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 43 ==== osd_op_reply(163 notify.5 [notify cookie 93922358619600 out=8b] v0'0 uv222 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.438+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 44 ==== osd_op_reply(164 notify.5 [notify-ack] v0'0 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.438+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 45 ==== watch-notify(notify_complete (2) cookie 93922358619600 notify 115964117016 ret 0) ==== 42+0+48 (crc 0 0 0) 0x556c00cee660 con 0x7fbb90016440 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.438+0000 7fba4e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.439+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:165 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 93922358374160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00cf5850 con 0x556c00cda320 2024-04-27T03:31:36.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.440+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 58 ==== osd_op_reply(165 data_loggenerations_metadata [watch unwatch cookie 93922358374160] v27'228 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.440+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:166 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 93922358347568] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ceaee0 con 0x7fbb8801ca60 2024-04-27T03:31:36.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.441+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 31 ==== osd_op_reply(166 notify.0 [watch unwatch cookie 93922358347568] v27'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.441+0000 7fbb9c53e800 20 remove_watcher() i=0 2024-04-27T03:31:36.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.441+0000 7fbb9c53e800 2 removed watcher, disabling cache 2024-04-27T03:31:36.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.441+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:167 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 93922358351744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce4730 con 0x556c00cda320 2024-04-27T03:31:36.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.442+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 59 ==== osd_op_reply(167 notify.1 [watch unwatch cookie 93922358351744] v27'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.442+0000 7fbb9c53e800 20 remove_watcher() i=1 2024-04-27T03:31:36.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.442+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:168 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 93922358356544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce4730 con 0x556c00cded10 2024-04-27T03:31:36.456 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.443+0000 7fbb9823f640 1 -- 172.21.15.165:0/1696544059 <== osd.3 v2:172.21.15.165:6808/2380918637 43 ==== osd_op_reply(168 notify.2 [watch unwatch cookie 93922358356544] v27'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb88048040 con 0x556c00cded10 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.443+0000 7fbb9c53e800 20 remove_watcher() i=2 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.443+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:169 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 93922358362576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce4730 con 0x7fbb8801ca60 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.444+0000 7fbb8ffff640 1 -- 172.21.15.165:0/1696544059 <== osd.2 v2:172.21.15.165:6816/975457786 32 ==== osd_op_reply(169 notify.3 [watch unwatch cookie 93922358362576] v27'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb80081070 con 0x7fbb8801ca60 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.444+0000 7fbb9c53e800 20 remove_watcher() i=3 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.444+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:170 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 93922358363632] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce81d0 con 0x556c00cda320 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.445+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.1 v2:172.21.15.165:6800/79879041 60 ==== osd_op_reply(170 notify.4 [watch unwatch cookie 93922358363632] v27'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x556c00cda320 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.445+0000 7fbb9c53e800 20 remove_watcher() i=4 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.445+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:171 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 93922358368464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce4730 con 0x7fbb90016440 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.447+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 46 ==== osd_op_reply(171 notify.5 [watch unwatch cookie 93922358368464] v27'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.447+0000 7fbb9c53e800 20 remove_watcher() i=5 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.447+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:172 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 93922358366128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce98d0 con 0x7fbb90016440 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.448+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 47 ==== osd_op_reply(172 notify.6 [watch unwatch cookie 93922358366128] v27'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.457 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.448+0000 7fbb9c53e800 20 remove_watcher() i=6 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.448+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:173 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 93922358370736] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x556c00ce4730 con 0x7fbb90016440 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 <== osd.0 v2:172.21.15.89:6800/2353668432 48 ==== osd_op_reply(173 notify.7 [watch unwatch cookie 93922358370736] v27'224 uv222 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbb90050060 con 0x7fbb90016440 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 20 remove_watcher() i=7 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fbb90016440 msgr2=0x7fbb900168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fbb90016440 0x7fbb900168c0 crc :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c00cda320 msgr2=0x556c00cdc7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c00cda320 0x556c00cdc7e0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fbb8801ca60 msgr2=0x7fbb88013a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fbb8801ca60 0x7fbb88013a60 crc :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c00cded10 msgr2=0x556c00ce1100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c00cded10 0x556c00ce1100 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba68036900 msgr2=0x7fba68038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba68036900 0x7fba68038dc0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbb80052fd0 tx=0x7fbb8004f000 comp rx=0 tx=0).stop 2024-04-27T03:31:36.458 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 msgr2=0x556c00ccec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb98a40640 1 -- 172.21.15.165:0/1696544059 reap_dead start 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 0x556c00ccec20 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fbb88026780 tx=0x7fbb880272b0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 shutdown_connections 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/1696544059 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cc7820 0x556c00ccec20 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 >> 172.21.15.165:0/1696544059 conn(0x556c00cc3af0 msgr2=0x556c00cc1150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 shutdown_connections 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/1696544059 wait complete. 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/3209572564 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba70036970 msgr2=0x7fba70038e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/3209572564 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba70036970 0x7fba70038e30 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x556c00ca0430 tx=0x7fbb80005a00 comp rx=0 tx=0).stop 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/3209572564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cba6a0 msgr2=0x556c00cbaac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/3209572564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cba6a0 0x556c00cbaac0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fbb8804b4b0 tx=0x7fbb8801aaf0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/3209572564 shutdown_connections 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/3209572564 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba70036970 0x7fba70038e30 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.459 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/3209572564 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00cba6a0 0x556c00cbaac0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.449+0000 7fbb9c53e800 1 -- 172.21.15.165:0/3209572564 >> 172.21.15.165:0/3209572564 conn(0x556c00cae740 msgr2=0x556c00caeb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 -- 172.21.15.165:0/3209572564 shutdown_connections 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 -- 172.21.15.165:0/3209572564 wait complete. 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c00ba54a0 msgr2=0x556c00ca2f40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x556c00ba54a0 0x556c00ca2f40 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x556c00ca96e0 msgr2=0x556c00cabb30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x556c00ca96e0 0x556c00cabb30 crc :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c00ca57a0 msgr2=0x556c00ca7b90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x556c00ca57a0 0x556c00ca7b90 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba80036900 msgr2=0x7fba80038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fba80036900 0x7fba80038dc0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fbb80054000 tx=0x7fbb8004e000 comp rx=0 tx=0).stop 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 msgr2=0x556c00c9bd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 --2- 172.21.15.165:0/900589583 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x556c00ba1990 0x556c00c9bd00 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fbb8800b210 tx=0x7fbb88000ed0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb98a40640 1 -- 172.21.15.165:0/900589583 reap_dead start 2024-04-27T03:31:36.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 shutdown_connections 2024-04-27T03:31:36.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 >> 172.21.15.165:0/900589583 conn(0x556c00b9b340 msgr2=0x556c00ba0170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 shutdown_connections 2024-04-27T03:31:36.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.450+0000 7fbb9c53e800 1 -- 172.21.15.165:0/900589583 wait complete. 2024-04-27T03:31:36.464 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'fud', '--purge-data'] 2024-04-27T03:31:36.464 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'fud', '--purge-data'] 2024-04-27T03:31:36.464 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid fud --purge-data 2024-04-27T03:31:36.548 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:31:36.548 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:31:36.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.543+0000 7f7aa4496800 1 Processor -- start 2024-04-27T03:31:36.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.543+0000 7f7aa4496800 1 -- start start 2024-04-27T03:31:36.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.543+0000 7f7aa4496800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d21ab2b7a0 0x55d21ab2bbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.543+0000 7f7aa4496800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d21a9ebd40 con 0x55d21ab2c100 2024-04-27T03:31:36.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.543+0000 7f7aa4496800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55d21a9be4d0 con 0x55d21ab2b7a0 2024-04-27T03:31:36.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.544+0000 7f7aa1f95640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d21ab2b7a0 0x55d21ab2bbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.544+0000 7f7a99794640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55d21ab2c100 0x55d21ab2c5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:41208/0 (socket says 172.21.15.165:41208) 2024-04-27T03:31:36.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.544+0000 7f7a99794640 1 -- 172.21.15.165:0/4155620186 learned_addr learned my addr 172.21.15.165:0/4155620186 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:31:36.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.544+0000 7f7aa1f95640 1 -- 172.21.15.165:0/4155620186 >> v1:172.21.15.89:6789/0 conn(0x55d21ab2c100 legacy=0x55d21ab2c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.544+0000 7f7aa1f95640 1 -- 172.21.15.165:0/4155620186 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d21a9ea590 con 0x55d21ab2b7a0 2024-04-27T03:31:36.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.546+0000 7f7aa1f95640 1 --2- 172.21.15.165:0/4155620186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ab2bbc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400a470 tx=0x7f7a9401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6142c13946613ce server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.546+0000 7f7aa1794640 1 -- 172.21.15.165:0/4155620186 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a94057070 con 0x55d21ab2b7a0 2024-04-27T03:31:36.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.546+0000 7f7aa1794640 1 -- 172.21.15.165:0/4155620186 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a9404adc0 con 0x55d21ab2b7a0 2024-04-27T03:31:36.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.546+0000 7f7aa1794640 1 -- 172.21.15.165:0/4155620186 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a94052590 con 0x55d21ab2b7a0 2024-04-27T03:31:36.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.547+0000 7f7aa4496800 1 -- 172.21.15.165:0/4155620186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 msgr2=0x55d21ab2bbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.547+0000 7f7aa4496800 1 --2- 172.21.15.165:0/4155620186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ab2bbc0 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f7a9400a470 tx=0x7f7a9401ce90 comp rx=0 tx=0).stop 2024-04-27T03:31:36.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.547+0000 7f7aa4496800 1 -- 172.21.15.165:0/4155620186 shutdown_connections 2024-04-27T03:31:36.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.547+0000 7f7aa4496800 1 --2- 172.21.15.165:0/4155620186 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ab2bbc0 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.547+0000 7f7aa4496800 1 -- 172.21.15.165:0/4155620186 >> 172.21.15.165:0/4155620186 conn(0x55d21ab273a0 msgr2=0x55d21ab297c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.547+0000 7f7aa4496800 1 -- 172.21.15.165:0/4155620186 shutdown_connections 2024-04-27T03:31:36.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.547+0000 7f7aa4496800 1 -- 172.21.15.165:0/4155620186 wait complete. 2024-04-27T03:31:36.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.558+0000 7f7aa4496800 1 Processor -- start 2024-04-27T03:31:36.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.558+0000 7f7aa4496800 1 -- start start 2024-04-27T03:31:36.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.559+0000 7f7aa4496800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ac20390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.559+0000 7f7aa4496800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d21aa783b0 con 0x55d21ab2b7a0 2024-04-27T03:31:36.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.559+0000 7f7aa1f95640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ac20390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.559+0000 7f7aa1f95640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ac20390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58560/0 (socket says 172.21.15.165:58560) 2024-04-27T03:31:36.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.559+0000 7f7aa1f95640 1 -- 172.21.15.165:0/1112120228 learned_addr learned my addr 172.21.15.165:0/1112120228 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.559+0000 7f7aa1f95640 1 -- 172.21.15.165:0/1112120228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d21ac20960 con 0x55d21ab2b7a0 2024-04-27T03:31:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.559+0000 7f7aa1f95640 1 --2- 172.21.15.165:0/1112120228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ac20390 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7a94055040 tx=0x7f7a9404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f415546d7885a03 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.560+0000 7f798fe07640 1 -- 172.21.15.165:0/1112120228 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a94057040 con 0x55d21ab2b7a0 2024-04-27T03:31:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.560+0000 7f798fe07640 1 -- 172.21.15.165:0/1112120228 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a94062070 con 0x55d21ab2b7a0 2024-04-27T03:31:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.560+0000 7f798fe07640 1 -- 172.21.15.165:0/1112120228 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a9405d470 con 0x55d21ab2b7a0 2024-04-27T03:31:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.560+0000 7f7aa4496800 1 -- 172.21.15.165:0/1112120228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 msgr2=0x55d21ac20390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.560+0000 7f7aa4496800 1 --2- 172.21.15.165:0/1112120228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ac20390 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f7a94055040 tx=0x7f7a9404c000 comp rx=0 tx=0).stop 2024-04-27T03:31:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.560+0000 7f7aa4496800 1 -- 172.21.15.165:0/1112120228 shutdown_connections 2024-04-27T03:31:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.560+0000 7f7aa4496800 1 --2- 172.21.15.165:0/1112120228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ac20390 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.560+0000 7f7aa4496800 1 -- 172.21.15.165:0/1112120228 >> 172.21.15.165:0/1112120228 conn(0x55d21ab273a0 msgr2=0x55d21ac1bcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.560+0000 7f7aa4496800 1 -- 172.21.15.165:0/1112120228 shutdown_connections 2024-04-27T03:31:36.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.560+0000 7f7aa4496800 1 -- 172.21.15.165:0/1112120228 wait complete. 2024-04-27T03:31:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.561+0000 7f7aa4496800 1 Processor -- start 2024-04-27T03:31:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.561+0000 7f7aa4496800 1 -- start start 2024-04-27T03:31:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.561+0000 7f7aa4496800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ac27d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.561+0000 7f7aa4496800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d21ab2cf00 con 0x55d21ab2b7a0 2024-04-27T03:31:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.561+0000 7f7aa1f95640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ac27d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.561+0000 7f7aa1f95640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ac27d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58564/0 (socket says 172.21.15.165:58564) 2024-04-27T03:31:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.561+0000 7f7aa1f95640 1 -- 172.21.15.165:0/1854296258 learned_addr learned my addr 172.21.15.165:0/1854296258 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.561+0000 7f7aa1f95640 1 -- 172.21.15.165:0/1854296258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d21ac282d0 con 0x55d21ab2b7a0 2024-04-27T03:31:36.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.562+0000 7f7aa1f95640 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ac27d90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7a9405f040 tx=0x7f7a94002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.562+0000 7f798e604640 1 -- 172.21.15.165:0/1854296258 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a9406b070 con 0x55d21ab2b7a0 2024-04-27T03:31:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.562+0000 7f798e604640 1 -- 172.21.15.165:0/1854296258 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a9405d960 con 0x55d21ab2b7a0 2024-04-27T03:31:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.562+0000 7f798e604640 1 -- 172.21.15.165:0/1854296258 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a94057070 con 0x55d21ab2b7a0 2024-04-27T03:31:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.562+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d21ac28560 con 0x55d21ab2b7a0 2024-04-27T03:31:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.562+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d21ac2b210 con 0x55d21ab2b7a0 2024-04-27T03:31:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.563+0000 7f798e604640 1 -- 172.21.15.165:0/1854296258 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f7a9405dc80 con 0x55d21ab2b7a0 2024-04-27T03:31:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.563+0000 7f798e604640 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7988036950 0x7f7988038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.563+0000 7f798e604640 1 -- 172.21.15.165:0/1854296258 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f7a9408e150 con 0x55d21ab2b7a0 2024-04-27T03:31:36.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.563+0000 7f7a99794640 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7988036950 0x7f7988038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.563+0000 7f7aa4496800 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d21ab2c100 0x55d21ac2efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.563+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac2f510 con 0x55d21ab2c100 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.563+0000 7f7aa2796640 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d21ab2c100 0x55d21ac2efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.563+0000 7f7a99794640 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7988036950 0x7f7988038e10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c000f60 tx=0x7f7a8c0012b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.563+0000 7f7aa2796640 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d21ab2c100 0x55d21ac2efd0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.564+0000 7f7aa2796640 1 -- 172.21.15.165:0/1854296258 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ab2c100 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.564+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac2c580 con 0x55d21ab2c100 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.564+0000 7f7aa2796640 1 -- 172.21.15.165:0/1854296258 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ab2c100 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.564+0000 7f7aa4496800 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d21ac317e0 0x55d21ac33bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.564+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac342a0 con 0x55d21ac317e0 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.564+0000 7f7aa1f95640 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d21ac317e0 0x55d21ac33bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.564+0000 7f7aa1f95640 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d21ac317e0 0x55d21ac33bd0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.565+0000 7f7aa1f95640 1 -- 172.21.15.165:0/1854296258 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f7a9405a040 con 0x55d21ac317e0 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.565+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac30790 con 0x55d21ab2c100 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.565+0000 7f7aa2796640 1 -- 172.21.15.165:0/1854296258 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ab2c100 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.565+0000 7f7aa4496800 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55d21ac356e0 0x55d21ac37ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.565+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac38330 con 0x55d21ac356e0 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.565+0000 7f7a99794640 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55d21ac356e0 0x55d21ac37ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.565+0000 7f7a99794640 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55d21ac356e0 0x55d21ac37ba0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.566+0000 7f7a99794640 1 -- 172.21.15.165:0/1854296258 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f7a8c009050 con 0x55d21ac356e0 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.566+0000 7f7aa4496800 1 Processor -- start 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.566+0000 7f7aa4496800 1 -- start start 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.566+0000 7f7aa4496800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac467e0 0x55d21ac46c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.566+0000 7f7aa4496800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d21aa1e6b0 con 0x55d21ac467e0 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.566+0000 7f7aa1f95640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac467e0 0x55d21ac46c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.566+0000 7f7aa1f95640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac467e0 0x55d21ac46c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58570/0 (socket says 172.21.15.165:58570) 2024-04-27T03:31:36.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.566+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2013271347 learned_addr learned my addr 172.21.15.165:0/2013271347 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.567+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2013271347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d21ac47140 con 0x55d21ac467e0 2024-04-27T03:31:36.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.567+0000 7f7aa1f95640 1 --2- 172.21.15.165:0/2013271347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac467e0 0x55d21ac46c00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7a94000c00 tx=0x7f7a94078850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.567+0000 7f79877fe640 1 -- 172.21.15.165:0/2013271347 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a94080040 con 0x55d21ac467e0 2024-04-27T03:31:36.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.567+0000 7f79877fe640 1 -- 172.21.15.165:0/2013271347 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a9405b370 con 0x55d21ac467e0 2024-04-27T03:31:36.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.567+0000 7f7aa4496800 1 -- 172.21.15.165:0/2013271347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d21ac47bb0 con 0x55d21ac467e0 2024-04-27T03:31:36.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.567+0000 7f79877fe640 1 -- 172.21.15.165:0/2013271347 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a94060a00 con 0x55d21ac467e0 2024-04-27T03:31:36.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.567+0000 7f7aa4496800 1 -- 172.21.15.165:0/2013271347 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d21ac480b0 con 0x55d21ac467e0 2024-04-27T03:31:36.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.568+0000 7f79877fe640 1 -- 172.21.15.165:0/2013271347 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f7a94060bc0 con 0x55d21ac467e0 2024-04-27T03:31:36.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.568+0000 7f79877fe640 1 --2- 172.21.15.165:0/2013271347 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f797c036950 0x7f797c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.568+0000 7f79877fe640 1 -- 172.21.15.165:0/2013271347 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f7a9407c360 con 0x55d21ac467e0 2024-04-27T03:31:36.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.568+0000 7f7a99794640 1 --2- 172.21.15.165:0/2013271347 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f797c036950 0x7f797c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.568+0000 7f7aa4496800 1 Processor -- start 2024-04-27T03:31:36.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.568+0000 7f7aa4496800 1 -- start start 2024-04-27T03:31:36.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.568+0000 7f7a99794640 1 --2- 172.21.15.165:0/2013271347 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f797c036950 0x7f797c038e10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00bf50 tx=0x7f7a8c00f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.569+0000 7f7aa4496800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 0x55d21ac53d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.569+0000 7f7aa4496800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d21ac20c20 con 0x55d21ac53920 2024-04-27T03:31:36.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.569+0000 7f7aa1f95640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 0x55d21ac53d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.569+0000 7f7aa1f95640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 0x55d21ac53d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58572/0 (socket says 172.21.15.165:58572) 2024-04-27T03:31:36.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.569+0000 7f7aa1f95640 1 -- 172.21.15.165:0/1179285451 learned_addr learned my addr 172.21.15.165:0/1179285451 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.569+0000 7f7aa1f95640 1 -- 172.21.15.165:0/1179285451 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d21ac54310 con 0x55d21ac53920 2024-04-27T03:31:36.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.569+0000 7f7aa1f95640 1 --2- 172.21.15.165:0/1179285451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 0x55d21ac53d40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7a94062040 tx=0x7f7a94079770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f02d8280b6f34324 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.569+0000 7f79857fa640 1 -- 172.21.15.165:0/1179285451 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a940720c0 con 0x55d21ac53920 2024-04-27T03:31:36.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.569+0000 7f79857fa640 1 -- 172.21.15.165:0/1179285451 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a9407c120 con 0x55d21ac53920 2024-04-27T03:31:36.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.569+0000 7f79857fa640 1 -- 172.21.15.165:0/1179285451 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a9405bd00 con 0x55d21ac53920 2024-04-27T03:31:36.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.570+0000 7f7aa4496800 1 -- 172.21.15.165:0/1179285451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 msgr2=0x55d21ac53d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.570+0000 7f7aa4496800 1 --2- 172.21.15.165:0/1179285451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 0x55d21ac53d40 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f7a94062040 tx=0x7f7a94079770 comp rx=0 tx=0).stop 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.570+0000 7f7aa4496800 1 -- 172.21.15.165:0/1179285451 shutdown_connections 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.570+0000 7f7aa4496800 1 --2- 172.21.15.165:0/1179285451 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 0x55d21ac53d40 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.570+0000 7f7aa4496800 1 -- 172.21.15.165:0/1179285451 >> 172.21.15.165:0/1179285451 conn(0x55d21ac4fc80 msgr2=0x55d21ac4dce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.570+0000 7f7aa4496800 1 -- 172.21.15.165:0/1179285451 shutdown_connections 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.570+0000 7f7aa4496800 1 -- 172.21.15.165:0/1179285451 wait complete. 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.570+0000 7f7aa4496800 1 Processor -- start 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.570+0000 7f7aa4496800 1 -- start start 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.570+0000 7f7aa4496800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 0x55d21ac5ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.570+0000 7f7aa4496800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d21ac549c0 con 0x55d21ac53920 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.570+0000 7f7aa1f95640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 0x55d21ac5ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.571+0000 7f7aa1f95640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 0x55d21ac5ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:58586/0 (socket says 172.21.15.165:58586) 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.571+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 learned_addr learned my addr 172.21.15.165:0/2877415245 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:31:36.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.571+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d21ac5b250 con 0x55d21ac53920 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.571+0000 7f7aa1f95640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 0x55d21ac5ad10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7a9407c2d0 tx=0x7f7a94051f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.571+0000 7f79737fe640 1 -- 172.21.15.165:0/2877415245 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a940b3070 con 0x55d21ac53920 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.571+0000 7f79737fe640 1 -- 172.21.15.165:0/2877415245 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7a94079950 con 0x55d21ac53920 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.571+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d21ac5b250 con 0x55d21ac53920 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.571+0000 7f79737fe640 1 -- 172.21.15.165:0/2877415245 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7a94090e40 con 0x55d21ac53920 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.571+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d21ac5b250 con 0x55d21ac53920 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.572+0000 7f79737fe640 1 -- 172.21.15.165:0/2877415245 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f7a940b6070 con 0x55d21ac53920 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.572+0000 7f79737fe640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f796c036900 0x7f796c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.572+0000 7f79737fe640 1 -- 172.21.15.165:0/2877415245 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f7a940a1070 con 0x55d21ac53920 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.572+0000 7f7a99794640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f796c036900 0x7f796c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.573+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.573+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d21ac664b0 0x55d21ac68970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.573+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac690c0 con 0x55d21ac664b0 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.573+0000 7f7aa2796640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d21ac664b0 0x55d21ac68970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.573+0000 7f7a99794640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f796c036900 0x7f796c038dc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c001b60 tx=0x7f7a8c010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.573+0000 7f7aa2796640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d21ac664b0 0x55d21ac68970 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.573+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.573+0000 7f7aa4496800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.573+0000 7f7aa4496800 20 realm 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.573+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.573+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac694d0 con 0x55d21ac664b0 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa4496800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa4496800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac69da0 con 0x55d21ac664b0 2024-04-27T03:31:36.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa4496800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac664b0 2024-04-27T03:31:36.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa4496800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:36.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d21ac6aea0 0x55d21ac6d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6d850 con 0x55d21ac6aea0 2024-04-27T03:31:36.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa1f95640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d21ac6aea0 0x55d21ac6d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.574+0000 7f7aa1f95640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d21ac6aea0 0x55d21ac6d290 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.575+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.575+0000 7f7aa4496800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.575+0000 7f7aa4496800 20 searching for the correct realm 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.575+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6f1b0 con 0x55d21ac6aea0 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.575+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.575+0000 7f7aa1f95640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7a940b8070 0x7f7a940b3210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.575+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6f1b0 con 0x7f7a940b8070 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.575+0000 7f7a99794640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7a940b8070 0x7f7a940b3210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.576+0000 7f7a99794640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7a940b8070 0x7f7a940b3210 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.576+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.576+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6f1b0 con 0x7f7a940b8070 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.576+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.576+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6f1b0 con 0x55d21ac664b0 2024-04-27T03:31:36.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.576+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.576+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac664b0 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.577+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.577+0000 7f7aa2796640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7a9c016440 0x7f7a9c0168a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.577+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a9c016440 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.577+0000 7f7aa2796640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7a9c016440 0x7f7a9c0168a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.577+0000 7f7aa2796640 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7a9c016440 0x7f7a9c0168a0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.577+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.577+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a9c016440 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.578+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.578+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a940b8070 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.578+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.578+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac6aea0 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.578+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.578+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac664b0 2024-04-27T03:31:36.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.578+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.578+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a940b8070 2024-04-27T03:31:36.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.579+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.579+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac664b0 2024-04-27T03:31:36.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.579+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.579+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a940b8070 2024-04-27T03:31:36.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.579+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.579+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac6aea0 2024-04-27T03:31:36.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.579+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.579+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac6aea0 2024-04-27T03:31:36.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.580+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.580+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a9c016440 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.580+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.580+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac6aea0 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.580+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.580+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac6aea0 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.580+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.580+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac664b0 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.580+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.581+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac664b0 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.581+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.581+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a9c016440 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.581+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.581+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac6aea0 2024-04-27T03:31:36.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.581+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.581+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a940b8070 2024-04-27T03:31:36.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.582+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.582+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac664b0 2024-04-27T03:31:36.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.582+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.582+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a9c016440 2024-04-27T03:31:36.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.582+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.582+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac6aea0 2024-04-27T03:31:36.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.582+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.582+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a9c016440 2024-04-27T03:31:36.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.583+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.583+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a940b8070 2024-04-27T03:31:36.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.583+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.583+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac6aea0 2024-04-27T03:31:36.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.583+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.583+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a9c016440 2024-04-27T03:31:36.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.583+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.583+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x55d21ac664b0 2024-04-27T03:31:36.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6a140 con 0x7f7a940b8070 2024-04-27T03:31:36.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:31:36.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:31:36.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6dc60 con 0x55d21ac664b0 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6e050 con 0x55d21ac664b0 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.584+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6e050 con 0x7f7a940b8070 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 20 zone default found 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 4 Realm: () 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 20 zonegroup default 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6f630 con 0x7f7a9c016440 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac6e580 con 0x55d21ac6aea0 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.585+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.586+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac715a0 con 0x55d21ac6aea0 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.586+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.586+0000 7f7aa4496800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.586+0000 7f7aa4496800 20 started sync module instance, tier type = 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.586+0000 7f7aa4496800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.586+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac70950 con 0x7f7a940b8070 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.587+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.587+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94360880745792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac71eb0 con 0x7f7a940b8070 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.587+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac72700 con 0x55d21ac664b0 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.588+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94360880745792] v27'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.588+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.588+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94360880749056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac72760 con 0x55d21ac664b0 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.588+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac73250 con 0x55d21ac6aea0 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.589+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94360880749056] v27'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.589+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.589+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94360880753952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac73670 con 0x55d21ac6aea0 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.589+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac74090 con 0x7f7a940b8070 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.590+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.590+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94360880759936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac74d50 con 0x7f7a940b8070 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.590+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac75770 con 0x55d21ac664b0 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.591+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94360880753952] v27'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.592+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.592+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94360880759936] v27'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.592+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94360880763488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac72250 con 0x55d21ac664b0 2024-04-27T03:31:36.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.592+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac76050 con 0x7f7a9c016440 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.593+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.593+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94360880765792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac76020 con 0x7f7a9c016440 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.593+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94360880763488] v27'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.593+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac748a0 con 0x7f7a9c016440 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.594+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94360880765792] v27'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.594+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.594+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94360880760992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac76b10 con 0x7f7a9c016440 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.594+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac774d0 con 0x7f7a9c016440 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.595+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(58 notify.7 [create] v27'225 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.595+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94360880771104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac77440 con 0x7f7a9c016440 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.595+0000 7f7aa4496800 20 add_watcher() i=0 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.595+0000 7f7aa4496800 20 add_watcher() i=1 2024-04-27T03:31:36.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.595+0000 7f7aa4496800 20 add_watcher() i=2 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.595+0000 7f7aa4496800 20 add_watcher() i=3 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.595+0000 7f7aa4496800 20 add_watcher() i=4 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.595+0000 7f7aa4496800 20 add_watcher() i=5 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.595+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94360880760992] v27'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.595+0000 7f7aa4496800 20 add_watcher() i=6 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.596+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94360880771104] v27'226 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.596+0000 7f7aa4496800 20 add_watcher() i=7 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.596+0000 7f7aa4496800 2 all 8 watchers are set, enabling cache 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.596+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac76440 con 0x55d21ac664b0 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.596+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.596+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94360880773232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac76440 con 0x55d21ac664b0 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.597+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94360880773232] v27'229 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.597+0000 7f79537fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.597+0000 7f7aa4496800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.597+0000 7f7aa4496800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.597+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac83310 con 0x55d21ac664b0 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.598+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'57 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.599+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac839e0 con 0x7f7a9c016440 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.600+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.600+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac83e00 con 0x55d21ac6aea0 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.600+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'34 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.600+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac83d20 con 0x55d21ac664b0 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.601+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'230 uv172 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.601+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac843b0 con 0x7f7a940b8070 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.602+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'77 uv59 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.602+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac84920 con 0x55d21ac6aea0 2024-04-27T03:31:36.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.605+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'61 uv49 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.605+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac84dd0 con 0x55d21ac664b0 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.606+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'35 uv29 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.606+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac84dd0 con 0x7f7a9c016440 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.608+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'53 uv42 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.608+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac84dd0 con 0x7f7a940b8070 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.609+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'27 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.609+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac81170 con 0x55d21ac6aea0 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.610+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.610+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac81170 con 0x7f7a940b8070 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.611+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'78 uv61 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.611+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac81170 con 0x7f7a9c016440 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.612+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'31 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.612+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x7f7a940b8070 2024-04-27T03:31:36.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.613+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.613+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x7f7a9c016440 2024-04-27T03:31:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.614+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'65 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.614+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x55d21ac664b0 2024-04-27T03:31:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.615+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'231 uv168 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.615+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x55d21ac6aea0 2024-04-27T03:31:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.616+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'72 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.616+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x7f7a940b8070 2024-04-27T03:31:36.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.617+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'79 uv57 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.617+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x55d21ac6aea0 2024-04-27T03:31:36.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.618+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'39 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.624 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.618+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x55d21ac664b0 2024-04-27T03:31:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.619+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'35 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.619+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x55d21ac664b0 2024-04-27T03:31:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.620+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'232 uv170 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.620+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x55d21ac6aea0 2024-04-27T03:31:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.621+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'73 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.621+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x7f7a940b8070 2024-04-27T03:31:36.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.622+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'43 uv37 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.622+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x55d21ac6aea0 2024-04-27T03:31:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.623+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'35 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.623+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x55d21ac6aea0 2024-04-27T03:31:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.624+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'46 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.624+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x55d21ac6aea0 2024-04-27T03:31:36.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.625+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'29 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.625+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x55d21ac6aea0 2024-04-27T03:31:36.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.626+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'62 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.626+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x7f7a9c016440 2024-04-27T03:31:36.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.627+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'37 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.627+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x55d21ac6aea0 2024-04-27T03:31:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.628+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'39 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.629+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x7f7a9c016440 2024-04-27T03:31:36.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.630+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'54 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.630+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x55d21ac6aea0 2024-04-27T03:31:36.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.631+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'74 uv56 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.631+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x7f7a9c016440 2024-04-27T03:31:36.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.632+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'66 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.632+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac85b50 con 0x7f7a940b8070 2024-04-27T03:31:36.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.633+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'33 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.633+0000 7f79527fc640 20 reqs_thread_entry: start 2024-04-27T03:31:36.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.633+0000 7f7aa4496800 20 init_complete bucket index max shards: 11 2024-04-27T03:31:36.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.634+0000 7f7aa4496800 20 Filter name: none 2024-04-27T03:31:36.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.634+0000 7f78c3fff640 20 reqs_thread_entry: start 2024-04-27T03:31:36.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.634+0000 7f7aa4496800 10 cache get: name=default.rgw.meta+users.uid+fud : miss 2024-04-27T03:31:36.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.634+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.634+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:94 8.e 8:75f48285:users.uid::fud:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acb0940 con 0x7f7a9c016440 2024-04-27T03:31:36.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.634+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(94 fud [call out=48b,read 0~380 out=380b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 231+0+503 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.634+0000 7f7aa4496800 20 rados_obj.operate() r=0 bl.length=380 2024-04-27T03:31:36.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.634+0000 7f7aa4496800 10 cache put: name=default.rgw.meta+users.uid+fud info.flags=0x13 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.634+0000 7f7aa4496800 10 adding default.rgw.meta+users.uid+fud to cache LRU end 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.634+0000 7f7aa4496800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.634+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:95 8.1 8:810902b1:users.uid::fud.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acb3310 con 0x7f7a9c016440 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.635+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(95 fud.buckets [call out=186b] v0'0 uv6 ondisk = 0) ==== 155+0+186 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.635+0000 7f7aa4496800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo7:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10 : miss 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.635+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.635+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:96 8.1 8:83eb05ee:root::.bucket.meta.myfoo7%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acb3160 con 0x7f7a9c016440 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.636+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(96 .bucket.meta.myfoo7:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10 [call out=48b,stat out=16b,read 0~382 out=382b,getxattrs out=218b] v0'0 uv5 ondisk = 0) ==== 334+0+664 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.636+0000 7f7aa4496800 20 rados_obj.operate() r=0 bl.length=382 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.636+0000 7f7aa4496800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo7:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10 info.flags=0x17 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.636+0000 7f7aa4496800 10 adding default.rgw.meta+root+.bucket.meta.myfoo7:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10 to cache LRU end 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.636+0000 7f7aa4496800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:31:36.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.636+0000 7f7aa4496800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:31:36.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.636+0000 7f7aa4496800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo7:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10 2024-04-27T03:31:36.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.636+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:97 4.f 4:f3263b70:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad30670 con 0x55d21ac664b0 2024-04-27T03:31:36.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.636+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(97 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.0 [call out=85b] v0'0 uv8 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.643 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.636+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:98 4.f 4:f3dcb8b3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad30670 con 0x55d21ac664b0 2024-04-27T03:31:36.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.636+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(98 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.1 [call out=85b] v0'0 uv9 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.636+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:99 4.2e 4:746d8d53:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad30670 con 0x55d21ac6aea0 2024-04-27T03:31:36.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.637+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(99 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.2 [call out=85b] v0'0 uv2 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.637+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:100 4.39 4:9fa74d8d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad30670 con 0x55d21ac6aea0 2024-04-27T03:31:36.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.637+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(100 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.3 [call out=85b] v0'0 uv8 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.637+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:101 4.2d 4:b77e433d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad305f0 con 0x7f7a9c016440 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.637+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(101 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.637+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:102 4.12 4:4bf62e70:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad305f0 con 0x55d21ac664b0 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.638+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(102 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5 [call out=377b] v0'0 uv3 ondisk = 0) ==== 195+0+377 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.638+0000 7f7aa4496800 10 cls_bucket_list_unordered: got fourteen 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.638+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:103 4.2d 4:b7944166:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad305f0 con 0x7f7a9c016440 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.638+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(103 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.6 [call out=85b] v0'0 uv4 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.638+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:104 4.13 4:c8d31d5c:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad305f0 con 0x7f7a940b8070 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.638+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== osd_op_reply(104 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.7 [call out=85b] v0'0 uv5 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.638+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:105 4.1b 4:dae57f5f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad305f0 con 0x7f7a940b8070 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.639+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(105 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.8 [call out=85b] v0'0 uv11 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.639+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:106 4.23 4:c5cad2b6:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad305f0 con 0x7f7a9c016440 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.639+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(106 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.639+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:107 4.4 4:22317043:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad305f0 con 0x7f7a9c016440 2024-04-27T03:31:36.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.640+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(107 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 196+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.640+0000 7f7aa4496800 20 get_obj_state: octx=0x7f7a8c02d040 obj=myfoo7:fourteen state=0x55d21ad2e738 s->prefetch_data=0 2024-04-27T03:31:36.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.640+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:108 3.1 3:83b25300:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10_fourteen:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad2f560 con 0x7f7a9c016440 2024-04-27T03:31:36.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.640+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== osd_op_reply(108 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10_fourteen [getxattrs out=856b,stat out=16b] v0'0 uv3 ondisk = 0) ==== 239+0+872 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.640+0000 7f7aa4496800 10 manifest: total_size = 8 2024-04-27T03:31:36.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.640+0000 7f7aa4496800 20 get_obj_state: setting s->obj_tag to 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.2194741913282182385 2024-04-27T03:31:36.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.640+0000 7f7aa4496800 20 get_obj_state: octx=0x7f7a8c02d040 obj=myfoo7:fourteen state=0x55d21ad2e738 s->prefetch_data=0 2024-04-27T03:31:36.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.640+0000 7f7aa4496800 20 prepare_atomic_modification: state is not atomic. state=0x55d21ad2e738 2024-04-27T03:31:36.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.640+0000 7f7aa4496800 20 bucket index object: :.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5 2024-04-27T03:31:36.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.640+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:109 4.12 4:4bf62e70:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=188b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad2fe20 con 0x55d21ac664b0 2024-04-27T03:31:36.652 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.641+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(109 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5 [stat,call,call] v27'4 uv4 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.641+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:110 3.1 3:83b25300:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10_fourteen:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad2fe20 con 0x7f7a9c016440 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 30 ==== osd_op_reply(110 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10_fourteen [call] v27'4 uv4 ondisk = 0) ==== 197+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:111 4.12 4:4bf62e70:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=265b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acb7840 con 0x55d21ac664b0 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 20 list_objects_ordered: starting attempt 1 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:112 4.f 4:f3263b70:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acb9120 con 0x55d21ac664b0 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:113 4.f 4:f3dcb8b3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acbb9e0 con 0x55d21ac664b0 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:114 4.2e 4:746d8d53:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acbe330 con 0x55d21ac6aea0 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:115 4.39 4:9fa74d8d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acc0c90 con 0x55d21ac6aea0 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:116 4.2d 4:b77e433d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acbb9e0 con 0x7f7a9c016440 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:117 4.12 4:4bf62e70:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acc0c90 con 0x55d21ac664b0 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:118 4.2d 4:b7944166:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acc7f20 con 0x7f7a9c016440 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:119 4.13 4:c8d31d5c:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21accac30 con 0x7f7a940b8070 2024-04-27T03:31:36.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:120 4.1b 4:dae57f5f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21accd640 con 0x7f7a940b8070 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:121 4.23 4:c5cad2b6:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x7f7a9c016440 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:122 4.4 4:22317043:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd32f0 con 0x7f7a9c016440 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 29 ==== osd_op_reply(114 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.2 [call out=85b] v0'0 uv2 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 30 ==== osd_op_reply(115 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.3 [call out=85b] v0'0 uv8 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a940bf070 con 0x55d21ac6aea0 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 31 ==== osd_op_reply(116 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(112 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.0 [call out=85b] v0'0 uv8 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(113 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.1 [call out=85b] v0'0 uv9 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.643+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 32 ==== osd_op_reply(118 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.6 [call out=85b] v0'0 uv4 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 33 ==== osd_op_reply(121 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 34 ==== osd_op_reply(122 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 196+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 23 ==== osd_op_reply(119 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.7 [call out=85b] v0'0 uv5 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 24 ==== osd_op_reply(120 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.8 [call out=85b] v0'0 uv11 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(111 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5 [stat,call,call] v27'5 uv5 ondisk = 0) ==== 279+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7984ff9640 20 handle_completion(): completion ok for obj=fourteen 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(117 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5 [call out=124b] v0'0 uv5 ondisk = 0) ==== 195+0+124 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo7[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10])"; uploads.size()=0; is_truncated=0 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:123 4.f 4:f3263b70:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x55d21ac664b0 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:124 4.f 4:f3dcb8b3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd2ca0 con 0x55d21ac664b0 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:125 4.2e 4:746d8d53:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acbe7c0 con 0x55d21ac6aea0 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:126 4.39 4:9fa74d8d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x55d21ac6aea0 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:127 4.2d 4:b77e433d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd2ca0 con 0x7f7a9c016440 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:128 4.12 4:4bf62e70:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acbe7c0 con 0x55d21ac664b0 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:129 4.2d 4:b7944166:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd2ca0 con 0x7f7a9c016440 2024-04-27T03:31:36.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:130 4.13 4:c8d31d5c:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acbe7c0 con 0x7f7a940b8070 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.644+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:131 4.1b 4:dae57f5f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd2ca0 con 0x7f7a940b8070 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(123 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.0 [call out=85b] v0'0 uv8 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 31 ==== osd_op_reply(125 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.2 [call out=85b] v0'0 uv2 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a940da060 con 0x55d21ac6aea0 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 36 ==== osd_op_reply(128 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5 [call out=124b] v0'0 uv5 ondisk = 0) ==== 195+0+124 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 32 ==== osd_op_reply(126 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.3 [call out=85b] v0'0 uv8 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a940da060 con 0x55d21ac6aea0 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 37 ==== osd_op_reply(124 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.1 [call out=85b] v0'0 uv9 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 25 ==== osd_op_reply(130 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.7 [call out=85b] v0'0 uv5 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:132 4.23 4:c5cad2b6:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acbe7c0 con 0x7f7a9c016440 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 35 ==== osd_op_reply(127 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.4 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:133 4.4 4:22317043:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x7f7a9c016440 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 36 ==== osd_op_reply(129 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.6 [call out=85b] v0'0 uv4 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 26 ==== osd_op_reply(131 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.8 [call out=85b] v0'0 uv11 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 37 ==== osd_op_reply(133 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.10 [call out=85b] v0'0 uv1 ondisk = 0) ==== 196+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 38 ==== osd_op_reply(132 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.9 [call out=85b] v0'0 uv1 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.645+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:134 8.1 8:810902b1:users.uid::fud.buckets:head [call user.set_buckets_info in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x7f7a9c016440 2024-04-27T03:31:36.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.646+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 39 ==== osd_op_reply(134 fud.buckets [call] v27'7 uv7 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.646+0000 7f7aa4496800 10 cache get: name=default.rgw.meta+root+myfoo7 : miss 2024-04-27T03:31:36.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.646+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.657 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.646+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:135 8.9 8:938c738d:root::myfoo7:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x7f7a940b8070 2024-04-27T03:31:36.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.647+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 27 ==== osd_op_reply(135 myfoo7 [call out=48b,read 0~168 out=168b] v0'0 uv44 ondisk = 0) ==== 192+0+216 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.647+0000 7f7aa4496800 20 rados_obj.operate() r=0 bl.length=168 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.647+0000 7f7aa4496800 10 cache put: name=default.rgw.meta+root+myfoo7 info.flags=0x11 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.647+0000 7f7aa4496800 10 adding default.rgw.meta+root+myfoo7 to cache LRU end 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.647+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:136 8.9 8:938c738d:root::myfoo7:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x7f7a940b8070 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.648+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 28 ==== osd_op_reply(136 myfoo7 [call,call,delete] v27'45 uv45 ondisk = 0) ==== 234+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.648+0000 7f7aa4496800 10 removing default.rgw.meta+root+myfoo7 from cache 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.648+0000 7f7aa4496800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:root:myfoo7, ofs0, ns] 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.648+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:137 7.e 7:7759931f:::notify.3:head [notify cookie 94360881134944 in=162b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x7f7a940b8070 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.648+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 29 ==== watch-notify(notify (1) cookie 94360880759936 notify 115964117012 ret 0) ==== 192+0+0 (crc 0 0 0) 0x55d21ac85930 con 0x7f7a940b8070 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.648+0000 7f79857fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117012 cookie 94360880759936 notifier 4749 bl.length()=150 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.648+0000 7f79857fa640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:138 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f7964003d60 con 0x7f7a940b8070 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.648+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 30 ==== osd_op_reply(137 notify.3 [notify cookie 94360881134944 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.648+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 31 ==== osd_op_reply(138 notify.3 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.649+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 32 ==== watch-notify(notify_complete (2) cookie 94360881134944 notify 115964117012 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d21ac84330 con 0x7f7a940b8070 2024-04-27T03:31:36.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.649+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:139 8.1 8:83eb05ee:root::.bucket.meta.myfoo7%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x7f7a9c016440 2024-04-27T03:31:36.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.650+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 40 ==== osd_op_reply(139 .bucket.meta.myfoo7:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10 [call,call,delete] v27'8 uv8 ondisk = 0) ==== 292+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.650+0000 7f7aa4496800 10 removing default.rgw.meta+root+.bucket.meta.myfoo7:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10 from cache 2024-04-27T03:31:36.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.650+0000 7f7aa4496800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo7:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10, ofs0, ns] 2024-04-27T03:31:36.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.650+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:140 7.3 7:c609908c:::notify.5:head [notify cookie 94360881017040 in=220b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x7f7a9c016440 2024-04-27T03:31:36.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.650+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 41 ==== watch-notify(notify (1) cookie 94360880765792 notify 115964117017 ret 0) ==== 250+0+0 (crc 0 0 0) 0x55d21acb26a0 con 0x7f7a9c016440 2024-04-27T03:31:36.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.650+0000 7f7984ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117017 cookie 94360880765792 notifier 4749 bl.length()=208 2024-04-27T03:31:36.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.650+0000 7f7984ff9640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:141 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f78cc0032c0 con 0x7f7a9c016440 2024-04-27T03:31:36.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.650+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 42 ==== osd_op_reply(140 notify.5 [notify cookie 94360881017040 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 43 ==== osd_op_reply(141 notify.5 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 44 ==== watch-notify(notify_complete (2) cookie 94360881017040 notify 115964117017 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d21ac47bb0 con 0x7f7a9c016440 2024-04-27T03:31:36.663 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:142 4.f 4:f3263b70:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x55d21ac664b0 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:143 4.f 4:f3dcb8b3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21accdb60 con 0x55d21ac664b0 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:144 4.2e 4:746d8d53:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acce3b0 con 0x55d21ac6aea0 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:145 4.39 4:9fa74d8d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x55d21ac6aea0 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:146 4.2d 4:b77e433d:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acce790 con 0x7f7a9c016440 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:147 4.12 4:4bf62e70:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21accdb60 con 0x55d21ac664b0 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:148 4.2d 4:b7944166:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acce3b0 con 0x7f7a9c016440 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:149 4.13 4:c8d31d5c:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acce790 con 0x7f7a940b8070 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:150 4.1b 4:dae57f5f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x7f7a940b8070 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:151 4.23 4:c5cad2b6:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21accdb60 con 0x7f7a9c016440 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.651+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:152 4.4 4:22317043:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acce3b0 con 0x7f7a9c016440 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.652+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 33 ==== osd_op_reply(144 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.2 [delete] v27'3 uv3 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f7a940da060 con 0x55d21ac6aea0 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.652+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 45 ==== osd_op_reply(151 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.9 [delete] v27'2 uv2 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.652+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 46 ==== osd_op_reply(146 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.4 [delete] v27'6 uv6 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.653+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 33 ==== osd_op_reply(149 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.7 [delete] v27'6 uv6 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.653+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 38 ==== osd_op_reply(142 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.0 [delete] v27'10 uv10 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.653+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 47 ==== osd_op_reply(148 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.6 [delete] v27'7 uv7 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 34 ==== osd_op_reply(150 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.8 [delete] v27'12 uv12 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 39 ==== osd_op_reply(143 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.1 [delete] v27'11 uv11 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 34 ==== osd_op_reply(145 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.3 [delete] v27'10 uv10 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f7a940da060 con 0x55d21ac6aea0 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 48 ==== osd_op_reply(152 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.10 [delete] v27'2 uv2 ondisk = 0) ==== 196+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 40 ==== osd_op_reply(147 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10.5 [delete] v27'6 uv6 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7aa4496800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo7/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10 : miss 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7aa4496800 20 rados->read ofs=0 len=0 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:153 6.b 6:d4c4523d:::pubsub..bucket.myfoo7%2f708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x55d21ac6aea0 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 35 ==== osd_op_reply(153 pubsub..bucket.myfoo7/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 210+0+0 (crc 0 0 0) 0x7f7a940da060 con 0x55d21ac6aea0 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7aa4496800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7aa4496800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo7/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10 info.flags=0x0 2024-04-27T03:31:36.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7aa4496800 10 adding default.rgw.log++pubsub..bucket.myfoo7/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10 to cache LRU end 2024-04-27T03:31:36.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.655+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:154 6.b 6:d4c4523d:::pubsub..bucket.myfoo7%2f708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x55d21ac6aea0 2024-04-27T03:31:36.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.658+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 36 ==== osd_op_reply(154 pubsub..bucket.myfoo7/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.10 [delete] v27'35 uv28 ondisk = -2 ((2) No such file or directory)) ==== 210+0+0 (crc 0 0 0) 0x7f7a940da060 con 0x55d21ac6aea0 2024-04-27T03:31:36.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.658+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:155 8.1 8:810902b1:users.uid::fud.buckets:head [call user.remove_bucket in=147b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x7f7a9c016440 2024-04-27T03:31:36.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.659+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 49 ==== osd_op_reply(155 fud.buckets [call] v27'9 uv9 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.659+0000 7f7aa4496800 10 removing key index: p5YnriCv1nAtykxBrupQ 2024-04-27T03:31:36.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.659+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:156 8.13 8:cc695d75:users.keys::p5YnriCv1nAtykxBrupQ:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x7f7a9c016440 2024-04-27T03:31:36.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.660+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 50 ==== osd_op_reply(156 p5YnriCv1nAtykxBrupQ [delete] v27'11 uv11 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.669 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.660+0000 7f7aa4496800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.keys:p5YnriCv1nAtykxBrupQ, ofs0, ns] 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.660+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:157 7.15 7:a93a5511:::notify.2:head [notify cookie 94360881017040 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acd04c0 con 0x55d21ac6aea0 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.660+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 37 ==== watch-notify(notify (1) cookie 94360880753952 notify 115964116999 ret 0) ==== 212+0+0 (crc 0 0 0) 0x55d21ac81840 con 0x55d21ac6aea0 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.660+0000 7f7984ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964116999 cookie 94360880753952 notifier 4749 bl.length()=170 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.660+0000 7f7984ff9640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:158 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f78cc0032c0 con 0x55d21ac6aea0 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.660+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 38 ==== osd_op_reply(157 notify.2 [notify cookie 94360881017040 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7a940da060 con 0x55d21ac6aea0 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.660+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 39 ==== osd_op_reply(158 notify.2 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a940da060 con 0x55d21ac6aea0 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.660+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 40 ==== watch-notify(notify_complete (2) cookie 94360881017040 notify 115964116999 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d21ac47140 con 0x55d21ac6aea0 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.660+0000 7f7aa4496800 10 removing email index: 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.660+0000 7f7aa4496800 10 removing user buckets index 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.660+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:159 8.1 8:810902b1:users.uid::fud.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ad2fea0 con 0x7f7a9c016440 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.661+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 51 ==== osd_op_reply(159 fud.buckets [delete] v27'10 uv10 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.661+0000 7f7aa4496800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:users.uid:fud.buckets, ofs0, ns] 2024-04-27T03:31:36.670 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.661+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:160 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94360881017040 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acb0e40 con 0x7f7a9c016440 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.662+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 52 ==== watch-notify(notify (1) cookie 94360880760992 notify 115964117018 ret 0) ==== 202+0+0 (crc 0 0 0) 0x55d21ac47bb0 con 0x7f7a9c016440 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.662+0000 7f79857fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117018 cookie 94360880760992 notifier 4749 bl.length()=160 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.662+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 53 ==== osd_op_reply(160 notify.6 [notify cookie 94360881017040 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.662+0000 7f79857fa640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:161 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f7964003d60 con 0x7f7a9c016440 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.662+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 54 ==== osd_op_reply(161 notify.6 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.662+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 55 ==== watch-notify(notify_complete (2) cookie 94360881017040 notify 115964117018 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d21ac834a0 con 0x7f7a9c016440 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.662+0000 7f7aa4496800 10 removing user index: fud 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.662+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:162 8.e 8:75f48285:users.uid::fud:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21acb30d0 con 0x7f7a9c016440 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.663+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 56 ==== osd_op_reply(162 fud [call,call,delete] v27'7 uv7 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.663+0000 7f7aa4496800 10 removing default.rgw.meta+users.uid+fud from cache 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.663+0000 7f7aa4496800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:fud, ofs0, ns] 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.663+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:163 7.3 7:c609908c:::notify.5:head [notify cookie 94360881017040 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d21acb0e40 con 0x7f7a9c016440 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.663+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 57 ==== watch-notify(notify (1) cookie 94360880765792 notify 115964117019 ret 0) ==== 194+0+0 (crc 0 0 0) 0x55d21ac834a0 con 0x7f7a9c016440 2024-04-27T03:31:36.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.663+0000 7f79857fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117019 cookie 94360880765792 notifier 4749 bl.length()=152 2024-04-27T03:31:36.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.663+0000 7f79857fa640 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f7964003d60 con 0x7f7a9c016440 2024-04-27T03:31:36.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.663+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 58 ==== osd_op_reply(163 notify.5 [notify cookie 94360881017040 out=8b] v0'0 uv225 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.664+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 59 ==== watch-notify(notify_complete (2) cookie 94360881017040 notify 115964117019 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55d21ac832d0 con 0x7f7a9c016440 2024-04-27T03:31:36.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.664+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 60 ==== osd_op_reply(164 notify.5 [notify-ack] v0'0 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.665+0000 7f79537fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:31:36.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.665+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:165 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94360880773232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac7fe80 con 0x55d21ac664b0 2024-04-27T03:31:36.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.666+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 41 ==== osd_op_reply(165 data_loggenerations_metadata [watch unwatch cookie 94360880773232] v27'233 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.666+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:166 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94360880745792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac77870 con 0x7f7a940b8070 2024-04-27T03:31:36.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.666+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 35 ==== osd_op_reply(166 notify.0 [watch unwatch cookie 94360880745792] v27'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.666+0000 7f7aa4496800 20 remove_watcher() i=0 2024-04-27T03:31:36.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.666+0000 7f7aa4496800 2 removed watcher, disabling cache 2024-04-27T03:31:36.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.666+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:167 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94360880749056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac77870 con 0x55d21ac664b0 2024-04-27T03:31:36.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.668+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 42 ==== osd_op_reply(167 notify.1 [watch unwatch cookie 94360880749056] v27'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.668+0000 7f7aa4496800 20 remove_watcher() i=1 2024-04-27T03:31:36.674 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.668+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:168 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94360880753952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac71a00 con 0x55d21ac6aea0 2024-04-27T03:31:36.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.669+0000 7f7aa1f95640 1 -- 172.21.15.165:0/2877415245 <== osd.3 v2:172.21.15.165:6808/2380918637 41 ==== osd_op_reply(168 notify.2 [watch unwatch cookie 94360880753952] v27'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a940da060 con 0x55d21ac6aea0 2024-04-27T03:31:36.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.669+0000 7f7aa4496800 20 remove_watcher() i=2 2024-04-27T03:31:36.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.669+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:169 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94360880759936] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac77870 con 0x7f7a940b8070 2024-04-27T03:31:36.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.670+0000 7f7a99794640 1 -- 172.21.15.165:0/2877415245 <== osd.2 v2:172.21.15.165:6816/975457786 36 ==== osd_op_reply(169 notify.3 [watch unwatch cookie 94360880759936] v27'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a8c009050 con 0x7f7a940b8070 2024-04-27T03:31:36.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.670+0000 7f7aa4496800 20 remove_watcher() i=3 2024-04-27T03:31:36.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.670+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:170 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94360880763488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac71a00 con 0x55d21ac664b0 2024-04-27T03:31:36.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.671+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.1 v2:172.21.15.165:6800/79879041 43 ==== osd_op_reply(170 notify.4 [watch unwatch cookie 94360880763488] v27'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x55d21ac664b0 2024-04-27T03:31:36.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.671+0000 7f7aa4496800 20 remove_watcher() i=4 2024-04-27T03:31:36.677 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.671+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:171 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94360880765792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac74480 con 0x7f7a9c016440 2024-04-27T03:31:36.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.672+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 61 ==== osd_op_reply(171 notify.5 [watch unwatch cookie 94360880765792] v27'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.672+0000 7f7aa4496800 20 remove_watcher() i=5 2024-04-27T03:31:36.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.672+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:172 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94360880760992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac75260 con 0x7f7a9c016440 2024-04-27T03:31:36.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.673+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 62 ==== osd_op_reply(172 notify.6 [watch unwatch cookie 94360880760992] v27'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.673+0000 7f7aa4496800 20 remove_watcher() i=6 2024-04-27T03:31:36.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.673+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:173 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94360880771104] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d21ac71a00 con 0x7f7a9c016440 2024-04-27T03:31:36.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.674+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 <== osd.0 v2:172.21.15.89:6800/2353668432 63 ==== osd_op_reply(173 notify.7 [watch unwatch cookie 94360880771104] v27'227 uv225 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7a9c00a4a0 con 0x7f7a9c016440 2024-04-27T03:31:36.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.674+0000 7f7aa4496800 20 remove_watcher() i=7 2024-04-27T03:31:36.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.674+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7a9c016440 msgr2=0x7f7a9c0168a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.674+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7a9c016440 0x7f7a9c0168a0 crc :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.674+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d21ac664b0 msgr2=0x55d21ac68970 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.674+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d21ac664b0 0x55d21ac68970 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7a940b8070 msgr2=0x7f7a940b3210 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7a940b8070 0x7f7a940b3210 crc :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d21ac6aea0 msgr2=0x55d21ac6d290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d21ac6aea0 0x55d21ac6d290 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f796c036900 msgr2=0x7f796c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f796c036900 0x7f796c038dc0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c001b60 tx=0x7f7a8c010000 comp rx=0 tx=0).stop 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 msgr2=0x55d21ac5ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa2796640 1 -- 172.21.15.165:0/2877415245 reap_dead start 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 0x55d21ac5ad10 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7f7a9407c2d0 tx=0x7f7a94051f40 comp rx=0 tx=0).stop 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 shutdown_connections 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2877415245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac53920 0x55d21ac5ad10 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 >> 172.21.15.165:0/2877415245 conn(0x55d21ac4fc80 msgr2=0x55d21ac4d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 shutdown_connections 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2877415245 wait complete. 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2013271347 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f797c036950 msgr2=0x7f797c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2013271347 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f797c036950 0x7f797c038e10 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00bf50 tx=0x7f7a8c00f000 comp rx=0 tx=0).stop 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2013271347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac467e0 msgr2=0x55d21ac46c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2013271347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac467e0 0x55d21ac46c00 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f7a94000c00 tx=0x7f7a94078850 comp rx=0 tx=0).stop 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2013271347 shutdown_connections 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2013271347 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f797c036950 0x7f797c038e10 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 --2- 172.21.15.165:0/2013271347 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ac467e0 0x55d21ac46c00 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2013271347 >> 172.21.15.165:0/2013271347 conn(0x55d21ac3a880 msgr2=0x55d21ac3ac80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2013271347 shutdown_connections 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.675+0000 7f7aa4496800 1 -- 172.21.15.165:0/2013271347 wait complete. 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d21ab2c100 msgr2=0x55d21ac2efd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d21ab2c100 0x55d21ac2efd0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55d21ac356e0 msgr2=0x55d21ac37ba0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55d21ac356e0 0x55d21ac37ba0 crc :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d21ac317e0 msgr2=0x55d21ac33bd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d21ac317e0 0x55d21ac33bd0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7988036950 msgr2=0x7f7988038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7988036950 0x7f7988038e10 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c000f60 tx=0x7f7a8c0012b0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 msgr2=0x55d21ac27d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 --2- 172.21.15.165:0/1854296258 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d21ab2b7a0 0x55d21ac27d90 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f7a9405f040 tx=0x7f7a94002fb0 comp rx=0 tx=0).stop 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa2796640 1 -- 172.21.15.165:0/1854296258 reap_dead start 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 shutdown_connections 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 >> 172.21.15.165:0/1854296258 conn(0x55d21ab273a0 msgr2=0x55d21ac1a530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 shutdown_connections 2024-04-27T03:31:36.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:31:36.676+0000 7f7aa4496800 1 -- 172.21.15.165:0/1854296258 wait complete. 2024-04-27T03:32:16.691 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'trim', '--uid', 'foo'] 2024-04-27T03:32:16.691 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'trim', '--uid', 'foo'] 2024-04-27T03:32:16.692 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage trim --uid foo 2024-04-27T03:32:16.742 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:16.742 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:16.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.737+0000 7fa49b12d800 1 Processor -- start 2024-04-27T03:32:16.743 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.737+0000 7fa49b12d800 1 -- start start 2024-04-27T03:32:16.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.738+0000 7fa49b12d800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562800bca130 0x562800bcc5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.738+0000 7fa49b12d800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562800a84d40 con 0x562800b391c0 2024-04-27T03:32:16.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.738+0000 7fa49b12d800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x562800a574d0 con 0x562800bca130 2024-04-27T03:32:16.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.738+0000 7fa48ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562800bca130 0x562800bcc5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.738+0000 7fa496d78640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x562800b391c0 0x562800bc89e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42260/0 (socket says 172.21.15.165:42260) 2024-04-27T03:32:16.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.738+0000 7fa496d78640 1 -- 172.21.15.165:0/4092719437 learned_addr learned my addr 172.21.15.165:0/4092719437 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:16.744 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.738+0000 7fa48ffff640 1 -- 172.21.15.165:0/4092719437 >> v1:172.21.15.89:6789/0 conn(0x562800b391c0 legacy=0x562800bc89e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.738+0000 7fa496577640 1 -- 172.21.15.165:0/4092719437 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 207841264 0 0) 0x562800a84d40 con 0x562800b391c0 2024-04-27T03:32:16.745 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.738+0000 7fa48ffff640 1 -- 172.21.15.165:0/4092719437 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562800a83590 con 0x562800bca130 2024-04-27T03:32:16.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.740+0000 7fa48ffff640 1 --2- 172.21.15.165:0/4092719437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800bca130 0x562800bcc5f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa48000a470 tx=0x7fa48001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e13c4706f3e132d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.741+0000 7fa496577640 1 -- 172.21.15.165:0/4092719437 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa480057070 con 0x562800bca130 2024-04-27T03:32:16.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.741+0000 7fa496577640 1 -- 172.21.15.165:0/4092719437 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa480052570 con 0x562800bca130 2024-04-27T03:32:16.746 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.741+0000 7fa496577640 1 -- 172.21.15.165:0/4092719437 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa480052870 con 0x562800bca130 2024-04-27T03:32:16.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.741+0000 7fa49b12d800 1 -- 172.21.15.165:0/4092719437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800bca130 msgr2=0x562800bcc5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.741+0000 7fa49b12d800 1 --2- 172.21.15.165:0/4092719437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800bca130 0x562800bcc5f0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fa48000a470 tx=0x7fa48001ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:16.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.741+0000 7fa49b12d800 1 -- 172.21.15.165:0/4092719437 shutdown_connections 2024-04-27T03:32:16.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.741+0000 7fa49b12d800 1 --2- 172.21.15.165:0/4092719437 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800bca130 0x562800bcc5f0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.747 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.741+0000 7fa49b12d800 1 -- 172.21.15.165:0/4092719437 >> 172.21.15.165:0/4092719437 conn(0x562800bc0380 msgr2=0x562800bc27a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:16.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.741+0000 7fa49b12d800 1 -- 172.21.15.165:0/4092719437 shutdown_connections 2024-04-27T03:32:16.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.741+0000 7fa49b12d800 1 -- 172.21.15.165:0/4092719437 wait complete. 2024-04-27T03:32:16.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.755+0000 7fa49b12d800 1 Processor -- start 2024-04-27T03:32:16.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.755+0000 7fa49b12d800 1 -- start start 2024-04-27T03:32:16.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.756+0000 7fa49b12d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 0x562800bc6310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.756+0000 7fa49b12d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562800b113c0 con 0x562800b391c0 2024-04-27T03:32:16.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.756+0000 7fa496d78640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 0x562800bc6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.756+0000 7fa496d78640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 0x562800bc6310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59360/0 (socket says 172.21.15.165:59360) 2024-04-27T03:32:16.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.756+0000 7fa496d78640 1 -- 172.21.15.165:0/675658617 learned_addr learned my addr 172.21.15.165:0/675658617 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.756+0000 7fa496d78640 1 -- 172.21.15.165:0/675658617 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562800bc48d0 con 0x562800b391c0 2024-04-27T03:32:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.757+0000 7fa496d78640 1 --2- 172.21.15.165:0/675658617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 0x562800bc6310 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa48800b210 tx=0x7fa48800b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae3d9cf2e5227729 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.757+0000 7fa384e09640 1 -- 172.21.15.165:0/675658617 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa488008070 con 0x562800b391c0 2024-04-27T03:32:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.757+0000 7fa384e09640 1 -- 172.21.15.165:0/675658617 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa48800d500 con 0x562800b391c0 2024-04-27T03:32:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.757+0000 7fa384e09640 1 -- 172.21.15.165:0/675658617 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa48800d840 con 0x562800b391c0 2024-04-27T03:32:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.757+0000 7fa49b12d800 1 -- 172.21.15.165:0/675658617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 msgr2=0x562800bc6310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.757+0000 7fa49b12d800 1 --2- 172.21.15.165:0/675658617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 0x562800bc6310 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fa48800b210 tx=0x7fa48800b6d0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.758+0000 7fa49b12d800 1 -- 172.21.15.165:0/675658617 shutdown_connections 2024-04-27T03:32:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.758+0000 7fa49b12d800 1 --2- 172.21.15.165:0/675658617 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 0x562800bc6310 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.758+0000 7fa49b12d800 1 -- 172.21.15.165:0/675658617 >> 172.21.15.165:0/675658617 conn(0x562800bc0380 msgr2=0x562800cb2820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.758+0000 7fa49b12d800 1 -- 172.21.15.165:0/675658617 shutdown_connections 2024-04-27T03:32:16.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.758+0000 7fa49b12d800 1 -- 172.21.15.165:0/675658617 wait complete. 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.758+0000 7fa49b12d800 1 Processor -- start 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.758+0000 7fa49b12d800 1 -- start start 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.758+0000 7fa49b12d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 0x562800cc0c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.758+0000 7fa49b12d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562800bccfa0 con 0x562800b391c0 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.758+0000 7fa496d78640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 0x562800cc0c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.758+0000 7fa496d78640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 0x562800cc0c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59372/0 (socket says 172.21.15.165:59372) 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.758+0000 7fa496d78640 1 -- 172.21.15.165:0/1032778391 learned_addr learned my addr 172.21.15.165:0/1032778391 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.759+0000 7fa496d78640 1 -- 172.21.15.165:0/1032778391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562800cc11d0 con 0x562800b391c0 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.759+0000 7fa496d78640 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 0x562800cc0c90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa488001e60 tx=0x7fa488000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.759+0000 7fa383606640 1 -- 172.21.15.165:0/1032778391 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa488019870 con 0x562800b391c0 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.759+0000 7fa383606640 1 -- 172.21.15.165:0/1032778391 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa488019a10 con 0x562800b391c0 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.759+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562800cc1460 con 0x562800b391c0 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.759+0000 7fa383606640 1 -- 172.21.15.165:0/1032778391 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa488019cf0 con 0x562800b391c0 2024-04-27T03:32:16.766 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.759+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562800cc4110 con 0x562800b391c0 2024-04-27T03:32:16.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.760+0000 7fa383606640 1 -- 172.21.15.165:0/1032778391 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa488017040 con 0x562800b391c0 2024-04-27T03:32:16.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.760+0000 7fa383606640 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa37c036950 0x7fa37c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.760+0000 7fa383606640 1 -- 172.21.15.165:0/1032778391 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fa488015020 con 0x562800b391c0 2024-04-27T03:32:16.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.760+0000 7fa48ffff640 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa37c036950 0x7fa37c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.760+0000 7fa49b12d800 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562800bca130 0x562800cc7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.760+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800cc8450 con 0x562800bca130 2024-04-27T03:32:16.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.760+0000 7fa497579640 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562800bca130 0x562800cc7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.761+0000 7fa48ffff640 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa37c036950 0x7fa37c038e10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa480054000 tx=0x7fa48004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.761+0000 7fa497579640 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562800bca130 0x562800cc7ed0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.761+0000 7fa497579640 1 -- 172.21.15.165:0/1032778391 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800bca130 2024-04-27T03:32:16.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.761+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800cc54a0 con 0x562800bca130 2024-04-27T03:32:16.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.761+0000 7fa497579640 1 -- 172.21.15.165:0/1032778391 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800bca130 2024-04-27T03:32:16.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.762+0000 7fa49b12d800 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562800cca780 0x562800cccb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.762+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800ccd260 con 0x562800cca780 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.762+0000 7fa496d78640 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562800cca780 0x562800cccb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.762+0000 7fa496d78640 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562800cca780 0x562800cccb70 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.762+0000 7fa496d78640 1 -- 172.21.15.165:0/1032778391 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa48802d070 con 0x562800cca780 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.762+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800cc9730 con 0x562800bca130 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.763+0000 7fa497579640 1 -- 172.21.15.165:0/1032778391 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800bca130 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.763+0000 7fa49b12d800 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562800cce710 0x562800cd0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.763+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800cd12c0 con 0x562800cce710 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.763+0000 7fa48ffff640 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562800cce710 0x562800cd0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.763+0000 7fa48ffff640 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562800cce710 0x562800cd0b60 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.763+0000 7fa48ffff640 1 -- 172.21.15.165:0/1032778391 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fa480052940 con 0x562800cce710 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.764+0000 7fa49b12d800 1 Processor -- start 2024-04-27T03:32:16.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.764+0000 7fa49b12d800 1 -- start start 2024-04-27T03:32:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.764+0000 7fa49b12d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800cdf720 0x562800cdfb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.764+0000 7fa49b12d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562800ab76b0 con 0x562800cdf720 2024-04-27T03:32:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.764+0000 7fa496d78640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800cdf720 0x562800cdfb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.764+0000 7fa496d78640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800cdf720 0x562800cdfb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59382/0 (socket says 172.21.15.165:59382) 2024-04-27T03:32:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.764+0000 7fa496d78640 1 -- 172.21.15.165:0/3384357919 learned_addr learned my addr 172.21.15.165:0/3384357919 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.764+0000 7fa496d78640 1 -- 172.21.15.165:0/3384357919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562800ce0080 con 0x562800cdf720 2024-04-27T03:32:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.765+0000 7fa496d78640 1 --2- 172.21.15.165:0/3384357919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800cdf720 0x562800cdfb40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa48804b370 tx=0x7fa48800ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.765+0000 7fa380e01640 1 -- 172.21.15.165:0/3384357919 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa4880449d0 con 0x562800cdf720 2024-04-27T03:32:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.765+0000 7fa380e01640 1 -- 172.21.15.165:0/3384357919 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa488044b70 con 0x562800cdf720 2024-04-27T03:32:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.765+0000 7fa380e01640 1 -- 172.21.15.165:0/3384357919 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa488044e50 con 0x562800cdf720 2024-04-27T03:32:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.765+0000 7fa49b12d800 1 -- 172.21.15.165:0/3384357919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562800ce0af0 con 0x562800cdf720 2024-04-27T03:32:16.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.765+0000 7fa49b12d800 1 -- 172.21.15.165:0/3384357919 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562800ce0ff0 con 0x562800cdf720 2024-04-27T03:32:16.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.766+0000 7fa380e01640 1 -- 172.21.15.165:0/3384357919 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa48803f1b0 con 0x562800cdf720 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.766+0000 7fa380e01640 1 --2- 172.21.15.165:0/3384357919 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa370036900 0x7fa370038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.766+0000 7fa380e01640 1 -- 172.21.15.165:0/3384357919 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fa48803cb90 con 0x562800cdf720 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.766+0000 7fa48ffff640 1 --2- 172.21.15.165:0/3384357919 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa370036900 0x7fa370038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.766+0000 7fa49b12d800 1 Processor -- start 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.766+0000 7fa49b12d800 1 -- start start 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.766+0000 7fa48ffff640 1 --2- 172.21.15.165:0/3384357919 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa370036900 0x7fa370038dc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x562800cc8d30 tx=0x7fa480005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.766+0000 7fa49b12d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 0x562800cece20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.766+0000 7fa49b12d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562800bc4c50 con 0x562800ceca00 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.767+0000 7fa496d78640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 0x562800cece20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.767+0000 7fa496d78640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 0x562800cece20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59386/0 (socket says 172.21.15.165:59386) 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.767+0000 7fa496d78640 1 -- 172.21.15.165:0/1593525851 learned_addr learned my addr 172.21.15.165:0/1593525851 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.767+0000 7fa496d78640 1 -- 172.21.15.165:0/1593525851 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562800ced3f0 con 0x562800ceca00 2024-04-27T03:32:16.785 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.767+0000 7fa496d78640 1 --2- 172.21.15.165:0/1593525851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 0x562800cece20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa48801aaa0 tx=0x7fa4880442e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e6339722f17346 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.767+0000 7fa37a7fc640 1 -- 172.21.15.165:0/1593525851 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa48805c020 con 0x562800ceca00 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.767+0000 7fa37a7fc640 1 -- 172.21.15.165:0/1593525851 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa488040850 con 0x562800ceca00 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.767+0000 7fa37a7fc640 1 -- 172.21.15.165:0/1593525851 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa48803f4a0 con 0x562800ceca00 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.768+0000 7fa49b12d800 1 -- 172.21.15.165:0/1593525851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 msgr2=0x562800cece20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.768+0000 7fa49b12d800 1 --2- 172.21.15.165:0/1593525851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 0x562800cece20 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa48801aaa0 tx=0x7fa4880442e0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.768+0000 7fa49b12d800 1 -- 172.21.15.165:0/1593525851 shutdown_connections 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.768+0000 7fa49b12d800 1 --2- 172.21.15.165:0/1593525851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 0x562800cece20 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.768+0000 7fa49b12d800 1 -- 172.21.15.165:0/1593525851 >> 172.21.15.165:0/1593525851 conn(0x562800ce8d20 msgr2=0x562800ce6d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.768+0000 7fa49b12d800 1 -- 172.21.15.165:0/1593525851 shutdown_connections 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.768+0000 7fa49b12d800 1 -- 172.21.15.165:0/1593525851 wait complete. 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.768+0000 7fa49b12d800 1 Processor -- start 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.768+0000 7fa49b12d800 1 -- start start 2024-04-27T03:32:16.786 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.768+0000 7fa49b12d800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 0x562800cf3e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.768+0000 7fa49b12d800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562800cedb10 con 0x562800ceca00 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.769+0000 7fa496d78640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 0x562800cf3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.769+0000 7fa496d78640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 0x562800cf3e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59396/0 (socket says 172.21.15.165:59396) 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.769+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 learned_addr learned my addr 172.21.15.165:0/2827401713 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.769+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562800cf4390 con 0x562800ceca00 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.769+0000 7fa496d78640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 0x562800cf3e50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa488026ec0 tx=0x7fa488043f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.769+0000 7fa378ff9640 1 -- 172.21.15.165:0/2827401713 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa48803ce30 con 0x562800ceca00 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.769+0000 7fa378ff9640 1 -- 172.21.15.165:0/2827401713 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa48802b270 con 0x562800ceca00 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.769+0000 7fa378ff9640 1 -- 172.21.15.165:0/2827401713 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa48800f740 con 0x562800ceca00 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.769+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562800cf4390 con 0x562800ceca00 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.769+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562800cf4390 con 0x562800ceca00 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.770+0000 7fa378ff9640 1 -- 172.21.15.165:0/2827401713 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa48805f020 con 0x562800ceca00 2024-04-27T03:32:16.787 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.770+0000 7fa378ff9640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa360036900 0x7fa360038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.770+0000 7fa378ff9640 1 -- 172.21.15.165:0/2827401713 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fa488052030 con 0x562800ceca00 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.771+0000 7fa48ffff640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa360036900 0x7fa360038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.771+0000 7fa49b12d800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.771+0000 7fa49b12d800 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562800cff3d0 0x562800d01890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.771+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d01fe0 con 0x562800cff3d0 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.771+0000 7fa48ffff640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa360036900 0x7fa360038dc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa480052fd0 tx=0x7fa48004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.771+0000 7fa497579640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562800cff3d0 0x562800d01890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.771+0000 7fa497579640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562800cff3d0 0x562800d01890 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 20 realm 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d023f0 con 0x562800cff3d0 2024-04-27T03:32:16.788 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d02cc0 con 0x562800cff3d0 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800cff3d0 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562800d03dc0 0x562800d061b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.772+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d067c0 con 0x562800d03dc0 2024-04-27T03:32:16.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.773+0000 7fa496d78640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562800d03dc0 0x562800d061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.773+0000 7fa496d78640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562800d03dc0 0x562800d061b0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.773+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.773+0000 7fa49b12d800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.773+0000 7fa49b12d800 20 searching for the correct realm 2024-04-27T03:32:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.773+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d080f0 con 0x562800d03dc0 2024-04-27T03:32:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.774+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.774+0000 7fa496d78640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa488061030 0x7fa488063860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.774+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d080f0 con 0x7fa488061030 2024-04-27T03:32:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.774+0000 7fa48ffff640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa488061030 0x7fa488063860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.774+0000 7fa48ffff640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa488061030 0x7fa488063860 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.774+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.774+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d080f0 con 0x7fa488061030 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.775+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.775+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d080f0 con 0x562800cff3d0 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.775+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.775+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800cff3d0 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.775+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.775+0000 7fa497579640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa490016440 0x7fa4900168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.775+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa490016440 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.776+0000 7fa497579640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa490016440 0x7fa4900168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.776+0000 7fa497579640 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa490016440 0x7fa4900168c0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.776+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.776+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa490016440 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.777+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.777+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa488061030 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.777+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.777+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800d03dc0 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.777+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.777+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800cff3d0 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.777+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.777+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa488061030 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.778+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.778+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800cff3d0 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.778+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.778+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa488061030 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.778+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.778+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800d03dc0 2024-04-27T03:32:16.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.779+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.779+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800d03dc0 2024-04-27T03:32:16.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.779+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.779+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa490016440 2024-04-27T03:32:16.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.779+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.779+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800d03dc0 2024-04-27T03:32:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.780+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.780+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800d03dc0 2024-04-27T03:32:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.780+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.780+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800cff3d0 2024-04-27T03:32:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.780+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.780+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800cff3d0 2024-04-27T03:32:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.780+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.801 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.780+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa490016440 2024-04-27T03:32:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.781+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.781+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800d03dc0 2024-04-27T03:32:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.781+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.781+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa488061030 2024-04-27T03:32:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.781+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.781+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800cff3d0 2024-04-27T03:32:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.781+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.782+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa490016440 2024-04-27T03:32:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.782+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.782+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800d03dc0 2024-04-27T03:32:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.782+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.802 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.782+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa490016440 2024-04-27T03:32:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.782+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.782+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa488061030 2024-04-27T03:32:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.783+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.783+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800d03dc0 2024-04-27T03:32:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.783+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.783+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa490016440 2024-04-27T03:32:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.783+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.783+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x562800cff3d0 2024-04-27T03:32:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562800d03060 con 0x7fa488061030 2024-04-27T03:32:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d07080 con 0x562800cff3d0 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d07470 con 0x562800cff3d0 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.784+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d07470 con 0x7fa488061030 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa49b12d800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa49b12d800 20 zone default found 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa49b12d800 4 Realm: () 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa49b12d800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa49b12d800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa49b12d800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa49b12d800 20 zonegroup default 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa49b12d800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d08570 con 0x7fa490016440 2024-04-27T03:32:16.805 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa49b12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa49b12d800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d09f50 con 0x562800d03dc0 2024-04-27T03:32:16.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.785+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.786+0000 7fa49b12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.786+0000 7fa49b12d800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.786+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d0a3d0 con 0x562800d03dc0 2024-04-27T03:32:16.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.786+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.786+0000 7fa49b12d800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.786+0000 7fa49b12d800 20 started sync module instance, tier type = 2024-04-27T03:32:16.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.786+0000 7fa49b12d800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:16.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.786+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d093f0 con 0x7fa488061030 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.787+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.787+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94729812350944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d09050 con 0x7fa488061030 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.787+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0b300 con 0x562800cff3d0 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.788+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94729812350944] v27'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.788+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.788+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94729812356688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0b720 con 0x562800cff3d0 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.788+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0c0e0 con 0x562800d03dc0 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.789+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94729812356688] v27'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.790+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.790+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94729812357744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0b690 con 0x562800d03dc0 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.790+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0cfa0 con 0x7fa488061030 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.791+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94729812357744] v27'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.792+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.792+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94729812362448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0d840 con 0x7fa488061030 2024-04-27T03:32:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.792+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0e200 con 0x562800cff3d0 2024-04-27T03:32:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.793+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94729812362448] v27'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.793+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.793+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94729812368720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0e620 con 0x562800cff3d0 2024-04-27T03:32:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.793+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0efe0 con 0x7fa490016440 2024-04-27T03:32:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.794+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94729812368720] v27'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.794+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.795+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94729812370256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0efe0 con 0x7fa490016440 2024-04-27T03:32:16.808 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.795+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0f9a0 con 0x7fa490016440 2024-04-27T03:32:16.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.796+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94729812370256] v27'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.796+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.812 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.796+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94729812372272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0fb80 con 0x7fa490016440 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.796+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d103d0 con 0x7fa490016440 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.797+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94729812372272] v27'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.797+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'228 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.797+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94729812375728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d10220 con 0x7fa490016440 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.797+0000 7fa49b12d800 20 add_watcher() i=0 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.797+0000 7fa49b12d800 20 add_watcher() i=1 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.797+0000 7fa49b12d800 20 add_watcher() i=2 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.797+0000 7fa49b12d800 20 add_watcher() i=3 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.797+0000 7fa49b12d800 20 add_watcher() i=4 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.797+0000 7fa49b12d800 20 add_watcher() i=5 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.797+0000 7fa49b12d800 20 add_watcher() i=6 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.799+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94729812375728] v27'229 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.813 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.799+0000 7fa49b12d800 20 add_watcher() i=7 2024-04-27T03:32:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.799+0000 7fa49b12d800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.799+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d13760 con 0x562800cff3d0 2024-04-27T03:32:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.799+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.799+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94729812376784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d10c50 con 0x562800cff3d0 2024-04-27T03:32:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.800+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94729812376784] v27'234 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.800+0000 7fa34d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.800+0000 7fa49b12d800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.800+0000 7fa49b12d800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.800+0000 7fa49b12d800 5 note: GC not initialized 2024-04-27T03:32:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.800+0000 7fa2c7fff640 20 reqs_thread_entry: start 2024-04-27T03:32:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.801+0000 7fa49b12d800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:16.814 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.801+0000 7fa49b12d800 20 Filter name: none 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.801+0000 7fa2c5ffb640 20 reqs_thread_entry: start 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.801+0000 7fa49b12d800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.801+0000 7fa49b12d800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.801+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562800d46aa0 con 0x562800cff3d0 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.801+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.801+0000 7fa49b12d800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.801+0000 7fa49b12d800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.801+0000 7fa49b12d800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.801+0000 7fa49b12d800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.801+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d49530 con 0x7fa490016440 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.803+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 usage.10 [call] v27'32 uv32 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.803+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:64 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_trim in=55b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d49530 con 0x7fa490016440 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.804+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(64 usage.10 [call] v27'33 uv32 ondisk = -61 ((61) No data available)) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.815 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.805+0000 7fa34d7e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.805+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94729812376784] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800cfc790 con 0x562800cff3d0 2024-04-27T03:32:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.806+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 data_loggenerations_metadata [watch unwatch cookie 94729812376784] v27'235 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.806+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94729812350944] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0fcd0 con 0x7fa488061030 2024-04-27T03:32:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.807+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(66 notify.0 [watch unwatch cookie 94729812350944] v27'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.807+0000 7fa49b12d800 20 remove_watcher() i=0 2024-04-27T03:32:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.807+0000 7fa49b12d800 2 removed watcher, disabling cache 2024-04-27T03:32:16.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.807+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94729812356688] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d46640 con 0x562800cff3d0 2024-04-27T03:32:16.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.808+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(67 notify.1 [watch unwatch cookie 94729812356688] v27'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.808+0000 7fa49b12d800 20 remove_watcher() i=1 2024-04-27T03:32:16.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.808+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94729812357744] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d46640 con 0x562800d03dc0 2024-04-27T03:32:16.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.809+0000 7fa496d78640 1 -- 172.21.15.165:0/2827401713 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(68 notify.2 [watch unwatch cookie 94729812357744] v27'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa488073020 con 0x562800d03dc0 2024-04-27T03:32:16.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.809+0000 7fa49b12d800 20 remove_watcher() i=2 2024-04-27T03:32:16.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.809+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:69 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94729812362448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0fcd0 con 0x7fa488061030 2024-04-27T03:32:16.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.810+0000 7fa48ffff640 1 -- 172.21.15.165:0/2827401713 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(69 notify.3 [watch unwatch cookie 94729812362448] v27'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa480081070 con 0x7fa488061030 2024-04-27T03:32:16.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.810+0000 7fa49b12d800 20 remove_watcher() i=3 2024-04-27T03:32:16.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.810+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94729812368720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0c4d0 con 0x562800cff3d0 2024-04-27T03:32:16.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.812+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(70 notify.4 [watch unwatch cookie 94729812368720] v27'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x562800cff3d0 2024-04-27T03:32:16.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.812+0000 7fa49b12d800 20 remove_watcher() i=4 2024-04-27T03:32:16.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.812+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94729812370256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0fcd0 con 0x7fa490016440 2024-04-27T03:32:16.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.813+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(71 notify.5 [watch unwatch cookie 94729812370256] v27'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.813+0000 7fa49b12d800 20 remove_watcher() i=5 2024-04-27T03:32:16.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.813+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94729812372272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0dd50 con 0x7fa490016440 2024-04-27T03:32:16.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.815+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 notify.6 [watch unwatch cookie 94729812372272] v27'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.815+0000 7fa49b12d800 20 remove_watcher() i=6 2024-04-27T03:32:16.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.815+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94729812375728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562800d0fcd0 con 0x7fa490016440 2024-04-27T03:32:16.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(73 notify.7 [watch unwatch cookie 94729812375728] v27'230 uv228 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa49000a4a0 con 0x7fa490016440 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 20 remove_watcher() i=7 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa490016440 msgr2=0x7fa4900168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa490016440 0x7fa4900168c0 crc :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562800cff3d0 msgr2=0x562800d01890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562800cff3d0 0x562800d01890 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa488061030 msgr2=0x7fa488063860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa488061030 0x7fa488063860 crc :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562800d03dc0 msgr2=0x562800d061b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562800d03dc0 0x562800d061b0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa360036900 msgr2=0x7fa360038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa360036900 0x7fa360038dc0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fa480052fd0 tx=0x7fa48004f000 comp rx=0 tx=0).stop 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 msgr2=0x562800cf3e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 0x562800cf3e50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa488026ec0 tx=0x7fa488043f90 comp rx=0 tx=0).stop 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa497579640 1 -- 172.21.15.165:0/2827401713 reap_dead start 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 shutdown_connections 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 --2- 172.21.15.165:0/2827401713 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800ceca00 0x562800cf3e50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 >> 172.21.15.165:0/2827401713 conn(0x562800ce8d20 msgr2=0x562800ce6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 shutdown_connections 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 -- 172.21.15.165:0/2827401713 wait complete. 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 -- 172.21.15.165:0/3384357919 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa370036900 msgr2=0x7fa370038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 --2- 172.21.15.165:0/3384357919 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa370036900 0x7fa370038dc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x562800cc8d30 tx=0x7fa480005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 -- 172.21.15.165:0/3384357919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800cdf720 msgr2=0x562800cdfb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.816+0000 7fa49b12d800 1 --2- 172.21.15.165:0/3384357919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800cdf720 0x562800cdfb40 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa48804b370 tx=0x7fa48800ed80 comp rx=0 tx=0).stop 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/3384357919 shutdown_connections 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 --2- 172.21.15.165:0/3384357919 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa370036900 0x7fa370038dc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 --2- 172.21.15.165:0/3384357919 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800cdf720 0x562800cdfb40 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.825 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/3384357919 >> 172.21.15.165:0/3384357919 conn(0x562800cd37c0 msgr2=0x562800cd3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:16.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/3384357919 shutdown_connections 2024-04-27T03:32:16.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/3384357919 wait complete. 2024-04-27T03:32:16.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562800bca130 msgr2=0x562800cc7ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562800bca130 0x562800cc7ed0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562800cce710 msgr2=0x562800cd0b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562800cce710 0x562800cd0b60 crc :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562800cca780 msgr2=0x562800cccb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562800cca780 0x562800cccb70 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa37c036950 msgr2=0x7fa37c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa37c036950 0x7fa37c038e10 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fa480054000 tx=0x7fa48004e000 comp rx=0 tx=0).stop 2024-04-27T03:32:16.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 msgr2=0x562800cc0c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.826 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 --2- 172.21.15.165:0/1032778391 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562800b391c0 0x562800cc0c90 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fa488001e60 tx=0x7fa488000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa497579640 1 -- 172.21.15.165:0/1032778391 reap_dead start 2024-04-27T03:32:16.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 shutdown_connections 2024-04-27T03:32:16.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 >> 172.21.15.165:0/1032778391 conn(0x562800bc0380 msgr2=0x562800bc51b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:16.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 shutdown_connections 2024-04-27T03:32:16.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.817+0000 7fa49b12d800 1 -- 172.21.15.165:0/1032778391 wait complete. 2024-04-27T03:32:16.831 INFO:tasks.util.rgw:rgwadmin: client.0 : ['usage', 'show', '--uid', 'foo'] 2024-04-27T03:32:16.831 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'usage', 'show', '--uid', 'foo'] 2024-04-27T03:32:16.831 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph usage show --uid foo 2024-04-27T03:32:16.876 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:16.876 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:16.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.871+0000 7f19f2fa2800 1 Processor -- start 2024-04-27T03:32:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.871+0000 7f19f2fa2800 1 -- start start 2024-04-27T03:32:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.871+0000 7f19f2fa2800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55e32e7f33c0 0x55e32e7f57b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.871+0000 7f19f2fa2800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e32e6add40 con 0x55e32e7ef8b0 2024-04-27T03:32:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.871+0000 7f19f2fa2800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55e32e6804d0 con 0x55e32e7f33c0 2024-04-27T03:32:16.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.872+0000 7f19f02a7640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55e32e7f33c0 0x55e32e7f57b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.872+0000 7f19f0aa8640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55e32e7ef8b0 0x55e32e7f1ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42272/0 (socket says 172.21.15.165:42272) 2024-04-27T03:32:16.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.872+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2230145316 learned_addr learned my addr 172.21.15.165:0/2230145316 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:16.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.872+0000 7f19f02a7640 1 -- 172.21.15.165:0/2230145316 >> v1:172.21.15.89:6789/0 conn(0x55e32e7ef8b0 legacy=0x55e32e7f1ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.872+0000 7f19f02a7640 1 -- 172.21.15.165:0/2230145316 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e32e6ac590 con 0x55e32e7f33c0 2024-04-27T03:32:16.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.874+0000 7f19f02a7640 1 --2- 172.21.15.165:0/2230145316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7f33c0 0x55e32e7f57b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f19e400a470 tx=0x7f19e401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873f14ba5fd98209 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.874+0000 7f19efaa6640 1 -- 172.21.15.165:0/2230145316 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e4057070 con 0x55e32e7f33c0 2024-04-27T03:32:16.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.874+0000 7f19efaa6640 1 -- 172.21.15.165:0/2230145316 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e4052570 con 0x55e32e7f33c0 2024-04-27T03:32:16.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.874+0000 7f19efaa6640 1 -- 172.21.15.165:0/2230145316 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e4052870 con 0x55e32e7f33c0 2024-04-27T03:32:16.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.874+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2230145316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7f33c0 msgr2=0x55e32e7f57b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.874+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2230145316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7f33c0 0x55e32e7f57b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f19e400a470 tx=0x7f19e401ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:16.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.874+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2230145316 shutdown_connections 2024-04-27T03:32:16.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.874+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2230145316 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7f33c0 0x55e32e7f57b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.874+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2230145316 >> 172.21.15.165:0/2230145316 conn(0x55e32e7e9380 msgr2=0x55e32e7eb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:16.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.874+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2230145316 shutdown_connections 2024-04-27T03:32:16.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.874+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2230145316 wait complete. 2024-04-27T03:32:16.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.886+0000 7f19f2fa2800 1 Processor -- start 2024-04-27T03:32:16.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.886+0000 7f19f2fa2800 1 -- start start 2024-04-27T03:32:16.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.886+0000 7f19f2fa2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 0x55e32e8e0070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.886+0000 7f19f2fa2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e32e73a3c0 con 0x55e32e7ef8b0 2024-04-27T03:32:16.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.887+0000 7f19f0aa8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 0x55e32e8e0070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.887+0000 7f19f0aa8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 0x55e32e8e0070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59414/0 (socket says 172.21.15.165:59414) 2024-04-27T03:32:16.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.887+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2236605495 learned_addr learned my addr 172.21.15.165:0/2236605495 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:16.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.887+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2236605495 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e32e8e0640 con 0x55e32e7ef8b0 2024-04-27T03:32:16.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.887+0000 7f19f0aa8640 1 --2- 172.21.15.165:0/2236605495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 0x55e32e8e0070 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f19e8001610 tx=0x7f19e8006f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39962e4e7f1c4c8f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.887+0000 7f18dee0d640 1 -- 172.21.15.165:0/2236605495 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e8004520 con 0x55e32e7ef8b0 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.887+0000 7f18dee0d640 1 -- 172.21.15.165:0/2236605495 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e80046e0 con 0x55e32e7ef8b0 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.887+0000 7f18dee0d640 1 -- 172.21.15.165:0/2236605495 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e8004a80 con 0x55e32e7ef8b0 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.888+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2236605495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 msgr2=0x55e32e8e0070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.888+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2236605495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 0x55e32e8e0070 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f19e8001610 tx=0x7f19e8006f90 comp rx=0 tx=0).stop 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.888+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2236605495 shutdown_connections 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.888+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2236605495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 0x55e32e8e0070 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.888+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2236605495 >> 172.21.15.165:0/2236605495 conn(0x55e32e7e9380 msgr2=0x55e32e8db990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.888+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2236605495 shutdown_connections 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.888+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2236605495 wait complete. 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.888+0000 7f19f2fa2800 1 Processor -- start 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.888+0000 7f19f2fa2800 1 -- start start 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.889+0000 7f19f2fa2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 0x55e32e7ef1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.889+0000 7f19f2fa2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e32e7f5ff0 con 0x55e32e7ef8b0 2024-04-27T03:32:16.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.889+0000 7f19f0aa8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 0x55e32e7ef1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.889+0000 7f19f0aa8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 0x55e32e7ef1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59430/0 (socket says 172.21.15.165:59430) 2024-04-27T03:32:16.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.889+0000 7f19f0aa8640 1 -- 172.21.15.165:0/3780685120 learned_addr learned my addr 172.21.15.165:0/3780685120 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:16.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.889+0000 7f19f0aa8640 1 -- 172.21.15.165:0/3780685120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e32e7ed8e0 con 0x55e32e7ef8b0 2024-04-27T03:32:16.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.889+0000 7f19f0aa8640 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 0x55e32e7ef1f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f19e800f000 tx=0x7f19e8000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.890+0000 7f18dd60a640 1 -- 172.21.15.165:0/3780685120 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e8022030 con 0x55e32e7ef8b0 2024-04-27T03:32:16.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.890+0000 7f18dd60a640 1 -- 172.21.15.165:0/3780685120 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8019040 con 0x55e32e7ef8b0 2024-04-27T03:32:16.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.890+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e32e7edb70 con 0x55e32e7ef8b0 2024-04-27T03:32:16.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.890+0000 7f18dd60a640 1 -- 172.21.15.165:0/3780685120 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e8014af0 con 0x55e32e7ef8b0 2024-04-27T03:32:16.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.890+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e32e7ee090 con 0x55e32e7ef8b0 2024-04-27T03:32:16.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.890+0000 7f18dd60a640 1 -- 172.21.15.165:0/3780685120 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f19e8025070 con 0x55e32e7ef8b0 2024-04-27T03:32:16.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.891+0000 7f18dd60a640 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18d8036900 0x7f18d8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.891+0000 7f18dd60a640 1 -- 172.21.15.165:0/3780685120 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f19e8023070 con 0x55e32e7ef8b0 2024-04-27T03:32:16.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.891+0000 7f19f02a7640 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18d8036900 0x7f18d8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.891+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e32e7f33c0 0x55e32e8f1010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.891+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e8f15e0 con 0x55e32e7f33c0 2024-04-27T03:32:16.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.891+0000 7f19f12a9640 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e32e7f33c0 0x55e32e8f1010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.891+0000 7f19f02a7640 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18d8036900 0x7f18d8038dc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f19e4054000 tx=0x7f19e404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.891+0000 7f19f12a9640 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e32e7f33c0 0x55e32e8f1010 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.892+0000 7f19f12a9640 1 -- 172.21.15.165:0/3780685120 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e7f33c0 2024-04-27T03:32:16.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.892+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e8f1ef0 con 0x55e32e7f33c0 2024-04-27T03:32:16.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.892+0000 7f19f12a9640 1 -- 172.21.15.165:0/3780685120 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e7f33c0 2024-04-27T03:32:16.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.892+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e32e8f3920 0x55e32e8f5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.892+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e8f6570 con 0x55e32e8f3920 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.892+0000 7f19f0aa8640 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e32e8f3920 0x55e32e8f5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.892+0000 7f19f0aa8640 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e32e8f3920 0x55e32e8f5de0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.893+0000 7f19f0aa8640 1 -- 172.21.15.165:0/3780685120 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e8f3920 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.893+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e8f2770 con 0x55e32e7f33c0 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.893+0000 7f19f12a9640 1 -- 172.21.15.165:0/3780685120 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e7f33c0 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.893+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e32e8f7a20 0x55e32e8f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.893+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e8fa640 con 0x55e32e8f7a20 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.893+0000 7f19f02a7640 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e32e8f7a20 0x55e32e8f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.893+0000 7f19f02a7640 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e32e8f7a20 0x55e32e8f9eb0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.894+0000 7f19f02a7640 1 -- 172.21.15.165:0/3780685120 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f19e4052980 con 0x55e32e8f7a20 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.894+0000 7f19f2fa2800 1 Processor -- start 2024-04-27T03:32:16.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.894+0000 7f19f2fa2800 1 -- start start 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.894+0000 7f19f2fa2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e9089d0 0x55e32e908df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.894+0000 7f19f2fa2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e32e7ef730 con 0x55e32e9089d0 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.895+0000 7f19f0aa8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e9089d0 0x55e32e908df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.895+0000 7f19f0aa8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e9089d0 0x55e32e908df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59444/0 (socket says 172.21.15.165:59444) 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.895+0000 7f19f0aa8640 1 -- 172.21.15.165:0/4136763483 learned_addr learned my addr 172.21.15.165:0/4136763483 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.895+0000 7f19f0aa8640 1 -- 172.21.15.165:0/4136763483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e32e909330 con 0x55e32e9089d0 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.895+0000 7f19f0aa8640 1 --2- 172.21.15.165:0/4136763483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e9089d0 0x55e32e908df0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f19e804ac00 tx=0x7f19e8026ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.895+0000 7f18d67fc640 1 -- 172.21.15.165:0/4136763483 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e801d9a0 con 0x55e32e9089d0 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.895+0000 7f18d67fc640 1 -- 172.21.15.165:0/4136763483 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8042040 con 0x55e32e9089d0 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.895+0000 7f19f2fa2800 1 -- 172.21.15.165:0/4136763483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e32e909da0 con 0x55e32e9089d0 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.895+0000 7f18d67fc640 1 -- 172.21.15.165:0/4136763483 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e801dc70 con 0x55e32e9089d0 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.895+0000 7f19f2fa2800 1 -- 172.21.15.165:0/4136763483 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e32e909da0 con 0x55e32e9089d0 2024-04-27T03:32:16.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.896+0000 7f18d67fc640 1 -- 172.21.15.165:0/4136763483 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f19e8010070 con 0x55e32e9089d0 2024-04-27T03:32:16.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.896+0000 7f18d67fc640 1 --2- 172.21.15.165:0/4136763483 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18c8036900 0x7f18c8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.896+0000 7f18d67fc640 1 -- 172.21.15.165:0/4136763483 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f19e803cbf0 con 0x55e32e9089d0 2024-04-27T03:32:16.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.896+0000 7f19f02a7640 1 --2- 172.21.15.165:0/4136763483 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18c8036900 0x7f18c8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.896+0000 7f19f2fa2800 1 Processor -- start 2024-04-27T03:32:16.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.897+0000 7f19f2fa2800 1 -- start start 2024-04-27T03:32:16.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.897+0000 7f19f02a7640 1 --2- 172.21.15.165:0/4136763483 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18c8036900 0x7f18c8038dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55e32e8f6910 tx=0x7f19e4005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.897+0000 7f19f2fa2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 0x55e32e915d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.897+0000 7f19f2fa2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e32e6e06b0 con 0x55e32e915940 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.897+0000 7f19f0aa8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 0x55e32e915d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.897+0000 7f19f0aa8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 0x55e32e915d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59450/0 (socket says 172.21.15.165:59450) 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.897+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2373408198 learned_addr learned my addr 172.21.15.165:0/2373408198 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.897+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2373408198 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e32e916330 con 0x55e32e915940 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.897+0000 7f19f0aa8640 1 --2- 172.21.15.165:0/2373408198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 0x55e32e915d60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f19e80437f0 tx=0x7f19e8044a90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff445c2744eeb1e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.898+0000 7f18c7fff640 1 -- 172.21.15.165:0/2373408198 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e805a070 con 0x55e32e915940 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.898+0000 7f18c7fff640 1 -- 172.21.15.165:0/2373408198 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8042420 con 0x55e32e915940 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.898+0000 7f18c7fff640 1 -- 172.21.15.165:0/2373408198 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e8011e50 con 0x55e32e915940 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.898+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2373408198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 msgr2=0x55e32e915d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.898+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2373408198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 0x55e32e915d60 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f19e80437f0 tx=0x7f19e8044a90 comp rx=0 tx=0).stop 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.898+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2373408198 shutdown_connections 2024-04-27T03:32:16.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.898+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2373408198 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 0x55e32e915d60 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.898+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2373408198 >> 172.21.15.165:0/2373408198 conn(0x55e32e911ca0 msgr2=0x55e32e90fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.898+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2373408198 shutdown_connections 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.898+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2373408198 wait complete. 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.898+0000 7f19f2fa2800 1 Processor -- start 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.898+0000 7f19f2fa2800 1 -- start start 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.899+0000 7f19f2fa2800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 0x55e32e91cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.899+0000 7f19f2fa2800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e32e916860 con 0x55e32e915940 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.899+0000 7f19f0aa8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 0x55e32e91cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.899+0000 7f19f0aa8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 0x55e32e91cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59456/0 (socket says 172.21.15.165:59456) 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.899+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 learned_addr learned my addr 172.21.15.165:0/2513595404 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.899+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e32e91d100 con 0x55e32e915940 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.899+0000 7f19f0aa8640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 0x55e32e91cbc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f19e8012c70 tx=0x7f19e8043750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.899+0000 7f18c67fc640 1 -- 172.21.15.165:0/2513595404 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e80128d0 con 0x55e32e915940 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.899+0000 7f18c67fc640 1 -- 172.21.15.165:0/2513595404 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f19e8011e50 con 0x55e32e915940 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.899+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e32e91d100 con 0x55e32e915940 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.899+0000 7f18c67fc640 1 -- 172.21.15.165:0/2513595404 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f19e8042b70 con 0x55e32e915940 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.899+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e32e91d100 con 0x55e32e915940 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.900+0000 7f18c67fc640 1 -- 172.21.15.165:0/2513595404 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f19e8070020 con 0x55e32e915940 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.900+0000 7f18c67fc640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18bc036900 0x7f18bc038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.900+0000 7f19f2fa2800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.900+0000 7f18c67fc640 1 -- 172.21.15.165:0/2513595404 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f19e806e020 con 0x55e32e915940 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.900+0000 7f19f02a7640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18bc036900 0x7f18bc038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e32e928180 0x55e32e92a620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ad70 con 0x55e32e928180 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f12a9640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e32e928180 0x55e32e92a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f02a7640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18bc036900 0x7f18bc038dc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f19e401ced0 tx=0x7f19e404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f12a9640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e32e928180 0x55e32e92a620 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f2fa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f2fa2800 20 realm 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f2fa2800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92b180 con 0x55e32e928180 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f2fa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f2fa2800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f2fa2800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.901+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e928180 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.902+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.902+0000 7f19f2fa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.902+0000 7f19f2fa2800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.902+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92bdc0 con 0x55e32e928180 2024-04-27T03:32:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.902+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.902+0000 7f19f2fa2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.902+0000 7f19f2fa2800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.902+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e32e92c780 0x55e32e92eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.902+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92f0b0 con 0x55e32e92c780 2024-04-27T03:32:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.902+0000 7f19f0aa8640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e32e92c780 0x55e32e92eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.902+0000 7f19f0aa8640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e32e92c780 0x55e32e92eb70 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.903+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.903+0000 7f19f2fa2800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:16.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.903+0000 7f19f2fa2800 20 searching for the correct realm 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.903+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e930ad0 con 0x55e32e92c780 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.903+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.903+0000 7f19f0aa8640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f19e8027030 0x7f19e8043900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.903+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e930ad0 con 0x7f19e8027030 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.903+0000 7f19f02a7640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f19e8027030 0x7f19e8043900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.903+0000 7f19f02a7640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f19e8027030 0x7f19e8043900 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.904+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.904+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e930ad0 con 0x7f19e8027030 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.904+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.904+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e930ad0 con 0x55e32e928180 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.904+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.904+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e928180 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.904+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.904+0000 7f19f12a9640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f19e0018410 0x7f19e0018870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.904+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e0018410 2024-04-27T03:32:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.905+0000 7f19f12a9640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f19e0018410 0x7f19e0018870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.905+0000 7f19f12a9640 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f19e0018410 0x7f19e0018870 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.905+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.905+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e0018410 2024-04-27T03:32:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.906+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.906+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e8027030 2024-04-27T03:32:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.906+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.906+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e92c780 2024-04-27T03:32:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.906+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.906+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e928180 2024-04-27T03:32:16.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.906+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.906+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e8027030 2024-04-27T03:32:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.907+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.907+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e928180 2024-04-27T03:32:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.907+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.907+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e8027030 2024-04-27T03:32:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.907+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.907+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e92c780 2024-04-27T03:32:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.907+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.907+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e92c780 2024-04-27T03:32:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.907+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.907+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e0018410 2024-04-27T03:32:16.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.908+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.908+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e92c780 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.908+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.908+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e92c780 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.908+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.908+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e928180 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.908+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.909+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e928180 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.909+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.909+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e0018410 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.909+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.909+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e92c780 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.909+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.909+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e8027030 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.910+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.910+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e928180 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.910+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.910+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e0018410 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.910+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.910+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e92c780 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.910+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.910+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e0018410 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.911+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.911+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e8027030 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.911+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.911+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e92c780 2024-04-27T03:32:16.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.911+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.911+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e0018410 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x55e32e928180 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92ba20 con 0x7f19e8027030 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:16.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92fa60 con 0x55e32e928180 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92fe50 con 0x55e32e928180 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.912+0000 7f19f2fa2800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e92fe50 con 0x7f19e8027030 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 20 zone default found 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 4 Realm: () 2024-04-27T03:32:16.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 20 zonegroup default 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e930fa0 con 0x7f19e0018410 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e932980 con 0x55e32e92c780 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.913+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.914+0000 7f19f2fa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.914+0000 7f19f2fa2800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.914+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e932d20 con 0x55e32e92c780 2024-04-27T03:32:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.914+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.914+0000 7f19f2fa2800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.914+0000 7f19f2fa2800 20 started sync module instance, tier type = 2024-04-27T03:32:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.914+0000 7f19f2fa2800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.914+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e931e20 con 0x7f19e8027030 2024-04-27T03:32:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.915+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.915+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94434227331600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e931a80 con 0x7f19e8027030 2024-04-27T03:32:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.915+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e933b90 con 0x55e32e928180 2024-04-27T03:32:16.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.916+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94434227331600] v27'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.916+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.916+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94434227336928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e933fb0 con 0x55e32e928180 2024-04-27T03:32:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.916+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e934970 con 0x55e32e92c780 2024-04-27T03:32:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.917+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94434227336928] v27'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.918+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.918+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94434227337984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e933f20 con 0x55e32e92c780 2024-04-27T03:32:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.918+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e9357e0 con 0x7f19e8027030 2024-04-27T03:32:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.918+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.919+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94434227337984] v27'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.919+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94434227344176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e935bd0 con 0x7f19e8027030 2024-04-27T03:32:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.919+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e9365d0 con 0x55e32e928180 2024-04-27T03:32:16.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.920+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.920+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94434227344176] v27'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.920+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94434227342608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e9369c0 con 0x55e32e928180 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.920+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e9373e0 con 0x7f19e0018410 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.921+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94434227342608] v27'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.921+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.921+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94434227351248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e9373a0 con 0x7f19e0018410 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.921+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e937e40 con 0x7f19e0018410 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.922+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94434227351248] v27'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.922+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.922+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94434227353904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e937e00 con 0x7f19e0018410 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.922+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e938820 con 0x7f19e0018410 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.923+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94434227353904] v27'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.923+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'231 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.923+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94434227356432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e9387e0 con 0x7f19e0018410 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.923+0000 7f19f2fa2800 20 add_watcher() i=0 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.923+0000 7f19f2fa2800 20 add_watcher() i=1 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.923+0000 7f19f2fa2800 20 add_watcher() i=2 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.923+0000 7f19f2fa2800 20 add_watcher() i=3 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.923+0000 7f19f2fa2800 20 add_watcher() i=4 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.923+0000 7f19f2fa2800 20 add_watcher() i=5 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.923+0000 7f19f2fa2800 20 add_watcher() i=6 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.924+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94434227356432] v27'232 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.924+0000 7f19f2fa2800 20 add_watcher() i=7 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.924+0000 7f19f2fa2800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.924+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e93bc40 con 0x55e32e928180 2024-04-27T03:32:16.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.925+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.925+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94434227347680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e939160 con 0x55e32e928180 2024-04-27T03:32:16.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.926+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94434227347680] v27'236 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.926+0000 7f18a6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:16.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.926+0000 7f19f2fa2800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:16.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.926+0000 7f19f2fa2800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:16.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.926+0000 7f19f2fa2800 5 note: GC not initialized 2024-04-27T03:32:16.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.926+0000 7f18a5fe3640 20 reqs_thread_entry: start 2024-04-27T03:32:16.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.926+0000 7f19f2fa2800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:16.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.926+0000 7f19f2fa2800 20 Filter name: none 2024-04-27T03:32:16.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.926+0000 7f18177fe640 20 reqs_thread_entry: start 2024-04-27T03:32:16.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.926+0000 7f19f2fa2800 20 rados->read ofs=0 len=0 2024-04-27T03:32:16.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.927+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e96eaf0 con 0x55e32e928180 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.927+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.927+0000 7f19f2fa2800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.927+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.13 6:cea599e4:usage::usage.10:head [call rgw.user_usage_log_read in=63b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e32e970040 con 0x7f19e0018410 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.927+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 usage.10 [call out=15b] v0'0 uv32 ondisk = 0) ==== 152+0+15 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.928+0000 7f18a6fe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.928+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94434227347680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e93b0f0 con 0x55e32e928180 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.929+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 data_loggenerations_metadata [watch unwatch cookie 94434227347680] v27'237 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.929+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94434227331600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e9360e0 con 0x7f19e8027030 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.930+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(65 notify.0 [watch unwatch cookie 94434227331600] v27'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.930+0000 7f19f2fa2800 20 remove_watcher() i=0 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.930+0000 7f19f2fa2800 2 removed watcher, disabling cache 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.930+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94434227336928] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e932210 con 0x55e32e928180 2024-04-27T03:32:16.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.931+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(66 notify.1 [watch unwatch cookie 94434227336928] v27'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.931+0000 7f19f2fa2800 20 remove_watcher() i=1 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.931+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94434227337984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e932210 con 0x55e32e92c780 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.933+0000 7f19f0aa8640 1 -- 172.21.15.165:0/2513595404 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(67 notify.2 [watch unwatch cookie 94434227337984] v27'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e800f030 con 0x55e32e92c780 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.933+0000 7f19f2fa2800 20 remove_watcher() i=2 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.933+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:68 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94434227344176] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e932210 con 0x7f19e8027030 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.933+0000 7f19f02a7640 1 -- 172.21.15.165:0/2513595404 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(68 notify.3 [watch unwatch cookie 94434227344176] v27'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e4081070 con 0x7f19e8027030 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.933+0000 7f19f2fa2800 20 remove_watcher() i=3 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.933+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94434227342608] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e932210 con 0x55e32e928180 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.935+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(69 notify.4 [watch unwatch cookie 94434227342608] v27'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x55e32e928180 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.935+0000 7f19f2fa2800 20 remove_watcher() i=4 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.935+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94434227351248] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e932210 con 0x7f19e0018410 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.936+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(70 notify.5 [watch unwatch cookie 94434227351248] v27'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.936+0000 7f19f2fa2800 20 remove_watcher() i=5 2024-04-27T03:32:16.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.936+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94434227353904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e936ed0 con 0x7f19e0018410 2024-04-27T03:32:16.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.937+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(71 notify.6 [watch unwatch cookie 94434227353904] v27'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e00083e0 con 0x7f19e0018410 2024-04-27T03:32:16.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.937+0000 7f19f2fa2800 20 remove_watcher() i=6 2024-04-27T03:32:16.949 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.937+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94434227356432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e32e932210 con 0x7f19e0018410 2024-04-27T03:32:16.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(72 notify.7 [watch unwatch cookie 94434227356432] v27'233 uv231 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f19e0044040 con 0x7f19e0018410 2024-04-27T03:32:16.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 20 remove_watcher() i=7 2024-04-27T03:32:16.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f19e0018410 msgr2=0x7f19e0018870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f19e0018410 0x7f19e0018870 crc :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e32e928180 msgr2=0x55e32e92a620 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e32e928180 0x55e32e92a620 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f19e8027030 msgr2=0x7f19e8043900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f19e8027030 0x7f19e8043900 crc :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e32e92c780 msgr2=0x55e32e92eb70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e32e92c780 0x55e32e92eb70 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18bc036900 msgr2=0x7f18bc038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18bc036900 0x7f18bc038dc0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f19e401ced0 tx=0x7f19e404f000 comp rx=0 tx=0).stop 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 msgr2=0x55e32e91cbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 0x55e32e91cbc0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f19e8012c70 tx=0x7f19e8043750 comp rx=0 tx=0).stop 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f12a9640 1 -- 172.21.15.165:0/2513595404 reap_dead start 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 shutdown_connections 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/2513595404 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e915940 0x55e32e91cbc0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 >> 172.21.15.165:0/2513595404 conn(0x55e32e911ca0 msgr2=0x55e32e90f300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:16.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 shutdown_connections 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.938+0000 7f19f2fa2800 1 -- 172.21.15.165:0/2513595404 wait complete. 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/4136763483 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18c8036900 msgr2=0x7f18c8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/4136763483 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18c8036900 0x7f18c8038dc0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x55e32e8f6910 tx=0x7f19e4005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/4136763483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e9089d0 msgr2=0x55e32e908df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/4136763483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e9089d0 0x55e32e908df0 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7f19e804ac00 tx=0x7f19e8026ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/4136763483 shutdown_connections 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/4136763483 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18c8036900 0x7f18c8038dc0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/4136763483 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e9089d0 0x55e32e908df0 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/4136763483 >> 172.21.15.165:0/4136763483 conn(0x55e32e8fca70 msgr2=0x55e32e8fce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/4136763483 shutdown_connections 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/4136763483 wait complete. 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e32e7f33c0 msgr2=0x55e32e8f1010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e32e7f33c0 0x55e32e8f1010 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e32e8f7a20 msgr2=0x55e32e8f9eb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e32e8f7a20 0x55e32e8f9eb0 crc :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e32e8f3920 msgr2=0x55e32e8f5de0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e32e8f3920 0x55e32e8f5de0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18d8036900 msgr2=0x7f18d8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f18d8036900 0x7f18d8038dc0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f19e4054000 tx=0x7f19e404e000 comp rx=0 tx=0).stop 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 msgr2=0x55e32e7ef1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 --2- 172.21.15.165:0/3780685120 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e32e7ef8b0 0x55e32e7ef1f0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f19e800f000 tx=0x7f19e8000f40 comp rx=0 tx=0).stop 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f12a9640 1 -- 172.21.15.165:0/3780685120 reap_dead start 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 shutdown_connections 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 >> 172.21.15.165:0/3780685120 conn(0x55e32e7e9380 msgr2=0x55e32e8da1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 shutdown_connections 2024-04-27T03:32:16.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.939+0000 7f19f2fa2800 1 -- 172.21.15.165:0/3780685120 wait complete. 2024-04-27T03:32:16.956 INFO:teuthology.orchestra.run.smithi165.stdout:{"entries":[],"summary":[]} 2024-04-27T03:32:16.957 DEBUG:tasks.util.rgw: json result: {'entries': [], 'summary': []} 2024-04-27T03:32:16.957 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'foo', '--purge-data'] 2024-04-27T03:32:16.957 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'foo', '--purge-data'] 2024-04-27T03:32:16.957 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid foo --purge-data 2024-04-27T03:32:17.000 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:17.001 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:17.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.996+0000 7f7275b37800 1 Processor -- start 2024-04-27T03:32:17.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.996+0000 7f7275b37800 1 -- start start 2024-04-27T03:32:17.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.996+0000 7f7275b37800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55de2817f350 0x55de28181740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.996+0000 7f7275b37800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55de28039d40 con 0x55de2817afb0 2024-04-27T03:32:17.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.996+0000 7f7275b37800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55de2800c4d0 con 0x55de2817f350 2024-04-27T03:32:17.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.996+0000 7f726bfff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55de2817f350 0x55de28181740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.996+0000 7f72718f9640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55de2817afb0 0x55de2817b3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42276/0 (socket says 172.21.15.165:42276) 2024-04-27T03:32:17.002 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.996+0000 7f72718f9640 1 -- 172.21.15.165:0/3666290970 learned_addr learned my addr 172.21.15.165:0/3666290970 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:17.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.997+0000 7f726bfff640 1 -- 172.21.15.165:0/3666290970 >> v1:172.21.15.89:6789/0 conn(0x55de2817afb0 legacy=0x55de2817b3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.003 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.997+0000 7f726bfff640 1 -- 172.21.15.165:0/3666290970 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de28038590 con 0x55de2817f350 2024-04-27T03:32:17.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.999+0000 7f726bfff640 1 --2- 172.21.15.165:0/3666290970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817f350 0x55de28181740 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f725c00a470 tx=0x7f725c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa18c3a9d74b0729 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.999+0000 7f72710f8640 1 -- 172.21.15.165:0/3666290970 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f725c057070 con 0x55de2817f350 2024-04-27T03:32:17.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.999+0000 7f72710f8640 1 -- 172.21.15.165:0/3666290970 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f725c052570 con 0x55de2817f350 2024-04-27T03:32:17.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.999+0000 7f72710f8640 1 -- 172.21.15.165:0/3666290970 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f725c052870 con 0x55de2817f350 2024-04-27T03:32:17.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.999+0000 7f7275b37800 1 -- 172.21.15.165:0/3666290970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817f350 msgr2=0x55de28181740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.999+0000 7f7275b37800 1 --2- 172.21.15.165:0/3666290970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817f350 0x55de28181740 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f725c00a470 tx=0x7f725c01ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:17.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.999+0000 7f7275b37800 1 -- 172.21.15.165:0/3666290970 shutdown_connections 2024-04-27T03:32:17.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.999+0000 7f7275b37800 1 --2- 172.21.15.165:0/3666290970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817f350 0x55de28181740 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.999+0000 7f7275b37800 1 -- 172.21.15.165:0/3666290970 >> 172.21.15.165:0/3666290970 conn(0x55de28176550 msgr2=0x55de28178970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.999+0000 7f7275b37800 1 -- 172.21.15.165:0/3666290970 shutdown_connections 2024-04-27T03:32:17.005 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:16.999+0000 7f7275b37800 1 -- 172.21.15.165:0/3666290970 wait complete. 2024-04-27T03:32:17.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.011+0000 7f7275b37800 1 Processor -- start 2024-04-27T03:32:17.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.011+0000 7f7275b37800 1 -- start start 2024-04-27T03:32:17.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.011+0000 7f7275b37800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817afb0 0x55de2826c110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.011+0000 7f7275b37800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55de280c63b0 con 0x55de2817afb0 2024-04-27T03:32:17.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.012+0000 7f72718f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817afb0 0x55de2826c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.012+0000 7f72718f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817afb0 0x55de2826c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59474/0 (socket says 172.21.15.165:59474) 2024-04-27T03:32:17.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.012+0000 7f72718f9640 1 -- 172.21.15.165:0/3813163394 learned_addr learned my addr 172.21.15.165:0/3813163394 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.012+0000 7f72718f9640 1 -- 172.21.15.165:0/3813163394 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de2826c6e0 con 0x55de2817afb0 2024-04-27T03:32:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.012+0000 7f72718f9640 1 --2- 172.21.15.165:0/3813163394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817afb0 0x55de2826c110 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7264001e10 tx=0x7f726400b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d1b82bb5e0d973 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.012+0000 7f715fe07640 1 -- 172.21.15.165:0/3813163394 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7264012020 con 0x55de2817afb0 2024-04-27T03:32:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.012+0000 7f715fe07640 1 -- 172.21.15.165:0/3813163394 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f726400e5a0 con 0x55de2817afb0 2024-04-27T03:32:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.012+0000 7f715fe07640 1 -- 172.21.15.165:0/3813163394 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f726400e8e0 con 0x55de2817afb0 2024-04-27T03:32:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.013+0000 7f7275b37800 1 -- 172.21.15.165:0/3813163394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817afb0 msgr2=0x55de2826c110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.013+0000 7f7275b37800 1 --2- 172.21.15.165:0/3813163394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817afb0 0x55de2826c110 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f7264001e10 tx=0x7f726400b780 comp rx=0 tx=0).stop 2024-04-27T03:32:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.013+0000 7f7275b37800 1 -- 172.21.15.165:0/3813163394 shutdown_connections 2024-04-27T03:32:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.013+0000 7f7275b37800 1 --2- 172.21.15.165:0/3813163394 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817afb0 0x55de2826c110 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.013+0000 7f7275b37800 1 -- 172.21.15.165:0/3813163394 >> 172.21.15.165:0/3813163394 conn(0x55de28176550 msgr2=0x55de28267a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.020 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.013+0000 7f7275b37800 1 -- 172.21.15.165:0/3813163394 shutdown_connections 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.013+0000 7f7275b37800 1 -- 172.21.15.165:0/3813163394 wait complete. 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.013+0000 7f7275b37800 1 Processor -- start 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.013+0000 7f7275b37800 1 -- start start 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.013+0000 7f7275b37800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817f350 0x55de28273ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.014+0000 7f7275b37800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55de28182010 con 0x55de2817f350 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.014+0000 7f72718f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817f350 0x55de28273ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.014+0000 7f72718f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817f350 0x55de28273ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59486/0 (socket says 172.21.15.165:59486) 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.014+0000 7f72718f9640 1 -- 172.21.15.165:0/1800477278 learned_addr learned my addr 172.21.15.165:0/1800477278 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.014+0000 7f72718f9640 1 -- 172.21.15.165:0/1800477278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de28274430 con 0x55de2817f350 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.014+0000 7f72718f9640 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817f350 0x55de28273ef0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7264001e10 tx=0x7f7264000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.014+0000 7f715e604640 1 -- 172.21.15.165:0/1800477278 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7264012030 con 0x55de2817f350 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.014+0000 7f715e604640 1 -- 172.21.15.165:0/1800477278 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f726401a700 con 0x55de2817f350 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.014+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55de282746c0 con 0x55de2817f350 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.014+0000 7f715e604640 1 -- 172.21.15.165:0/1800477278 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f726401aa20 con 0x55de2817f350 2024-04-27T03:32:17.021 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.014+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de28277370 con 0x55de2817f350 2024-04-27T03:32:17.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.015+0000 7f715e604640 1 -- 172.21.15.165:0/1800477278 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f7264015020 con 0x55de2817f350 2024-04-27T03:32:17.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.015+0000 7f715e604640 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7158036900 0x7f7158038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.015+0000 7f715e604640 1 -- 172.21.15.165:0/1800477278 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f7264020030 con 0x55de2817f350 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.015+0000 7f726bfff640 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7158036900 0x7f7158038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.016+0000 7f7275b37800 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55de28279120 0x55de2827b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.016+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2827ba50 con 0x55de28279120 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.016+0000 7f72720fa640 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55de28279120 0x55de2827b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.016+0000 7f726bfff640 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7158036900 0x7f7158038dc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f725c054000 tx=0x7f725c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.016+0000 7f72720fa640 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55de28279120 0x55de2827b510 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.016+0000 7f72720fa640 1 -- 172.21.15.165:0/1800477278 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de28279120 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.016+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28278700 con 0x55de28279120 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.017+0000 7f72720fa640 1 -- 172.21.15.165:0/1800477278 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de28279120 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.017+0000 7f7275b37800 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55de2827dd60 0x55de28280150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.017+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28280840 con 0x55de2827dd60 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.017+0000 7f72718f9640 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55de2827dd60 0x55de28280150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.017+0000 7f72718f9640 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55de2827dd60 0x55de28280150 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.017+0000 7f72718f9640 1 -- 172.21.15.165:0/1800477278 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f726400dcd0 con 0x55de2827dd60 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.017+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2827cd10 con 0x55de28279120 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.017+0000 7f72720fa640 1 -- 172.21.15.165:0/1800477278 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de28279120 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.018+0000 7f7275b37800 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55de28281cf0 0x55de28284140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.018+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282848a0 con 0x55de28281cf0 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.018+0000 7f726bfff640 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55de28281cf0 0x55de28284140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.018+0000 7f726bfff640 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55de28281cf0 0x55de28284140 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.018+0000 7f726bfff640 1 -- 172.21.15.165:0/1800477278 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f725c052940 con 0x55de28281cf0 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.018+0000 7f7275b37800 1 Processor -- start 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.019+0000 7f7275b37800 1 -- start start 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.019+0000 7f7275b37800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de280f0c40 0x55de280ef0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.019+0000 7f7275b37800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55de2806c6b0 con 0x55de280f0c40 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.019+0000 7f72718f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de280f0c40 0x55de280ef0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.019+0000 7f72718f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de280f0c40 0x55de280ef0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59488/0 (socket says 172.21.15.165:59488) 2024-04-27T03:32:17.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.019+0000 7f72718f9640 1 -- 172.21.15.165:0/3132725868 learned_addr learned my addr 172.21.15.165:0/3132725868 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.019+0000 7f72718f9640 1 -- 172.21.15.165:0/3132725868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de280ef5f0 con 0x55de280f0c40 2024-04-27T03:32:17.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.019+0000 7f72718f9640 1 --2- 172.21.15.165:0/3132725868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de280f0c40 0x55de280ef0b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f726404c010 tx=0x7f72640184f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.020+0000 7f71577fe640 1 -- 172.21.15.165:0/3132725868 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7264017d80 con 0x55de280f0c40 2024-04-27T03:32:17.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.020+0000 7f7275b37800 1 -- 172.21.15.165:0/3132725868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55de280ef880 con 0x55de280f0c40 2024-04-27T03:32:17.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.020+0000 7f71577fe640 1 -- 172.21.15.165:0/3132725868 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7264043070 con 0x55de280f0c40 2024-04-27T03:32:17.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.020+0000 7f71577fe640 1 -- 172.21.15.165:0/3132725868 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7264047180 con 0x55de280f0c40 2024-04-27T03:32:17.028 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.020+0000 7f7275b37800 1 -- 172.21.15.165:0/3132725868 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de280efd80 con 0x55de280f0c40 2024-04-27T03:32:17.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.020+0000 7f71577fe640 1 -- 172.21.15.165:0/3132725868 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f726403c060 con 0x55de280f0c40 2024-04-27T03:32:17.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.020+0000 7f71577fe640 1 --2- 172.21.15.165:0/3132725868 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f714c036900 0x7f714c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.020+0000 7f71577fe640 1 -- 172.21.15.165:0/3132725868 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f726402e060 con 0x55de280f0c40 2024-04-27T03:32:17.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.021+0000 7f726bfff640 1 --2- 172.21.15.165:0/3132725868 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f714c036900 0x7f714c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.021+0000 7f7275b37800 1 Processor -- start 2024-04-27T03:32:17.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.021+0000 7f7275b37800 1 -- start start 2024-04-27T03:32:17.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.021+0000 7f726bfff640 1 --2- 172.21.15.165:0/3132725868 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f714c036900 0x7f714c038dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55de2827c310 tx=0x7f725c005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.021+0000 7f7275b37800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 0x55de282a2440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.021+0000 7f7275b37800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55de2826ca60 con 0x55de282a2020 2024-04-27T03:32:17.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.021+0000 7f72718f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 0x55de282a2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.021+0000 7f72718f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 0x55de282a2440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59502/0 (socket says 172.21.15.165:59502) 2024-04-27T03:32:17.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.021+0000 7f72718f9640 1 -- 172.21.15.165:0/559163139 learned_addr learned my addr 172.21.15.165:0/559163139 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.022+0000 7f72718f9640 1 -- 172.21.15.165:0/559163139 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de282a2a10 con 0x55de282a2020 2024-04-27T03:32:17.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.022+0000 7f72718f9640 1 --2- 172.21.15.165:0/559163139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 0x55de282a2440 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7264018740 tx=0x7f726400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd545436617c113e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.022+0000 7f71557fa640 1 -- 172.21.15.165:0/559163139 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f726405c020 con 0x55de282a2020 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.022+0000 7f71557fa640 1 -- 172.21.15.165:0/559163139 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7264041e20 con 0x55de282a2020 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.022+0000 7f71557fa640 1 -- 172.21.15.165:0/559163139 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f72640479b0 con 0x55de282a2020 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.022+0000 7f7275b37800 1 -- 172.21.15.165:0/559163139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 msgr2=0x55de282a2440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.022+0000 7f7275b37800 1 --2- 172.21.15.165:0/559163139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 0x55de282a2440 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f7264018740 tx=0x7f726400dbe0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.022+0000 7f7275b37800 1 -- 172.21.15.165:0/559163139 shutdown_connections 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.022+0000 7f7275b37800 1 --2- 172.21.15.165:0/559163139 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 0x55de282a2440 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.022+0000 7f7275b37800 1 -- 172.21.15.165:0/559163139 >> 172.21.15.165:0/559163139 conn(0x55de2829e380 msgr2=0x55de2829c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.022+0000 7f7275b37800 1 -- 172.21.15.165:0/559163139 shutdown_connections 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.022+0000 7f7275b37800 1 -- 172.21.15.165:0/559163139 wait complete. 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.023+0000 7f7275b37800 1 Processor -- start 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.023+0000 7f7275b37800 1 -- start start 2024-04-27T03:32:17.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.023+0000 7f7275b37800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 0x55de282a9430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.023+0000 7f7275b37800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55de282a30c0 con 0x55de282a2020 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.023+0000 7f72718f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 0x55de282a9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.023+0000 7f72718f9640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 0x55de282a9430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59516/0 (socket says 172.21.15.165:59516) 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.023+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 learned_addr learned my addr 172.21.15.165:0/4253761301 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.023+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de282a9970 con 0x55de282a2020 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.024+0000 7f72718f9640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 0x55de282a9430 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7264018740 tx=0x7f7264014f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.024+0000 7f71437fe640 1 -- 172.21.15.165:0/4253761301 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7264071070 con 0x55de282a2020 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.024+0000 7f71437fe640 1 -- 172.21.15.165:0/4253761301 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f726400d7a0 con 0x55de282a2020 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.024+0000 7f71437fe640 1 -- 172.21.15.165:0/4253761301 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f726400d3d0 con 0x55de282a2020 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.024+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55de282a9970 con 0x55de282a2020 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.024+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de282a9970 con 0x55de282a2020 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.025+0000 7f71437fe640 1 -- 172.21.15.165:0/4253761301 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f7264074070 con 0x55de282a2020 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.025+0000 7f71437fe640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f713c036900 0x7f713c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.040 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.025+0000 7f71437fe640 1 -- 172.21.15.165:0/4253761301 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f7264069070 con 0x55de282a2020 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.025+0000 7f726bfff640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f713c036900 0x7f713c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.025+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.025+0000 7f7275b37800 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55de282b4b30 0x55de282b6ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.025+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282b7740 con 0x55de282b4b30 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.025+0000 7f72720fa640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55de282b4b30 0x55de282b6ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.025+0000 7f726bfff640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f713c036900 0x7f713c038dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f725c052fd0 tx=0x7f725c04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f72720fa640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55de282b4b30 0x55de282b6ff0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f7275b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f7275b37800 20 realm 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282b7b50 con 0x55de282b4b30 2024-04-27T03:32:17.041 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f7275b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f7275b37800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282b8420 con 0x55de282b4b30 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f7275b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.026+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b4b30 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.027+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.027+0000 7f7275b37800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.027+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.027+0000 7f7275b37800 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55de282b9520 0x55de282bb910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.042 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.027+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282bbf20 con 0x55de282b9520 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.027+0000 7f72718f9640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55de282b9520 0x55de282bb910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.027+0000 7f72718f9640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55de282b9520 0x55de282bb910 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.027+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.027+0000 7f7275b37800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.027+0000 7f7275b37800 20 searching for the correct realm 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.027+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282bd830 con 0x55de282b9520 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.028+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.028+0000 7f72718f9640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7264076060 0x7f726405d980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.028+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282bd830 con 0x7f7264076060 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.028+0000 7f726bfff640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7264076060 0x7f726405d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.028+0000 7f726bfff640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7264076060 0x7f726405d980 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.029+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.043 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.029+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282bd830 con 0x7f7264076060 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.029+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.029+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282bd830 con 0x55de282b4b30 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.029+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.029+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b4b30 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.029+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.029+0000 7f72720fa640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f726c016440 0x7f726c0168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.029+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f726c016440 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.030+0000 7f72720fa640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f726c016440 0x7f726c0168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.030+0000 7f72720fa640 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f726c016440 0x7f726c0168c0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.030+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.030+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f726c016440 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.030+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.030+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f7264076060 2024-04-27T03:32:17.044 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.031+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.031+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b9520 2024-04-27T03:32:17.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.031+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.031+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b4b30 2024-04-27T03:32:17.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.031+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.031+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f7264076060 2024-04-27T03:32:17.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.032+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.032+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b4b30 2024-04-27T03:32:17.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.032+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.032+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f7264076060 2024-04-27T03:32:17.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.032+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.032+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b9520 2024-04-27T03:32:17.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.032+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.032+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b9520 2024-04-27T03:32:17.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.033+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.033+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f726c016440 2024-04-27T03:32:17.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.033+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.033+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b9520 2024-04-27T03:32:17.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.033+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.033+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b9520 2024-04-27T03:32:17.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.033+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.054 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.033+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b4b30 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.034+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.034+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b4b30 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.034+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.034+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f726c016440 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.034+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.034+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b9520 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.034+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.034+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f7264076060 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.035+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.035+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b4b30 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.035+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.035+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f726c016440 2024-04-27T03:32:17.055 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.035+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.035+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b9520 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.035+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.035+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f726c016440 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.036+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.036+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f7264076060 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.036+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.036+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b9520 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.036+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.036+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f726c016440 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.036+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.036+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x55de282b4b30 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55de282b87c0 con 0x7f7264076060 2024-04-27T03:32:17.056 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282bc7e0 con 0x55de282b4b30 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282bcbd0 con 0x55de282b4b30 2024-04-27T03:32:17.057 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.037+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282bcbd0 con 0x7f7264076060 2024-04-27T03:32:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f7275b37800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f7275b37800 20 zone default found 2024-04-27T03:32:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f7275b37800 4 Realm: () 2024-04-27T03:32:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f7275b37800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f7275b37800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f7275b37800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:17.058 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f7275b37800 20 zonegroup default 2024-04-27T03:32:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282bdcb0 con 0x7f726c016440 2024-04-27T03:32:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f7275b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.038+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282bf690 con 0x55de282b9520 2024-04-27T03:32:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.039+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.039+0000 7f7275b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.039+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.039+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282bfa80 con 0x55de282b9520 2024-04-27T03:32:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.039+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.059 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.039+0000 7f7275b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.039+0000 7f7275b37800 20 started sync module instance, tier type = 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.039+0000 7f7275b37800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.039+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282beb30 con 0x7f7264076060 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.040+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.040+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94412645068624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282be790 con 0x7f7264076060 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.040+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c09e0 con 0x55de282b4b30 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.041+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94412645068624] v27'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.041+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.041+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94412645074224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c0950 con 0x55de282b4b30 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.041+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c18a0 con 0x55de282b9520 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.042+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94412645074224] v27'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.043+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.043+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94412645078000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c1c90 con 0x55de282b9520 2024-04-27T03:32:17.060 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.043+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c2690 con 0x7f7264076060 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.044+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.044+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94412645083776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c3350 con 0x7f7264076060 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.044+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c3d90 con 0x55de282b4b30 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.044+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94412645078000] v27'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.045+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.045+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94412645087328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c4180 con 0x55de282b4b30 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.045+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c4be0 con 0x7f726c016440 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.046+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.046+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94412645083776] v27'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.046+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94412645091008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c4be0 con 0x7f726c016440 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.046+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c5630 con 0x7f726c016440 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.046+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94412645087328] v27'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.047+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94412645091008] v27'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.048+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.048+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94412645084832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c5140 con 0x7f726c016440 2024-04-27T03:32:17.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.048+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c5b00 con 0x7f726c016440 2024-04-27T03:32:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.049+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94412645084832] v27'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.049+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'234 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.049+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94412645076432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c5b00 con 0x7f726c016440 2024-04-27T03:32:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.049+0000 7f7275b37800 20 add_watcher() i=0 2024-04-27T03:32:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.049+0000 7f7275b37800 20 add_watcher() i=1 2024-04-27T03:32:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.049+0000 7f7275b37800 20 add_watcher() i=2 2024-04-27T03:32:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.049+0000 7f7275b37800 20 add_watcher() i=3 2024-04-27T03:32:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.049+0000 7f7275b37800 20 add_watcher() i=4 2024-04-27T03:32:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.049+0000 7f7275b37800 20 add_watcher() i=5 2024-04-27T03:32:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.049+0000 7f7275b37800 20 add_watcher() i=6 2024-04-27T03:32:17.065 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.050+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94412645076432] v27'235 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.050+0000 7f7275b37800 20 add_watcher() i=7 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.050+0000 7f7275b37800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.050+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282c8e70 con 0x55de282b4b30 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.050+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.050+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94412645107808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c5c70 con 0x55de282b4b30 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.051+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94412645107808] v27'238 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.051+0000 7f71237fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.051+0000 7f7275b37800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.051+0000 7f7275b37800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.052+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d1810 con 0x55de282b4b30 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.053+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'58 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.053+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d1ee0 con 0x7f726c016440 2024-04-27T03:32:17.066 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.054+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.054+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d2300 con 0x55de282b9520 2024-04-27T03:32:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.055+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'36 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.055+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d2230 con 0x55de282b4b30 2024-04-27T03:32:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.056+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'239 uv172 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.056+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d2900 con 0x7f7264076060 2024-04-27T03:32:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.057+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'80 uv59 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.057+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d2d20 con 0x55de282b9520 2024-04-27T03:32:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.058+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'63 uv49 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.058+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d2d20 con 0x55de282b4b30 2024-04-27T03:32:17.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.060+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'36 uv29 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.060+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d2db0 con 0x7f726c016440 2024-04-27T03:32:17.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.061+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'55 uv42 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.061+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d2e40 con 0x7f7264076060 2024-04-27T03:32:17.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.062+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'28 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.062+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3470 con 0x55de282b9520 2024-04-27T03:32:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.063+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.063+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3470 con 0x7f7264076060 2024-04-27T03:32:17.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.064+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'81 uv61 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.064+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x7f726c016440 2024-04-27T03:32:17.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.065+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'34 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.065+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x7f7264076060 2024-04-27T03:32:17.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.066+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.071 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.066+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x7f726c016440 2024-04-27T03:32:17.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.067+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'67 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.073 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.067+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x55de282b4b30 2024-04-27T03:32:17.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.068+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'240 uv168 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.074 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.068+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x55de282b9520 2024-04-27T03:32:17.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.070+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'75 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.075 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.070+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x7f7264076060 2024-04-27T03:32:17.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.071+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'82 uv57 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.076 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.071+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x55de282b9520 2024-04-27T03:32:17.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.072+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'40 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.077 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.072+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x55de282b4b30 2024-04-27T03:32:17.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.073+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'36 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.079 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.073+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x55de282b4b30 2024-04-27T03:32:17.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.074+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'241 uv170 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.080 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.074+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x55de282b9520 2024-04-27T03:32:17.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.076+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'76 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.081 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.076+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x7f7264076060 2024-04-27T03:32:17.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.077+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'44 uv37 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.082 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.077+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x55de282b9520 2024-04-27T03:32:17.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.078+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'36 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.078+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x55de282b9520 2024-04-27T03:32:17.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.079+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'47 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.079+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x55de282b9520 2024-04-27T03:32:17.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.079+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'30 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.080+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x55de282b9520 2024-04-27T03:32:17.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.080+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'64 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.080+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x7f726c016440 2024-04-27T03:32:17.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.082+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'38 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.082+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x55de282b9520 2024-04-27T03:32:17.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.083+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'41 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.083+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x7f726c016440 2024-04-27T03:32:17.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.084+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'56 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.084+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x55de282b9520 2024-04-27T03:32:17.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.085+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'77 uv56 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.085+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x7f726c016440 2024-04-27T03:32:17.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.086+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'68 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.086+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282d3920 con 0x7f7264076060 2024-04-27T03:32:17.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.087+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'34 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f71227fc640 20 reqs_thread_entry: start 2024-04-27T03:32:17.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f7275b37800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:17.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f7275b37800 20 Filter name: none 2024-04-27T03:32:17.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f7093fff640 20 reqs_thread_entry: start 2024-04-27T03:32:17.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f7275b37800 10 cache get: name=default.rgw.meta+users.uid+foo : miss 2024-04-27T03:32:17.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:94 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282fe8c0 con 0x55de282b4b30 2024-04-27T03:32:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(94 foo [call out=48b,read 0~391 out=391b,getxattrs out=75b] v0'0 uv15 ondisk = 0) ==== 231+0+514 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f7275b37800 20 rados_obj.operate() r=0 bl.length=391 2024-04-27T03:32:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f7275b37800 10 cache put: name=default.rgw.meta+users.uid+foo info.flags=0x13 2024-04-27T03:32:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f7275b37800 10 adding default.rgw.meta+users.uid+foo to cache LRU end 2024-04-27T03:32:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f7275b37800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.088+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:95 8.9 8:96d98480:users.uid::foo.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282fe880 con 0x7f7264076060 2024-04-27T03:32:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.089+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== osd_op_reply(95 foo.buckets [call out=183b] v0'0 uv43 ondisk = 0) ==== 155+0+183 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.089+0000 7f7275b37800 10 cache get: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7 : miss 2024-04-27T03:32:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.089+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.089+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:96 8.13 8:cb80e7bd:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7:head [call version.read in=11b,stat,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28300ee0 con 0x7f726c016440 2024-04-27T03:32:17.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.089+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(96 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7 [call out=48b,stat out=16b,read 0~379 out=379b,getxattrs out=218b] v0'0 uv9 ondisk = 0) ==== 332+0+661 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.089+0000 7f7275b37800 20 rados_obj.operate() r=0 bl.length=379 2024-04-27T03:32:17.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.089+0000 7f7275b37800 10 cache put: name=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7 info.flags=0x17 2024-04-27T03:32:17.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.089+0000 7f7275b37800 10 adding default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7 to cache LRU end 2024-04-27T03:32:17.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.089+0000 7f7275b37800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:17.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.089+0000 7f7275b37800 10 updating xattr: name=user.rgw.acl bl.length()=123 2024-04-27T03:32:17.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.089+0000 7f7275b37800 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7 2024-04-27T03:32:17.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.089+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:97 4.26 4:64b8297a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2837e9f0 con 0x7f726c016440 2024-04-27T03:32:17.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.090+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(97 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.090+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:98 4.f 4:f24dbbfc:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2837e9f0 con 0x55de282b4b30 2024-04-27T03:32:17.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.090+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(98 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.1 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c00a4a0 con 0x55de282b4b30 2024-04-27T03:32:17.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.090+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:99 4.2d 4:b6586b9f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2837e9f0 con 0x7f726c016440 2024-04-27T03:32:17.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.091+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(99 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c00a4a0 con 0x7f726c016440 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.091+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:100 4.2e 4:779e3a5e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2837ea30 con 0x55de282b9520 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.091+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(100 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.091+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:101 4.36 4:6ed8ef8f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2837ea30 con 0x55de282b4b30 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.091+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(101 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4 [call out=426b] v0'0 uv7 ondisk = 0) ==== 194+0+426 (crc 0 0 0) 0x7f726c0501c0 con 0x55de282b4b30 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.091+0000 7f7275b37800 10 cls_bucket_list_unordered: got 15f6f8dc036519d7fe15b39338f6e5db 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.091+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:102 4.16 4:68001f6a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2837ea30 con 0x55de282b9520 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.092+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(102 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.5 [call out=85b] v0'0 uv7 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.092+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:103 4.21 4:8581bfa3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2837ea30 con 0x7f726c016440 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.092+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(103 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c0501c0 con 0x7f726c016440 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.092+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:104 4.d 4:b11585fa:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2837f040 con 0x55de282b9520 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.093+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 29 ==== osd_op_reply(104 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.093+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:105 4.17 4:eb32c994:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2837f040 con 0x7f726c016440 2024-04-27T03:32:17.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.093+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(105 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c0501c0 con 0x7f726c016440 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.093+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:106 4.1c 4:39c0a4a8:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2837f040 con 0x7f726c016440 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.094+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(106 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.9 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c0501c0 con 0x7f726c016440 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.094+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:107 4.37 4:efd4b2ac:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2837f040 con 0x7f726c016440 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.094+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(107 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.094+0000 7f7275b37800 20 get_obj_state: octx=0x7f725c07b040 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x55de2837cc88 s->prefetch_data=0 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.094+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:108 3.2 3:439dba6a:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7_15f6f8dc036519d7fe15b39338f6e5db:head [getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2837db10 con 0x55de282b9520 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.094+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 30 ==== osd_op_reply(108 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7_15f6f8dc036519d7fe15b39338f6e5db [getxattrs out=879b,stat out=16b] v0'0 uv1 ondisk = 0) ==== 262+0+895 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.094+0000 7f7275b37800 10 manifest: total_size = 6 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.094+0000 7f7275b37800 20 get_obj_state: setting s->obj_tag to 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11184265405899080943 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.094+0000 7f7275b37800 20 get_obj_state: octx=0x7f725c07b040 obj=myfoo:15f6f8dc036519d7fe15b39338f6e5db state=0x55de2837cc88 s->prefetch_data=0 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.094+0000 7f7275b37800 20 prepare_atomic_modification: state is not atomic. state=0x55de2837cc88 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.094+0000 7f7275b37800 20 bucket index object: :.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4 2024-04-27T03:32:17.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.094+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:109 4.36 4:6ed8ef8f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_prepare_op in=210b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de2837e4d0 con 0x55de282b4b30 2024-04-27T03:32:17.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.095+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(109 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4 [stat,call,call] v27'10 uv10 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.095+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:110 3.2 3:439dba6a:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7_15f6f8dc036519d7fe15b39338f6e5db:head [call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de28303b50 con 0x55de282b9520 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.096+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 31 ==== osd_op_reply(110 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7_15f6f8dc036519d7fe15b39338f6e5db [call] v27'2 uv2 ondisk = 0) ==== 220+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.096+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:111 4.36 4:6ed8ef8f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4:head [stat,call rgw.guard_bucket_resharding in=36b,call rgw.bucket_complete_op in=287b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de28306240 con 0x55de282b4b30 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 20 list_objects_ordered: starting attempt 1 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 10 cls_bucket_list_ordered: request from each of 11 shard(s) for 112 entries to get 1001 total entries 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:112 4.26 4:64b8297a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.0:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28306240 con 0x7f726c016440 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:113 4.f 4:f24dbbfc:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.1:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2830a510 con 0x55de282b4b30 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:114 4.2d 4:b6586b9f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.2:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28306240 con 0x7f726c016440 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:115 4.2e 4:779e3a5e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.3:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2830f2e0 con 0x55de282b9520 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:116 4.36 4:6ed8ef8f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28311bf0 con 0x55de282b4b30 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:117 4.16 4:68001f6a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.5:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de283144c0 con 0x55de282b9520 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:118 4.21 4:8581bfa3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.6:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28316db0 con 0x7f726c016440 2024-04-27T03:32:17.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:119 4.d 4:b11585fa:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.7:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28319b70 con 0x55de282b9520 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:120 4.17 4:eb32c994:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.8:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2831c9f0 con 0x7f726c016440 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:121 4.1c 4:39c0a4a8:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.9:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2831f2e0 con 0x7f726c016440 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:122 4.37 4:efd4b2ac:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.10:head [call rgw.bucket_list in=69b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28321da0 con 0x7f726c016440 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 32 ==== osd_op_reply(115 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(113 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.1 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 33 ==== osd_op_reply(119 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 34 ==== osd_op_reply(117 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.5 [call out=85b] v0'0 uv7 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== osd_op_reply(112 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 30 ==== osd_op_reply(114 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 31 ==== osd_op_reply(118 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 32 ==== osd_op_reply(122 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 33 ==== osd_op_reply(120 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.097+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 34 ==== osd_op_reply(121 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.9 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(111 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4 [stat,call,call] v27'11 uv11 ondisk = 0) ==== 278+0+0 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f71557fa640 20 handle_completion(): completion ok for obj=15f6f8dc036519d7fe15b39338f6e5db 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(116 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4 [call out=124b] v0'0 uv11 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 20 cls_bucket_list_ordered: returning, count=0, is_truncated=0 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 20 cls_bucket_list_ordered: returning, last_entry NOT SET 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 10 list_objects_ordered: end of outer loop, truncated=0, count=0, attempt=1 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 10 list_objects_ordered: exiting attempt loop because we reached end (0) or we're returning half the requested entries (0 of 1000) 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 20 abort_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":myfoo[708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7])"; uploads.size()=0; is_truncated=0 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:123 4.26 4:64b8297a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.0:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28321da0 con 0x7f726c016440 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:124 4.f 4:f24dbbfc:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.1:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28320790 con 0x55de282b4b30 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:125 4.2d 4:b6586b9f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.2:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28313450 con 0x7f726c016440 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:126 4.2e 4:779e3a5e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.3:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28321da0 con 0x55de282b9520 2024-04-27T03:32:17.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:127 4.36 4:6ed8ef8f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28313450 con 0x55de282b4b30 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:128 4.16 4:68001f6a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.5:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28320790 con 0x55de282b9520 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:129 4.21 4:8581bfa3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.6:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28313450 con 0x7f726c016440 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:130 4.d 4:b11585fa:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.7:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28314220 con 0x55de282b9520 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:131 4.17 4:eb32c994:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.8:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28313450 con 0x7f726c016440 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:132 4.1c 4:39c0a4a8:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.9:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28314220 con 0x7f726c016440 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.098+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:133 4.37 4:efd4b2ac:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.10:head [call rgw.bucket_list in=47b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2831eec0 con 0x7f726c016440 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.099+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 35 ==== osd_op_reply(123 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.0 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.099+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 36 ==== osd_op_reply(129 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.6 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.099+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 35 ==== osd_op_reply(126 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.3 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.099+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 37 ==== osd_op_reply(125 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.2 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.099+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 36 ==== osd_op_reply(128 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.5 [call out=85b] v0'0 uv7 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.099+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(124 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.1 [call out=85b] v0'0 uv5 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.099+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(127 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4 [call out=124b] v0'0 uv11 ondisk = 0) ==== 194+0+124 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.099+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 37 ==== osd_op_reply(130 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.7 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.099+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 38 ==== osd_op_reply(131 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.8 [call out=85b] v0'0 uv1 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.099+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 39 ==== osd_op_reply(132 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.9 [call out=85b] v0'0 uv3 ondisk = 0) ==== 194+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.099+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 40 ==== osd_op_reply(133 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.10 [call out=85b] v0'0 uv3 ondisk = 0) ==== 195+0+85 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.099+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:134 8.9 8:96d98480:users.uid::foo.buckets:head [call user.set_buckets_info in=207b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de283138c0 con 0x7f7264076060 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.100+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(134 foo.buckets [call] v27'46 uv46 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.100+0000 7f7275b37800 10 cache get: name=default.rgw.meta+root+myfoo : miss 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.100+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.100+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:135 8.9 8:91b71841:root::myfoo:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de283138c0 con 0x7f7264076060 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.100+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 23 ==== osd_op_reply(135 myfoo [call out=48b,read 0~165 out=165b] v0'0 uv40 ondisk = 0) ==== 191+0+213 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.100+0000 7f7275b37800 20 rados_obj.operate() r=0 bl.length=165 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.100+0000 7f7275b37800 10 cache put: name=default.rgw.meta+root+myfoo info.flags=0x11 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.100+0000 7f7275b37800 10 adding default.rgw.meta+root+myfoo to cache LRU end 2024-04-27T03:32:17.117 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.100+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:136 8.9 8:91b71841:root::myfoo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de283203a0 con 0x7f7264076060 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.101+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 24 ==== osd_op_reply(136 myfoo [call,call,delete] v27'47 uv47 ondisk = 0) ==== 233+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.101+0000 7f7275b37800 10 removing default.rgw.meta+root+myfoo from cache 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.101+0000 7f7275b37800 10 distributing notification oid=default.rgw.control:notify.6 cni=[op: 1, obj: default.rgw.meta:root:myfoo, ofs0, ns] 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.101+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:137 7.14 7:2b04a3e9:::notify.6:head [notify cookie 94412645438512 in=161b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de2831f840 con 0x7f726c016440 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.102+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 41 ==== watch-notify(notify (1) cookie 94412645084832 notify 115964117020 ret 0) ==== 191+0+0 (crc 0 0 0) 0x55de28300430 con 0x7f726c016440 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.102+0000 7f71557fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117020 cookie 94412645084832 notifier 4770 bl.length()=149 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.102+0000 7f71557fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:138 7.14 7:2b04a3e9:::notify.6:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f7134003d60 con 0x7f726c016440 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.102+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 42 ==== osd_op_reply(137 notify.6 [notify cookie 94412645438512 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.102+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 43 ==== osd_op_reply(138 notify.6 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.102+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 44 ==== watch-notify(notify_complete (2) cookie 94412645438512 notify 115964117020 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55de280ef880 con 0x7f726c016440 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.102+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:139 8.13 8:cb80e7bd:root::.bucket.meta.myfoo%3a708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7:head [call version.check_conds in=122b,call version.inc in=38b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de283203a0 con 0x7f726c016440 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.103+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 45 ==== osd_op_reply(139 .bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7 [call,call,delete] v27'12 uv12 ondisk = 0) ==== 290+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.118 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.103+0000 7f7275b37800 10 removing default.rgw.meta+root+.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7 from cache 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.103+0000 7f7275b37800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:root:.bucket.meta.myfoo:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7, ofs0, ns] 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.103+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:140 7.0 7:05bf5b68:::notify.1:head [notify cookie 94412645438512 in=218b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28320fa0 con 0x55de282b4b30 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.103+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 36 ==== watch-notify(notify (1) cookie 94412645074224 notify 115964117035 ret 0) ==== 248+0+0 (crc 0 0 0) 0x55de280ef880 con 0x55de282b4b30 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.103+0000 7f7154ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117035 cookie 94412645074224 notifier 4770 bl.length()=206 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f7154ff9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:141 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f709c0032c0 con 0x55de282b4b30 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 37 ==== osd_op_reply(140 notify.1 [notify cookie 94412645438512 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 38 ==== osd_op_reply(141 notify.1 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 39 ==== watch-notify(notify_complete (2) cookie 94412645438512 notify 115964117035 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55de282d19a0 con 0x55de282b4b30 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:142 4.26 4:64b8297a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282ff420 con 0x7f726c016440 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:143 4.f 4:f24dbbfc:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282ffc70 con 0x55de282b4b30 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:144 4.2d 4:b6586b9f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.2:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de28300090 con 0x7f726c016440 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:145 4.2e 4:779e3a5e:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.3:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de2837cef0 con 0x55de282b9520 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:146 4.36 4:6ed8ef8f:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282ff420 con 0x55de282b4b30 2024-04-27T03:32:17.119 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:147 4.16 4:68001f6a:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.5:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de28300090 con 0x55de282b9520 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:148 4.21 4:8581bfa3:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.6:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de2831f390 con 0x7f726c016440 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:149 4.d 4:b11585fa:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282ffc70 con 0x55de282b9520 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:150 4.17 4:eb32c994:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.8:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de283200d0 con 0x7f726c016440 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:151 4.1c 4:39c0a4a8:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.9:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282ff420 con 0x7f726c016440 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.104+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:152 4.37 4:efd4b2ac:::.dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.10:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de2837cef0 con 0x7f726c016440 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 40 ==== osd_op_reply(143 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.1 [delete] v27'12 uv12 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 38 ==== osd_op_reply(145 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.3 [delete] v27'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 39 ==== osd_op_reply(149 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.7 [delete] v27'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 41 ==== osd_op_reply(146 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.4 [delete] v27'12 uv12 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 40 ==== osd_op_reply(147 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.5 [delete] v27'10 uv10 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 46 ==== osd_op_reply(144 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.2 [delete] v27'8 uv8 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 47 ==== osd_op_reply(150 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.8 [delete] v27'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 48 ==== osd_op_reply(152 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.10 [delete] v27'4 uv4 ondisk = 0) ==== 195+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 49 ==== osd_op_reply(151 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.9 [delete] v27'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 50 ==== osd_op_reply(142 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.0 [delete] v27'2 uv2 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 51 ==== osd_op_reply(148 .dir.708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7.6 [delete] v27'4 uv4 ondisk = 0) ==== 194+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f7275b37800 10 cache get: name=default.rgw.log++pubsub..bucket.myfoo/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7 : miss 2024-04-27T03:32:17.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f7275b37800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.106+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:153 6.18 6:1806bc35:::pubsub..bucket.myfoo%2f708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de282ff420 con 0x7f726c016440 2024-04-27T03:32:17.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.107+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 52 ==== osd_op_reply(153 pubsub..bucket.myfoo/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7 [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.107+0000 7f7275b37800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.107+0000 7f7275b37800 10 cache put: name=default.rgw.log++pubsub..bucket.myfoo/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7 info.flags=0x0 2024-04-27T03:32:17.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.107+0000 7f7275b37800 10 adding default.rgw.log++pubsub..bucket.myfoo/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7 to cache LRU end 2024-04-27T03:32:17.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.107+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:154 6.18 6:1806bc35:::pubsub..bucket.myfoo%2f708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282ff420 con 0x7f726c016440 2024-04-27T03:32:17.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.109+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 53 ==== osd_op_reply(154 pubsub..bucket.myfoo/708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.7 [delete] v27'35 uv27 ondisk = -2 ((2) No such file or directory)) ==== 208+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.109+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:155 8.9 8:96d98480:users.uid::foo.buckets:head [call user.remove_bucket in=144b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de28321da0 con 0x7f7264076060 2024-04-27T03:32:17.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.110+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 25 ==== osd_op_reply(155 foo.buckets [call] v27'48 uv48 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.110+0000 7f7275b37800 10 removing key index: 9te6NH5mcdcq0Tc5i8i1 2024-04-27T03:32:17.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.110+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:156 8.7 8:e4e0e270:users.keys::9te6NH5mcdcq0Tc5i8i1:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de28321da0 con 0x7f726c016440 2024-04-27T03:32:17.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.111+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 54 ==== osd_op_reply(156 9te6NH5mcdcq0Tc5i8i1 [delete] v27'2 uv2 ondisk = 0) ==== 164+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.111+0000 7f7275b37800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.keys:9te6NH5mcdcq0Tc5i8i1, ofs0, ns] 2024-04-27T03:32:17.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.111+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:157 7.e 7:7759931f:::notify.3:head [notify cookie 94412645438512 in=182b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28321da0 con 0x7f7264076060 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.111+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 26 ==== watch-notify(notify (1) cookie 94412645083776 notify 115964117013 ret 0) ==== 212+0+0 (crc 0 0 0) 0x55de282d2950 con 0x7f7264076060 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.111+0000 7f7154ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117013 cookie 94412645083776 notifier 4770 bl.length()=170 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.111+0000 7f7154ff9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:158 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f709c0032a0 con 0x7f7264076060 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.111+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 27 ==== osd_op_reply(157 notify.3 [notify cookie 94412645438512 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.111+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 28 ==== watch-notify(notify_complete (2) cookie 94412645438512 notify 115964117013 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55de282d31e0 con 0x7f7264076060 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.111+0000 7f7275b37800 10 removing email index: foo@foo.com 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.111+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 29 ==== osd_op_reply(158 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.111+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:159 8.3 8:c162d5d1:users.email::foo@foo.com:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de28321da0 con 0x55de282b4b30 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.112+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 42 ==== osd_op_reply(159 foo@foo.com [delete] v27'10 uv10 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.112+0000 7f7275b37800 10 distributing notification oid=default.rgw.control:notify.3 cni=[op: 1, obj: default.rgw.meta:users.email:foo@foo.com, ofs0, ns] 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.112+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:160 7.e 7:7759931f:::notify.3:head [notify cookie 94412645438512 in=174b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28321da0 con 0x7f7264076060 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.113+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 30 ==== watch-notify(notify (1) cookie 94412645083776 notify 115964117014 ret 0) ==== 204+0+0 (crc 0 0 0) 0x55de282d31e0 con 0x7f7264076060 2024-04-27T03:32:17.128 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.113+0000 7f71557fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117014 cookie 94412645083776 notifier 4770 bl.length()=162 2024-04-27T03:32:17.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.113+0000 7f71557fa640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:161 7.e 7:7759931f:::notify.3:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f7134003d60 con 0x7f7264076060 2024-04-27T03:32:17.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.113+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 31 ==== osd_op_reply(160 notify.3 [notify cookie 94412645438512 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.113+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 32 ==== osd_op_reply(161 notify.3 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.113+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 33 ==== watch-notify(notify_complete (2) cookie 94412645438512 notify 115964117014 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55de282d1ee0 con 0x7f7264076060 2024-04-27T03:32:17.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.113+0000 7f7275b37800 10 removing user buckets index 2024-04-27T03:32:17.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.113+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:162 8.9 8:96d98480:users.uid::foo.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de28321da0 con 0x7f7264076060 2024-04-27T03:32:17.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.114+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 34 ==== osd_op_reply(162 foo.buckets [delete] v27'49 uv49 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.114+0000 7f7275b37800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.uid:foo.buckets, ofs0, ns] 2024-04-27T03:32:17.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.114+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:163 7.3 7:c609908c:::notify.5:head [notify cookie 94412645438512 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28321da0 con 0x7f726c016440 2024-04-27T03:32:17.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.114+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 55 ==== watch-notify(notify (1) cookie 94412645091008 notify 115964117021 ret 0) ==== 202+0+0 (crc 0 0 0) 0x55de282d19a0 con 0x7f726c016440 2024-04-27T03:32:17.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.114+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 56 ==== osd_op_reply(163 notify.5 [notify cookie 94412645438512 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.129 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.114+0000 7f7154ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117021 cookie 94412645091008 notifier 4770 bl.length()=160 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.114+0000 7f7154ff9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:164 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f709c0032a0 con 0x7f726c016440 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.115+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 57 ==== osd_op_reply(164 notify.5 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.115+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 58 ==== watch-notify(notify_complete (2) cookie 94412645438512 notify 115964117021 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55de282d17d0 con 0x7f726c016440 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.115+0000 7f7275b37800 10 removing user index: foo 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.115+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:165 8.18 8:1e6ca5ed:users.uid::foo:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de28321da0 con 0x55de282b4b30 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.116+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 43 ==== osd_op_reply(165 foo [call,call,delete] v27'19 uv19 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.116+0000 7f7275b37800 10 removing default.rgw.meta+users.uid+foo from cache 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.116+0000 7f7275b37800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 1, obj: default.rgw.meta:users.uid:foo, ofs0, ns] 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.116+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:166 7.d 7:b4812045:::notify.4:head [notify cookie 94412645438512 in=164b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55de28321da0 con 0x55de282b4b30 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.116+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 44 ==== watch-notify(notify (1) cookie 94412645087328 notify 115964117036 ret 0) ==== 194+0+0 (crc 0 0 0) 0x55de282d17d0 con 0x55de282b4b30 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.116+0000 7f7154ff9640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117036 cookie 94412645087328 notifier 4770 bl.length()=152 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.116+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 45 ==== osd_op_reply(166 notify.4 [notify cookie 94412645438512 out=8b] v0'0 uv234 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.130 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.116+0000 7f7154ff9640 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:167 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f709c0032a0 con 0x55de282b4b30 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.116+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 46 ==== osd_op_reply(167 notify.4 [notify-ack] v0'0 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.116+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 47 ==== watch-notify(notify_complete (2) cookie 94412645438512 notify 115964117036 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55de282d1bb0 con 0x55de282b4b30 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.117+0000 7f71237fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.117+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:168 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94412645107808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de28321d60 con 0x55de282b4b30 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.118+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 48 ==== osd_op_reply(168 data_loggenerations_metadata [watch unwatch cookie 94412645107808] v27'242 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.118+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:169 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94412645068624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de28321d30 con 0x7f7264076060 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.119+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 35 ==== osd_op_reply(169 notify.0 [watch unwatch cookie 94412645068624] v27'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.119+0000 7f7275b37800 20 remove_watcher() i=0 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.119+0000 7f7275b37800 2 removed watcher, disabling cache 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.119+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:170 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94412645074224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282bef50 con 0x55de282b4b30 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.120+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 49 ==== osd_op_reply(170 notify.1 [watch unwatch cookie 94412645074224] v27'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.120+0000 7f7275b37800 20 remove_watcher() i=1 2024-04-27T03:32:17.131 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.120+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:171 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94412645078000] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282bef50 con 0x55de282b9520 2024-04-27T03:32:17.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.121+0000 7f72718f9640 1 -- 172.21.15.165:0/4253761301 <== osd.3 v2:172.21.15.165:6808/2380918637 41 ==== osd_op_reply(171 notify.2 [watch unwatch cookie 94412645078000] v27'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7264073030 con 0x55de282b9520 2024-04-27T03:32:17.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.121+0000 7f7275b37800 20 remove_watcher() i=2 2024-04-27T03:32:17.132 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.121+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:172 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94412645083776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c13f0 con 0x7f7264076060 2024-04-27T03:32:17.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.122+0000 7f726bfff640 1 -- 172.21.15.165:0/4253761301 <== osd.2 v2:172.21.15.165:6816/975457786 36 ==== osd_op_reply(172 notify.3 [watch unwatch cookie 94412645083776] v27'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f725c081070 con 0x7f7264076060 2024-04-27T03:32:17.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.122+0000 7f7275b37800 20 remove_watcher() i=3 2024-04-27T03:32:17.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.122+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:173 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94412645087328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282bef50 con 0x55de282b4b30 2024-04-27T03:32:17.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.124+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.1 v2:172.21.15.165:6800/79879041 50 ==== osd_op_reply(173 notify.4 [watch unwatch cookie 94412645087328] v27'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c056060 con 0x55de282b4b30 2024-04-27T03:32:17.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.124+0000 7f7275b37800 20 remove_watcher() i=4 2024-04-27T03:32:17.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.124+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:174 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94412645091008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c2a80 con 0x7f726c016440 2024-04-27T03:32:17.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.125+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 59 ==== osd_op_reply(174 notify.5 [watch unwatch cookie 94412645091008] v27'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.125+0000 7f7275b37800 20 remove_watcher() i=5 2024-04-27T03:32:17.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.125+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:175 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94412645084832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282c13f0 con 0x7f726c016440 2024-04-27T03:32:17.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.127+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 60 ==== osd_op_reply(175 notify.6 [watch unwatch cookie 94412645084832] v27'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.136 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.127+0000 7f7275b37800 20 remove_watcher() i=6 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.127+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:176 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94412645076432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55de282bef50 con 0x7f726c016440 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 <== osd.0 v2:172.21.15.89:6800/2353668432 61 ==== osd_op_reply(176 notify.7 [watch unwatch cookie 94412645076432] v27'236 uv234 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f726c056060 con 0x7f726c016440 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 20 remove_watcher() i=7 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f726c016440 msgr2=0x7f726c0168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f726c016440 0x7f726c0168c0 crc :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55de282b4b30 msgr2=0x55de282b6ff0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55de282b4b30 0x55de282b6ff0 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7264076060 msgr2=0x7f726405d980 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f7264076060 0x7f726405d980 crc :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55de282b9520 msgr2=0x55de282bb910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55de282b9520 0x55de282bb910 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f713c036900 msgr2=0x7f713c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.137 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f713c036900 0x7f713c038dc0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f725c052fd0 tx=0x7f725c04f000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 msgr2=0x55de282a9430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f72720fa640 1 -- 172.21.15.165:0/4253761301 reap_dead start 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 0x55de282a9430 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f7264018740 tx=0x7f7264014f00 comp rx=0 tx=0).stop 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 shutdown_connections 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 --2- 172.21.15.165:0/4253761301 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de282a2020 0x55de282a9430 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 >> 172.21.15.165:0/4253761301 conn(0x55de2829e380 msgr2=0x55de2829b9e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 shutdown_connections 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.128+0000 7f7275b37800 1 -- 172.21.15.165:0/4253761301 wait complete. 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/3132725868 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f714c036900 msgr2=0x7f714c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 --2- 172.21.15.165:0/3132725868 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f714c036900 0x7f714c038dc0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x55de2827c310 tx=0x7f725c005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/3132725868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de280f0c40 msgr2=0x55de280ef0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 --2- 172.21.15.165:0/3132725868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de280f0c40 0x55de280ef0b0 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f726404c010 tx=0x7f72640184f0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.138 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/3132725868 shutdown_connections 2024-04-27T03:32:17.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 --2- 172.21.15.165:0/3132725868 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f714c036900 0x7f714c038dc0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 --2- 172.21.15.165:0/3132725868 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de280f0c40 0x55de280ef0b0 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/3132725868 >> 172.21.15.165:0/3132725868 conn(0x55de28286da0 msgr2=0x55de282871a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/3132725868 shutdown_connections 2024-04-27T03:32:17.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/3132725868 wait complete. 2024-04-27T03:32:17.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55de28279120 msgr2=0x55de2827b510 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55de28279120 0x55de2827b510 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55de28281cf0 msgr2=0x55de28284140 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55de28281cf0 0x55de28284140 crc :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55de2827dd60 msgr2=0x55de28280150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55de2827dd60 0x55de28280150 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.139 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7158036900 msgr2=0x7f7158038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7158036900 0x7f7158038dc0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f725c054000 tx=0x7f725c04e000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817f350 msgr2=0x55de28273ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 --2- 172.21.15.165:0/1800477278 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55de2817f350 0x55de28273ef0 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f7264001e10 tx=0x7f7264000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f72720fa640 1 -- 172.21.15.165:0/1800477278 reap_dead start 2024-04-27T03:32:17.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 shutdown_connections 2024-04-27T03:32:17.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 >> 172.21.15.165:0/1800477278 conn(0x55de28176550 msgr2=0x55de28266230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 shutdown_connections 2024-04-27T03:32:17.140 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.129+0000 7f7275b37800 1 -- 172.21.15.165:0/1800477278 wait complete. 2024-04-27T03:32:17.143 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'info', '--uid', 'foo'] 2024-04-27T03:32:17.143 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'info', '--uid', 'foo'] 2024-04-27T03:32:17.143 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user info --uid foo 2024-04-27T03:32:17.227 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:17.227 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:17.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.221+0000 7f720eb23800 1 Processor -- start 2024-04-27T03:32:17.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.221+0000 7f720eb23800 1 -- start start 2024-04-27T03:32:17.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.221+0000 7f720eb23800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55f7f6e06480 0x55f7f6e08870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.221+0000 7f720eb23800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55f7f6cc0d40 con 0x55f7f6e02970 2024-04-27T03:32:17.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.221+0000 7f720eb23800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55f7f6c934d0 con 0x55f7f6e06480 2024-04-27T03:32:17.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.221+0000 7f720a0b6640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55f7f6e06480 0x55f7f6e08870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.221+0000 7f720a8b7640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55f7f6e02970 0x55f7f6e04d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42280/0 (socket says 172.21.15.165:42280) 2024-04-27T03:32:17.228 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.221+0000 7f720a8b7640 1 -- 172.21.15.165:0/1603948980 learned_addr learned my addr 172.21.15.165:0/1603948980 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:17.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.222+0000 7f720a0b6640 1 -- 172.21.15.165:0/1603948980 >> v1:172.21.15.89:6789/0 conn(0x55f7f6e02970 legacy=0x55f7f6e04d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.229 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.222+0000 7f720a0b6640 1 -- 172.21.15.165:0/1603948980 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f6cbf590 con 0x55f7f6e06480 2024-04-27T03:32:17.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.223+0000 7f720a0b6640 1 --2- 172.21.15.165:0/1603948980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e06480 0x55f7f6e08870 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f720000a470 tx=0x7f720001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11fee7d6a4521b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.224+0000 7f72098b5640 1 -- 172.21.15.165:0/1603948980 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7200057070 con 0x55f7f6e06480 2024-04-27T03:32:17.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.224+0000 7f72098b5640 1 -- 172.21.15.165:0/1603948980 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7200052570 con 0x55f7f6e06480 2024-04-27T03:32:17.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.224+0000 7f72098b5640 1 -- 172.21.15.165:0/1603948980 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7200052870 con 0x55f7f6e06480 2024-04-27T03:32:17.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.224+0000 7f720eb23800 1 -- 172.21.15.165:0/1603948980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e06480 msgr2=0x55f7f6e08870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.224+0000 7f720eb23800 1 --2- 172.21.15.165:0/1603948980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e06480 0x55f7f6e08870 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f720000a470 tx=0x7f720001ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:17.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.224+0000 7f720eb23800 1 -- 172.21.15.165:0/1603948980 shutdown_connections 2024-04-27T03:32:17.230 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.224+0000 7f720eb23800 1 --2- 172.21.15.165:0/1603948980 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e06480 0x55f7f6e08870 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.224+0000 7f720eb23800 1 -- 172.21.15.165:0/1603948980 >> 172.21.15.165:0/1603948980 conn(0x55f7f6dfc320 msgr2=0x55f7f6dfe740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.224+0000 7f720eb23800 1 -- 172.21.15.165:0/1603948980 shutdown_connections 2024-04-27T03:32:17.231 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.224+0000 7f720eb23800 1 -- 172.21.15.165:0/1603948980 wait complete. 2024-04-27T03:32:17.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.236+0000 7f720eb23800 1 Processor -- start 2024-04-27T03:32:17.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.236+0000 7f720eb23800 1 -- start start 2024-04-27T03:32:17.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.236+0000 7f720eb23800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 0x55f7f6ef3280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.236+0000 7f720eb23800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7f6d4d3c0 con 0x55f7f6e02970 2024-04-27T03:32:17.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.236+0000 7f720a8b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 0x55f7f6ef3280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.236+0000 7f720a8b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 0x55f7f6ef3280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59542/0 (socket says 172.21.15.165:59542) 2024-04-27T03:32:17.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.236+0000 7f720a8b7640 1 -- 172.21.15.165:0/2059720700 learned_addr learned my addr 172.21.15.165:0/2059720700 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.237+0000 7f720a8b7640 1 -- 172.21.15.165:0/2059720700 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f6ef3850 con 0x55f7f6e02970 2024-04-27T03:32:17.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.237+0000 7f720a8b7640 1 --2- 172.21.15.165:0/2059720700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 0x55f7f6ef3280 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f71fc001dd0 tx=0x7f71fc00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52131da181b18ff8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.237+0000 7f70f8e09640 1 -- 172.21.15.165:0/2059720700 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc012020 con 0x55f7f6e02970 2024-04-27T03:32:17.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.237+0000 7f70f8e09640 1 -- 172.21.15.165:0/2059720700 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71fc00e5a0 con 0x55f7f6e02970 2024-04-27T03:32:17.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.237+0000 7f70f8e09640 1 -- 172.21.15.165:0/2059720700 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc00e900 con 0x55f7f6e02970 2024-04-27T03:32:17.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720eb23800 1 -- 172.21.15.165:0/2059720700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 msgr2=0x55f7f6ef3280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720eb23800 1 --2- 172.21.15.165:0/2059720700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 0x55f7f6ef3280 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f71fc001dd0 tx=0x7f71fc00b6c0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720eb23800 1 -- 172.21.15.165:0/2059720700 shutdown_connections 2024-04-27T03:32:17.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720eb23800 1 --2- 172.21.15.165:0/2059720700 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 0x55f7f6ef3280 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720eb23800 1 -- 172.21.15.165:0/2059720700 >> 172.21.15.165:0/2059720700 conn(0x55f7f6dfc320 msgr2=0x55f7f6eeeb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720eb23800 1 -- 172.21.15.165:0/2059720700 shutdown_connections 2024-04-27T03:32:17.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720eb23800 1 -- 172.21.15.165:0/2059720700 wait complete. 2024-04-27T03:32:17.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720eb23800 1 Processor -- start 2024-04-27T03:32:17.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720eb23800 1 -- start start 2024-04-27T03:32:17.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720eb23800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 0x55f7f6efadf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720eb23800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7f6e0b0c0 con 0x55f7f6e02970 2024-04-27T03:32:17.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720a8b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 0x55f7f6efadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720a8b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 0x55f7f6efadf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59554/0 (socket says 172.21.15.165:59554) 2024-04-27T03:32:17.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.238+0000 7f720a8b7640 1 -- 172.21.15.165:0/3910996250 learned_addr learned my addr 172.21.15.165:0/3910996250 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.239+0000 7f720a8b7640 1 -- 172.21.15.165:0/3910996250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f6efb330 con 0x55f7f6e02970 2024-04-27T03:32:17.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.239+0000 7f720a8b7640 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 0x55f7f6efadf0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f71fc001dd0 tx=0x7f71fc000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.239+0000 7f70f7606640 1 -- 172.21.15.165:0/3910996250 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc012030 con 0x55f7f6e02970 2024-04-27T03:32:17.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.239+0000 7f70f7606640 1 -- 172.21.15.165:0/3910996250 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71fc01a7a0 con 0x55f7f6e02970 2024-04-27T03:32:17.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.239+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f7f6efb5c0 con 0x55f7f6e02970 2024-04-27T03:32:17.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.239+0000 7f70f7606640 1 -- 172.21.15.165:0/3910996250 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc01a990 con 0x55f7f6e02970 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.239+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f6efe270 con 0x55f7f6e02970 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.240+0000 7f70f7606640 1 -- 172.21.15.165:0/3910996250 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f71fc015020 con 0x55f7f6e02970 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.240+0000 7f70f7606640 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70f0036950 0x7f70f0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.240+0000 7f70f7606640 1 -- 172.21.15.165:0/3910996250 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f71fc020030 con 0x55f7f6e02970 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.240+0000 7f720a0b6640 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70f0036950 0x7f70f0038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.240+0000 7f720eb23800 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7f6f00020 0x55f7f6f02410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.240+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f02950 con 0x55f7f6f00020 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.240+0000 7f720b0b8640 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7f6f00020 0x55f7f6f02410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.241+0000 7f720a0b6640 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70f0036950 0x7f70f0038e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7200054000 tx=0x7f720004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.241+0000 7f720b0b8640 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7f6f00020 0x55f7f6f02410 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.241+0000 7f720b0b8640 1 -- 172.21.15.165:0/3910996250 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f00020 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.241+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6eff600 con 0x55f7f6f00020 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.241+0000 7f720b0b8640 1 -- 172.21.15.165:0/3910996250 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f00020 2024-04-27T03:32:17.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.241+0000 7f720eb23800 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7f6f04c80 0x55f7f6f07070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.241+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f07760 con 0x55f7f6f04c80 2024-04-27T03:32:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.241+0000 7f720a8b7640 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7f6f04c80 0x55f7f6f07070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.242+0000 7f720a8b7640 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7f6f04c80 0x55f7f6f07070 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.242+0000 7f720a8b7640 1 -- 172.21.15.165:0/3910996250 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f04c80 2024-04-27T03:32:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.242+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f03c30 con 0x55f7f6f00020 2024-04-27T03:32:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.242+0000 7f720b0b8640 1 -- 172.21.15.165:0/3910996250 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f00020 2024-04-27T03:32:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.242+0000 7f720eb23800 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55f7f6f08c10 0x55f7f6f0b060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.242+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f0b7c0 con 0x55f7f6f08c10 2024-04-27T03:32:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.242+0000 7f720a0b6640 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55f7f6f08c10 0x55f7f6f0b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.243+0000 7f720a0b6640 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55f7f6f08c10 0x55f7f6f0b060 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.243+0000 7f720a0b6640 1 -- 172.21.15.165:0/3910996250 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f72000529e0 con 0x55f7f6f08c10 2024-04-27T03:32:17.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.243+0000 7f720eb23800 1 Processor -- start 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.243+0000 7f720eb23800 1 -- start start 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.243+0000 7f720eb23800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f19c20 0x55f7f6f1a040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.243+0000 7f720eb23800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7f6cf36b0 con 0x55f7f6f19c20 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.244+0000 7f720a8b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f19c20 0x55f7f6f1a040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.244+0000 7f720a8b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f19c20 0x55f7f6f1a040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59560/0 (socket says 172.21.15.165:59560) 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.244+0000 7f720a8b7640 1 -- 172.21.15.165:0/4062771041 learned_addr learned my addr 172.21.15.165:0/4062771041 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.244+0000 7f720a8b7640 1 -- 172.21.15.165:0/4062771041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f6f1a580 con 0x55f7f6f19c20 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.244+0000 7f720a8b7640 1 --2- 172.21.15.165:0/4062771041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f19c20 0x55f7f6f1a040 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f71fc04ad80 tx=0x7f71fc00db20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.244+0000 7f70f4e01640 1 -- 172.21.15.165:0/4062771041 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc03d070 con 0x55f7f6f19c20 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.244+0000 7f70f4e01640 1 -- 172.21.15.165:0/4062771041 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71fc0445c0 con 0x55f7f6f19c20 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.244+0000 7f720eb23800 1 -- 172.21.15.165:0/4062771041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f7f6f1aff0 con 0x55f7f6f19c20 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.244+0000 7f70f4e01640 1 -- 172.21.15.165:0/4062771041 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc0448a0 con 0x55f7f6f19c20 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.245+0000 7f720eb23800 1 -- 172.21.15.165:0/4062771041 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f6f1b4f0 con 0x55f7f6f19c20 2024-04-27T03:32:17.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.245+0000 7f70f4e01640 1 -- 172.21.15.165:0/4062771041 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f71fc04c1a0 con 0x55f7f6f19c20 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.245+0000 7f70f4e01640 1 --2- 172.21.15.165:0/4062771041 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70e4036950 0x7f70e4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.245+0000 7f70f4e01640 1 -- 172.21.15.165:0/4062771041 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f71fc02ec70 con 0x55f7f6f19c20 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.245+0000 7f720a0b6640 1 --2- 172.21.15.165:0/4062771041 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70e4036950 0x7f70e4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.246+0000 7f720eb23800 1 Processor -- start 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.246+0000 7f720eb23800 1 -- start start 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.246+0000 7f720a0b6640 1 --2- 172.21.15.165:0/4062771041 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70e4036950 0x7f70e4038e10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55f7f6f03230 tx=0x7f7200005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.246+0000 7f720eb23800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 0x55f7f6f27140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.246+0000 7f720eb23800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7f6ef3bd0 con 0x55f7f6f26d20 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.246+0000 7f720a8b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 0x55f7f6f27140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.246+0000 7f720a8b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 0x55f7f6f27140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59572/0 (socket says 172.21.15.165:59572) 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.246+0000 7f720a8b7640 1 -- 172.21.15.165:0/1945241417 learned_addr learned my addr 172.21.15.165:0/1945241417 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.246+0000 7f720a8b7640 1 -- 172.21.15.165:0/1945241417 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f6f27710 con 0x55f7f6f26d20 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.247+0000 7f720a8b7640 1 --2- 172.21.15.165:0/1945241417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 0x55f7f6f27140 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f71fc044aa0 tx=0x7f71fc02ebd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=281eadc54bcab665 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.247+0000 7f70ee7fc640 1 -- 172.21.15.165:0/1945241417 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc016c80 con 0x55f7f6f26d20 2024-04-27T03:32:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.247+0000 7f70ee7fc640 1 -- 172.21.15.165:0/1945241417 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71fc04e070 con 0x55f7f6f26d20 2024-04-27T03:32:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.247+0000 7f70ee7fc640 1 -- 172.21.15.165:0/1945241417 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc04ebb0 con 0x55f7f6f26d20 2024-04-27T03:32:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.247+0000 7f720eb23800 1 -- 172.21.15.165:0/1945241417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 msgr2=0x55f7f6f27140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.247+0000 7f720eb23800 1 --2- 172.21.15.165:0/1945241417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 0x55f7f6f27140 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7f71fc044aa0 tx=0x7f71fc02ebd0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.247+0000 7f720eb23800 1 -- 172.21.15.165:0/1945241417 shutdown_connections 2024-04-27T03:32:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.247+0000 7f720eb23800 1 --2- 172.21.15.165:0/1945241417 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 0x55f7f6f27140 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.247+0000 7f720eb23800 1 -- 172.21.15.165:0/1945241417 >> 172.21.15.165:0/1945241417 conn(0x55f7f6f22ff0 msgr2=0x55f7f6f21050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.247+0000 7f720eb23800 1 -- 172.21.15.165:0/1945241417 shutdown_connections 2024-04-27T03:32:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.247+0000 7f720eb23800 1 -- 172.21.15.165:0/1945241417 wait complete. 2024-04-27T03:32:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.248+0000 7f720eb23800 1 Processor -- start 2024-04-27T03:32:17.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.248+0000 7f720eb23800 1 -- start start 2024-04-27T03:32:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.248+0000 7f720eb23800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 0x55f7f6f2e060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.248+0000 7f720eb23800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55f7f6f27d70 con 0x55f7f6f26d20 2024-04-27T03:32:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.248+0000 7f720a8b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 0x55f7f6f2e060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.248+0000 7f720a8b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 0x55f7f6f2e060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59588/0 (socket says 172.21.15.165:59588) 2024-04-27T03:32:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.248+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 learned_addr learned my addr 172.21.15.165:0/130853645 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.248+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f7f6f2e5a0 con 0x55f7f6f26d20 2024-04-27T03:32:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.248+0000 7f720a8b7640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 0x55f7f6f2e060 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f71fc01cfd0 tx=0x7f71fc02e970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.249+0000 7f70ecff9640 1 -- 172.21.15.165:0/130853645 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc016c80 con 0x55f7f6f26d20 2024-04-27T03:32:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.249+0000 7f70ecff9640 1 -- 172.21.15.165:0/130853645 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f71fc037ba0 con 0x55f7f6f26d20 2024-04-27T03:32:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.249+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55f7f6f2e5a0 con 0x55f7f6f26d20 2024-04-27T03:32:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.249+0000 7f70ecff9640 1 -- 172.21.15.165:0/130853645 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f71fc02d3f0 con 0x55f7f6f26d20 2024-04-27T03:32:17.262 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.249+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f7f6f2e5a0 con 0x55f7f6f26d20 2024-04-27T03:32:17.302 DEBUG:teuthology.orchestra.run:got remote process result: 22 2024-04-27T03:32:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.250+0000 7f70ecff9640 1 -- 172.21.15.165:0/130853645 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f71fc072030 con 0x55f7f6f26d20 2024-04-27T03:32:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.250+0000 7f720eb23800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.250+0000 7f70ecff9640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70d4036950 0x7f70d4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.250+0000 7f70ecff9640 1 -- 172.21.15.165:0/130853645 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f71fc070070 con 0x55f7f6f26d20 2024-04-27T03:32:17.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.250+0000 7f720a0b6640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70d4036950 0x7f70d4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.250+0000 7f720eb23800 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7f6f39680 0x55f7f6f3bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.250+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3c290 con 0x55f7f6f39680 2024-04-27T03:32:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.250+0000 7f720b0b8640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7f6f39680 0x55f7f6f3bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.250+0000 7f720a0b6640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70d4036950 0x7f70d4038e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55f7f6f0bbc0 tx=0x7f720004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.250+0000 7f720b0b8640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7f6f39680 0x55f7f6f3bb40 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 20 realm 2024-04-27T03:32:17.304 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3c6a0 con 0x55f7f6f39680 2024-04-27T03:32:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3cf40 con 0x55f7f6f39680 2024-04-27T03:32:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f39680 2024-04-27T03:32:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7f6f3e040 0x55f7f6f40430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f409b0 con 0x55f7f6f3e040 2024-04-27T03:32:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.251+0000 7f720a8b7640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7f6f3e040 0x55f7f6f40430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.252+0000 7f720a8b7640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7f6f3e040 0x55f7f6f40430 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.252+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.252+0000 7f720eb23800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.252+0000 7f720eb23800 20 searching for the correct realm 2024-04-27T03:32:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.252+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f42310 con 0x55f7f6f3e040 2024-04-27T03:32:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.252+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.253+0000 7f720a8b7640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f71fc057040 0x7f71fc0522b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.253+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f42310 con 0x7f71fc057040 2024-04-27T03:32:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.253+0000 7f720a0b6640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f71fc057040 0x7f71fc0522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.307 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.253+0000 7f720a0b6640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f71fc057040 0x7f71fc0522b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.253+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.253+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f42310 con 0x7f71fc057040 2024-04-27T03:32:17.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.253+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.253+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f42310 con 0x55f7f6f39680 2024-04-27T03:32:17.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.254+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.254+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f39680 2024-04-27T03:32:17.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.254+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.308 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.254+0000 7f720b0b8640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7204016440 0x7f72040168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.254+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f7204016440 2024-04-27T03:32:17.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.254+0000 7f720b0b8640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7204016440 0x7f72040168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.254+0000 7f720b0b8640 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7204016440 0x7f72040168c0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.255+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.309 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.255+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f7204016440 2024-04-27T03:32:17.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.255+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.255+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f71fc057040 2024-04-27T03:32:17.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.255+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.255+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f3e040 2024-04-27T03:32:17.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.255+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.310 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.255+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f39680 2024-04-27T03:32:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.256+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.256+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f71fc057040 2024-04-27T03:32:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.256+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.256+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f39680 2024-04-27T03:32:17.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.256+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.256+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f71fc057040 2024-04-27T03:32:17.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.256+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.256+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f3e040 2024-04-27T03:32:17.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.257+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.257+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f3e040 2024-04-27T03:32:17.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.257+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.257+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f7204016440 2024-04-27T03:32:17.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.257+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.257+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f3e040 2024-04-27T03:32:17.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.258+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.258+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f3e040 2024-04-27T03:32:17.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.258+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.258+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f39680 2024-04-27T03:32:17.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.258+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.258+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f39680 2024-04-27T03:32:17.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.258+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.258+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f7204016440 2024-04-27T03:32:17.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.259+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.259+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f3e040 2024-04-27T03:32:17.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.259+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.259+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f71fc057040 2024-04-27T03:32:17.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.259+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.259+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f39680 2024-04-27T03:32:17.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.259+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.259+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f7204016440 2024-04-27T03:32:17.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.260+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.260+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f3e040 2024-04-27T03:32:17.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.260+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.260+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f7204016440 2024-04-27T03:32:17.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.260+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.260+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f71fc057040 2024-04-27T03:32:17.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.260+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.261+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f3e040 2024-04-27T03:32:17.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.261+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.261+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f7204016440 2024-04-27T03:32:17.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.261+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.261+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x55f7f6f39680 2024-04-27T03:32:17.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.261+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.318 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.261+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f3d2e0 con 0x7f71fc057040 2024-04-27T03:32:17.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:17.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:17.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:17.319 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:17.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:17.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:17.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f41400 con 0x55f7f6f39680 2024-04-27T03:32:17.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.320 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f41880 con 0x55f7f6f39680 2024-04-27T03:32:17.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:17.321 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f41880 con 0x7f71fc057040 2024-04-27T03:32:17.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:17.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 zone default found 2024-04-27T03:32:17.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 4 Realm: () 2024-04-27T03:32:17.322 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:17.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:17.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:17.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 zonegroup default 2024-04-27T03:32:17.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.262+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f421a0 con 0x7f7204016440 2024-04-27T03:32:17.323 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720eb23800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720eb23800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f441c0 con 0x55f7f6f3e040 2024-04-27T03:32:17.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720eb23800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.324 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720eb23800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f44560 con 0x55f7f6f3e040 2024-04-27T03:32:17.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720eb23800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720eb23800 20 started sync module instance, tier type = 2024-04-27T03:32:17.325 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720eb23800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:17.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.263+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f43660 con 0x7f71fc057040 2024-04-27T03:32:17.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.264+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.264+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94523488483920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f432c0 con 0x7f71fc057040 2024-04-27T03:32:17.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.264+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f45260 con 0x55f7f6f39680 2024-04-27T03:32:17.326 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.266+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94523488483920] v27'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.266+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.266+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94523488488160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f45650 con 0x55f7f6f39680 2024-04-27T03:32:17.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.266+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f460a0 con 0x55f7f6f3e040 2024-04-27T03:32:17.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.267+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.267+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94523488492432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f46940 con 0x55f7f6f3e040 2024-04-27T03:32:17.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.267+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f47440 con 0x7f71fc057040 2024-04-27T03:32:17.327 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.267+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94523488488160] v27'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.268+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.268+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94523488489344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f47ce0 con 0x7f71fc057040 2024-04-27T03:32:17.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.268+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f486a0 con 0x55f7f6f39680 2024-04-27T03:32:17.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.268+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94523488492432] v27'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.269+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.269+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94523488502256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f48ac0 con 0x55f7f6f39680 2024-04-27T03:32:17.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.269+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f49480 con 0x7f7204016440 2024-04-27T03:32:17.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.269+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94523488489344] v27'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.270+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.270+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94523488502256] v27'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.270+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94523488499760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f49480 con 0x7f7204016440 2024-04-27T03:32:17.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.270+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f48610 con 0x7f7204016440 2024-04-27T03:32:17.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.271+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94523488499760] v27'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.271+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.271+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94523488505808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f493f0 con 0x7f7204016440 2024-04-27T03:32:17.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.271+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f4a2f0 con 0x7f7204016440 2024-04-27T03:32:17.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.273+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94523488505808] v27'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.273+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'237 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.273+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94523488508304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f4a260 con 0x7f7204016440 2024-04-27T03:32:17.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.273+0000 7f720eb23800 20 add_watcher() i=0 2024-04-27T03:32:17.330 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.273+0000 7f720eb23800 20 add_watcher() i=1 2024-04-27T03:32:17.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.273+0000 7f720eb23800 20 add_watcher() i=2 2024-04-27T03:32:17.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.273+0000 7f720eb23800 20 add_watcher() i=3 2024-04-27T03:32:17.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.273+0000 7f720eb23800 20 add_watcher() i=4 2024-04-27T03:32:17.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.273+0000 7f720eb23800 20 add_watcher() i=5 2024-04-27T03:32:17.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.273+0000 7f720eb23800 20 add_watcher() i=6 2024-04-27T03:32:17.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.274+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94523488508304] v27'238 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.274+0000 7f720eb23800 20 add_watcher() i=7 2024-04-27T03:32:17.331 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.274+0000 7f720eb23800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:17.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.274+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f4d740 con 0x55f7f6f39680 2024-04-27T03:32:17.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.274+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.274+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94523488509824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f4ac00 con 0x55f7f6f39680 2024-04-27T03:32:17.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.275+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94523488509824] v27'243 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.275+0000 7f70c17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:17.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.275+0000 7f720eb23800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:17.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.275+0000 7f720eb23800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:17.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.275+0000 7f720eb23800 5 note: GC not initialized 2024-04-27T03:32:17.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.276+0000 7f703bfff640 20 reqs_thread_entry: start 2024-04-27T03:32:17.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.276+0000 7f720eb23800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:17.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.276+0000 7f720eb23800 20 Filter name: none 2024-04-27T03:32:17.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.276+0000 7f7039ffb640 20 reqs_thread_entry: start 2024-04-27T03:32:17.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.276+0000 7f720eb23800 20 rados->read ofs=0 len=0 2024-04-27T03:32:17.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.276+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 8.18 8:1e6ca5ed:users.uid::foo:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f80560 con 0x55f7f6f39680 2024-04-27T03:32:17.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.276+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 foo [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.276+0000 7f720eb23800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:17.334 INFO:teuthology.orchestra.run.smithi165.stderr:could not fetch user info: no user info saved 2024-04-27T03:32:17.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.277+0000 7f70c17e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:17.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.277+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94523488509824] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f80520 con 0x55f7f6f39680 2024-04-27T03:32:17.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.278+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(63 data_loggenerations_metadata [watch unwatch cookie 94523488509824] v27'244 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.278+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:64 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94523488483920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f80520 con 0x7f71fc057040 2024-04-27T03:32:17.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.279+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(64 notify.0 [watch unwatch cookie 94523488483920] v27'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.279+0000 7f720eb23800 20 remove_watcher() i=0 2024-04-27T03:32:17.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.279+0000 7f720eb23800 2 removed watcher, disabling cache 2024-04-27T03:32:17.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.279+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94523488488160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f80520 con 0x55f7f6f39680 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.280+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.1 [watch unwatch cookie 94523488488160] v27'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.280+0000 7f720eb23800 20 remove_watcher() i=1 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.280+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94523488492432] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f80520 con 0x55f7f6f3e040 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.281+0000 7f720a8b7640 1 -- 172.21.15.165:0/130853645 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(66 notify.2 [watch unwatch cookie 94523488492432] v27'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f71fc018430 con 0x55f7f6f3e040 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.281+0000 7f720eb23800 20 remove_watcher() i=2 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.281+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94523488489344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f80520 con 0x7f71fc057040 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.282+0000 7f720a0b6640 1 -- 172.21.15.165:0/130853645 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(67 notify.3 [watch unwatch cookie 94523488489344] v27'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f7200081070 con 0x7f71fc057040 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.282+0000 7f720eb23800 20 remove_watcher() i=3 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.282+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94523488502256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f80520 con 0x55f7f6f39680 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.283+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(68 notify.4 [watch unwatch cookie 94523488502256] v27'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x55f7f6f39680 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.283+0000 7f720eb23800 20 remove_watcher() i=4 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.283+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94523488499760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f80520 con 0x7f7204016440 2024-04-27T03:32:17.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.284+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(69 notify.5 [watch unwatch cookie 94523488499760] v27'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.284+0000 7f720eb23800 20 remove_watcher() i=5 2024-04-27T03:32:17.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.284+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:70 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94523488505808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f80520 con 0x7f7204016440 2024-04-27T03:32:17.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.285+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(70 notify.6 [watch unwatch cookie 94523488505808] v27'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.285+0000 7f720eb23800 20 remove_watcher() i=6 2024-04-27T03:32:17.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.285+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:71 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94523488508304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55f7f6f80520 con 0x7f7204016440 2024-04-27T03:32:17.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.286+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(71 notify.7 [watch unwatch cookie 94523488508304] v27'239 uv237 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f720400a4a0 con 0x7f7204016440 2024-04-27T03:32:17.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.286+0000 7f720eb23800 20 remove_watcher() i=7 2024-04-27T03:32:17.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.286+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7204016440 msgr2=0x7f72040168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.286+0000 7f720eb23800 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f7204016440 0x7f72040168c0 crc :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.286+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7f6f39680 msgr2=0x55f7f6f3bb40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.286+0000 7f720eb23800 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7f6f39680 0x55f7f6f3bb40 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.336 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.286+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f71fc057040 msgr2=0x7f71fc0522b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.286+0000 7f720eb23800 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f71fc057040 0x7f71fc0522b0 crc :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.286+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7f6f3e040 msgr2=0x55f7f6f40430 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.286+0000 7f720eb23800 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7f6f3e040 0x55f7f6f40430 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70d4036950 msgr2=0x7f70d4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70d4036950 0x7f70d4038e10 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x55f7f6f0bbc0 tx=0x7f720004f000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 msgr2=0x55f7f6f2e060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 --2- 172.21.15.165:0/130853645 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f26d20 0x55f7f6f2e060 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7f71fc01cfd0 tx=0x7f71fc02e970 comp rx=0 tx=0).stop 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720b0b8640 1 -- 172.21.15.165:0/130853645 reap_dead start 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 shutdown_connections 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 >> 172.21.15.165:0/130853645 conn(0x55f7f6f22ff0 msgr2=0x55f7f6f20650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 shutdown_connections 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/130853645 wait complete. 2024-04-27T03:32:17.337 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/4062771041 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70e4036950 msgr2=0x7f70e4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 --2- 172.21.15.165:0/4062771041 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70e4036950 0x7f70e4038e10 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x55f7f6f03230 tx=0x7f7200005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:17.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/4062771041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f19c20 msgr2=0x55f7f6f1a040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 --2- 172.21.15.165:0/4062771041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f19c20 0x55f7f6f1a040 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7f71fc04ad80 tx=0x7f71fc00db20 comp rx=0 tx=0).stop 2024-04-27T03:32:17.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/4062771041 shutdown_connections 2024-04-27T03:32:17.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 --2- 172.21.15.165:0/4062771041 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70e4036950 0x7f70e4038e10 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 --2- 172.21.15.165:0/4062771041 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6f19c20 0x55f7f6f1a040 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/4062771041 >> 172.21.15.165:0/4062771041 conn(0x55f7f6f0dcc0 msgr2=0x55f7f6f0e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/4062771041 shutdown_connections 2024-04-27T03:32:17.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/4062771041 wait complete. 2024-04-27T03:32:17.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7f6f00020 msgr2=0x55f7f6f02410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55f7f6f00020 0x55f7f6f02410 crc :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.338 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55f7f6f08c10 msgr2=0x55f7f6f0b060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55f7f6f08c10 0x55f7f6f0b060 crc :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7f6f04c80 msgr2=0x55f7f6f07070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.287+0000 7f720eb23800 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55f7f6f04c80 0x55f7f6f07070 crc :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.288+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70f0036950 msgr2=0x7f70f0038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.288+0000 7f720eb23800 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f70f0036950 0x7f70f0038e10 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f7200054000 tx=0x7f720004e000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.288+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 msgr2=0x55f7f6efadf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.288+0000 7f720eb23800 1 --2- 172.21.15.165:0/3910996250 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55f7f6e02970 0x55f7f6efadf0 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7f71fc001dd0 tx=0x7f71fc000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.288+0000 7f720b0b8640 1 -- 172.21.15.165:0/3910996250 reap_dead start 2024-04-27T03:32:17.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.288+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 shutdown_connections 2024-04-27T03:32:17.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.288+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 >> 172.21.15.165:0/3910996250 conn(0x55f7f6dfc320 msgr2=0x55f7f6eed300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.288+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 shutdown_connections 2024-04-27T03:32:17.339 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.288+0000 7f720eb23800 1 -- 172.21.15.165:0/3910996250 wait complete. 2024-04-27T03:32:17.340 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get', '--rgw-zone', 'default'] 2024-04-27T03:32:17.340 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get', '--rgw-zone', 'default'] 2024-04-27T03:32:17.340 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get --rgw-zone default 2024-04-27T03:32:17.385 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:17.385 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:17.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.380+0000 7f04b761f800 1 Processor -- start 2024-04-27T03:32:17.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.380+0000 7f04b761f800 1 -- start start 2024-04-27T03:32:17.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.380+0000 7f04b761f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x563885752de0 0x563885753200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.380+0000 7f04b761f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x563885612d40 con 0x563885754040 2024-04-27T03:32:17.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.380+0000 7f04b761f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5638855e54d0 con 0x563885752de0 2024-04-27T03:32:17.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.381+0000 7f04b511e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x563885752de0 0x563885753200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.381+0000 7f04b491d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x563885754040 0x5638857544c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42292/0 (socket says 172.21.15.165:42292) 2024-04-27T03:32:17.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.381+0000 7f04b511e640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x563885752de0 0x563885753200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59600/0 (socket says 172.21.15.165:59600) 2024-04-27T03:32:17.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.381+0000 7f04b491d640 1 -- 172.21.15.165:0/1961906166 learned_addr learned my addr 172.21.15.165:0/1961906166 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:17.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.381+0000 7f04b511e640 1 -- 172.21.15.165:0/1961906166 >> v1:172.21.15.89:6789/0 conn(0x563885754040 legacy=0x5638857544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.381+0000 7f04b511e640 1 -- 172.21.15.165:0/1961906166 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x563885611590 con 0x563885752de0 2024-04-27T03:32:17.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.381+0000 7f04a7fff640 1 -- 172.21.15.165:0/1961906166 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3376259786 0 0) 0x563885612d40 con 0x563885754040 2024-04-27T03:32:17.392 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.383+0000 7f04b511e640 1 --2- 172.21.15.165:0/1961906166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x563885753200 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f04a800a470 tx=0x7f04a801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c070be75d2108459 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.383+0000 7f04a7fff640 1 -- 172.21.15.165:0/1961906166 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04a8057070 con 0x563885752de0 2024-04-27T03:32:17.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.383+0000 7f04a7fff640 1 -- 172.21.15.165:0/1961906166 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a8052570 con 0x563885752de0 2024-04-27T03:32:17.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.383+0000 7f04a7fff640 1 -- 172.21.15.165:0/1961906166 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04a8052870 con 0x563885752de0 2024-04-27T03:32:17.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.383+0000 7f04b761f800 1 -- 172.21.15.165:0/1961906166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 msgr2=0x563885753200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.393 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.384+0000 7f04b761f800 1 --2- 172.21.15.165:0/1961906166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x563885753200 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f04a800a470 tx=0x7f04a801ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:17.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.384+0000 7f04b761f800 1 -- 172.21.15.165:0/1961906166 shutdown_connections 2024-04-27T03:32:17.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.384+0000 7f04b761f800 1 --2- 172.21.15.165:0/1961906166 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x563885753200 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.384+0000 7f04b761f800 1 -- 172.21.15.165:0/1961906166 >> 172.21.15.165:0/1961906166 conn(0x56388574e420 msgr2=0x563885750840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.384+0000 7f04b761f800 1 -- 172.21.15.165:0/1961906166 shutdown_connections 2024-04-27T03:32:17.394 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.384+0000 7f04b761f800 1 -- 172.21.15.165:0/1961906166 wait complete. 2024-04-27T03:32:17.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.397+0000 7f04b761f800 1 Processor -- start 2024-04-27T03:32:17.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.397+0000 7f04b761f800 1 -- start start 2024-04-27T03:32:17.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.397+0000 7f04b761f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x563885845490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.397+0000 7f04b761f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56388569f3c0 con 0x563885752de0 2024-04-27T03:32:17.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.397+0000 7f04b511e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x563885845490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.397+0000 7f04b511e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x563885845490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59616/0 (socket says 172.21.15.165:59616) 2024-04-27T03:32:17.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.397+0000 7f04b511e640 1 -- 172.21.15.165:0/2931731368 learned_addr learned my addr 172.21.15.165:0/2931731368 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.397+0000 7f04b511e640 1 -- 172.21.15.165:0/2931731368 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563885845a60 con 0x563885752de0 2024-04-27T03:32:17.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.398+0000 7f04b511e640 1 --2- 172.21.15.165:0/2931731368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x563885845490 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f04a8055040 tx=0x7f04a804c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33c7e4bf41d6d645 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.398+0000 7f03a3606640 1 -- 172.21.15.165:0/2931731368 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04a8057070 con 0x563885752de0 2024-04-27T03:32:17.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.398+0000 7f03a3606640 1 -- 172.21.15.165:0/2931731368 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a8063070 con 0x563885752de0 2024-04-27T03:32:17.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.398+0000 7f03a3606640 1 -- 172.21.15.165:0/2931731368 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04a8052720 con 0x563885752de0 2024-04-27T03:32:17.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b761f800 1 -- 172.21.15.165:0/2931731368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 msgr2=0x563885845490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.412 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b761f800 1 --2- 172.21.15.165:0/2931731368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x563885845490 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f04a8055040 tx=0x7f04a804c000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b761f800 1 -- 172.21.15.165:0/2931731368 shutdown_connections 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b761f800 1 --2- 172.21.15.165:0/2931731368 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x563885845490 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b761f800 1 -- 172.21.15.165:0/2931731368 >> 172.21.15.165:0/2931731368 conn(0x56388574e420 msgr2=0x563885840db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b761f800 1 -- 172.21.15.165:0/2931731368 shutdown_connections 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b761f800 1 -- 172.21.15.165:0/2931731368 wait complete. 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b761f800 1 Processor -- start 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b761f800 1 -- start start 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b761f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x56388584cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b761f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56388575a500 con 0x563885752de0 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b511e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x56388584cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b511e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x56388584cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59626/0 (socket says 172.21.15.165:59626) 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.399+0000 7f04b511e640 1 -- 172.21.15.165:0/2900976197 learned_addr learned my addr 172.21.15.165:0/2900976197 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.413 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.400+0000 7f04b511e640 1 -- 172.21.15.165:0/2900976197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56388584d480 con 0x563885752de0 2024-04-27T03:32:17.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.400+0000 7f04b511e640 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x56388584cf40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f04a8060040 tx=0x7f04a8002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.400+0000 7f03a1e03640 1 -- 172.21.15.165:0/2900976197 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04a8052c70 con 0x563885752de0 2024-04-27T03:32:17.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.401+0000 7f03a1e03640 1 -- 172.21.15.165:0/2900976197 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a806c070 con 0x563885752de0 2024-04-27T03:32:17.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.401+0000 7f04b761f800 1 -- 172.21.15.165:0/2900976197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56388584d710 con 0x563885752de0 2024-04-27T03:32:17.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.401+0000 7f04b761f800 1 -- 172.21.15.165:0/2900976197 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5638858503c0 con 0x563885752de0 2024-04-27T03:32:17.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.401+0000 7f03a1e03640 1 -- 172.21.15.165:0/2900976197 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04a805e480 con 0x563885752de0 2024-04-27T03:32:17.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.401+0000 7f04b761f800 1 Processor -- start 2024-04-27T03:32:17.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.401+0000 7f04b761f800 1 -- start start 2024-04-27T03:32:17.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.401+0000 7f04b761f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 0x56388585ac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.401+0000 7f04b761f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5638856456b0 con 0x56388585a7f0 2024-04-27T03:32:17.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.401+0000 7f03a1e03640 1 -- 172.21.15.165:0/2900976197 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f04a805e620 con 0x563885752de0 2024-04-27T03:32:17.414 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.402+0000 7f04b591f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 0x56388585ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.402+0000 7f04b591f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 0x56388585ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59630/0 (socket says 172.21.15.165:59630) 2024-04-27T03:32:17.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.402+0000 7f04b591f640 1 -- 172.21.15.165:0/278217793 learned_addr learned my addr 172.21.15.165:0/278217793 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.402+0000 7f03a1e03640 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f039c036900 0x7f039c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.402+0000 7f03a1e03640 1 -- 172.21.15.165:0/2900976197 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f04a808e680 con 0x563885752de0 2024-04-27T03:32:17.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.402+0000 7f04b511e640 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f039c036900 0x7f039c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.402+0000 7f04b591f640 1 -- 172.21.15.165:0/278217793 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56388585b150 con 0x56388585a7f0 2024-04-27T03:32:17.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.402+0000 7f04b591f640 1 --2- 172.21.15.165:0/278217793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 0x56388585ac10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f04b000a430 tx=0x7f04b000a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=613ed6e54422f930 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.402+0000 7f04b511e640 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f039c036900 0x7f039c038dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f04a80524b0 tx=0x7f04a805bef0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.415 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.402+0000 7f039b7fe640 1 -- 172.21.15.165:0/278217793 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04b0015070 con 0x56388585a7f0 2024-04-27T03:32:17.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.402+0000 7f039b7fe640 1 -- 172.21.15.165:0/278217793 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b0001670 con 0x56388585a7f0 2024-04-27T03:32:17.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.402+0000 7f039b7fe640 1 -- 172.21.15.165:0/278217793 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04b0001990 con 0x56388585a7f0 2024-04-27T03:32:17.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b761f800 1 -- 172.21.15.165:0/278217793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 msgr2=0x56388585ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b761f800 1 --2- 172.21.15.165:0/278217793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 0x56388585ac10 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f04b000a430 tx=0x7f04b000a8f0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b761f800 1 -- 172.21.15.165:0/278217793 shutdown_connections 2024-04-27T03:32:17.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b761f800 1 --2- 172.21.15.165:0/278217793 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 0x56388585ac10 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b761f800 1 -- 172.21.15.165:0/278217793 >> 172.21.15.165:0/278217793 conn(0x563885754040 msgr2=0x563885857290 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b761f800 1 -- 172.21.15.165:0/278217793 shutdown_connections 2024-04-27T03:32:17.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b761f800 1 -- 172.21.15.165:0/278217793 wait complete. 2024-04-27T03:32:17.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b761f800 1 Processor -- start 2024-04-27T03:32:17.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b761f800 1 -- start start 2024-04-27T03:32:17.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b761f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 0x56388575cdf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b761f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56388585bae0 con 0x56388585a7f0 2024-04-27T03:32:17.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b591f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 0x56388575cdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b591f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 0x56388575cdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59646/0 (socket says 172.21.15.165:59646) 2024-04-27T03:32:17.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.403+0000 7f04b591f640 1 -- 172.21.15.165:0/1307700325 learned_addr learned my addr 172.21.15.165:0/1307700325 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.404+0000 7f04b591f640 1 -- 172.21.15.165:0/1307700325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56388575b4e0 con 0x56388585a7f0 2024-04-27T03:32:17.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.404+0000 7f04b591f640 1 --2- 172.21.15.165:0/1307700325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 0x56388575cdf0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f04b000a930 tx=0x7f04b0003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.404+0000 7f0399ffb640 1 -- 172.21.15.165:0/1307700325 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04b0020020 con 0x56388585a7f0 2024-04-27T03:32:17.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.404+0000 7f0399ffb640 1 -- 172.21.15.165:0/1307700325 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04b00108b0 con 0x56388585a7f0 2024-04-27T03:32:17.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.404+0000 7f0399ffb640 1 -- 172.21.15.165:0/1307700325 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04b0015040 con 0x56388585a7f0 2024-04-27T03:32:17.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.404+0000 7f04b761f800 1 -- 172.21.15.165:0/1307700325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56388575b770 con 0x56388585a7f0 2024-04-27T03:32:17.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.404+0000 7f04b761f800 1 -- 172.21.15.165:0/1307700325 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56388575bc10 con 0x56388585a7f0 2024-04-27T03:32:17.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.405+0000 7f0399ffb640 1 -- 172.21.15.165:0/1307700325 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f04b0023020 con 0x56388585a7f0 2024-04-27T03:32:17.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.405+0000 7f0399ffb640 1 --2- 172.21.15.165:0/1307700325 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f038c036950 0x7f038c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.405+0000 7f0399ffb640 1 -- 172.21.15.165:0/1307700325 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f04b004bf40 con 0x56388585a7f0 2024-04-27T03:32:17.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.405+0000 7f04b491d640 1 --2- 172.21.15.165:0/1307700325 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f038c036950 0x7f038c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.405+0000 7f04b491d640 1 --2- 172.21.15.165:0/1307700325 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f038c036950 0x7f038c038e10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f04ac000c00 tx=0x7f04ac001330 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.405+0000 7f04b761f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:17.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.405+0000 7f04b761f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:17.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.406+0000 7f04b761f800 1 Processor -- start 2024-04-27T03:32:17.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.406+0000 7f04b761f800 1 -- start start 2024-04-27T03:32:17.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.406+0000 7f04b761f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388587e540 0x56388587e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.406+0000 7f04b761f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563885845d20 con 0x56388587e540 2024-04-27T03:32:17.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.406+0000 7f04b511e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388587e540 0x56388587e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.406+0000 7f04b511e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388587e540 0x56388587e960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59654/0 (socket says 172.21.15.165:59654) 2024-04-27T03:32:17.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.406+0000 7f04b511e640 1 -- 172.21.15.165:0/2446754178 learned_addr learned my addr 172.21.15.165:0/2446754178 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.406+0000 7f04b511e640 1 -- 172.21.15.165:0/2446754178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56388587eea0 con 0x56388587e540 2024-04-27T03:32:17.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.406+0000 7f04b511e640 1 --2- 172.21.15.165:0/2446754178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388587e540 0x56388587e960 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f04a807bfd0 tx=0x7f04a8068cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.407+0000 7f037a7dc640 1 -- 172.21.15.165:0/2446754178 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04a8092070 con 0x56388587e540 2024-04-27T03:32:17.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.407+0000 7f037a7dc640 1 -- 172.21.15.165:0/2446754178 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f04a8058040 con 0x56388587e540 2024-04-27T03:32:17.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.407+0000 7f04b761f800 1 -- 172.21.15.165:0/2446754178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x56388587f910 con 0x56388587e540 2024-04-27T03:32:17.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.407+0000 7f037a7dc640 1 -- 172.21.15.165:0/2446754178 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f04a8068e00 con 0x56388587e540 2024-04-27T03:32:17.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.407+0000 7f04b761f800 1 -- 172.21.15.165:0/2446754178 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56388587fe10 con 0x56388587e540 2024-04-27T03:32:17.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.407+0000 7f037a7dc640 1 -- 172.21.15.165:0/2446754178 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f04a805bb10 con 0x56388587e540 2024-04-27T03:32:17.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.408+0000 7f037a7dc640 1 --2- 172.21.15.165:0/2446754178 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f02f8036950 0x7f02f8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.408+0000 7f037a7dc640 1 -- 172.21.15.165:0/2446754178 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f04a8082070 con 0x56388587e540 2024-04-27T03:32:17.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.408+0000 7f04b491d640 1 --2- 172.21.15.165:0/2446754178 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f02f8036950 0x7f02f8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.408+0000 7f02ebfff640 20 reqs_thread_entry: start 2024-04-27T03:32:17.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.408+0000 7f04b761f800 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56388588bf90 0x56388588e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.408+0000 7f04b761f800 1 -- 172.21.15.165:0/2900976197 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x56388588eb20 con 0x56388588bf90 2024-04-27T03:32:17.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.408+0000 7f04b491d640 1 --2- 172.21.15.165:0/2446754178 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f02f8036950 0x7f02f8038e10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f04ac000f60 tx=0x7f04ac006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.408+0000 7f04b591f640 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56388588bf90 0x56388588e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.408+0000 7f04b591f640 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56388588bf90 0x56388588e450 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.409+0000 7f04b591f640 1 -- 172.21.15.165:0/2900976197 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f04b0010d70 con 0x56388588bf90 2024-04-27T03:32:17.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.409+0000 7f04b761f800 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563885890940 0x563885892d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.409+0000 7f04b761f800 1 -- 172.21.15.165:0/2900976197 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:2 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563885893420 con 0x563885890940 2024-04-27T03:32:17.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.409+0000 7f04b511e640 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563885890940 0x563885892d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.409+0000 7f04b511e640 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563885890940 0x563885892d30 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.409+0000 7f04b511e640 1 -- 172.21.15.165:0/2900976197 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(2 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f04a805b2e0 con 0x563885890940 2024-04-27T03:32:17.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.410+0000 7f04b761f800 1 -- 172.21.15.165:0/1307700325 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f038c036950 msgr2=0x7f038c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.410+0000 7f04b761f800 1 --2- 172.21.15.165:0/1307700325 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f038c036950 0x7f038c038e10 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f04ac000c00 tx=0x7f04ac001330 comp rx=0 tx=0).stop 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.410+0000 7f04b761f800 1 -- 172.21.15.165:0/1307700325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 msgr2=0x56388575cdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.410+0000 7f04b761f800 1 --2- 172.21.15.165:0/1307700325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 0x56388575cdf0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f04b000a930 tx=0x7f04b0003370 comp rx=0 tx=0).stop 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.410+0000 7f04b761f800 1 -- 172.21.15.165:0/1307700325 shutdown_connections 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.410+0000 7f04b761f800 1 --2- 172.21.15.165:0/1307700325 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f038c036950 0x7f038c038e10 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.410+0000 7f04b761f800 1 --2- 172.21.15.165:0/1307700325 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388585a7f0 0x56388575cdf0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.410+0000 7f04b761f800 1 -- 172.21.15.165:0/1307700325 >> 172.21.15.165:0/1307700325 conn(0x563885754040 msgr2=0x5638858548d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.410+0000 7f04b761f800 1 -- 172.21.15.165:0/1307700325 shutdown_connections 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.410+0000 7f04b761f800 1 -- 172.21.15.165:0/1307700325 wait complete. 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2446754178 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f02f8036950 msgr2=0x7f02f8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 --2- 172.21.15.165:0/2446754178 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f02f8036950 0x7f02f8038e10 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f04ac000f60 tx=0x7f04ac006000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2446754178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388587e540 msgr2=0x56388587e960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 --2- 172.21.15.165:0/2446754178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388587e540 0x56388587e960 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f04a807bfd0 tx=0x7f04a8068cf0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2446754178 shutdown_connections 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 --2- 172.21.15.165:0/2446754178 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f02f8036950 0x7f02f8038e10 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 --2- 172.21.15.165:0/2446754178 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x56388587e540 0x56388587e960 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2446754178 >> 172.21.15.165:0/2446754178 conn(0x563885872fd0 msgr2=0x5638858739f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2446754178 shutdown_connections 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2446754178 wait complete. 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56388588bf90 msgr2=0x56388588e450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56388588bf90 0x56388588e450 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563885890940 msgr2=0x563885892d30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563885890940 0x563885892d30 crc :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f039c036900 msgr2=0x7f039c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f039c036900 0x7f039c038dc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f04a80524b0 tx=0x7f04a805bef0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2900976197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 msgr2=0x56388584cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x56388584cf40 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f04a8060040 tx=0x7f04a8002fb0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2900976197 shutdown_connections 2024-04-27T03:32:17.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563885890940 0x563885892d30 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x56388588bf90 0x56388588e450 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f039c036900 0x7f039c038dc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 --2- 172.21.15.165:0/2900976197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563885752de0 0x56388584cf40 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2900976197 >> 172.21.15.165:0/2900976197 conn(0x56388574e420 msgr2=0x56388583f5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2900976197 shutdown_connections 2024-04-27T03:32:17.430 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.411+0000 7f04b761f800 1 -- 172.21.15.165:0/2900976197 wait complete. 2024-04-27T03:32:17.431 INFO:teuthology.orchestra.run.smithi165.stdout:{"id":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-04-27T03:32:17.432 DEBUG:tasks.util.rgw: json result: {'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-04-27T03:32:17.432 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'set'] 2024-04-27T03:32:17.432 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'set'] 2024-04-27T03:32:17.432 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone set 2024-04-27T03:32:17.512 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:17.513 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.507+0000 7fe46f133800 1 Processor -- start 2024-04-27T03:32:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.508+0000 7fe46f133800 1 -- start start 2024-04-27T03:32:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.508+0000 7fe46f133800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d1812cc8a0 0x55d1812cec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.508+0000 7fe46f133800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d18118ad40 con 0x55d1812cf1d0 2024-04-27T03:32:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.508+0000 7fe46f133800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55d18115d4d0 con 0x55d1812cc8a0 2024-04-27T03:32:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.508+0000 7fe46adf5640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d1812cc8a0 0x55d1812cec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.508+0000 7fe46a5f4640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55d1812cf1d0 0x55d1812d15c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42298/0 (socket says 172.21.15.165:42298) 2024-04-27T03:32:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.508+0000 7fe46a5f4640 1 -- 172.21.15.165:0/1078867252 learned_addr learned my addr 172.21.15.165:0/1078867252 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:17.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.508+0000 7fe46adf5640 1 -- 172.21.15.165:0/1078867252 >> v1:172.21.15.89:6789/0 conn(0x55d1812cf1d0 legacy=0x55d1812d15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.508+0000 7fe46adf5640 1 -- 172.21.15.165:0/1078867252 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d181189590 con 0x55d1812cc8a0 2024-04-27T03:32:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.510+0000 7fe46adf5640 1 --2- 172.21.15.165:0/1078867252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1812cec90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00a470 tx=0x7fe45c01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=922badb01a83a65a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.511+0000 7fe469df3640 1 -- 172.21.15.165:0/1078867252 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe45c057070 con 0x55d1812cc8a0 2024-04-27T03:32:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.511+0000 7fe469df3640 1 -- 172.21.15.165:0/1078867252 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45c052570 con 0x55d1812cc8a0 2024-04-27T03:32:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.511+0000 7fe469df3640 1 -- 172.21.15.165:0/1078867252 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe45c052870 con 0x55d1812cc8a0 2024-04-27T03:32:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.511+0000 7fe46f133800 1 -- 172.21.15.165:0/1078867252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 msgr2=0x55d1812cec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.511+0000 7fe46f133800 1 --2- 172.21.15.165:0/1078867252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1812cec90 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fe45c00a470 tx=0x7fe45c01ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.511+0000 7fe46f133800 1 -- 172.21.15.165:0/1078867252 shutdown_connections 2024-04-27T03:32:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.511+0000 7fe46f133800 1 --2- 172.21.15.165:0/1078867252 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1812cec90 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.511+0000 7fe46f133800 1 -- 172.21.15.165:0/1078867252 >> 172.21.15.165:0/1078867252 conn(0x55d1812c6370 msgr2=0x55d1812c8790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.511+0000 7fe46f133800 1 -- 172.21.15.165:0/1078867252 shutdown_connections 2024-04-27T03:32:17.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.511+0000 7fe46f133800 1 -- 172.21.15.165:0/1078867252 wait complete. 2024-04-27T03:32:17.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.523+0000 7fe46f133800 1 Processor -- start 2024-04-27T03:32:17.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.523+0000 7fe46f133800 1 -- start start 2024-04-27T03:32:17.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.523+0000 7fe46f133800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1813bd0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.523+0000 7fe46f133800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d1812173b0 con 0x55d1812cc8a0 2024-04-27T03:32:17.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.523+0000 7fe46adf5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1813bd0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.523+0000 7fe46adf5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1813bd0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59660/0 (socket says 172.21.15.165:59660) 2024-04-27T03:32:17.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.523+0000 7fe46adf5640 1 -- 172.21.15.165:0/2189355668 learned_addr learned my addr 172.21.15.165:0/2189355668 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.524+0000 7fe46adf5640 1 -- 172.21.15.165:0/2189355668 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1813bd690 con 0x55d1812cc8a0 2024-04-27T03:32:17.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.524+0000 7fe46adf5640 1 --2- 172.21.15.165:0/2189355668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1813bd0c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe45c055040 tx=0x7fe45c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23e0dd52707b8c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.524+0000 7fe358e09640 1 -- 172.21.15.165:0/2189355668 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe45c057070 con 0x55d1812cc8a0 2024-04-27T03:32:17.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.524+0000 7fe358e09640 1 -- 172.21.15.165:0/2189355668 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45c063070 con 0x55d1812cc8a0 2024-04-27T03:32:17.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.524+0000 7fe358e09640 1 -- 172.21.15.165:0/2189355668 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe45c052720 con 0x55d1812cc8a0 2024-04-27T03:32:17.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46f133800 1 -- 172.21.15.165:0/2189355668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 msgr2=0x55d1813bd0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46f133800 1 --2- 172.21.15.165:0/2189355668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1813bd0c0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7fe45c055040 tx=0x7fe45c04c000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46f133800 1 -- 172.21.15.165:0/2189355668 shutdown_connections 2024-04-27T03:32:17.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46f133800 1 --2- 172.21.15.165:0/2189355668 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1813bd0c0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46f133800 1 -- 172.21.15.165:0/2189355668 >> 172.21.15.165:0/2189355668 conn(0x55d1812c6370 msgr2=0x55d1813b89e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46f133800 1 -- 172.21.15.165:0/2189355668 shutdown_connections 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46f133800 1 -- 172.21.15.165:0/2189355668 wait complete. 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46f133800 1 Processor -- start 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46f133800 1 -- start start 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46f133800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1813c4bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46f133800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d1812d2f80 con 0x55d1812cc8a0 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46adf5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1813c4bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46adf5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1813c4bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59670/0 (socket says 172.21.15.165:59670) 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.525+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 learned_addr learned my addr 172.21.15.165:0/2878031156 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.526+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1813c5110 con 0x55d1812cc8a0 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.526+0000 7fe46adf5640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1813c4bd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe45c060040 tx=0x7fe45c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.526+0000 7fe357606640 1 -- 172.21.15.165:0/2878031156 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe45c052e30 con 0x55d1812cc8a0 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.526+0000 7fe357606640 1 -- 172.21.15.165:0/2878031156 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45c06c070 con 0x55d1812cc8a0 2024-04-27T03:32:17.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.526+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d1813c53a0 con 0x55d1812cc8a0 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.526+0000 7fe357606640 1 -- 172.21.15.165:0/2878031156 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe45c05e5f0 con 0x55d1812cc8a0 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.526+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1813c8050 con 0x55d1812cc8a0 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.527+0000 7fe46f133800 1 Processor -- start 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.527+0000 7fe46f133800 1 -- start start 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.527+0000 7fe357606640 1 -- 172.21.15.165:0/2878031156 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fe45c05e790 con 0x55d1812cc8a0 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.527+0000 7fe46f133800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 0x55d1812ca880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.527+0000 7fe46f133800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d1811bd6b0 con 0x55d1812cc330 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.527+0000 7fe357606640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe350036950 0x7fe350038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.527+0000 7fe357606640 1 -- 172.21.15.165:0/2878031156 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fe45c08e840 con 0x55d1812cc8a0 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.527+0000 7fe46b5f6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 0x55d1812ca880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.527+0000 7fe46b5f6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 0x55d1812ca880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59680/0 (socket says 172.21.15.165:59680) 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.527+0000 7fe46b5f6640 1 -- 172.21.15.165:0/1251218694 learned_addr learned my addr 172.21.15.165:0/1251218694 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.527+0000 7fe46adf5640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe350036950 0x7fe350038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.527+0000 7fe46b5f6640 1 -- 172.21.15.165:0/1251218694 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1812cadc0 con 0x55d1812cc330 2024-04-27T03:32:17.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.528+0000 7fe46adf5640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe350036950 0x7fe350038e10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe45c071fd0 tx=0x7fe45c07af90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.528+0000 7fe46b5f6640 1 --2- 172.21.15.165:0/1251218694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 0x55d1812ca880 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe46400a430 tx=0x7fe46400a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbd62679a19aff41 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.528+0000 7fe355602640 1 -- 172.21.15.165:0/1251218694 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe464015070 con 0x55d1812cc330 2024-04-27T03:32:17.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.528+0000 7fe355602640 1 -- 172.21.15.165:0/1251218694 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe464001670 con 0x55d1812cc330 2024-04-27T03:32:17.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.528+0000 7fe355602640 1 -- 172.21.15.165:0/1251218694 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe464001990 con 0x55d1812cc330 2024-04-27T03:32:17.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.528+0000 7fe46f133800 1 -- 172.21.15.165:0/1251218694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 msgr2=0x55d1812ca880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.528+0000 7fe46f133800 1 --2- 172.21.15.165:0/1251218694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 0x55d1812ca880 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7fe46400a430 tx=0x7fe46400a8f0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.528+0000 7fe46f133800 1 -- 172.21.15.165:0/1251218694 shutdown_connections 2024-04-27T03:32:17.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.528+0000 7fe46f133800 1 --2- 172.21.15.165:0/1251218694 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 0x55d1812ca880 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.528+0000 7fe46f133800 1 -- 172.21.15.165:0/1251218694 >> 172.21.15.165:0/1251218694 conn(0x55d1812cf1d0 msgr2=0x55d1813cef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.528+0000 7fe46f133800 1 -- 172.21.15.165:0/1251218694 shutdown_connections 2024-04-27T03:32:17.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.528+0000 7fe46f133800 1 -- 172.21.15.165:0/1251218694 wait complete. 2024-04-27T03:32:17.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.529+0000 7fe46f133800 1 Processor -- start 2024-04-27T03:32:17.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.529+0000 7fe46f133800 1 -- start start 2024-04-27T03:32:17.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.529+0000 7fe46f133800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 0x55d1813dbb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.529+0000 7fe46f133800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d1812cb050 con 0x55d1812cc330 2024-04-27T03:32:17.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.529+0000 7fe46b5f6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 0x55d1813dbb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.529+0000 7fe46b5f6640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 0x55d1813dbb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59694/0 (socket says 172.21.15.165:59694) 2024-04-27T03:32:17.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.529+0000 7fe46b5f6640 1 -- 172.21.15.165:0/1847406022 learned_addr learned my addr 172.21.15.165:0/1847406022 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.529+0000 7fe46b5f6640 1 -- 172.21.15.165:0/1847406022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1813dc060 con 0x55d1812cc330 2024-04-27T03:32:17.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.529+0000 7fe46b5f6640 1 --2- 172.21.15.165:0/1847406022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 0x55d1813dbb20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe46400a930 tx=0x7fe464003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.530+0000 7fe34f7fe640 1 -- 172.21.15.165:0/1847406022 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe464020020 con 0x55d1812cc330 2024-04-27T03:32:17.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.530+0000 7fe34f7fe640 1 -- 172.21.15.165:0/1847406022 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe4640108b0 con 0x55d1812cc330 2024-04-27T03:32:17.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.530+0000 7fe46f133800 1 -- 172.21.15.165:0/1847406022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d1813dc2f0 con 0x55d1812cc330 2024-04-27T03:32:17.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.530+0000 7fe34f7fe640 1 -- 172.21.15.165:0/1847406022 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe464015040 con 0x55d1812cc330 2024-04-27T03:32:17.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.530+0000 7fe46f133800 1 -- 172.21.15.165:0/1847406022 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1813defa0 con 0x55d1812cc330 2024-04-27T03:32:17.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.530+0000 7fe34f7fe640 1 -- 172.21.15.165:0/1847406022 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fe464023020 con 0x55d1812cc330 2024-04-27T03:32:17.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.531+0000 7fe34f7fe640 1 --2- 172.21.15.165:0/1847406022 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe344036900 0x7fe344038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.531+0000 7fe34f7fe640 1 -- 172.21.15.165:0/1847406022 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fe46404beb0 con 0x55d1812cc330 2024-04-27T03:32:17.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.531+0000 7fe46a5f4640 1 --2- 172.21.15.165:0/1847406022 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe344036900 0x7fe344038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.531+0000 7fe46f133800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:17.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.531+0000 7fe46f133800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:17.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.531+0000 7fe46a5f4640 1 --2- 172.21.15.165:0/1847406022 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe344036900 0x7fe344038dc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe460000f60 tx=0x7fe460001230 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.531+0000 7fe46f133800 1 Processor -- start 2024-04-27T03:32:17.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.531+0000 7fe46f133800 1 -- start start 2024-04-27T03:32:17.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.531+0000 7fe46f133800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1813f6210 0x55d1813f6630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.531+0000 7fe46f133800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d1813bd950 con 0x55d1813f6210 2024-04-27T03:32:17.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.532+0000 7fe46adf5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1813f6210 0x55d1813f6630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.532+0000 7fe46adf5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1813f6210 0x55d1813f6630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59696/0 (socket says 172.21.15.165:59696) 2024-04-27T03:32:17.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.532+0000 7fe46adf5640 1 -- 172.21.15.165:0/1404477164 learned_addr learned my addr 172.21.15.165:0/1404477164 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.532+0000 7fe46adf5640 1 -- 172.21.15.165:0/1404477164 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1813f6b70 con 0x55d1813f6210 2024-04-27T03:32:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.532+0000 7fe46adf5640 1 --2- 172.21.15.165:0/1404477164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1813f6210 0x55d1813f6630 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe45c08ccf0 tx=0x7fe45c002e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.532+0000 7fe32ffe7640 1 -- 172.21.15.165:0/1404477164 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe45c092030 con 0x55d1813f6210 2024-04-27T03:32:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.532+0000 7fe32ffe7640 1 -- 172.21.15.165:0/1404477164 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe45c058040 con 0x55d1813f6210 2024-04-27T03:32:17.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.532+0000 7fe46f133800 1 -- 172.21.15.165:0/1404477164 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d1813f75e0 con 0x55d1813f6210 2024-04-27T03:32:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.532+0000 7fe32ffe7640 1 -- 172.21.15.165:0/1404477164 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe45c067ce0 con 0x55d1813f6210 2024-04-27T03:32:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.532+0000 7fe46f133800 1 -- 172.21.15.165:0/1404477164 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1813f75e0 con 0x55d1813f6210 2024-04-27T03:32:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.533+0000 7fe32ffe7640 1 -- 172.21.15.165:0/1404477164 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fe45c095030 con 0x55d1813f6210 2024-04-27T03:32:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.533+0000 7fe32ffe7640 1 --2- 172.21.15.165:0/1404477164 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe2a8036900 0x7fe2a8038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.533+0000 7fe32ffe7640 1 -- 172.21.15.165:0/1404477164 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fe45c093070 con 0x55d1813f6210 2024-04-27T03:32:17.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.533+0000 7fe46a5f4640 1 --2- 172.21.15.165:0/1404477164 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe2a8036900 0x7fe2a8038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.533+0000 7fe32dfe3640 20 reqs_thread_entry: start 2024-04-27T03:32:17.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.534+0000 7fe46f133800 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d181403a80 0x55d181405e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.534+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d181406510 con 0x55d181403a80 2024-04-27T03:32:17.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.534+0000 7fe46a5f4640 1 --2- 172.21.15.165:0/1404477164 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe2a8036900 0x7fe2a8038dc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe460001ed0 tx=0x7fe46000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.534+0000 7fe46b5f6640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d181403a80 0x55d181405e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.534+0000 7fe46b5f6640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d181403a80 0x55d181405e90 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.534+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.534+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d181406df0 con 0x55d181403a80 2024-04-27T03:32:17.556 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.534+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.535+0000 7fe46f133800 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d181408700 0x55d18140ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.535+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d18140b320 con 0x55d181408700 2024-04-27T03:32:17.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.535+0000 7fe46adf5640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d181408700 0x55d18140ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.535+0000 7fe46adf5640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d181408700 0x55d18140ab90 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.557 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.535+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe45c09e070 con 0x55d181408700 2024-04-27T03:32:17.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.535+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d181418460 con 0x55d181403a80 2024-04-27T03:32:17.558 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.536+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.559 INFO:teuthology.orchestra.run.smithi165.stderr:NOTICE: set zone's realm_id= 2024-04-27T03:32:17.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.536+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181408700 2024-04-27T03:32:17.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.536+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(5 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe45c09e070 con 0x55d181408700 2024-04-27T03:32:17.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.536+0000 7fe46adf5640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe45c066380 0x7fe45c0021a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.536+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:6 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe45c066380 2024-04-27T03:32:17.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.536+0000 7fe46a5f4640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe45c066380 0x7fe45c0021a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.537+0000 7fe46a5f4640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe45c066380 0x7fe45c0021a0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.537+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe460009050 con 0x7fe45c066380 2024-04-27T03:32:17.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.537+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe45c066380 2024-04-27T03:32:17.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.537+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(7 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe460009050 con 0x7fe45c066380 2024-04-27T03:32:17.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.537+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:8 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181403a80 2024-04-27T03:32:17.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.537+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(8 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.537+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181403a80 2024-04-27T03:32:17.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.537+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.537+0000 7fe46b5f6640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe464025020 0x7fe46403e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.537+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:10 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe464025020 2024-04-27T03:32:17.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.538+0000 7fe46b5f6640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe464025020 0x7fe46403e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.538+0000 7fe46b5f6640 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe464025020 0x7fe46403e860 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.538+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x7fe464025020 2024-04-27T03:32:17.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.538+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe464025020 2024-04-27T03:32:17.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.538+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x7fe464025020 2024-04-27T03:32:17.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.539+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:12 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe45c066380 2024-04-27T03:32:17.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.539+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe460009050 con 0x7fe45c066380 2024-04-27T03:32:17.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.539+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:13 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181408700 2024-04-27T03:32:17.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.539+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe45c09e070 con 0x55d181408700 2024-04-27T03:32:17.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.539+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:14 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181403a80 2024-04-27T03:32:17.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.539+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.539+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:15 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe45c066380 2024-04-27T03:32:17.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.539+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe460009050 con 0x7fe45c066380 2024-04-27T03:32:17.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.539+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:16 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181403a80 2024-04-27T03:32:17.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.540+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.540+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:17 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe45c066380 2024-04-27T03:32:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.540+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(17 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe460009050 con 0x7fe45c066380 2024-04-27T03:32:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.540+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:18 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181408700 2024-04-27T03:32:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.540+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(18 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe45c09e070 con 0x55d181408700 2024-04-27T03:32:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.540+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181408700 2024-04-27T03:32:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.540+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe45c09e070 con 0x55d181408700 2024-04-27T03:32:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.540+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:20 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe464025020 2024-04-27T03:32:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.541+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x7fe464025020 2024-04-27T03:32:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.541+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:21 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181408700 2024-04-27T03:32:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.541+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(21 [pgnls start_epoch 27 out=107b] v25'2 uv2 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe45c09e070 con 0x55d181408700 2024-04-27T03:32:17.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.541+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181408700 2024-04-27T03:32:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.541+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(22 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe45c09e070 con 0x55d181408700 2024-04-27T03:32:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.541+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:23 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181403a80 2024-04-27T03:32:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.542+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.542+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181403a80 2024-04-27T03:32:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.542+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=74b] v18'1 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.542+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:25 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe464025020 2024-04-27T03:32:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.542+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(25 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x7fe464025020 2024-04-27T03:32:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.542+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:26 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181408700 2024-04-27T03:32:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.542+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe45c09e070 con 0x55d181408700 2024-04-27T03:32:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.542+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:27 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe45c066380 2024-04-27T03:32:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.543+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe460009050 con 0x7fe45c066380 2024-04-27T03:32:17.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.543+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:28 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181403a80 2024-04-27T03:32:17.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.543+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.543+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:29 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe464025020 2024-04-27T03:32:17.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.543+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x7fe464025020 2024-04-27T03:32:17.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.543+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:30 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181408700 2024-04-27T03:32:17.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.544+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe45c09e070 con 0x55d181408700 2024-04-27T03:32:17.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.544+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:31 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe464025020 2024-04-27T03:32:17.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.544+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x7fe464025020 2024-04-27T03:32:17.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.544+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:32 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe45c066380 2024-04-27T03:32:17.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.544+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe460009050 con 0x7fe45c066380 2024-04-27T03:32:17.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.544+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:33 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181408700 2024-04-27T03:32:17.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.544+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe45c09e070 con 0x55d181408700 2024-04-27T03:32:17.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.545+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:34 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe464025020 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.545+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46400e070 con 0x7fe464025020 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.545+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:35 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x55d181403a80 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.545+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(35 [pgnls start_epoch 27 out=115b] v18'2 uv2 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.545+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:36 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d18140f820 con 0x7fe45c066380 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.545+0000 7fe46a5f4640 1 -- 172.21.15.165:0/2878031156 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(36 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe460009050 con 0x7fe45c066380 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.545+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:37 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d1814122d0 con 0x55d181403a80 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.545+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(37 zone_names.default [read 0~46 out=46b] v0'0 uv1 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.546+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:38 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d181416330 con 0x55d181408700 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.546+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(38 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv2 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe45c09e070 con 0x55d181408700 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.546+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:39 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [create,call version.set in=58b,writefull 0~1155 in=1155b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d1814071c0 con 0x55d181408700 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.547+0000 7fe46adf5640 1 -- 172.21.15.165:0/2878031156 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(39 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [create,call,writefull 0~1155] v27'3 uv3 ondisk = 0) ==== 274+0+0 (crc 0 0 0) 0x7fe45c09e070 con 0x55d181408700 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.547+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:40 5.f 5:f43b6ece:::zone_names.default:head [create,call version.set in=58b,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d181414cb0 con 0x55d181403a80 2024-04-27T03:32:17.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.548+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(40 zone_names.default [create,call,writefull 0~46] v27'3 uv3 ondisk = 0) ==== 246+0+0 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.548+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:41 5.19 5:9f02474b:::default.zone.:head [create,writefull 0~46 in=46b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d181412750 con 0x55d181403a80 2024-04-27T03:32:17.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.549+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(41 default.zone. [create,writefull 0~46] v27'2 uv1 ondisk = -17 ((17) File exists)) ==== 199+0+0 (crc 0 0 0) 0x7fe46400e070 con 0x55d181403a80 2024-04-27T03:32:17.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 -- 172.21.15.165:0/1847406022 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe344036900 msgr2=0x7fe344038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 --2- 172.21.15.165:0/1847406022 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe344036900 0x7fe344038dc0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7fe460000f60 tx=0x7fe460001230 comp rx=0 tx=0).stop 2024-04-27T03:32:17.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 -- 172.21.15.165:0/1847406022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 msgr2=0x55d1813dbb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 --2- 172.21.15.165:0/1847406022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 0x55d1813dbb20 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7fe46400a930 tx=0x7fe464003370 comp rx=0 tx=0).stop 2024-04-27T03:32:17.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 -- 172.21.15.165:0/1847406022 shutdown_connections 2024-04-27T03:32:17.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 --2- 172.21.15.165:0/1847406022 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe344036900 0x7fe344038dc0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 --2- 172.21.15.165:0/1847406022 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc330 0x55d1813dbb20 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 -- 172.21.15.165:0/1847406022 >> 172.21.15.165:0/1847406022 conn(0x55d1812cf1d0 msgr2=0x55d1813cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 -- 172.21.15.165:0/1847406022 shutdown_connections 2024-04-27T03:32:17.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 -- 172.21.15.165:0/1847406022 wait complete. 2024-04-27T03:32:17.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 -- 172.21.15.165:0/1404477164 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe2a8036900 msgr2=0x7fe2a8038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 --2- 172.21.15.165:0/1404477164 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe2a8036900 0x7fe2a8038dc0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fe460001ed0 tx=0x7fe46000f000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 -- 172.21.15.165:0/1404477164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1813f6210 msgr2=0x55d1813f6630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 --2- 172.21.15.165:0/1404477164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1813f6210 0x55d1813f6630 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7fe45c08ccf0 tx=0x7fe45c002e40 comp rx=0 tx=0).stop 2024-04-27T03:32:17.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 -- 172.21.15.165:0/1404477164 shutdown_connections 2024-04-27T03:32:17.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 --2- 172.21.15.165:0/1404477164 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe2a8036900 0x7fe2a8038dc0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 --2- 172.21.15.165:0/1404477164 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1813f6210 0x55d1813f6630 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 -- 172.21.15.165:0/1404477164 >> 172.21.15.165:0/1404477164 conn(0x55d1813eacc0 msgr2=0x55d1813eb6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 -- 172.21.15.165:0/1404477164 shutdown_connections 2024-04-27T03:32:17.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.550+0000 7fe46f133800 1 -- 172.21.15.165:0/1404477164 wait complete. 2024-04-27T03:32:17.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe464025020 msgr2=0x7fe46403e860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe464025020 0x7fe46403e860 crc :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d181403a80 msgr2=0x55d181405e90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d181403a80 0x55d181405e90 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe45c066380 msgr2=0x7fe45c0021a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe45c066380 0x7fe45c0021a0 crc :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d181408700 msgr2=0x55d18140ab90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d181408700 0x55d18140ab90 crc :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe350036950 msgr2=0x7fe350038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe350036950 0x7fe350038e10 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe45c071fd0 tx=0x7fe45c07af90 comp rx=0 tx=0).stop 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 msgr2=0x55d1813c4bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 --2- 172.21.15.165:0/2878031156 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d1812cc8a0 0x55d1813c4bd0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7fe45c060040 tx=0x7fe45c002fb0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46b5f6640 1 -- 172.21.15.165:0/2878031156 reap_dead start 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 shutdown_connections 2024-04-27T03:32:17.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 >> 172.21.15.165:0/2878031156 conn(0x55d1812c6370 msgr2=0x55d1813b7200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 shutdown_connections 2024-04-27T03:32:17.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.551+0000 7fe46f133800 1 -- 172.21.15.165:0/2878031156 wait complete. 2024-04-27T03:32:17.581 INFO:teuthology.orchestra.run.smithi165.stdout:{"id":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}},{"key":"new-placement","val":{"index_pool":".rgw.buckets.index.2","storage_classes":{"STANDARD":{"data_pool":".rgw.buckets.2"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-04-27T03:32:17.581 DEBUG:tasks.util.rgw: json result: {'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-04-27T03:32:17.581 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'get'] 2024-04-27T03:32:17.581 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'get'] 2024-04-27T03:32:17.581 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone get 2024-04-27T03:32:17.662 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:17.662 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.657+0000 7fc32ee13800 1 Processor -- start 2024-04-27T03:32:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.657+0000 7fc32ee13800 1 -- start start 2024-04-27T03:32:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.657+0000 7fc32ee13800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a783f312d0 0x55a783f39dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.657+0000 7fc32ee13800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a783defd40 con 0x55a783f2f790 2024-04-27T03:32:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.657+0000 7fc32ee13800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55a783dc24d0 con 0x55a783f312d0 2024-04-27T03:32:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.658+0000 7fc323fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a783f312d0 0x55a783f39dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.658+0000 7fc32a46e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55a783f2f790 0x55a783f2fbb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42304/0 (socket says 172.21.15.165:42304) 2024-04-27T03:32:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.658+0000 7fc32a46e640 1 -- 172.21.15.165:0/606967840 learned_addr learned my addr 172.21.15.165:0/606967840 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.658+0000 7fc323fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a783f312d0 0x55a783f39dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59706/0 (socket says 172.21.15.165:59706) 2024-04-27T03:32:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.658+0000 7fc329c6d640 1 -- 172.21.15.165:0/606967840 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1136870387 0 0) 0x55a783defd40 con 0x55a783f2f790 2024-04-27T03:32:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.658+0000 7fc329c6d640 1 -- 172.21.15.165:0/606967840 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fc318001e10 con 0x55a783f2f790 2024-04-27T03:32:17.664 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.658+0000 7fc323fff640 1 -- 172.21.15.165:0/606967840 >> v1:172.21.15.89:6789/0 conn(0x55a783f2f790 legacy=0x55a783f2fbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.658+0000 7fc323fff640 1 -- 172.21.15.165:0/606967840 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a783dee590 con 0x55a783f312d0 2024-04-27T03:32:17.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.660+0000 7fc323fff640 1 --2- 172.21.15.165:0/606967840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f312d0 0x55a783f39dd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc31400a470 tx=0x7fc31401cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b9c7edc503afb server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.660+0000 7fc329c6d640 1 -- 172.21.15.165:0/606967840 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc314057070 con 0x55a783f312d0 2024-04-27T03:32:17.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.660+0000 7fc329c6d640 1 -- 172.21.15.165:0/606967840 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc314052440 con 0x55a783f312d0 2024-04-27T03:32:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.660+0000 7fc329c6d640 1 -- 172.21.15.165:0/606967840 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc314052740 con 0x55a783f312d0 2024-04-27T03:32:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.661+0000 7fc32ee13800 1 -- 172.21.15.165:0/606967840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f312d0 msgr2=0x55a783f39dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.661+0000 7fc32ee13800 1 --2- 172.21.15.165:0/606967840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f312d0 0x55a783f39dd0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fc31400a470 tx=0x7fc31401cdb0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.661+0000 7fc32ee13800 1 -- 172.21.15.165:0/606967840 shutdown_connections 2024-04-27T03:32:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.661+0000 7fc32ee13800 1 --2- 172.21.15.165:0/606967840 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f312d0 0x55a783f39dd0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.661+0000 7fc32ee13800 1 -- 172.21.15.165:0/606967840 >> 172.21.15.165:0/606967840 conn(0x55a783f2b310 msgr2=0x55a783f2d730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.661+0000 7fc32ee13800 1 -- 172.21.15.165:0/606967840 shutdown_connections 2024-04-27T03:32:17.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.661+0000 7fc32ee13800 1 -- 172.21.15.165:0/606967840 wait complete. 2024-04-27T03:32:17.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.672+0000 7fc32ee13800 1 Processor -- start 2024-04-27T03:32:17.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.672+0000 7fc32ee13800 1 -- start start 2024-04-27T03:32:17.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.673+0000 7fc32ee13800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 0x55a7840242f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.673+0000 7fc32ee13800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a783e7c3b0 con 0x55a783f2f790 2024-04-27T03:32:17.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.673+0000 7fc32a46e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 0x55a7840242f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.673+0000 7fc32a46e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 0x55a7840242f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59720/0 (socket says 172.21.15.165:59720) 2024-04-27T03:32:17.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.673+0000 7fc32a46e640 1 -- 172.21.15.165:0/1560681312 learned_addr learned my addr 172.21.15.165:0/1560681312 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.673+0000 7fc32a46e640 1 -- 172.21.15.165:0/1560681312 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7840248c0 con 0x55a783f2f790 2024-04-27T03:32:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.674+0000 7fc32a46e640 1 --2- 172.21.15.165:0/1560681312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 0x55a7840242f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc31c001de0 tx=0x7fc31c00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e337179ddf06273a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.674+0000 7fc218608640 1 -- 172.21.15.165:0/1560681312 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc31c012020 con 0x55a783f2f790 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.674+0000 7fc218608640 1 -- 172.21.15.165:0/1560681312 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c00e5a0 con 0x55a783f2f790 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.674+0000 7fc218608640 1 -- 172.21.15.165:0/1560681312 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc31c00e900 con 0x55a783f2f790 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.674+0000 7fc32ee13800 1 -- 172.21.15.165:0/1560681312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 msgr2=0x55a7840242f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.674+0000 7fc32ee13800 1 --2- 172.21.15.165:0/1560681312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 0x55a7840242f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fc31c001de0 tx=0x7fc31c00b6c0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.674+0000 7fc32ee13800 1 -- 172.21.15.165:0/1560681312 shutdown_connections 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.674+0000 7fc32ee13800 1 --2- 172.21.15.165:0/1560681312 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 0x55a7840242f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.674+0000 7fc32ee13800 1 -- 172.21.15.165:0/1560681312 >> 172.21.15.165:0/1560681312 conn(0x55a783f2b310 msgr2=0x55a78401fc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.675+0000 7fc32ee13800 1 -- 172.21.15.165:0/1560681312 shutdown_connections 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.675+0000 7fc32ee13800 1 -- 172.21.15.165:0/1560681312 wait complete. 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.675+0000 7fc32ee13800 1 Processor -- start 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.675+0000 7fc32ee13800 1 -- start start 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.675+0000 7fc32ee13800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 0x55a78402beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.675+0000 7fc32ee13800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a783f3c5f0 con 0x55a783f2f790 2024-04-27T03:32:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.675+0000 7fc32a46e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 0x55a78402beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.675+0000 7fc32a46e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 0x55a78402beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59732/0 (socket says 172.21.15.165:59732) 2024-04-27T03:32:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.675+0000 7fc32a46e640 1 -- 172.21.15.165:0/2889802356 learned_addr learned my addr 172.21.15.165:0/2889802356 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.676+0000 7fc32a46e640 1 -- 172.21.15.165:0/2889802356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a78402c3f0 con 0x55a783f2f790 2024-04-27T03:32:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.676+0000 7fc32a46e640 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 0x55a78402beb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00c010 tx=0x7fc31c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.676+0000 7fc216e05640 1 -- 172.21.15.165:0/2889802356 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc31c012030 con 0x55a783f2f790 2024-04-27T03:32:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.676+0000 7fc216e05640 1 -- 172.21.15.165:0/2889802356 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c01a7a0 con 0x55a783f2f790 2024-04-27T03:32:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.676+0000 7fc216e05640 1 -- 172.21.15.165:0/2889802356 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc31c01a950 con 0x55a783f2f790 2024-04-27T03:32:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.676+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a78402c680 con 0x55a783f2f790 2024-04-27T03:32:17.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.676+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a78402f290 con 0x55a783f2f790 2024-04-27T03:32:17.690 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc32ee13800 1 Processor -- start 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc32ee13800 1 -- start start 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc32ee13800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 0x55a784039ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc32ee13800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a783e226b0 con 0x55a7840396c0 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc216e05640 1 -- 172.21.15.165:0/2889802356 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc31c015020 con 0x55a783f2f790 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc216e05640 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc210036950 0x7fc210038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc32ac6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 0x55a784039ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc32ac6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 0x55a784039ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59742/0 (socket says 172.21.15.165:59742) 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc32ac6f640 1 -- 172.21.15.165:0/1312456231 learned_addr learned my addr 172.21.15.165:0/1312456231 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc216e05640 1 -- 172.21.15.165:0/2889802356 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fc31c020030 con 0x55a783f2f790 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc32a46e640 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc210036950 0x7fc210038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc32ac6f640 1 -- 172.21.15.165:0/1312456231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a78403a020 con 0x55a7840396c0 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.677+0000 7fc32a46e640 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc210036950 0x7fc210038e10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc31c01afd0 tx=0x7fc31c02df90 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.678+0000 7fc32ac6f640 1 --2- 172.21.15.165:0/1312456231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 0x55a784039ae0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc32400a430 tx=0x7fc32400a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=668435f5c9d5eadd server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.678+0000 7fc214e01640 1 -- 172.21.15.165:0/1312456231 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc324015070 con 0x55a7840396c0 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.678+0000 7fc214e01640 1 -- 172.21.15.165:0/1312456231 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc324001670 con 0x55a7840396c0 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.678+0000 7fc214e01640 1 -- 172.21.15.165:0/1312456231 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc324001990 con 0x55a7840396c0 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.678+0000 7fc32ee13800 1 -- 172.21.15.165:0/1312456231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 msgr2=0x55a784039ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.678+0000 7fc32ee13800 1 --2- 172.21.15.165:0/1312456231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 0x55a784039ae0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7fc32400a430 tx=0x7fc32400a8f0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.678+0000 7fc32ee13800 1 -- 172.21.15.165:0/1312456231 shutdown_connections 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.678+0000 7fc32ee13800 1 --2- 172.21.15.165:0/1312456231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 0x55a784039ae0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.678+0000 7fc32ee13800 1 -- 172.21.15.165:0/1312456231 >> 172.21.15.165:0/1312456231 conn(0x55a783f312d0 msgr2=0x55a784036160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.678+0000 7fc32ee13800 1 -- 172.21.15.165:0/1312456231 shutdown_connections 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.678+0000 7fc32ee13800 1 -- 172.21.15.165:0/1312456231 wait complete. 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.678+0000 7fc32ee13800 1 Processor -- start 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.679+0000 7fc32ee13800 1 -- start start 2024-04-27T03:32:17.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.679+0000 7fc32ee13800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 0x55a784040bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.679+0000 7fc32ee13800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a78403a9b0 con 0x55a7840396c0 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.679+0000 7fc32ac6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 0x55a784040bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.679+0000 7fc32ac6f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 0x55a784040bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59748/0 (socket says 172.21.15.165:59748) 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.679+0000 7fc32ac6f640 1 -- 172.21.15.165:0/748877662 learned_addr learned my addr 172.21.15.165:0/748877662 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.679+0000 7fc32ac6f640 1 -- 172.21.15.165:0/748877662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7840410f0 con 0x55a7840396c0 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.679+0000 7fc32ac6f640 1 --2- 172.21.15.165:0/748877662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 0x55a784040bb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc32400a930 tx=0x7fc324003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.679+0000 7fc20effd640 1 -- 172.21.15.165:0/748877662 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc324020020 con 0x55a7840396c0 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.679+0000 7fc20effd640 1 -- 172.21.15.165:0/748877662 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc3240108b0 con 0x55a7840396c0 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.679+0000 7fc32ee13800 1 -- 172.21.15.165:0/748877662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a784041380 con 0x55a7840396c0 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.679+0000 7fc20effd640 1 -- 172.21.15.165:0/748877662 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc324015040 con 0x55a7840396c0 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.680+0000 7fc32ee13800 1 -- 172.21.15.165:0/748877662 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a784044030 con 0x55a7840396c0 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.680+0000 7fc20effd640 1 -- 172.21.15.165:0/748877662 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc324023020 con 0x55a7840396c0 2024-04-27T03:32:17.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.680+0000 7fc20effd640 1 --2- 172.21.15.165:0/748877662 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc204036950 0x7fc204038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.680+0000 7fc20effd640 1 -- 172.21.15.165:0/748877662 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fc32404bf40 con 0x55a7840396c0 2024-04-27T03:32:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.681+0000 7fc323fff640 1 --2- 172.21.15.165:0/748877662 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc204036950 0x7fc204038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.681+0000 7fc32ee13800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.681+0000 7fc32ee13800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.681+0000 7fc323fff640 1 --2- 172.21.15.165:0/748877662 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc204036950 0x7fc204038e10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc314054000 tx=0x7fc31404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.681+0000 7fc32ee13800 1 Processor -- start 2024-04-27T03:32:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.681+0000 7fc32ee13800 1 -- start start 2024-04-27T03:32:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.681+0000 7fc32ee13800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a78405b2c0 0x55a78405b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.681+0000 7fc32ee13800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a784024c40 con 0x55a78405b2c0 2024-04-27T03:32:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.681+0000 7fc32a46e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a78405b2c0 0x55a78405b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.682+0000 7fc32a46e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a78405b2c0 0x55a78405b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59752/0 (socket says 172.21.15.165:59752) 2024-04-27T03:32:17.694 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.682+0000 7fc32a46e640 1 -- 172.21.15.165:0/2796338490 learned_addr learned my addr 172.21.15.165:0/2796338490 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.682+0000 7fc32a46e640 1 -- 172.21.15.165:0/2796338490 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a78405bc20 con 0x55a78405b2c0 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.682+0000 7fc32a46e640 1 --2- 172.21.15.165:0/2796338490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a78405b2c0 0x55a78405b6e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00bc10 tx=0x7fc31c02d7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.682+0000 7fc1eefe5640 1 -- 172.21.15.165:0/2796338490 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc31c049070 con 0x55a78405b2c0 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.682+0000 7fc1eefe5640 1 -- 172.21.15.165:0/2796338490 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc31c0164e0 con 0x55a78405b2c0 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.682+0000 7fc32ee13800 1 -- 172.21.15.165:0/2796338490 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a78405c690 con 0x55a78405b2c0 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.682+0000 7fc1eefe5640 1 -- 172.21.15.165:0/2796338490 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc31c02da20 con 0x55a78405b2c0 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.682+0000 7fc32ee13800 1 -- 172.21.15.165:0/2796338490 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a78405cb90 con 0x55a78405b2c0 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.683+0000 7fc1eefe5640 1 -- 172.21.15.165:0/2796338490 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc31c017c50 con 0x55a78405b2c0 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.683+0000 7fc1eefe5640 1 --2- 172.21.15.165:0/2796338490 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc16c036900 0x7fc16c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.683+0000 7fc1eefe5640 1 -- 172.21.15.165:0/2796338490 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fc31c035040 con 0x55a78405b2c0 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.683+0000 7fc323fff640 1 --2- 172.21.15.165:0/2796338490 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc16c036900 0x7fc16c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.683+0000 7fc1ecfe1640 20 reqs_thread_entry: start 2024-04-27T03:32:17.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.683+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a784068cc0 0x55a78406b0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.683+0000 7fc323fff640 1 --2- 172.21.15.165:0/2796338490 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc16c036900 0x7fc16c038dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc31401cdf0 tx=0x7fc314005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.683+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a78406b7a0 con 0x55a784068cc0 2024-04-27T03:32:17.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.684+0000 7fc32ac6f640 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a784068cc0 0x55a78406b0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.684+0000 7fc32ac6f640 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a784068cc0 0x55a78406b0f0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.684+0000 7fc32ac6f640 1 -- 172.21.15.165:0/2889802356 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc32400e070 con 0x55a784068cc0 2024-04-27T03:32:17.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.684+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a78406c080 con 0x55a784068cc0 2024-04-27T03:32:17.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.684+0000 7fc32ac6f640 1 -- 172.21.15.165:0/2889802356 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc32400e070 con 0x55a784068cc0 2024-04-27T03:32:17.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.684+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a78406d990 0x55a78406fe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.684+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a7840705b0 con 0x55a78406d990 2024-04-27T03:32:17.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.685+0000 7fc32a46e640 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a78406d990 0x55a78406fe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.685+0000 7fc32a46e640 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a78406d990 0x55a78406fe20 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.685+0000 7fc32a46e640 1 -- 172.21.15.165:0/2889802356 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~1155 out=1155b] v0'0 uv3 ondisk = 0) ==== 232+0+1203 (crc 0 0 0) 0x7fc31c042070 con 0x55a78406d990 2024-04-27T03:32:17.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 -- 172.21.15.165:0/748877662 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc204036950 msgr2=0x7fc204038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 --2- 172.21.15.165:0/748877662 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc204036950 0x7fc204038e10 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fc314054000 tx=0x7fc31404e000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 -- 172.21.15.165:0/748877662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 msgr2=0x55a784040bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 --2- 172.21.15.165:0/748877662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 0x55a784040bb0 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7fc32400a930 tx=0x7fc324003370 comp rx=0 tx=0).stop 2024-04-27T03:32:17.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 -- 172.21.15.165:0/748877662 shutdown_connections 2024-04-27T03:32:17.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 --2- 172.21.15.165:0/748877662 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc204036950 0x7fc204038e10 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 --2- 172.21.15.165:0/748877662 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7840396c0 0x55a784040bb0 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 -- 172.21.15.165:0/748877662 >> 172.21.15.165:0/748877662 conn(0x55a783f312d0 msgr2=0x55a7840337a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 -- 172.21.15.165:0/748877662 shutdown_connections 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 -- 172.21.15.165:0/748877662 wait complete. 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 -- 172.21.15.165:0/2796338490 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc16c036900 msgr2=0x7fc16c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2796338490 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc16c036900 0x7fc16c038dc0 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fc31401cdf0 tx=0x7fc314005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 -- 172.21.15.165:0/2796338490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a78405b2c0 msgr2=0x55a78405b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2796338490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a78405b2c0 0x55a78405b6e0 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00bc10 tx=0x7fc31c02d7c0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 -- 172.21.15.165:0/2796338490 shutdown_connections 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2796338490 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc16c036900 0x7fc16c038dc0 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2796338490 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a78405b2c0 0x55a78405b6e0 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 -- 172.21.15.165:0/2796338490 >> 172.21.15.165:0/2796338490 conn(0x55a78404fd70 msgr2=0x55a784050790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 -- 172.21.15.165:0/2796338490 shutdown_connections 2024-04-27T03:32:17.700 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.686+0000 7fc32ee13800 1 -- 172.21.15.165:0/2796338490 wait complete. 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a784068cc0 msgr2=0x55a78406b0f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a784068cc0 0x55a78406b0f0 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a78406d990 msgr2=0x55a78406fe20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a78406d990 0x55a78406fe20 crc :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc210036950 msgr2=0x7fc210038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc210036950 0x7fc210038e10 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7fc31c01afd0 tx=0x7fc31c02df90 comp rx=0 tx=0).stop 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 msgr2=0x55a78402beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 0x55a78402beb0 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x7fc31c00c010 tx=0x7fc31c000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 shutdown_connections 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a78406d990 0x55a78406fe20 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a784068cc0 0x55a78406b0f0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc210036950 0x7fc210038e10 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 --2- 172.21.15.165:0/2889802356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a783f2f790 0x55a78402beb0 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 >> 172.21.15.165:0/2889802356 conn(0x55a783f2b310 msgr2=0x55a78401e410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 shutdown_connections 2024-04-27T03:32:17.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.687+0000 7fc32ee13800 1 -- 172.21.15.165:0/2889802356 wait complete. 2024-04-27T03:32:17.702 INFO:teuthology.orchestra.run.smithi165.stdout:{"id":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}},{"key":"new-placement","val":{"index_pool":".rgw.buckets.index.2","storage_classes":{"STANDARD":{"data_pool":".rgw.buckets.2"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":false}}],"realm_id":""} 2024-04-27T03:32:17.703 DEBUG:tasks.util.rgw: json result: {'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}, {'key': 'new-placement', 'val': {'index_pool': '.rgw.buckets.index.2', 'storage_classes': {'STANDARD': {'data_pool': '.rgw.buckets.2'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': False}}], 'realm_id': ''} 2024-04-27T03:32:17.703 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-27T03:32:17.703 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'rm', '--rgw-zone', 'default', '--placement-id', 'new-placement'] 2024-04-27T03:32:17.703 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement rm --rgw-zone default --placement-id new-placement 2024-04-27T03:32:17.787 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:17.787 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:17.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.782+0000 7f0ccf487800 1 Processor -- start 2024-04-27T03:32:17.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.782+0000 7f0ccf487800 1 -- start start 2024-04-27T03:32:17.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.782+0000 7f0ccf487800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55e6c343a970 0x55e6c343cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.782+0000 7f0ccf487800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e6c32f8d40 con 0x55e6c343d2a0 2024-04-27T03:32:17.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.782+0000 7f0ccf487800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55e6c32cb4d0 con 0x55e6c343a970 2024-04-27T03:32:17.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.783+0000 7f0cccf86640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55e6c343a970 0x55e6c343cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.783+0000 7f0cc4785640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55e6c343d2a0 0x55e6c343f690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42308/0 (socket says 172.21.15.165:42308) 2024-04-27T03:32:17.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.783+0000 7f0cc4785640 1 -- 172.21.15.165:0/2983188068 learned_addr learned my addr 172.21.15.165:0/2983188068 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:17.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.783+0000 7f0cc7fff640 1 -- 172.21.15.165:0/2983188068 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3538264098 0 0) 0x55e6c32f8d40 con 0x55e6c343d2a0 2024-04-27T03:32:17.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.783+0000 7f0cc7fff640 1 -- 172.21.15.165:0/2983188068 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0cb8001e10 con 0x55e6c343d2a0 2024-04-27T03:32:17.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.783+0000 7f0cccf86640 1 -- 172.21.15.165:0/2983188068 >> v1:172.21.15.89:6789/0 conn(0x55e6c343d2a0 legacy=0x55e6c343f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.783+0000 7f0cccf86640 1 -- 172.21.15.165:0/2983188068 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c32f7590 con 0x55e6c343a970 2024-04-27T03:32:17.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.785+0000 7f0cccf86640 1 --2- 172.21.15.165:0/2983188068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c343cd60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00a470 tx=0x7f0cbc01cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e76b7792e35ad87 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.785+0000 7f0cc7fff640 1 -- 172.21.15.165:0/2983188068 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cbc057070 con 0x55e6c343a970 2024-04-27T03:32:17.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.785+0000 7f0cc7fff640 1 -- 172.21.15.165:0/2983188068 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cbc052570 con 0x55e6c343a970 2024-04-27T03:32:17.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.785+0000 7f0cc7fff640 1 -- 172.21.15.165:0/2983188068 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cbc052870 con 0x55e6c343a970 2024-04-27T03:32:17.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.785+0000 7f0ccf487800 1 -- 172.21.15.165:0/2983188068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 msgr2=0x55e6c343cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.785+0000 7f0ccf487800 1 --2- 172.21.15.165:0/2983188068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c343cd60 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc00a470 tx=0x7f0cbc01cdb0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.785+0000 7f0ccf487800 1 -- 172.21.15.165:0/2983188068 shutdown_connections 2024-04-27T03:32:17.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.785+0000 7f0ccf487800 1 --2- 172.21.15.165:0/2983188068 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c343cd60 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.785+0000 7f0ccf487800 1 -- 172.21.15.165:0/2983188068 >> 172.21.15.165:0/2983188068 conn(0x55e6c3434440 msgr2=0x55e6c3436860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.785+0000 7f0ccf487800 1 -- 172.21.15.165:0/2983188068 shutdown_connections 2024-04-27T03:32:17.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.785+0000 7f0ccf487800 1 -- 172.21.15.165:0/2983188068 wait complete. 2024-04-27T03:32:17.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.797+0000 7f0ccf487800 1 Processor -- start 2024-04-27T03:32:17.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.797+0000 7f0ccf487800 1 -- start start 2024-04-27T03:32:17.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.797+0000 7f0ccf487800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c343a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.803 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.797+0000 7f0ccf487800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e6c33853e0 con 0x55e6c343a970 2024-04-27T03:32:17.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.798+0000 7f0cccf86640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c343a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.798+0000 7f0cccf86640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c343a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59772/0 (socket says 172.21.15.165:59772) 2024-04-27T03:32:17.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.798+0000 7f0cccf86640 1 -- 172.21.15.165:0/2576622915 learned_addr learned my addr 172.21.15.165:0/2576622915 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.804 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.798+0000 7f0cccf86640 1 -- 172.21.15.165:0/2576622915 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c3438870 con 0x55e6c343a970 2024-04-27T03:32:17.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.798+0000 7f0cccf86640 1 --2- 172.21.15.165:0/2576622915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c343a2b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc055040 tx=0x7f0cbc04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c1365f9eaccc37e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.798+0000 7f0bbae0d640 1 -- 172.21.15.165:0/2576622915 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cbc057070 con 0x55e6c343a970 2024-04-27T03:32:17.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.798+0000 7f0bbae0d640 1 -- 172.21.15.165:0/2576622915 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cbc063070 con 0x55e6c343a970 2024-04-27T03:32:17.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.798+0000 7f0bbae0d640 1 -- 172.21.15.165:0/2576622915 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cbc052720 con 0x55e6c343a970 2024-04-27T03:32:17.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.799+0000 7f0ccf487800 1 -- 172.21.15.165:0/2576622915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 msgr2=0x55e6c343a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.799+0000 7f0ccf487800 1 --2- 172.21.15.165:0/2576622915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c343a2b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc055040 tx=0x7f0cbc04c000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.806 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.799+0000 7f0ccf487800 1 -- 172.21.15.165:0/2576622915 shutdown_connections 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.799+0000 7f0ccf487800 1 --2- 172.21.15.165:0/2576622915 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c343a2b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.799+0000 7f0ccf487800 1 -- 172.21.15.165:0/2576622915 >> 172.21.15.165:0/2576622915 conn(0x55e6c3434440 msgr2=0x55e6c35268c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.799+0000 7f0ccf487800 1 -- 172.21.15.165:0/2576622915 shutdown_connections 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.799+0000 7f0ccf487800 1 -- 172.21.15.165:0/2576622915 wait complete. 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.799+0000 7f0ccf487800 1 Processor -- start 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.799+0000 7f0ccf487800 1 -- start start 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.800+0000 7f0ccf487800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c3534cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.800+0000 7f0ccf487800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e6c3441050 con 0x55e6c343a970 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.800+0000 7f0cccf86640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c3534cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.800+0000 7f0cccf86640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c3534cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59786/0 (socket says 172.21.15.165:59786) 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.800+0000 7f0cccf86640 1 -- 172.21.15.165:0/1137309242 learned_addr learned my addr 172.21.15.165:0/1137309242 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.800+0000 7f0cccf86640 1 -- 172.21.15.165:0/1137309242 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c3535210 con 0x55e6c343a970 2024-04-27T03:32:17.807 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.801+0000 7f0cccf86640 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c3534cd0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc055040 tx=0x7f0cbc002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.801+0000 7f0bb960a640 1 -- 172.21.15.165:0/1137309242 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cbc052e30 con 0x55e6c343a970 2024-04-27T03:32:17.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.801+0000 7f0bb960a640 1 -- 172.21.15.165:0/1137309242 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cbc06c070 con 0x55e6c343a970 2024-04-27T03:32:17.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.801+0000 7f0bb960a640 1 -- 172.21.15.165:0/1137309242 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cbc05e5f0 con 0x55e6c343a970 2024-04-27T03:32:17.816 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.801+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e6c35354a0 con 0x55e6c343a970 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.801+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c3538150 con 0x55e6c343a970 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.801+0000 7f0ccf487800 1 Processor -- start 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0ccf487800 1 -- start start 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0ccf487800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 0x55e6c3542930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0ccf487800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e6c332b6b0 con 0x55e6c3542510 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0bb960a640 1 -- 172.21.15.165:0/1137309242 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0cbc05e790 con 0x55e6c343a970 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0ccd787640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 0x55e6c3542930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0ccd787640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 0x55e6c3542930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59802/0 (socket says 172.21.15.165:59802) 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0ccd787640 1 -- 172.21.15.165:0/1413741124 learned_addr learned my addr 172.21.15.165:0/1413741124 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0bb960a640 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0bb4036900 0x7f0bb4038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0bb960a640 1 -- 172.21.15.165:0/1137309242 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0cbc08e8f0 con 0x55e6c343a970 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0ccd787640 1 -- 172.21.15.165:0/1413741124 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c3542e70 con 0x55e6c3542510 2024-04-27T03:32:17.817 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0cccf86640 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0bb4036900 0x7f0bb4038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0ccd787640 1 --2- 172.21.15.165:0/1413741124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 0x55e6c3542930 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800a430 tx=0x7f0cc800a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee12da31771e2d26 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.802+0000 7f0bb2ffd640 1 -- 172.21.15.165:0/1413741124 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cc8015070 con 0x55e6c3542510 2024-04-27T03:32:17.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0bb2ffd640 1 -- 172.21.15.165:0/1413741124 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc8001670 con 0x55e6c3542510 2024-04-27T03:32:17.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0bb2ffd640 1 -- 172.21.15.165:0/1413741124 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cc8001990 con 0x55e6c3542510 2024-04-27T03:32:17.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0ccf487800 1 -- 172.21.15.165:0/1413741124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 msgr2=0x55e6c3542930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0cccf86640 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0bb4036900 0x7f0bb4038dc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc068010 tx=0x7f0cbc005330 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1413741124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 0x55e6c3542930 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800a430 tx=0x7f0cc800a8f0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0ccf487800 1 -- 172.21.15.165:0/1413741124 shutdown_connections 2024-04-27T03:32:17.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1413741124 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 0x55e6c3542930 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0ccf487800 1 -- 172.21.15.165:0/1413741124 >> 172.21.15.165:0/1413741124 conn(0x55e6c343d2a0 msgr2=0x55e6c353efb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0ccf487800 1 -- 172.21.15.165:0/1413741124 shutdown_connections 2024-04-27T03:32:17.818 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0ccf487800 1 -- 172.21.15.165:0/1413741124 wait complete. 2024-04-27T03:32:17.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0ccf487800 1 Processor -- start 2024-04-27T03:32:17.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0ccf487800 1 -- start start 2024-04-27T03:32:17.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0ccf487800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 0x55e6c3549a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.803+0000 7f0ccf487800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e6c3543800 con 0x55e6c3542510 2024-04-27T03:32:17.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.804+0000 7f0ccd787640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 0x55e6c3549a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.804+0000 7f0ccd787640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 0x55e6c3549a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59818/0 (socket says 172.21.15.165:59818) 2024-04-27T03:32:17.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.804+0000 7f0ccd787640 1 -- 172.21.15.165:0/1931762481 learned_addr learned my addr 172.21.15.165:0/1931762481 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.804+0000 7f0ccd787640 1 -- 172.21.15.165:0/1931762481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c3549f50 con 0x55e6c3542510 2024-04-27T03:32:17.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.804+0000 7f0ccd787640 1 --2- 172.21.15.165:0/1931762481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 0x55e6c3549a10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800a930 tx=0x7f0cc8003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.804+0000 7f0bb17fa640 1 -- 172.21.15.165:0/1931762481 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cc8020020 con 0x55e6c3542510 2024-04-27T03:32:17.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.804+0000 7f0bb17fa640 1 -- 172.21.15.165:0/1931762481 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cc80108b0 con 0x55e6c3542510 2024-04-27T03:32:17.819 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.804+0000 7f0ccf487800 1 -- 172.21.15.165:0/1931762481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e6c354a1e0 con 0x55e6c3542510 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.804+0000 7f0bb17fa640 1 -- 172.21.15.165:0/1931762481 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cc8015040 con 0x55e6c3542510 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.804+0000 7f0ccf487800 1 -- 172.21.15.165:0/1931762481 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c354ce90 con 0x55e6c3542510 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.805+0000 7f0bb17fa640 1 -- 172.21.15.165:0/1931762481 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0cc8023020 con 0x55e6c3542510 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.805+0000 7f0bb17fa640 1 --2- 172.21.15.165:0/1931762481 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ba4036950 0x7f0ba4038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.805+0000 7f0bb17fa640 1 -- 172.21.15.165:0/1931762481 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0cc804be80 con 0x55e6c3542510 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.805+0000 7f0ccf487800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.805+0000 7f0ccf487800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.805+0000 7f0cc4785640 1 --2- 172.21.15.165:0/1931762481 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ba4036950 0x7f0ba4038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.806+0000 7f0ccf487800 1 Processor -- start 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.806+0000 7f0ccf487800 1 -- start start 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.806+0000 7f0cc4785640 1 --2- 172.21.15.165:0/1931762481 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ba4036950 0x7f0ba4038e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4000f60 tx=0x7f0cb40010a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.806+0000 7f0ccf487800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3564130 0x55e6c3564550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.820 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.806+0000 7f0ccf487800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e6c3438b30 con 0x55e6c3564130 2024-04-27T03:32:17.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.806+0000 7f0cccf86640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3564130 0x55e6c3564550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.806+0000 7f0cccf86640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3564130 0x55e6c3564550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59826/0 (socket says 172.21.15.165:59826) 2024-04-27T03:32:17.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.806+0000 7f0cccf86640 1 -- 172.21.15.165:0/2752660101 learned_addr learned my addr 172.21.15.165:0/2752660101 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.806+0000 7f0cccf86640 1 -- 172.21.15.165:0/2752660101 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e6c3564a90 con 0x55e6c3564130 2024-04-27T03:32:17.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.807+0000 7f0cccf86640 1 --2- 172.21.15.165:0/2752660101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3564130 0x55e6c3564550 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc08ce10 tx=0x7f0cbc052950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.807+0000 7f0b917e2640 1 -- 172.21.15.165:0/2752660101 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cbc05ce20 con 0x55e6c3564130 2024-04-27T03:32:17.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.807+0000 7f0ccf487800 1 -- 172.21.15.165:0/2752660101 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e6c3565500 con 0x55e6c3564130 2024-04-27T03:32:17.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.807+0000 7f0b917e2640 1 -- 172.21.15.165:0/2752660101 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0cbc092070 con 0x55e6c3564130 2024-04-27T03:32:17.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.807+0000 7f0b917e2640 1 -- 172.21.15.165:0/2752660101 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0cbc009ab0 con 0x55e6c3564130 2024-04-27T03:32:17.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.807+0000 7f0ccf487800 1 -- 172.21.15.165:0/2752660101 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e6c3565a00 con 0x55e6c3564130 2024-04-27T03:32:17.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.807+0000 7f0b917e2640 1 -- 172.21.15.165:0/2752660101 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0cbc05ec60 con 0x55e6c3564130 2024-04-27T03:32:17.821 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.808+0000 7f0b917e2640 1 --2- 172.21.15.165:0/2752660101 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0b08036900 0x7f0b08038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.808+0000 7f0b917e2640 1 -- 172.21.15.165:0/2752660101 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0cbc082040 con 0x55e6c3564130 2024-04-27T03:32:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.808+0000 7f0cc4785640 1 --2- 172.21.15.165:0/2752660101 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0b08036900 0x7f0b08038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.808+0000 7f0b12ffd640 20 reqs_thread_entry: start 2024-04-27T03:32:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.808+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e6c3571b80 0x55e6c3574040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.808+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6c3574710 con 0x55e6c3571b80 2024-04-27T03:32:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.808+0000 7f0cc4785640 1 --2- 172.21.15.165:0/2752660101 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0b08036900 0x7f0b08038dc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4004170 tx=0x7f0cb400f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.808+0000 7f0ccd787640 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e6c3571b80 0x55e6c3574040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.808+0000 7f0ccd787640 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e6c3571b80 0x55e6c3574040 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.809+0000 7f0ccd787640 1 -- 172.21.15.165:0/1137309242 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0cc8010d70 con 0x55e6c3571b80 2024-04-27T03:32:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.809+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e6c3576530 0x55e6c3578920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.809+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:2 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e6c3579010 con 0x55e6c3576530 2024-04-27T03:32:17.822 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.809+0000 7f0cccf86640 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e6c3576530 0x55e6c3578920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.809+0000 7f0cccf86640 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e6c3576530 0x55e6c3578920 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.809+0000 7f0cccf86640 1 -- 172.21.15.165:0/1137309242 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(2 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~1155 out=1155b] v0'0 uv3 ondisk = 0) ==== 232+0+1203 (crc 0 0 0) 0x7f0cbc097d70 con 0x55e6c3576530 2024-04-27T03:32:17.823 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.809+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [stat,call version.check_conds in=122b,call version.inc in=38b,writefull 0~993 in=993b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e6c3575c00 con 0x55e6c3576530 2024-04-27T03:32:17.824 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.811+0000 7f0cccf86640 1 -- 172.21.15.165:0/1137309242 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [stat,call,call,writefull 0~993] v27'4 uv4 ondisk = 0) ==== 316+0+0 (crc 0 0 0) 0x7f0cbc097d70 con 0x55e6c3576530 2024-04-27T03:32:17.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.811+0000 7f0ccf487800 1 -- 172.21.15.165:0/1931762481 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ba4036950 msgr2=0x7f0ba4038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.811+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1931762481 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ba4036950 0x7f0ba4038e10 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4000f60 tx=0x7f0cb40010a0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.811+0000 7f0ccf487800 1 -- 172.21.15.165:0/1931762481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 msgr2=0x55e6c3549a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.811+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1931762481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 0x55e6c3549a10 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f0cc800a930 tx=0x7f0cc8003370 comp rx=0 tx=0).stop 2024-04-27T03:32:17.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/1931762481 shutdown_connections 2024-04-27T03:32:17.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1931762481 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0ba4036950 0x7f0ba4038e10 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1931762481 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3542510 0x55e6c3549a10 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/1931762481 >> 172.21.15.165:0/1931762481 conn(0x55e6c343d2a0 msgr2=0x55e6c353c5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/1931762481 shutdown_connections 2024-04-27T03:32:17.827 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/1931762481 wait complete. 2024-04-27T03:32:17.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/2752660101 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0b08036900 msgr2=0x7f0b08038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/2752660101 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0b08036900 0x7f0b08038dc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0cb4004170 tx=0x7f0cb400f000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/2752660101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3564130 msgr2=0x55e6c3564550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/2752660101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3564130 0x55e6c3564550 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc08ce10 tx=0x7f0cbc052950 comp rx=0 tx=0).stop 2024-04-27T03:32:17.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/2752660101 shutdown_connections 2024-04-27T03:32:17.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/2752660101 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0b08036900 0x7f0b08038dc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.829 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/2752660101 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c3564130 0x55e6c3564550 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/2752660101 >> 172.21.15.165:0/2752660101 conn(0x55e6c3558c00 msgr2=0x55e6c3559620 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/2752660101 shutdown_connections 2024-04-27T03:32:17.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/2752660101 wait complete. 2024-04-27T03:32:17.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e6c3571b80 msgr2=0x55e6c3574040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e6c3571b80 0x55e6c3574040 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e6c3576530 msgr2=0x55e6c3578920 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e6c3576530 0x55e6c3578920 crc :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0bb4036900 msgr2=0x7f0bb4038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0bb4036900 0x7f0bb4038dc0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc068010 tx=0x7f0cbc005330 comp rx=0 tx=0).stop 2024-04-27T03:32:17.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 msgr2=0x55e6c3534cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.830 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c3534cd0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc055040 tx=0x7f0cbc002fb0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 shutdown_connections 2024-04-27T03:32:17.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e6c3576530 0x55e6c3578920 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e6c3571b80 0x55e6c3574040 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0bb4036900 0x7f0bb4038dc0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 --2- 172.21.15.165:0/1137309242 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e6c343a970 0x55e6c3534cd0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 >> 172.21.15.165:0/1137309242 conn(0x55e6c3434440 msgr2=0x55e6c3439090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 shutdown_connections 2024-04-27T03:32:17.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.812+0000 7f0ccf487800 1 -- 172.21.15.165:0/1137309242 wait complete. 2024-04-27T03:32:17.833 INFO:teuthology.orchestra.run.smithi165.stdout:{"id":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","notif_pool":"default.rgw.log:notif","topics_pool":"default.rgw.meta:topics","account_pool":"default.rgw.meta:accounts","group_pool":"default.rgw.meta:groups","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0,"inline_data":true}}],"realm_id":""} 2024-04-27T03:32:17.833 DEBUG:tasks.util.rgw: json result: {'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa', 'name': 'default', 'domain_root': 'default.rgw.meta:root', 'control_pool': 'default.rgw.control', 'gc_pool': 'default.rgw.log:gc', 'lc_pool': 'default.rgw.log:lc', 'log_pool': 'default.rgw.log', 'intent_log_pool': 'default.rgw.log:intent', 'usage_log_pool': 'default.rgw.log:usage', 'roles_pool': 'default.rgw.meta:roles', 'reshard_pool': 'default.rgw.log:reshard', 'user_keys_pool': 'default.rgw.meta:users.keys', 'user_email_pool': 'default.rgw.meta:users.email', 'user_swift_pool': 'default.rgw.meta:users.swift', 'user_uid_pool': 'default.rgw.meta:users.uid', 'otp_pool': 'default.rgw.otp', 'notif_pool': 'default.rgw.log:notif', 'topics_pool': 'default.rgw.meta:topics', 'account_pool': 'default.rgw.meta:accounts', 'group_pool': 'default.rgw.meta:groups', 'system_key': {'access_key': '', 'secret_key': ''}, 'placement_pools': [{'key': 'default-placement', 'val': {'index_pool': 'default.rgw.buckets.index', 'storage_classes': {'STANDARD': {'data_pool': 'default.rgw.buckets.data'}}, 'data_extra_pool': 'default.rgw.buckets.non-ec', 'index_type': 0, 'inline_data': True}}], 'realm_id': ''} 2024-04-27T03:32:17.833 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'get'] 2024-04-27T03:32:17.833 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'get'] 2024-04-27T03:32:17.833 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup get 2024-04-27T03:32:17.874 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:17.874 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.869+0000 7f0b48923800 1 Processor -- start 2024-04-27T03:32:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.869+0000 7f0b48923800 1 -- start start 2024-04-27T03:32:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.869+0000 7f0b48923800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5633d0034960 0x5633d0036d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.869+0000 7f0b48923800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5633cfef2d40 con 0x5633d0037290 2024-04-27T03:32:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.869+0000 7f0b48923800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5633cfec54d0 con 0x5633d0034960 2024-04-27T03:32:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.870+0000 7f0b44659640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5633d0034960 0x5633d0036d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.870+0000 7f0b3bfff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5633d0037290 0x5633d0039680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42322/0 (socket says 172.21.15.165:42322) 2024-04-27T03:32:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.870+0000 7f0b3bfff640 1 -- 172.21.15.165:0/2567792123 learned_addr learned my addr 172.21.15.165:0/2567792123 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.870+0000 7f0b44659640 1 -- 172.21.15.165:0/2567792123 >> v1:172.21.15.89:6789/0 conn(0x5633d0037290 legacy=0x5633d0039680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.870+0000 7f0b43e58640 1 -- 172.21.15.165:0/2567792123 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2305647025 0 0) 0x5633cfef2d40 con 0x5633d0037290 2024-04-27T03:32:17.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.870+0000 7f0b44659640 1 -- 172.21.15.165:0/2567792123 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633cfef1590 con 0x5633d0034960 2024-04-27T03:32:17.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.872+0000 7f0b44659640 1 --2- 172.21.15.165:0/2567792123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d0036d50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400a470 tx=0x7f0b3401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1cfa4dd4779938 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.872+0000 7f0b43e58640 1 -- 172.21.15.165:0/2567792123 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34057070 con 0x5633d0034960 2024-04-27T03:32:17.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.872+0000 7f0b43e58640 1 -- 172.21.15.165:0/2567792123 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b34052570 con 0x5633d0034960 2024-04-27T03:32:17.878 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.872+0000 7f0b43e58640 1 -- 172.21.15.165:0/2567792123 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34052870 con 0x5633d0034960 2024-04-27T03:32:17.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.872+0000 7f0b48923800 1 -- 172.21.15.165:0/2567792123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 msgr2=0x5633d0036d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.872+0000 7f0b48923800 1 --2- 172.21.15.165:0/2567792123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d0036d50 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f0b3400a470 tx=0x7f0b3401ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:17.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.873+0000 7f0b48923800 1 -- 172.21.15.165:0/2567792123 shutdown_connections 2024-04-27T03:32:17.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.873+0000 7f0b48923800 1 --2- 172.21.15.165:0/2567792123 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d0036d50 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.873+0000 7f0b48923800 1 -- 172.21.15.165:0/2567792123 >> 172.21.15.165:0/2567792123 conn(0x5633d002e310 msgr2=0x5633d0030730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.873+0000 7f0b48923800 1 -- 172.21.15.165:0/2567792123 shutdown_connections 2024-04-27T03:32:17.879 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.873+0000 7f0b48923800 1 -- 172.21.15.165:0/2567792123 wait complete. 2024-04-27T03:32:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.884+0000 7f0b48923800 1 Processor -- start 2024-04-27T03:32:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.884+0000 7f0b48923800 1 -- start start 2024-04-27T03:32:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.885+0000 7f0b48923800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d0124e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.885+0000 7f0b48923800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5633cff7f3b0 con 0x5633d0034960 2024-04-27T03:32:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.885+0000 7f0b44659640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d0124e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.885+0000 7f0b44659640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d0124e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59844/0 (socket says 172.21.15.165:59844) 2024-04-27T03:32:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.885+0000 7f0b44659640 1 -- 172.21.15.165:0/4119029859 learned_addr learned my addr 172.21.15.165:0/4119029859 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.885+0000 7f0b44659640 1 -- 172.21.15.165:0/4119029859 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d0125430 con 0x5633d0034960 2024-04-27T03:32:17.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.885+0000 7f0b44659640 1 --2- 172.21.15.165:0/4119029859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d0124e60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0b34055040 tx=0x7f0b3404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d25618aedb8817d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.886+0000 7f0a3260c640 1 -- 172.21.15.165:0/4119029859 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34063070 con 0x5633d0034960 2024-04-27T03:32:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.886+0000 7f0a3260c640 1 -- 172.21.15.165:0/4119029859 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b34058040 con 0x5633d0034960 2024-04-27T03:32:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.886+0000 7f0a3260c640 1 -- 172.21.15.165:0/4119029859 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34057070 con 0x5633d0034960 2024-04-27T03:32:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.886+0000 7f0b48923800 1 -- 172.21.15.165:0/4119029859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 msgr2=0x5633d0124e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.886+0000 7f0b48923800 1 --2- 172.21.15.165:0/4119029859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d0124e60 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f0b34055040 tx=0x7f0b3404c000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.886+0000 7f0b48923800 1 -- 172.21.15.165:0/4119029859 shutdown_connections 2024-04-27T03:32:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.886+0000 7f0b48923800 1 --2- 172.21.15.165:0/4119029859 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d0124e60 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.886+0000 7f0b48923800 1 -- 172.21.15.165:0/4119029859 >> 172.21.15.165:0/4119029859 conn(0x5633d002e310 msgr2=0x5633d0120780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.886+0000 7f0b48923800 1 -- 172.21.15.165:0/4119029859 shutdown_connections 2024-04-27T03:32:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.886+0000 7f0b48923800 1 -- 172.21.15.165:0/4119029859 wait complete. 2024-04-27T03:32:17.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.887+0000 7f0b48923800 1 Processor -- start 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.887+0000 7f0b48923800 1 -- start start 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.887+0000 7f0b48923800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d00342a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.887+0000 7f0b48923800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5633d003d0b0 con 0x5633d0034960 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.887+0000 7f0b44659640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d00342a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.887+0000 7f0b44659640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d00342a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59852/0 (socket says 172.21.15.165:59852) 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.887+0000 7f0b44659640 1 -- 172.21.15.165:0/1613898511 learned_addr learned my addr 172.21.15.165:0/1613898511 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.887+0000 7f0b44659640 1 -- 172.21.15.165:0/1613898511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d0032990 con 0x5633d0034960 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.888+0000 7f0b44659640 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d00342a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0b34061040 tx=0x7f0b34002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.888+0000 7f0a30e09640 1 -- 172.21.15.165:0/1613898511 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34063030 con 0x5633d0034960 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.888+0000 7f0a30e09640 1 -- 172.21.15.165:0/1613898511 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b34052d90 con 0x5633d0034960 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.888+0000 7f0a30e09640 1 -- 172.21.15.165:0/1613898511 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b3405e5b0 con 0x5633d0034960 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.888+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5633d0032c20 con 0x5633d0034960 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.888+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633d0033140 con 0x5633d0034960 2024-04-27T03:32:17.901 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.888+0000 7f0b48923800 1 Processor -- start 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.888+0000 7f0b48923800 1 -- start start 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.888+0000 7f0b48923800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 0x5633d013c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.888+0000 7f0b48923800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5633cff256b0 con 0x5633d013c410 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.889+0000 7f0a30e09640 1 -- 172.21.15.165:0/1613898511 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0b3405e750 con 0x5633d0034960 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.889+0000 7f0a30e09640 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0a2c036950 0x7f0a2c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.889+0000 7f0a30e09640 1 -- 172.21.15.165:0/1613898511 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0b34064070 con 0x5633d0034960 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.889+0000 7f0b44e5a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 0x5633d013c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.889+0000 7f0b44659640 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0a2c036950 0x7f0a2c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.889+0000 7f0b44e5a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 0x5633d013c830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59856/0 (socket says 172.21.15.165:59856) 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.889+0000 7f0b44e5a640 1 -- 172.21.15.165:0/2669556933 learned_addr learned my addr 172.21.15.165:0/2669556933 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.889+0000 7f0b44e5a640 1 -- 172.21.15.165:0/2669556933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d013cd70 con 0x5633d013c410 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.889+0000 7f0b44659640 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0a2c036950 0x7f0a2c038e10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0b34072510 tx=0x7f0b34080040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.889+0000 7f0b44e5a640 1 --2- 172.21.15.165:0/2669556933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 0x5633d013c830 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00a430 tx=0x7f0b3c00a8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c7ef7de558b32c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.902 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0a2a7fc640 1 -- 172.21.15.165:0/2669556933 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b3c015070 con 0x5633d013c410 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0a2a7fc640 1 -- 172.21.15.165:0/2669556933 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3c001670 con 0x5633d013c410 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0a2a7fc640 1 -- 172.21.15.165:0/2669556933 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b3c001990 con 0x5633d013c410 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0b48923800 1 -- 172.21.15.165:0/2669556933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 msgr2=0x5633d013c830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0b48923800 1 --2- 172.21.15.165:0/2669556933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 0x5633d013c830 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c00a430 tx=0x7f0b3c00a8f0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0b48923800 1 -- 172.21.15.165:0/2669556933 shutdown_connections 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0b48923800 1 --2- 172.21.15.165:0/2669556933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 0x5633d013c830 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0b48923800 1 -- 172.21.15.165:0/2669556933 >> 172.21.15.165:0/2669556933 conn(0x5633d0037290 msgr2=0x5633d0138eb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0b48923800 1 -- 172.21.15.165:0/2669556933 shutdown_connections 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0b48923800 1 -- 172.21.15.165:0/2669556933 wait complete. 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0b48923800 1 Processor -- start 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0b48923800 1 -- start start 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.890+0000 7f0b48923800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 0x5633d01438a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.903 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.891+0000 7f0b48923800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5633d013d700 con 0x5633d013c410 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.891+0000 7f0b44e5a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 0x5633d01438a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.891+0000 7f0b44e5a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 0x5633d01438a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59870/0 (socket says 172.21.15.165:59870) 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.891+0000 7f0b44e5a640 1 -- 172.21.15.165:0/4078018245 learned_addr learned my addr 172.21.15.165:0/4078018245 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.891+0000 7f0b44e5a640 1 -- 172.21.15.165:0/4078018245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d0143de0 con 0x5633d013c410 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.891+0000 7f0b44e5a640 1 --2- 172.21.15.165:0/4078018245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 0x5633d01438a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c013040 tx=0x7f0b3c003370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.891+0000 7f0a28ff9640 1 -- 172.21.15.165:0/4078018245 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b3c020020 con 0x5633d013c410 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.891+0000 7f0a28ff9640 1 -- 172.21.15.165:0/4078018245 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b3c0108e0 con 0x5633d013c410 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.891+0000 7f0b48923800 1 -- 172.21.15.165:0/4078018245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5633d0144070 con 0x5633d013c410 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.891+0000 7f0a28ff9640 1 -- 172.21.15.165:0/4078018245 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b3c015040 con 0x5633d013c410 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.891+0000 7f0b48923800 1 -- 172.21.15.165:0/4078018245 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633d0146d20 con 0x5633d013c410 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.892+0000 7f0a28ff9640 1 -- 172.21.15.165:0/4078018245 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0b3c023020 con 0x5633d013c410 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.892+0000 7f0a28ff9640 1 --2- 172.21.15.165:0/4078018245 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0a1c036900 0x7f0a1c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.892+0000 7f0a28ff9640 1 -- 172.21.15.165:0/4078018245 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0b3c04bf80 con 0x5633d013c410 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.893+0000 7f0b3bfff640 1 --2- 172.21.15.165:0/4078018245 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0a1c036900 0x7f0a1c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.893+0000 7f0b48923800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.893+0000 7f0b48923800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.893+0000 7f0b48923800 1 Processor -- start 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.893+0000 7f0b3bfff640 1 --2- 172.21.15.165:0/4078018245 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0a1c036900 0x7f0a1c038dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c000c00 tx=0x7f0b2c001330 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.893+0000 7f0b48923800 1 -- start start 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.893+0000 7f0b48923800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d015df70 0x5633d015e390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.893+0000 7f0b48923800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5633d01256f0 con 0x5633d015df70 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.893+0000 7f0b44659640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d015df70 0x5633d015e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.893+0000 7f0b44659640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d015df70 0x5633d015e390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59872/0 (socket says 172.21.15.165:59872) 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.893+0000 7f0b44659640 1 -- 172.21.15.165:0/1805526512 learned_addr learned my addr 172.21.15.165:0/1805526512 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.894+0000 7f0b44659640 1 -- 172.21.15.165:0/1805526512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5633d015e8d0 con 0x5633d015df70 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.894+0000 7f0b44659640 1 --2- 172.21.15.165:0/1805526512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d015df70 0x5633d015e390 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0b3405ec00 tx=0x7f0b34069690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.894+0000 7f0a08fe1640 1 -- 172.21.15.165:0/1805526512 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b34073c20 con 0x5633d015df70 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.894+0000 7f0a08fe1640 1 -- 172.21.15.165:0/1805526512 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0b34092070 con 0x5633d015df70 2024-04-27T03:32:17.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.894+0000 7f0b48923800 1 -- 172.21.15.165:0/1805526512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5633d015f340 con 0x5633d015df70 2024-04-27T03:32:17.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.894+0000 7f0a08fe1640 1 -- 172.21.15.165:0/1805526512 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0b340725d0 con 0x5633d015df70 2024-04-27T03:32:17.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.894+0000 7f0b48923800 1 -- 172.21.15.165:0/1805526512 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5633d015f340 con 0x5633d015df70 2024-04-27T03:32:17.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.895+0000 7f0a08fe1640 1 -- 172.21.15.165:0/1805526512 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0b34072770 con 0x5633d015df70 2024-04-27T03:32:17.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.895+0000 7f0a08fe1640 1 --2- 172.21.15.165:0/1805526512 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f097c036940 0x7f097c038e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.895+0000 7f0a08fe1640 1 -- 172.21.15.165:0/1805526512 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0b34085070 con 0x5633d015df70 2024-04-27T03:32:17.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.895+0000 7f0b3bfff640 1 --2- 172.21.15.165:0/1805526512 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f097c036940 0x7f097c038e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.895+0000 7f098a7fc640 20 reqs_thread_entry: start 2024-04-27T03:32:17.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.895+0000 7f0b48923800 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5633d016b7e0 0x5633d016dbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.895+0000 7f0b3bfff640 1 --2- 172.21.15.165:0/1805526512 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f097c036940 0x7f097c038e00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c000f60 tx=0x7f0b2c006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.895+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5633d016e1b0 con 0x5633d016b7e0 2024-04-27T03:32:17.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.895+0000 7f0b44e5a640 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5633d016b7e0 0x5633d016dbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.896+0000 7f0b44e5a640 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5633d016b7e0 0x5633d016dbd0 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.896+0000 7f0b44e5a640 1 -- 172.21.15.165:0/1613898511 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0b3c00e070 con 0x5633d016b7e0 2024-04-27T03:32:17.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.896+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5633d016e730 con 0x5633d016b7e0 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.896+0000 7f0b44e5a640 1 -- 172.21.15.165:0/1613898511 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0b3c00e070 con 0x5633d016b7e0 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.896+0000 7f0b48923800 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5633d0170350 0x5633d01727e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.896+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:3 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5633d0172f70 con 0x5633d0170350 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.896+0000 7f0b44659640 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5633d0170350 0x5633d01727e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.897+0000 7f0b44659640 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5633d0170350 0x5633d01727e0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.897+0000 7f0b44659640 1 -- 172.21.15.165:0/1613898511 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(3 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0b34070020 con 0x5633d0170350 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/4078018245 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0a1c036900 msgr2=0x7f0a1c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/4078018245 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0a1c036900 0x7f0a1c038dc0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c000c00 tx=0x7f0b2c001330 comp rx=0 tx=0).stop 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/4078018245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 msgr2=0x5633d01438a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/4078018245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 0x5633d01438a0 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f0b3c013040 tx=0x7f0b3c003370 comp rx=0 tx=0).stop 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/4078018245 shutdown_connections 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/4078018245 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0a1c036900 0x7f0a1c038dc0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/4078018245 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d013c410 0x5633d01438a0 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/4078018245 >> 172.21.15.165:0/4078018245 conn(0x5633d0037290 msgr2=0x5633d01364f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/4078018245 shutdown_connections 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/4078018245 wait complete. 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/1805526512 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f097c036940 msgr2=0x7f097c038e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/1805526512 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f097c036940 0x7f097c038e00 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f0b2c000f60 tx=0x7f0b2c006000 comp rx=0 tx=0).stop 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/1805526512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d015df70 msgr2=0x5633d015e390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/1805526512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d015df70 0x5633d015e390 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f0b3405ec00 tx=0x7f0b34069690 comp rx=0 tx=0).stop 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/1805526512 shutdown_connections 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/1805526512 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f097c036940 0x7f097c038e00 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/1805526512 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d015df70 0x5633d015e390 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/1805526512 >> 172.21.15.165:0/1805526512 conn(0x5633d0152a40 msgr2=0x5633d0153460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/1805526512 shutdown_connections 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/1805526512 wait complete. 2024-04-27T03:32:17.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5633d016b7e0 msgr2=0x5633d016dbd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5633d016b7e0 0x5633d016dbd0 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5633d0170350 msgr2=0x5633d01727e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5633d0170350 0x5633d01727e0 crc :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0a2c036950 msgr2=0x7f0a2c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0a2c036950 0x7f0a2c038e10 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f0b34072510 tx=0x7f0b34080040 comp rx=0 tx=0).stop 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 msgr2=0x5633d00342a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d00342a0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f0b34061040 tx=0x7f0b34002fb0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 shutdown_connections 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5633d0170350 0x5633d01727e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5633d016b7e0 0x5633d016dbd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0a2c036950 0x7f0a2c038e10 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 --2- 172.21.15.165:0/1613898511 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5633d0034960 0x5633d00342a0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.898+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 >> 172.21.15.165:0/1613898511 conn(0x5633d002e310 msgr2=0x5633d011ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:17.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.899+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 shutdown_connections 2024-04-27T03:32:17.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:17.899+0000 7f0b48923800 1 -- 172.21.15.165:0/1613898511 wait complete. 2024-04-27T03:32:17.914 INFO:teuthology.orchestra.run.smithi165.stdout:{"id":"fd642a41-253a-48d4-9473-6eca68174247","name":"default","api_name":"default","is_master":true,"endpoints":[],"hostnames":[],"hostnames_s3website":[],"master_zone":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa","zones":[{"id":"708f831a-84ca-40fe-a0c7-f3b916a7f9aa","name":"default","endpoints":[],"log_meta":false,"log_data":false,"bucket_index_max_shards":11,"read_only":false,"tier_type":"","sync_from_all":true,"sync_from":[],"redirect_zone":"","supported_features":["notification_v2","resharding"]}],"placement_targets":[{"name":"default-placement","tags":[],"storage_classes":["STANDARD"]}],"default_placement":"default-placement","realm_id":"","sync_policy":{"groups":[]},"enabled_features":["notification_v2","resharding"]} 2024-04-27T03:32:17.915 DEBUG:tasks.util.rgw: json result: {'id': 'fd642a41-253a-48d4-9473-6eca68174247', 'name': 'default', 'api_name': 'default', 'is_master': True, 'endpoints': [], 'hostnames': [], 'hostnames_s3website': [], 'master_zone': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa', 'zones': [{'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa', 'name': 'default', 'endpoints': [], 'log_meta': False, 'log_data': False, 'bucket_index_max_shards': 11, 'read_only': False, 'tier_type': '', 'sync_from_all': True, 'sync_from': [], 'redirect_zone': '', 'supported_features': ['notification_v2', 'resharding']}], 'placement_targets': [{'name': 'default-placement', 'tags': [], 'storage_classes': ['STANDARD']}], 'default_placement': 'default-placement', 'realm_id': '', 'sync_policy': {'groups': []}, 'enabled_features': ['notification_v2', 'resharding']} 2024-04-27T03:32:17.917 INFO:teuthology.run_tasks:Running task radosgw-admin-rest... 2024-04-27T03:32:17.936 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-04-27T03:32:17.936 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'ada', '--display-name', 'Ms. Admin User', '--access-key', 'MH1WC2XQ1S8UISFDZC8W', '--secret', 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', '--max-buckets', '0', '--caps', 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write'] 2024-04-27T03:32:17.936 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid ada --display-name 'Ms. Admin User' --access-key MH1WC2XQ1S8UISFDZC8W --secret dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG --max-buckets 0 --caps 'users=read, write; usage=read, write; buckets=read, write; zone=read, write; info=read;ratelimit=read, write' 2024-04-27T03:32:18.008 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:18.009 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:18.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.004+0000 7f01d6b33800 1 Processor -- start 2024-04-27T03:32:18.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.004+0000 7f01d6b33800 1 -- start start 2024-04-27T03:32:18.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.004+0000 7f01d6b33800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55aa953fa150 0x55aa953fc540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.004+0000 7f01d6b33800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55aa952b5d40 con 0x55aa953f5cc0 2024-04-27T03:32:18.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.004+0000 7f01d6b33800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55aa952884d0 con 0x55aa953fa150 2024-04-27T03:32:18.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.004+0000 7f01c9fd0640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55aa953fa150 0x55aa953fc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.004+0000 7f01d27d1640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55aa953f5cc0 0x55aa953f60e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42330/0 (socket says 172.21.15.165:42330) 2024-04-27T03:32:18.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.004+0000 7f01d27d1640 1 -- 172.21.15.165:0/2792741557 learned_addr learned my addr 172.21.15.165:0/2792741557 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:18.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.004+0000 7f01c9fd0640 1 -- 172.21.15.165:0/2792741557 >> v1:172.21.15.89:6789/0 conn(0x55aa953f5cc0 legacy=0x55aa953f60e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.004+0000 7f01c9fd0640 1 -- 172.21.15.165:0/2792741557 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa952b4590 con 0x55aa953fa150 2024-04-27T03:32:18.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.004+0000 7f01d1fd0640 1 -- 172.21.15.165:0/2792741557 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1807939841 0 0) 0x55aa952b5d40 con 0x55aa953f5cc0 2024-04-27T03:32:18.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.006+0000 7f01c9fd0640 1 --2- 172.21.15.165:0/2792741557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953fa150 0x55aa953fc540 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00a470 tx=0x7f01bc01ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2248446663e49df2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.007+0000 7f01d1fd0640 1 -- 172.21.15.165:0/2792741557 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01bc057070 con 0x55aa953fa150 2024-04-27T03:32:18.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.007+0000 7f01d1fd0640 1 -- 172.21.15.165:0/2792741557 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01bc052570 con 0x55aa953fa150 2024-04-27T03:32:18.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.007+0000 7f01d1fd0640 1 -- 172.21.15.165:0/2792741557 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01bc052870 con 0x55aa953fa150 2024-04-27T03:32:18.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.007+0000 7f01d6b33800 1 -- 172.21.15.165:0/2792741557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953fa150 msgr2=0x55aa953fc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.007+0000 7f01d6b33800 1 --2- 172.21.15.165:0/2792741557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953fa150 0x55aa953fc540 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f01bc00a470 tx=0x7f01bc01ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:18.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.007+0000 7f01d6b33800 1 -- 172.21.15.165:0/2792741557 shutdown_connections 2024-04-27T03:32:18.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.007+0000 7f01d6b33800 1 --2- 172.21.15.165:0/2792741557 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953fa150 0x55aa953fc540 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.007+0000 7f01d6b33800 1 -- 172.21.15.165:0/2792741557 >> 172.21.15.165:0/2792741557 conn(0x55aa953f1320 msgr2=0x55aa953f3740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.007+0000 7f01d6b33800 1 -- 172.21.15.165:0/2792741557 shutdown_connections 2024-04-27T03:32:18.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.007+0000 7f01d6b33800 1 -- 172.21.15.165:0/2792741557 wait complete. 2024-04-27T03:32:18.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.019+0000 7f01d6b33800 1 Processor -- start 2024-04-27T03:32:18.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.019+0000 7f01d6b33800 1 -- start start 2024-04-27T03:32:18.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.019+0000 7f01d6b33800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 0x55aa95369f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.083 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.019+0000 7f01d6b33800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aa95342360 con 0x55aa953f5cc0 2024-04-27T03:32:18.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.019+0000 7f01d27d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 0x55aa95369f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.019+0000 7f01d27d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 0x55aa95369f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59892/0 (socket says 172.21.15.165:59892) 2024-04-27T03:32:18.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.019+0000 7f01d27d1640 1 -- 172.21.15.165:0/1541093949 learned_addr learned my addr 172.21.15.165:0/1541093949 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.020+0000 7f01d27d1640 1 -- 172.21.15.165:0/1541093949 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa954e8b50 con 0x55aa953f5cc0 2024-04-27T03:32:18.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.020+0000 7f01d27d1640 1 --2- 172.21.15.165:0/1541093949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 0x55aa95369f90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f01c400b240 tx=0x7f01c400b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5ea1dc1a2ea0a58 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.020+0000 7f00c0608640 1 -- 172.21.15.165:0/1541093949 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c4008070 con 0x55aa953f5cc0 2024-04-27T03:32:18.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.020+0000 7f00c0608640 1 -- 172.21.15.165:0/1541093949 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c400d500 con 0x55aa953f5cc0 2024-04-27T03:32:18.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.020+0000 7f00c0608640 1 -- 172.21.15.165:0/1541093949 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c400d860 con 0x55aa953f5cc0 2024-04-27T03:32:18.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d6b33800 1 -- 172.21.15.165:0/1541093949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 msgr2=0x55aa95369f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1541093949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 0x55aa95369f90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f01c400b240 tx=0x7f01c400b700 comp rx=0 tx=0).stop 2024-04-27T03:32:18.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d6b33800 1 -- 172.21.15.165:0/1541093949 shutdown_connections 2024-04-27T03:32:18.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1541093949 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 0x55aa95369f90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d6b33800 1 -- 172.21.15.165:0/1541093949 >> 172.21.15.165:0/1541093949 conn(0x55aa953f1320 msgr2=0x55aa954e2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d6b33800 1 -- 172.21.15.165:0/1541093949 shutdown_connections 2024-04-27T03:32:18.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d6b33800 1 -- 172.21.15.165:0/1541093949 wait complete. 2024-04-27T03:32:18.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d6b33800 1 Processor -- start 2024-04-27T03:32:18.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d6b33800 1 -- start start 2024-04-27T03:32:18.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d6b33800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 0x55aa954f1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d6b33800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aa953fce10 con 0x55aa953f5cc0 2024-04-27T03:32:18.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d27d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 0x55aa954f1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d27d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 0x55aa954f1dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59902/0 (socket says 172.21.15.165:59902) 2024-04-27T03:32:18.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.021+0000 7f01d27d1640 1 -- 172.21.15.165:0/1551658358 learned_addr learned my addr 172.21.15.165:0/1551658358 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.022+0000 7f01d27d1640 1 -- 172.21.15.165:0/1551658358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa954f2310 con 0x55aa953f5cc0 2024-04-27T03:32:18.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.022+0000 7f01d27d1640 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 0x55aa954f1dd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x55aa953695c0 tx=0x7f01c4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.022+0000 7f00bee05640 1 -- 172.21.15.165:0/1551658358 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c4019870 con 0x55aa953f5cc0 2024-04-27T03:32:18.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.022+0000 7f00bee05640 1 -- 172.21.15.165:0/1551658358 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c4019a10 con 0x55aa953f5cc0 2024-04-27T03:32:18.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.022+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aa954f25a0 con 0x55aa953f5cc0 2024-04-27T03:32:18.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.022+0000 7f00bee05640 1 -- 172.21.15.165:0/1551658358 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c4019d30 con 0x55aa953f5cc0 2024-04-27T03:32:18.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.023+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa954f5250 con 0x55aa953f5cc0 2024-04-27T03:32:18.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.023+0000 7f00bee05640 1 -- 172.21.15.165:0/1551658358 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f01c4017040 con 0x55aa953f5cc0 2024-04-27T03:32:18.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.023+0000 7f00bee05640 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f00b8036950 0x7f00b8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.023+0000 7f00bee05640 1 -- 172.21.15.165:0/1551658358 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f01c4015020 con 0x55aa953f5cc0 2024-04-27T03:32:18.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.023+0000 7f01c9fd0640 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f00b8036950 0x7f00b8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.024+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aa953fa150 0x55aa954f9010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.024+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa954f9590 con 0x55aa953fa150 2024-04-27T03:32:18.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.024+0000 7f01d2fd2640 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aa953fa150 0x55aa954f9010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.024+0000 7f01c9fd0640 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f00b8036950 0x7f00b8038e10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f01bc054000 tx=0x7f01bc04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.024+0000 7f01d2fd2640 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aa953fa150 0x55aa954f9010 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.024+0000 7f01d2fd2640 1 -- 172.21.15.165:0/1551658358 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa953fa150 2024-04-27T03:32:18.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.024+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa954f65e0 con 0x55aa953fa150 2024-04-27T03:32:18.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.025+0000 7f01d2fd2640 1 -- 172.21.15.165:0/1551658358 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa953fa150 2024-04-27T03:32:18.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.025+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aa954fb8c0 0x55aa954fdcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.025+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa954fe3a0 con 0x55aa954fb8c0 2024-04-27T03:32:18.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.025+0000 7f01d27d1640 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aa954fb8c0 0x55aa954fdcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.025+0000 7f01d27d1640 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aa954fb8c0 0x55aa954fdcb0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.025+0000 7f01d27d1640 1 -- 172.21.15.165:0/1551658358 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f01c402d070 con 0x55aa954fb8c0 2024-04-27T03:32:18.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.025+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa954fa840 con 0x55aa953fa150 2024-04-27T03:32:18.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.026+0000 7f01d2fd2640 1 -- 172.21.15.165:0/1551658358 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa953fa150 2024-04-27T03:32:18.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.026+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55aa954ff800 0x55aa95501c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.026+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa955023d0 con 0x55aa954ff800 2024-04-27T03:32:18.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.026+0000 7f01c9fd0640 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55aa954ff800 0x55aa95501c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.026+0000 7f01c9fd0640 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55aa954ff800 0x55aa95501c70 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.026+0000 7f01c9fd0640 1 -- 172.21.15.165:0/1551658358 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f01bc052940 con 0x55aa954ff800 2024-04-27T03:32:18.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.027+0000 7f01d6b33800 1 Processor -- start 2024-04-27T03:32:18.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.027+0000 7f01d6b33800 1 -- start start 2024-04-27T03:32:18.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.027+0000 7f01d6b33800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa95510830 0x55aa95510c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.027+0000 7f01d6b33800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aa952e86b0 con 0x55aa95510830 2024-04-27T03:32:18.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.027+0000 7f01d27d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa95510830 0x55aa95510c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.027+0000 7f01d27d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa95510830 0x55aa95510c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59914/0 (socket says 172.21.15.165:59914) 2024-04-27T03:32:18.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.027+0000 7f01d27d1640 1 -- 172.21.15.165:0/1575454538 learned_addr learned my addr 172.21.15.165:0/1575454538 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.027+0000 7f01d27d1640 1 -- 172.21.15.165:0/1575454538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa95511190 con 0x55aa95510830 2024-04-27T03:32:18.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.028+0000 7f01d27d1640 1 --2- 172.21.15.165:0/1575454538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa95510830 0x55aa95510c50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f01c404b3f0 tx=0x7f01c400f840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.028+0000 7f00b7fff640 1 -- 172.21.15.165:0/1575454538 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c401a8f0 con 0x55aa95510830 2024-04-27T03:32:18.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.028+0000 7f00b7fff640 1 -- 172.21.15.165:0/1575454538 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c4041e10 con 0x55aa95510830 2024-04-27T03:32:18.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.028+0000 7f00b7fff640 1 -- 172.21.15.165:0/1575454538 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c4044b80 con 0x55aa95510830 2024-04-27T03:32:18.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.028+0000 7f01d6b33800 1 -- 172.21.15.165:0/1575454538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aa95511c00 con 0x55aa95510830 2024-04-27T03:32:18.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.028+0000 7f01d6b33800 1 -- 172.21.15.165:0/1575454538 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa95512100 con 0x55aa95510830 2024-04-27T03:32:18.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.028+0000 7f00b7fff640 1 -- 172.21.15.165:0/1575454538 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f01c4044d20 con 0x55aa95510830 2024-04-27T03:32:18.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.029+0000 7f00b7fff640 1 --2- 172.21.15.165:0/1575454538 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f00ac036900 0x7f00ac038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.029+0000 7f00b7fff640 1 -- 172.21.15.165:0/1575454538 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f01c403bbf0 con 0x55aa95510830 2024-04-27T03:32:18.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.029+0000 7f01c9fd0640 1 --2- 172.21.15.165:0/1575454538 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f00ac036900 0x7f00ac038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.029+0000 7f01d6b33800 1 Processor -- start 2024-04-27T03:32:18.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.029+0000 7f01d6b33800 1 -- start start 2024-04-27T03:32:18.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.029+0000 7f01c9fd0640 1 --2- 172.21.15.165:0/1575454538 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f00ac036900 0x7f00ac038dc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55aa954f9e70 tx=0x7f01bc005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.029+0000 7f01d6b33800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 0x55aa9551df50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.029+0000 7f01d6b33800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aa954ed2e0 con 0x55aa9551db30 2024-04-27T03:32:18.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.030+0000 7f01d27d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 0x55aa9551df50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.030+0000 7f01d27d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 0x55aa9551df50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59926/0 (socket says 172.21.15.165:59926) 2024-04-27T03:32:18.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.030+0000 7f01d27d1640 1 -- 172.21.15.165:0/3269117047 learned_addr learned my addr 172.21.15.165:0/3269117047 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.030+0000 7f01d27d1640 1 -- 172.21.15.165:0/3269117047 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa9551e520 con 0x55aa9551db30 2024-04-27T03:32:18.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.030+0000 7f01d27d1640 1 --2- 172.21.15.165:0/3269117047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 0x55aa9551df50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f01c400e3b0 tx=0x7f01c4018940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adf8bfdf1419531a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.030+0000 7f00b5ffb640 1 -- 172.21.15.165:0/3269117047 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c4050070 con 0x55aa9551db30 2024-04-27T03:32:18.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.030+0000 7f00b5ffb640 1 -- 172.21.15.165:0/3269117047 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c405c070 con 0x55aa9551db30 2024-04-27T03:32:18.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.030+0000 7f00b5ffb640 1 -- 172.21.15.165:0/3269117047 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c403f050 con 0x55aa9551db30 2024-04-27T03:32:18.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.031+0000 7f01d6b33800 1 -- 172.21.15.165:0/3269117047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 msgr2=0x55aa9551df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.031+0000 7f01d6b33800 1 --2- 172.21.15.165:0/3269117047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 0x55aa9551df50 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7f01c400e3b0 tx=0x7f01c4018940 comp rx=0 tx=0).stop 2024-04-27T03:32:18.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.031+0000 7f01d6b33800 1 -- 172.21.15.165:0/3269117047 shutdown_connections 2024-04-27T03:32:18.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.031+0000 7f01d6b33800 1 --2- 172.21.15.165:0/3269117047 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 0x55aa9551df50 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.031+0000 7f01d6b33800 1 -- 172.21.15.165:0/3269117047 >> 172.21.15.165:0/3269117047 conn(0x55aa95519e90 msgr2=0x55aa95517ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.031+0000 7f01d6b33800 1 -- 172.21.15.165:0/3269117047 shutdown_connections 2024-04-27T03:32:18.097 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.031+0000 7f01d6b33800 1 -- 172.21.15.165:0/3269117047 wait complete. 2024-04-27T03:32:18.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.031+0000 7f01d6b33800 1 Processor -- start 2024-04-27T03:32:18.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.031+0000 7f01d6b33800 1 -- start start 2024-04-27T03:32:18.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.031+0000 7f01d6b33800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 0x55aa95524f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.031+0000 7f01d6b33800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55aa9551ebd0 con 0x55aa9551db30 2024-04-27T03:32:18.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.031+0000 7f01d27d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 0x55aa95524f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.032+0000 7f01d27d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 0x55aa95524f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59938/0 (socket says 172.21.15.165:59938) 2024-04-27T03:32:18.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.032+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 learned_addr learned my addr 172.21.15.165:0/3954986473 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.032+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55aa95525440 con 0x55aa9551db30 2024-04-27T03:32:18.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.032+0000 7f01d27d1640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 0x55aa95524f00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f01c4030fd0 tx=0x7f01c400e670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.098 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.032+0000 7f00a3fff640 1 -- 172.21.15.165:0/3954986473 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c406b020 con 0x55aa9551db30 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.032+0000 7f00a3fff640 1 -- 172.21.15.165:0/3954986473 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f01c4030a10 con 0x55aa9551db30 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.032+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55aa95525440 con 0x55aa9551db30 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.032+0000 7f00a3fff640 1 -- 172.21.15.165:0/3954986473 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f01c404b960 con 0x55aa9551db30 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.032+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55aa95525440 con 0x55aa9551db30 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.033+0000 7f00a3fff640 1 -- 172.21.15.165:0/3954986473 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f01c406e020 con 0x55aa9551db30 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.033+0000 7f00a3fff640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f009c036900 0x7f009c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.033+0000 7f00a3fff640 1 -- 172.21.15.165:0/3954986473 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f01c4071ba0 con 0x55aa9551db30 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.033+0000 7f01c9fd0640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f009c036900 0x7f009c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.033+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d6b33800 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aa95530570 0x55aa95532a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa95533180 con 0x55aa95530570 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01c9fd0640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f009c036900 0x7f009c038dc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f01bc052fd0 tx=0x7f01bc04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.099 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d2fd2640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aa95530570 0x55aa95532a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d2fd2640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aa95530570 0x55aa95532a30 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d6b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d6b33800 20 realm 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa95533590 con 0x55aa95530570 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d6b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d6b33800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.034+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa95533e60 con 0x55aa95530570 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.035+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.035+0000 7f01d6b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.100 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.035+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.035+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95530570 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.035+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.035+0000 7f01d6b33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.035+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.035+0000 7f01d6b33800 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aa95534f60 0x55aa95537350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.035+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa95537960 con 0x55aa95534f60 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.035+0000 7f01d27d1640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aa95534f60 0x55aa95537350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.035+0000 7f01d27d1640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aa95534f60 0x55aa95537350 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.035+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.036+0000 7f01d6b33800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.036+0000 7f01d6b33800 20 searching for the correct realm 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.036+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95539290 con 0x55aa95534f60 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.036+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.036+0000 7f01d27d1640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f01c4070020 0x7f01c402f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.101 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.036+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95539290 con 0x7f01c4070020 2024-04-27T03:32:18.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.036+0000 7f01c9fd0640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f01c4070020 0x7f01c402f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.036+0000 7f01c9fd0640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f01c4070020 0x7f01c402f720 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.037+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.037+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95539290 con 0x7f01c4070020 2024-04-27T03:32:18.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.037+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.037+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95539290 con 0x55aa95530570 2024-04-27T03:32:18.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.037+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.037+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95530570 2024-04-27T03:32:18.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.037+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.037+0000 7f01d2fd2640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f01cc016440 0x7f01cc0168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.037+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01cc016440 2024-04-27T03:32:18.102 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.038+0000 7f01d2fd2640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f01cc016440 0x7f01cc0168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.038+0000 7f01d2fd2640 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f01cc016440 0x7f01cc0168c0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.038+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.038+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01cc016440 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.039+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.039+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01c4070020 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.039+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.039+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95534f60 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.039+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.039+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95530570 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.039+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.039+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01c4070020 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.040+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.040+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95530570 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.040+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.103 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.040+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01c4070020 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.040+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.040+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95534f60 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.040+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.040+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95534f60 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.041+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.041+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01cc016440 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.041+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.041+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95534f60 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.041+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.041+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95534f60 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.041+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.041+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95530570 2024-04-27T03:32:18.104 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.042+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.042+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95530570 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.042+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.042+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01cc016440 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.042+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.042+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95534f60 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.042+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.042+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01c4070020 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.043+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.043+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95530570 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.043+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.043+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01cc016440 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.043+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.043+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95534f60 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.044+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.105 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.044+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01cc016440 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.044+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.044+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01c4070020 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.044+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.044+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95534f60 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.044+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.044+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01cc016440 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x55aa95530570 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55aa95534200 con 0x7f01c4070020 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d6b33800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:18.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d6b33800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d6b33800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d6b33800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d6b33800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d6b33800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa95538220 con 0x55aa95530570 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d6b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.045+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa95538610 con 0x55aa95530570 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa95538610 con 0x7f01c4070020 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 20 zone default found 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 4 Realm: () 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 20 zonegroup default 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa95539710 con 0x7f01cc016440 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.046+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553b0f0 con 0x55aa95534f60 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.047+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.047+0000 7f01d6b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.047+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.047+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553b510 con 0x55aa95534f60 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.047+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.047+0000 7f01d6b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.047+0000 7f01d6b33800 20 started sync module instance, tier type = 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.047+0000 7f01d6b33800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.047+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553a590 con 0x7f01c4070020 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.048+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.048+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94191138089344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553be00 con 0x7f01c4070020 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.048+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553c650 con 0x55aa95530570 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.051+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.051+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94191138095520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553cc10 con 0x55aa95530570 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.051+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553d730 con 0x55aa95534f60 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.052+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94191138089344] v27'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.053+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94191138095520] v27'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.053+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.053+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94191138099712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553c660 con 0x55aa95534f60 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.053+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553e140 con 0x7f01c4070020 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.054+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.054+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94191138104544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553edb0 con 0x7f01c4070020 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.054+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553f770 con 0x55aa95530570 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.054+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94191138099712] v27'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.055+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.055+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94191138104544] v27'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.055+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94191138105600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553f770 con 0x55aa95530570 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.055+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa95540050 con 0x7f01cc016440 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.056+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.056+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94191138110304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa95540000 con 0x7f01cc016440 2024-04-27T03:32:18.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.056+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa95541020 con 0x7f01cc016440 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.057+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94191138105600] v27'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.057+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94191138110304] v27'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.058+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.058+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94191138114320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa955403f0 con 0x7f01cc016440 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.058+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa95541600 con 0x7f01cc016440 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.059+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94191138114320] v27'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.059+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'240 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.059+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94191138108096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa95541600 con 0x7f01cc016440 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.059+0000 7f01d6b33800 20 add_watcher() i=0 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.059+0000 7f01d6b33800 20 add_watcher() i=1 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.059+0000 7f01d6b33800 20 add_watcher() i=2 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.059+0000 7f01d6b33800 20 add_watcher() i=3 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.059+0000 7f01d6b33800 20 add_watcher() i=4 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.059+0000 7f01d6b33800 20 add_watcher() i=5 2024-04-27T03:32:18.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.059+0000 7f01d6b33800 20 add_watcher() i=6 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.060+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94191138108096] v27'241 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.060+0000 7f01d6b33800 20 add_watcher() i=7 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.060+0000 7f01d6b33800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.060+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa95544850 con 0x55aa95530570 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.060+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.060+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94191138116288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa95541b90 con 0x55aa95530570 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.061+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94191138116288] v27'245 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.061+0000 7f0088fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.061+0000 7f01d6b33800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.061+0000 7f01d6b33800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.061+0000 7f01d6b33800 5 note: GC not initialized 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f00037fe640 20 reqs_thread_entry: start 2024-04-27T03:32:18.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f01d6b33800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f01d6b33800 20 Filter name: none 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f00017fa640 20 reqs_thread_entry: start 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f01d6b33800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa95577cc0 con 0x7f01cc016440 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 ada [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 231+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f01d6b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f01d6b33800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x0 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f01d6b33800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f01d6b33800 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : miss 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f01d6b33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.062+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:63 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa95576460 con 0x55aa95530570 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.063+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(63 MH1WC2XQ1S8UISFDZC8W [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 206+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.063+0000 7f01d6b33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.063+0000 7f01d6b33800 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x0 2024-04-27T03:32:18.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.063+0000 7f01d6b33800 10 adding default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-04-27T03:32:18.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.063+0000 7f01d6b33800 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-04-27T03:32:18.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.063+0000 7f01d6b33800 10 cache get: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W : hit (negative entry) 2024-04-27T03:32:18.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.063+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:64 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.set in=58b,writefull 0~473 in=473b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa95576e60 con 0x7f01cc016440 2024-04-27T03:32:18.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.064+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(64 ada [delete,create,call,writefull 0~473] v27'3 uv3 ondisk = 0) ==== 273+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.064+0000 7f01d6b33800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:18.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.064+0000 7f01d6b33800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:18.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.064+0000 7f01d6b33800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-27T03:32:18.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.064+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify cookie 94191138334848 in=661b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa9557b7c0 con 0x55aa95530570 2024-04-27T03:32:18.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.064+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== watch-notify(notify (1) cookie 94191138105600 notify 115964117037 ret 0) ==== 691+0+0 (crc 0 0 0) 0x55aa95577200 con 0x55aa95530570 2024-04-27T03:32:18.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.064+0000 7f00b5ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117037 cookie 94191138105600 notifier 4819 bl.length()=649 2024-04-27T03:32:18.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.064+0000 7f00b5ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.064+0000 7f00b5ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.064+0000 7f00b5ffb640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f0098003ea0 con 0x55aa95530570 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.064+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.4 [notify cookie 94191138334848 out=8b] v0'0 uv240 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.065+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(66 notify.4 [notify-ack] v0'0 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.065+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== watch-notify(notify_complete (2) cookie 94191138334848 notify 115964117037 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55aa955778e0 con 0x55aa95530570 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.065+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:67 8.18 8:1f190392:users.keys::MH1WC2XQ1S8UISFDZC8W:head [delete,create,writefull 0~7 in=7b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa95576800 con 0x55aa95530570 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.066+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(67 MH1WC2XQ1S8UISFDZC8W [delete,create,writefull 0~7] v27'20 uv20 ondisk = 0) ==== 248+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.066+0000 7f01d6b33800 10 cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.066+0000 7f01d6b33800 10 moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.066+0000 7f01d6b33800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.keys:MH1WC2XQ1S8UISFDZC8W, ofs0, ns] 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.066+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:68 7.15 7:a93a5511:::notify.2:head [notify cookie 94191138334720 in=189b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55aa9557b7a0 con 0x55aa95534f60 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.066+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== watch-notify(notify (1) cookie 94191138099712 notify 115964117000 ret 0) ==== 219+0+0 (crc 0 0 0) 0x55aa95512100 con 0x55aa95534f60 2024-04-27T03:32:18.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.066+0000 7f00b57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117000 cookie 94191138099712 notifier 4819 bl.length()=177 2024-04-27T03:32:18.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.066+0000 7f00b57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W info.flags=0x7 2024-04-27T03:32:18.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.066+0000 7f00b57fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+MH1WC2XQ1S8UISFDZC8W to cache LRU end 2024-04-27T03:32:18.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.066+0000 7f00b57fa640 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f0008003400 con 0x55aa95534f60 2024-04-27T03:32:18.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.066+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(68 notify.2 [notify cookie 94191138334720 out=8b] v0'0 uv240 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.066+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(69 notify.2 [notify-ack] v0'0 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.116 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.067+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== watch-notify(notify_complete (2) cookie 94191138334720 notify 115964117000 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55aa95511c00 con 0x55aa95534f60 2024-04-27T03:32:18.116 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-27T03:32:18.064161Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:32:18.064141Z","tags":[],"group_ids":[]} 2024-04-27T03:32:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.067+0000 7f0088fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.067+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94191138116288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa95543fc0 con 0x55aa95530570 2024-04-27T03:32:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.069+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94191138116288] v27'246 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.069+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94191138089344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa955412c0 con 0x7f01c4070020 2024-04-27T03:32:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.070+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94191138089344] v27'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.070+0000 7f01d6b33800 20 remove_watcher() i=0 2024-04-27T03:32:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.071+0000 7f01d6b33800 2 removed watcher, disabling cache 2024-04-27T03:32:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.071+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94191138095520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa955412c0 con 0x55aa95530570 2024-04-27T03:32:18.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.072+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94191138095520] v27'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.072+0000 7f01d6b33800 20 remove_watcher() i=1 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.072+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94191138099712] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553c1a0 con 0x55aa95534f60 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.073+0000 7f01d27d1640 1 -- 172.21.15.165:0/3954986473 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94191138099712] v27'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01c4053040 con 0x55aa95534f60 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.073+0000 7f01d6b33800 20 remove_watcher() i=2 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.073+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94191138104544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa955412c0 con 0x7f01c4070020 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.074+0000 7f01c9fd0640 1 -- 172.21.15.165:0/3954986473 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94191138104544] v27'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01bc081070 con 0x7f01c4070020 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.074+0000 7f01d6b33800 20 remove_watcher() i=3 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.074+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94191138105600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553c1a0 con 0x55aa95530570 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.075+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94191138105600] v27'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x55aa95530570 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.075+0000 7f01d6b33800 20 remove_watcher() i=4 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.075+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94191138110304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa955412c0 con 0x7f01cc016440 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.076+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94191138110304] v27'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.076+0000 7f01d6b33800 20 remove_watcher() i=5 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.076+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94191138114320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553c1a0 con 0x7f01cc016440 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.077+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94191138114320] v27'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.077+0000 7f01d6b33800 20 remove_watcher() i=6 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.078+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94191138108096] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55aa9553e990 con 0x7f01cc016440 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94191138108096] v27'242 uv240 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f01cc00a4a0 con 0x7f01cc016440 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 20 remove_watcher() i=7 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f01cc016440 msgr2=0x7f01cc0168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f01cc016440 0x7f01cc0168c0 crc :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aa95530570 msgr2=0x55aa95532a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aa95530570 0x55aa95532a30 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f01c4070020 msgr2=0x7f01c402f720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f01c4070020 0x7f01c402f720 crc :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aa95534f60 msgr2=0x55aa95537350 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aa95534f60 0x55aa95537350 crc :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f009c036900 msgr2=0x7f009c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f009c036900 0x7f009c038dc0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7f01bc052fd0 tx=0x7f01bc04f000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d2fd2640 1 -- 172.21.15.165:0/3954986473 reap_dead start 2024-04-27T03:32:18.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 msgr2=0x55aa95524f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 0x55aa95524f00 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7f01c4030fd0 tx=0x7f01c400e670 comp rx=0 tx=0).stop 2024-04-27T03:32:18.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 shutdown_connections 2024-04-27T03:32:18.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 --2- 172.21.15.165:0/3954986473 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa9551db30 0x55aa95524f00 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 >> 172.21.15.165:0/3954986473 conn(0x55aa95519e90 msgr2=0x55aa955174f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 shutdown_connections 2024-04-27T03:32:18.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/3954986473 wait complete. 2024-04-27T03:32:18.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/1575454538 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f00ac036900 msgr2=0x7f00ac038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1575454538 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f00ac036900 0x7f00ac038dc0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x55aa954f9e70 tx=0x7f01bc005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/1575454538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa95510830 msgr2=0x55aa95510c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1575454538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa95510830 0x55aa95510c50 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7f01c404b3f0 tx=0x7f01c400f840 comp rx=0 tx=0).stop 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/1575454538 shutdown_connections 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1575454538 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f00ac036900 0x7f00ac038dc0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1575454538 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa95510830 0x55aa95510c50 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/1575454538 >> 172.21.15.165:0/1575454538 conn(0x55aa955048d0 msgr2=0x55aa95504cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/1575454538 shutdown_connections 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.079+0000 7f01d6b33800 1 -- 172.21.15.165:0/1575454538 wait complete. 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aa953fa150 msgr2=0x55aa954f9010 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55aa953fa150 0x55aa954f9010 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55aa954ff800 msgr2=0x55aa95501c70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.126 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55aa954ff800 0x55aa95501c70 crc :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aa954fb8c0 msgr2=0x55aa954fdcb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55aa954fb8c0 0x55aa954fdcb0 crc :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f00b8036950 msgr2=0x7f00b8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f00b8036950 0x7f00b8038e10 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f01bc054000 tx=0x7f01bc04e000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 msgr2=0x55aa954f1dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 --2- 172.21.15.165:0/1551658358 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55aa953f5cc0 0x55aa954f1dd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x55aa953695c0 tx=0x7f01c4000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d2fd2640 1 -- 172.21.15.165:0/1551658358 reap_dead start 2024-04-27T03:32:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 shutdown_connections 2024-04-27T03:32:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 >> 172.21.15.165:0/1551658358 conn(0x55aa953f1320 msgr2=0x55aa954e1110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 shutdown_connections 2024-04-27T03:32:18.127 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.080+0000 7f01d6b33800 1 -- 172.21.15.165:0/1551658358 wait complete. 2024-04-27T03:32:18.127 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:18.128 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:18.128 ERROR:root:0 2024-04-27T03:32:18.128 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:18.134 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx000002da86f6de3b631ed-00662c71c2-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:32:18.135 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'foo', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '4'} 2024-04-27T03:32:18.145 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:18.146 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 0} 2024-04-27T03:32:18.158 INFO:tasks.radosgw_admin_rest: json result: {'keys': [], 'truncated': True, 'count': 0, 'marker': '8:1e6ca5ed:users.uid::foo:head'} 2024-04-27T03:32:18.158 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1} 2024-04-27T03:32:18.174 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['foo'], 'truncated': True, 'count': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-04-27T03:32:18.174 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'list'] {'list': '', 'max-entries': 1, 'marker': '8:a00000ee:users.uid::bar.buckets:head'} 2024-04-27T03:32:18.182 INFO:tasks.radosgw_admin_rest: json result: {'keys': ['ada'], 'truncated': False, 'count': 1} 2024-04-27T03:32:18.182 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:18.185 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:18.186 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'access-key': '9te6NH5mcdcq0Tc5i8i1', 'uid': 'uid_not_exist'} 2024-04-27T03:32:18.190 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:18.190 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-27T03:32:18.190 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-27T03:32:18.190 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid ada --caps 'users=read, write' 2024-04-27T03:32:18.236 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:18.237 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:18.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.232+0000 7f96bab75800 1 Processor -- start 2024-04-27T03:32:18.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.232+0000 7f96bab75800 1 -- start start 2024-04-27T03:32:18.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.232+0000 7f96bab75800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ddbef7a300 0x55ddbef82e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.232+0000 7f96bab75800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55ddbee38d40 con 0x55ddbef787c0 2024-04-27T03:32:18.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.232+0000 7f96bab75800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55ddbee0b4d0 con 0x55ddbef7a300 2024-04-27T03:32:18.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.232+0000 7f96b7e73640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55ddbef7a300 0x55ddbef82e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.232+0000 7f96b8674640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55ddbef787c0 0x55ddbef78be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42332/0 (socket says 172.21.15.165:42332) 2024-04-27T03:32:18.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.232+0000 7f96b8674640 1 -- 172.21.15.165:0/3681263061 learned_addr learned my addr 172.21.15.165:0/3681263061 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:18.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.232+0000 7f96b7e73640 1 -- 172.21.15.165:0/3681263061 >> v1:172.21.15.89:6789/0 conn(0x55ddbef787c0 legacy=0x55ddbef78be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.232+0000 7f96b7672640 1 -- 172.21.15.165:0/3681263061 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3235599610 0 0) 0x55ddbee38d40 con 0x55ddbef787c0 2024-04-27T03:32:18.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.232+0000 7f96b7e73640 1 -- 172.21.15.165:0/3681263061 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbee37590 con 0x55ddbef7a300 2024-04-27T03:32:18.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.234+0000 7f96b7e73640 1 --2- 172.21.15.165:0/3681263061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef7a300 0x55ddbef82e00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f96a000a470 tx=0x7f96a001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8e0f8aa9049eb9d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.234+0000 7f96b7672640 1 -- 172.21.15.165:0/3681263061 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a0057070 con 0x55ddbef7a300 2024-04-27T03:32:18.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.234+0000 7f96b7672640 1 -- 172.21.15.165:0/3681263061 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a0052440 con 0x55ddbef7a300 2024-04-27T03:32:18.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.235+0000 7f96b7672640 1 -- 172.21.15.165:0/3681263061 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a0052740 con 0x55ddbef7a300 2024-04-27T03:32:18.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.235+0000 7f96bab75800 1 -- 172.21.15.165:0/3681263061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef7a300 msgr2=0x55ddbef82e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.235+0000 7f96bab75800 1 --2- 172.21.15.165:0/3681263061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef7a300 0x55ddbef82e00 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f96a000a470 tx=0x7f96a001ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:18.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.235+0000 7f96bab75800 1 -- 172.21.15.165:0/3681263061 shutdown_connections 2024-04-27T03:32:18.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.235+0000 7f96bab75800 1 --2- 172.21.15.165:0/3681263061 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef7a300 0x55ddbef82e00 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.235+0000 7f96bab75800 1 -- 172.21.15.165:0/3681263061 >> 172.21.15.165:0/3681263061 conn(0x55ddbef74340 msgr2=0x55ddbef76760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.235+0000 7f96bab75800 1 -- 172.21.15.165:0/3681263061 shutdown_connections 2024-04-27T03:32:18.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.235+0000 7f96bab75800 1 -- 172.21.15.165:0/3681263061 wait complete. 2024-04-27T03:32:18.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.247+0000 7f96bab75800 1 Processor -- start 2024-04-27T03:32:18.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.247+0000 7f96bab75800 1 -- start start 2024-04-27T03:32:18.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.247+0000 7f96bab75800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 0x55ddbf06d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.247+0000 7f96bab75800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ddbeec53b0 con 0x55ddbef787c0 2024-04-27T03:32:18.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.247+0000 7f96b8674640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 0x55ddbf06d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.247+0000 7f96b8674640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 0x55ddbf06d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59958/0 (socket says 172.21.15.165:59958) 2024-04-27T03:32:18.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.247+0000 7f96b8674640 1 -- 172.21.15.165:0/3775163137 learned_addr learned my addr 172.21.15.165:0/3775163137 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.248+0000 7f96b8674640 1 -- 172.21.15.165:0/3775163137 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbf06d8f0 con 0x55ddbef787c0 2024-04-27T03:32:18.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.248+0000 7f96b8674640 1 --2- 172.21.15.165:0/3775163137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 0x55ddbf06d320 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f96a8001630 tx=0x7f96a8001af0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f159699684e452b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.248+0000 7f95a660c640 1 -- 172.21.15.165:0/3775163137 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8003450 con 0x55ddbef787c0 2024-04-27T03:32:18.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.248+0000 7f95a660c640 1 -- 172.21.15.165:0/3775163137 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a80035f0 con 0x55ddbef787c0 2024-04-27T03:32:18.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.248+0000 7f95a660c640 1 -- 172.21.15.165:0/3775163137 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8003990 con 0x55ddbef787c0 2024-04-27T03:32:18.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96bab75800 1 -- 172.21.15.165:0/3775163137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 msgr2=0x55ddbf06d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.258 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96bab75800 1 --2- 172.21.15.165:0/3775163137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 0x55ddbf06d320 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f96a8001630 tx=0x7f96a8001af0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96bab75800 1 -- 172.21.15.165:0/3775163137 shutdown_connections 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96bab75800 1 --2- 172.21.15.165:0/3775163137 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 0x55ddbf06d320 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96bab75800 1 -- 172.21.15.165:0/3775163137 >> 172.21.15.165:0/3775163137 conn(0x55ddbef74340 msgr2=0x55ddbf068c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96bab75800 1 -- 172.21.15.165:0/3775163137 shutdown_connections 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96bab75800 1 -- 172.21.15.165:0/3775163137 wait complete. 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96bab75800 1 Processor -- start 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96bab75800 1 -- start start 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96bab75800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 0x55ddbf074ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96bab75800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ddbef85620 con 0x55ddbef787c0 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96b8674640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 0x55ddbf074ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96b8674640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 0x55ddbf074ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59960/0 (socket says 172.21.15.165:59960) 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.249+0000 7f96b8674640 1 -- 172.21.15.165:0/391830108 learned_addr learned my addr 172.21.15.165:0/391830108 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.259 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.250+0000 7f96b8674640 1 -- 172.21.15.165:0/391830108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbf075430 con 0x55ddbef787c0 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.250+0000 7f96b8674640 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 0x55ddbf074ef0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f96a8006fa0 tx=0x7f96a8001250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.250+0000 7f95a4e09640 1 -- 172.21.15.165:0/391830108 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8013820 con 0x55ddbef787c0 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.250+0000 7f95a4e09640 1 -- 172.21.15.165:0/391830108 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a8023070 con 0x55ddbef787c0 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.250+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ddbf0756c0 con 0x55ddbef787c0 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.250+0000 7f95a4e09640 1 -- 172.21.15.165:0/391830108 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8013af0 con 0x55ddbef787c0 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.250+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddbf078370 con 0x55ddbef787c0 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.251+0000 7f95a4e09640 1 -- 172.21.15.165:0/391830108 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f96a8010070 con 0x55ddbef787c0 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.251+0000 7f95a4e09640 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f95a0036950 0x7f95a0038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.251+0000 7f95a4e09640 1 -- 172.21.15.165:0/391830108 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f96a8018070 con 0x55ddbef787c0 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.251+0000 7f96b7e73640 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f95a0036950 0x7f95a0038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.251+0000 7f96bab75800 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ddbef7a300 0x55ddbf07c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.251+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf07c670 con 0x55ddbef7a300 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.251+0000 7f96b8e75640 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ddbef7a300 0x55ddbf07c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.260 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.252+0000 7f96b7e73640 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f95a0036950 0x7f95a0038e10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f96a0054000 tx=0x7f96a004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.252+0000 7f96b8e75640 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ddbef7a300 0x55ddbf07c000 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.252+0000 7f96b8e75640 1 -- 172.21.15.165:0/391830108 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbef7a300 2024-04-27T03:32:18.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.252+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0796e0 con 0x55ddbef7a300 2024-04-27T03:32:18.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.252+0000 7f96b8e75640 1 -- 172.21.15.165:0/391830108 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbef7a300 2024-04-27T03:32:18.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.252+0000 7f96bab75800 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ddbf07e9a0 0x55ddbf080d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.252+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf081480 con 0x55ddbf07e9a0 2024-04-27T03:32:18.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.252+0000 7f96b8674640 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ddbf07e9a0 0x55ddbf080d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.253+0000 7f96b8674640 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ddbf07e9a0 0x55ddbf080d90 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.253+0000 7f96b8674640 1 -- 172.21.15.165:0/391830108 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f96a801b420 con 0x55ddbf07e9a0 2024-04-27T03:32:18.261 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.253+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf07d950 con 0x55ddbef7a300 2024-04-27T03:32:18.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.253+0000 7f96b8e75640 1 -- 172.21.15.165:0/391830108 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbef7a300 2024-04-27T03:32:18.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.253+0000 7f96bab75800 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55ddbf082930 0x55ddbf084d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.253+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0854f0 con 0x55ddbf082930 2024-04-27T03:32:18.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.253+0000 7f96b7e73640 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55ddbf082930 0x55ddbf084d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.254+0000 7f96b7e73640 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55ddbf082930 0x55ddbf084d60 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.254+0000 7f96b7e73640 1 -- 172.21.15.165:0/391830108 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f96a00527b0 con 0x55ddbf082930 2024-04-27T03:32:18.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.254+0000 7f96bab75800 1 Processor -- start 2024-04-27T03:32:18.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.254+0000 7f96bab75800 1 -- start start 2024-04-27T03:32:18.273 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.255+0000 7f96bab75800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf093950 0x55ddbf093d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.255+0000 7f96bab75800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ddbee6b6b0 con 0x55ddbf093950 2024-04-27T03:32:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.255+0000 7f96b8674640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf093950 0x55ddbf093d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.255+0000 7f96b8674640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf093950 0x55ddbf093d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59966/0 (socket says 172.21.15.165:59966) 2024-04-27T03:32:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.255+0000 7f96b8674640 1 -- 172.21.15.165:0/430799764 learned_addr learned my addr 172.21.15.165:0/430799764 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.255+0000 7f96b8674640 1 -- 172.21.15.165:0/430799764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbf0942b0 con 0x55ddbf093950 2024-04-27T03:32:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.255+0000 7f96b8674640 1 --2- 172.21.15.165:0/430799764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf093950 0x55ddbf093d70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f96a8001b30 tx=0x7f96a8010e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.255+0000 7f959dffb640 1 -- 172.21.15.165:0/430799764 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a80490e0 con 0x55ddbf093950 2024-04-27T03:32:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.256+0000 7f959dffb640 1 -- 172.21.15.165:0/430799764 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a8049280 con 0x55ddbf093950 2024-04-27T03:32:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.256+0000 7f959dffb640 1 -- 172.21.15.165:0/430799764 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8049430 con 0x55ddbf093950 2024-04-27T03:32:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.256+0000 7f96bab75800 1 -- 172.21.15.165:0/430799764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ddbf094d20 con 0x55ddbf093950 2024-04-27T03:32:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.256+0000 7f96bab75800 1 -- 172.21.15.165:0/430799764 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddbf095220 con 0x55ddbf093950 2024-04-27T03:32:18.274 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.256+0000 7f959dffb640 1 -- 172.21.15.165:0/430799764 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f96a8028460 con 0x55ddbf093950 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.256+0000 7f959dffb640 1 --2- 172.21.15.165:0/430799764 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9590036900 0x7f9590038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.257+0000 7f959dffb640 1 -- 172.21.15.165:0/430799764 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f96a804dd00 con 0x55ddbf093950 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.257+0000 7f96b7e73640 1 --2- 172.21.15.165:0/430799764 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9590036900 0x7f9590038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.257+0000 7f96bab75800 1 Processor -- start 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.257+0000 7f96bab75800 1 -- start start 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.257+0000 7f96b7e73640 1 --2- 172.21.15.165:0/430799764 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9590036900 0x7f9590038dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55ddbf07cb40 tx=0x7f96a0005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.257+0000 7f96bab75800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 0x55ddbf0a1080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.257+0000 7f96bab75800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ddbf06dc70 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.257+0000 7f96b8674640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 0x55ddbf0a1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.257+0000 7f96b8674640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 0x55ddbf0a1080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59980/0 (socket says 172.21.15.165:59980) 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.257+0000 7f96b8674640 1 -- 172.21.15.165:0/3556007625 learned_addr learned my addr 172.21.15.165:0/3556007625 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.257+0000 7f96b8674640 1 -- 172.21.15.165:0/3556007625 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbf0a1650 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.258+0000 7f96b8674640 1 --2- 172.21.15.165:0/3556007625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 0x55ddbf0a1080 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f96a8000f80 tx=0x7f96a804efe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45cfb9a37a496f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.275 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.258+0000 7f958f7fe640 1 -- 172.21.15.165:0/3556007625 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a805b020 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.258+0000 7f958f7fe640 1 -- 172.21.15.165:0/3556007625 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a80273c0 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.258+0000 7f958f7fe640 1 -- 172.21.15.165:0/3556007625 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8033470 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.258+0000 7f96bab75800 1 -- 172.21.15.165:0/3556007625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 msgr2=0x55ddbf0a1080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.258+0000 7f96bab75800 1 --2- 172.21.15.165:0/3556007625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 0x55ddbf0a1080 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f96a8000f80 tx=0x7f96a804efe0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.258+0000 7f96bab75800 1 -- 172.21.15.165:0/3556007625 shutdown_connections 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.258+0000 7f96bab75800 1 --2- 172.21.15.165:0/3556007625 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 0x55ddbf0a1080 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.258+0000 7f96bab75800 1 -- 172.21.15.165:0/3556007625 >> 172.21.15.165:0/3556007625 conn(0x55ddbf09cf70 msgr2=0x55ddbf09afd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.258+0000 7f96bab75800 1 -- 172.21.15.165:0/3556007625 shutdown_connections 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.258+0000 7f96bab75800 1 -- 172.21.15.165:0/3556007625 wait complete. 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.259+0000 7f96bab75800 1 Processor -- start 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.259+0000 7f96bab75800 1 -- start start 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.259+0000 7f96bab75800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 0x55ddbf0a7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.259+0000 7f96bab75800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55ddbf0a1d60 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.276 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.259+0000 7f96b8674640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 0x55ddbf0a7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.259+0000 7f96b8674640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 0x55ddbf0a7ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:59984/0 (socket says 172.21.15.165:59984) 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.259+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 learned_addr learned my addr 172.21.15.165:0/4186098321 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.259+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ddbf0a8530 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.259+0000 7f96b8674640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 0x55ddbf0a7ff0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f96a80496e0 tx=0x7f96a801df70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.260+0000 7f958dffb640 1 -- 172.21.15.165:0/4186098321 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a8070070 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.260+0000 7f958dffb640 1 -- 172.21.15.165:0/4186098321 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96a801bda0 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.260+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55ddbf0a8530 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.260+0000 7f958dffb640 1 -- 172.21.15.165:0/4186098321 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96a804e0c0 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.260+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ddbf0a8530 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.260+0000 7f958dffb640 1 -- 172.21.15.165:0/4186098321 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f96a8073070 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.261+0000 7f958dffb640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9580036900 0x7f9580038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.261+0000 7f958dffb640 1 -- 172.21.15.165:0/4186098321 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f96a8067030 con 0x55ddbf0a0ca0 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.261+0000 7f96b7e73640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9580036900 0x7f9580038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.277 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.261+0000 7f96bab75800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.261+0000 7f96bab75800 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ddbf0b3730 0x55ddbf0b5bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.261+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b6340 con 0x55ddbf0b3730 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.261+0000 7f96b7e73640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9580036900 0x7f9580038dc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55ddbf0858f0 tx=0x7f96a004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.261+0000 7f96b8e75640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ddbf0b3730 0x55ddbf0b5bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.261+0000 7f96b8e75640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ddbf0b3730 0x55ddbf0b5bf0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 20 realm 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b6750 con 0x55ddbf0b3730 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.278 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b3730 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b73c0 con 0x55ddbf0b3730 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ddbf0b7d80 0x55ddbf0ba170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0ba7c0 con 0x55ddbf0b7d80 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.262+0000 7f96b8674640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ddbf0b7d80 0x55ddbf0ba170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.279 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.263+0000 7f96b8674640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ddbf0b7d80 0x55ddbf0ba170 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.263+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.263+0000 7f96bab75800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.263+0000 7f96bab75800 20 searching for the correct realm 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.263+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bc0d0 con 0x55ddbf0b7d80 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.264+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.264+0000 7f96b8674640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f96a8075060 0x7f96a80501b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.264+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bc0d0 con 0x7f96a8075060 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.264+0000 7f96b7e73640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f96a8075060 0x7f96a80501b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.264+0000 7f96b7e73640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f96a8075060 0x7f96a80501b0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.264+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.264+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bc0d0 con 0x7f96a8075060 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.264+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.280 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.264+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bc0d0 con 0x55ddbf0b3730 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.265+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.265+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b3730 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.265+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.265+0000 7f96b8e75640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f96b0018410 0x7f96b0018890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.265+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96b0018410 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.265+0000 7f96b8e75640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f96b0018410 0x7f96b0018890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.265+0000 7f96b8e75640 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f96b0018410 0x7f96b0018890 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.266+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.266+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96b0018410 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.266+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.266+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96a8075060 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.266+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.266+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b7d80 2024-04-27T03:32:18.281 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.267+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.267+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b3730 2024-04-27T03:32:18.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.267+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.267+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96a8075060 2024-04-27T03:32:18.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.267+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.267+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b3730 2024-04-27T03:32:18.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.267+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.267+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96a8075060 2024-04-27T03:32:18.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.268+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.282 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.268+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b7d80 2024-04-27T03:32:18.293 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.268+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.268+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b7d80 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.268+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.268+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96b0018410 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.268+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.268+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b7d80 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.269+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.269+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b7d80 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.269+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.269+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b3730 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.269+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.269+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b3730 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.269+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.294 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.269+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96b0018410 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.270+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.270+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b7d80 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.270+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.270+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96a8075060 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.270+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.270+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b3730 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.270+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.270+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96b0018410 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.271+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.271+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b7d80 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.271+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.271+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96b0018410 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.271+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.295 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.271+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96a8075060 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.271+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.271+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b7d80 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.272+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.272+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96b0018410 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.272+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.272+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x55ddbf0b3730 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0b7020 con 0x7f96a8075060 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:18.296 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bab60 con 0x55ddbf0b3730 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0baf50 con 0x55ddbf0b3730 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.273+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0baf50 con 0x7f96a8075060 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:18.297 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 20 zone default found 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 4 Realm: () 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 20 zonegroup default 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bc550 con 0x7f96b0018410 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bdf30 con 0x55ddbf0b7d80 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.298 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.274+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0be390 con 0x55ddbf0b7d80 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.275+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.275+0000 7f96bab75800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.275+0000 7f96bab75800 20 started sync module instance, tier type = 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.275+0000 7f96bab75800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.275+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bd3d0 con 0x7f96a8075060 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.276+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.276+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94410881357760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bd030 con 0x7f96a8075060 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.276+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bf2a0 con 0x55ddbf0b3730 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.277+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94410881357760] v27'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.277+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.277+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94410881363440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bf6c0 con 0x55ddbf0b3730 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.277+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c0080 con 0x55ddbf0b7d80 2024-04-27T03:32:18.299 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.278+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94410881363440] v27'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.279+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.279+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94410881366992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c0420 con 0x55ddbf0b7d80 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.279+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c0e70 con 0x7f96a8075060 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.280+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94410881366992] v27'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.280+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.280+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94410881370464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c1290 con 0x7f96a8075060 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.280+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c1c90 con 0x55ddbf0b3730 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.281+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94410881370464] v27'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.281+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.281+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94410881364496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c2080 con 0x55ddbf0b3730 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.281+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c2ac0 con 0x7f96b0018410 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.282+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94410881364496] v27'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.282+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.300 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.282+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94410881377680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c2a80 con 0x7f96b0018410 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.282+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c34f0 con 0x7f96b0018410 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.283+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94410881377680] v27'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.283+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.283+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94410881380384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c34f0 con 0x7f96b0018410 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.284+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c3f30 con 0x7f96b0018410 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.284+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94410881380384] v27'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.284+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'243 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.285+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94410881374112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c3f30 con 0x7f96b0018410 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.285+0000 7f96bab75800 20 add_watcher() i=0 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.285+0000 7f96bab75800 20 add_watcher() i=1 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.285+0000 7f96bab75800 20 add_watcher() i=2 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.285+0000 7f96bab75800 20 add_watcher() i=3 2024-04-27T03:32:18.301 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.285+0000 7f96bab75800 20 add_watcher() i=4 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.285+0000 7f96bab75800 20 add_watcher() i=5 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.285+0000 7f96bab75800 20 add_watcher() i=6 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.285+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94410881374112] v27'244 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.286+0000 7f96bab75800 20 add_watcher() i=7 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.286+0000 7f96bab75800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.286+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c7330 con 0x55ddbf0b3730 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.286+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.286+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94410881383504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c4850 con 0x55ddbf0b3730 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.287+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94410881383504] v27'247 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.287+0000 7f956e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.287+0000 7f96bab75800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.287+0000 7f96bab75800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.287+0000 7f96bab75800 5 note: GC not initialized 2024-04-27T03:32:18.302 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.288+0000 7f956d7e2640 20 reqs_thread_entry: start 2024-04-27T03:32:18.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.288+0000 7f96bab75800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:18.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.288+0000 7f96bab75800 20 Filter name: none 2024-04-27T03:32:18.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.288+0000 7f94deffd640 20 reqs_thread_entry: start 2024-04-27T03:32:18.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.288+0000 7f96bab75800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-27T03:32:18.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.288+0000 7f96bab75800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.303 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.288+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0fa660 con 0x7f96b0018410 2024-04-27T03:32:18.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.289+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 ada [call out=48b,read 0~473 out=473b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 231+0+596 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.289+0000 7f96bab75800 20 rados_obj.operate() r=0 bl.length=473 2024-04-27T03:32:18.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.289+0000 7f96bab75800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-27T03:32:18.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.289+0000 7f96bab75800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:18.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.289+0000 7f96bab75800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:18.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.289+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~460 in=460b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf100f60 con 0x7f96b0018410 2024-04-27T03:32:18.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.290+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~460] v27'5 uv5 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.290+0000 7f96bab75800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:18.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.290+0000 7f96bab75800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:18.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.290+0000 7f96bab75800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-27T03:32:18.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.290+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94410881632720 in=648b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0fc670 con 0x55ddbf0b3730 2024-04-27T03:32:18.305 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.290+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== watch-notify(notify (1) cookie 94410881364496 notify 115964117039 ret 0) ==== 678+0+0 (crc 0 0 0) 0x7f96b0010040 con 0x55ddbf0b3730 2024-04-27T03:32:18.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.290+0000 7f958f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117039 cookie 94410881364496 notifier 4826 bl.length()=636 2024-04-27T03:32:18.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.290+0000 7f958f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:18.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.290+0000 7f958f7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:18.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.290+0000 7f958f7fe640 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f957c003ea0 con 0x55ddbf0b3730 2024-04-27T03:32:18.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.290+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 notify.4 [notify cookie 94410881632720 out=8b] v0'0 uv243 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.290+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.306 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.291+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify_complete (2) cookie 94410881632720 notify 115964117039 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f96b0007740 con 0x55ddbf0b3730 2024-04-27T03:32:18.306 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-27T03:32:18.064161Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:32:18.064141Z","tags":[],"group_ids":[]} 2024-04-27T03:32:18.311 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.291+0000 7f956e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:18.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.291+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94410881383504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0fca80 con 0x55ddbf0b3730 2024-04-27T03:32:18.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.292+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94410881383504] v27'248 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.292+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94410881357760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0c3c20 con 0x7f96a8075060 2024-04-27T03:32:18.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.293+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94410881357760] v27'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.294+0000 7f96bab75800 20 remove_watcher() i=0 2024-04-27T03:32:18.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.294+0000 7f96bab75800 2 removed watcher, disabling cache 2024-04-27T03:32:18.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.294+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94410881363440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bd7c0 con 0x55ddbf0b3730 2024-04-27T03:32:18.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.295+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94410881363440] v27'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.295+0000 7f96bab75800 20 remove_watcher() i=1 2024-04-27T03:32:18.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.295+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94410881366992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bd7c0 con 0x55ddbf0b7d80 2024-04-27T03:32:18.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.296+0000 7f96b8674640 1 -- 172.21.15.165:0/4186098321 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94410881366992] v27'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96a8072030 con 0x55ddbf0b7d80 2024-04-27T03:32:18.312 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.296+0000 7f96bab75800 20 remove_watcher() i=2 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.296+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94410881370464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bfbd0 con 0x7f96a8075060 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.297+0000 7f96b7e73640 1 -- 172.21.15.165:0/4186098321 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94410881370464] v27'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96a0081070 con 0x7f96a8075060 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.297+0000 7f96bab75800 20 remove_watcher() i=3 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.297+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94410881364496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bd7c0 con 0x55ddbf0b3730 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.298+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94410881364496] v27'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x55ddbf0b3730 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.298+0000 7f96bab75800 20 remove_watcher() i=4 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.298+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94410881377680] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bd7c0 con 0x7f96b0018410 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.299+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94410881377680] v27'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.299+0000 7f96bab75800 20 remove_watcher() i=5 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.299+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94410881380384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bd7c0 con 0x7f96b0018410 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.299+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94410881380384] v27'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.300+0000 7f96bab75800 20 remove_watcher() i=6 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.300+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94410881374112] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55ddbf0bd7c0 con 0x7f96b0018410 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.300+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94410881374112] v27'245 uv243 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f96b00083e0 con 0x7f96b0018410 2024-04-27T03:32:18.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.300+0000 7f96bab75800 20 remove_watcher() i=7 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f96b0018410 msgr2=0x7f96b0018890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f96b0018410 0x7f96b0018890 crc :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ddbf0b3730 msgr2=0x55ddbf0b5bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ddbf0b3730 0x55ddbf0b5bf0 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f96a8075060 msgr2=0x7f96a80501b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f96a8075060 0x7f96a80501b0 crc :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ddbf0b7d80 msgr2=0x55ddbf0ba170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ddbf0b7d80 0x55ddbf0ba170 crc :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9580036900 msgr2=0x7f9580038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9580036900 0x7f9580038dc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x55ddbf0858f0 tx=0x7f96a004f000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96b8e75640 1 -- 172.21.15.165:0/4186098321 reap_dead start 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 msgr2=0x55ddbf0a7ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 0x55ddbf0a7ff0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7f96a80496e0 tx=0x7f96a801df70 comp rx=0 tx=0).stop 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 shutdown_connections 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 --2- 172.21.15.165:0/4186098321 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf0a0ca0 0x55ddbf0a7ff0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 >> 172.21.15.165:0/4186098321 conn(0x55ddbf09cf70 msgr2=0x55ddbf09a5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 shutdown_connections 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/4186098321 wait complete. 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/430799764 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9590036900 msgr2=0x7f9590038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 --2- 172.21.15.165:0/430799764 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9590036900 0x7f9590038dc0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x55ddbf07cb40 tx=0x7f96a0005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/430799764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf093950 msgr2=0x55ddbf093d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 --2- 172.21.15.165:0/430799764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf093950 0x55ddbf093d70 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f96a8001b30 tx=0x7f96a8010e80 comp rx=0 tx=0).stop 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/430799764 shutdown_connections 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 --2- 172.21.15.165:0/430799764 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9590036900 0x7f9590038dc0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 --2- 172.21.15.165:0/430799764 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbf093950 0x55ddbf093d70 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/430799764 >> 172.21.15.165:0/430799764 conn(0x55ddbf0879f0 msgr2=0x55ddbf087df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.315 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/430799764 shutdown_connections 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.301+0000 7f96bab75800 1 -- 172.21.15.165:0/430799764 wait complete. 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ddbef7a300 msgr2=0x55ddbf07c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55ddbef7a300 0x55ddbf07c000 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55ddbf082930 msgr2=0x55ddbf084d60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55ddbf082930 0x55ddbf084d60 crc :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ddbf07e9a0 msgr2=0x55ddbf080d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55ddbf07e9a0 0x55ddbf080d90 crc :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f95a0036950 msgr2=0x7f95a0038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f95a0036950 0x7f95a0038e10 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f96a0054000 tx=0x7f96a004e000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 msgr2=0x55ddbf074ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 --2- 172.21.15.165:0/391830108 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55ddbef787c0 0x55ddbf074ef0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7f96a8006fa0 tx=0x7f96a8001250 comp rx=0 tx=0).stop 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96b8e75640 1 -- 172.21.15.165:0/391830108 reap_dead start 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 shutdown_connections 2024-04-27T03:32:18.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 >> 172.21.15.165:0/391830108 conn(0x55ddbef74340 msgr2=0x55ddbf067440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 shutdown_connections 2024-04-27T03:32:18.317 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.302+0000 7f96bab75800 1 -- 172.21.15.165:0/391830108 wait complete. 2024-04-27T03:32:18.317 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:18.318 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:18.318 ERROR:root:0 2024-04-27T03:32:18.318 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'system_user', '--display-name', 'System User', '--access-key', 'system_akey', '--secret', 'system_skey', '--caps', 'user-info-without-keys=read', '--system'] 2024-04-27T03:32:18.318 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'system_user', '--display-name', 'System User', '--access-key', 'system_akey', '--secret', 'system_skey', '--caps', 'user-info-without-keys=read', '--system'] 2024-04-27T03:32:18.318 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid system_user --display-name 'System User' --access-key system_akey --secret system_skey --caps user-info-without-keys=read --system 2024-04-27T03:32:18.369 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:18.369 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:18.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.364+0000 7f3ec6212800 1 Processor -- start 2024-04-27T03:32:18.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.364+0000 7f3ec6212800 1 -- start start 2024-04-27T03:32:18.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.364+0000 7f3ec6212800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562dd798a0d0 0x562dd798c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.364+0000 7f3ec6212800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562dd7844d40 con 0x562dd78f9160 2024-04-27T03:32:18.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.364+0000 7f3ec6212800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x562dd78174d0 con 0x562dd798a0d0 2024-04-27T03:32:18.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.365+0000 7f3ec107c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562dd798a0d0 0x562dd798c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.365+0000 7f3ec187d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x562dd78f9160 0x562dd7988980 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42344/0 (socket says 172.21.15.165:42344) 2024-04-27T03:32:18.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.365+0000 7f3ec187d640 1 -- 172.21.15.165:0/898742851 learned_addr learned my addr 172.21.15.165:0/898742851 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:18.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.365+0000 7f3ec087b640 1 -- 172.21.15.165:0/898742851 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2900992682 0 0) 0x562dd7844d40 con 0x562dd78f9160 2024-04-27T03:32:18.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.365+0000 7f3ec087b640 1 -- 172.21.15.165:0/898742851 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f3eac001e10 con 0x562dd78f9160 2024-04-27T03:32:18.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.365+0000 7f3ec107c640 1 -- 172.21.15.165:0/898742851 >> v1:172.21.15.89:6789/0 conn(0x562dd78f9160 legacy=0x562dd7988980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.365+0000 7f3ec107c640 1 -- 172.21.15.165:0/898742851 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dd7843590 con 0x562dd798a0d0 2024-04-27T03:32:18.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.367+0000 7f3ec107c640 1 --2- 172.21.15.165:0/898742851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd798a0d0 0x562dd798c590 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800a470 tx=0x7f3eb801cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b0b75bdeece0b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.367+0000 7f3ec087b640 1 -- 172.21.15.165:0/898742851 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb8057070 con 0x562dd798a0d0 2024-04-27T03:32:18.373 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.367+0000 7f3ec087b640 1 -- 172.21.15.165:0/898742851 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb8052570 con 0x562dd798a0d0 2024-04-27T03:32:18.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.367+0000 7f3ec087b640 1 -- 172.21.15.165:0/898742851 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb8052870 con 0x562dd798a0d0 2024-04-27T03:32:18.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.368+0000 7f3ec6212800 1 -- 172.21.15.165:0/898742851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd798a0d0 msgr2=0x562dd798c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.368+0000 7f3ec6212800 1 --2- 172.21.15.165:0/898742851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd798a0d0 0x562dd798c590 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f3eb800a470 tx=0x7f3eb801cdb0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.368+0000 7f3ec6212800 1 -- 172.21.15.165:0/898742851 shutdown_connections 2024-04-27T03:32:18.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.368+0000 7f3ec6212800 1 --2- 172.21.15.165:0/898742851 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd798a0d0 0x562dd798c590 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.368+0000 7f3ec6212800 1 -- 172.21.15.165:0/898742851 >> 172.21.15.165:0/898742851 conn(0x562dd7980320 msgr2=0x562dd7982740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.368+0000 7f3ec6212800 1 -- 172.21.15.165:0/898742851 shutdown_connections 2024-04-27T03:32:18.374 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.368+0000 7f3ec6212800 1 -- 172.21.15.165:0/898742851 wait complete. 2024-04-27T03:32:18.385 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.379+0000 7f3ec6212800 1 Processor -- start 2024-04-27T03:32:18.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.379+0000 7f3ec6212800 1 -- start start 2024-04-27T03:32:18.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.380+0000 7f3ec6212800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 0x562dd79862b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.380+0000 7f3ec6212800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562dd78d1360 con 0x562dd78f9160 2024-04-27T03:32:18.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.380+0000 7f3ec187d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 0x562dd79862b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.380+0000 7f3ec187d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 0x562dd79862b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60004/0 (socket says 172.21.15.165:60004) 2024-04-27T03:32:18.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.380+0000 7f3ec187d640 1 -- 172.21.15.165:0/4271617927 learned_addr learned my addr 172.21.15.165:0/4271617927 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.380+0000 7f3ec187d640 1 -- 172.21.15.165:0/4271617927 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dd7984870 con 0x562dd78f9160 2024-04-27T03:32:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.380+0000 7f3ec187d640 1 --2- 172.21.15.165:0/4271617927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 0x562dd79862b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4001de0 tx=0x7f3eb400b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7ceccaeb4882571 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.381+0000 7f3dafe07640 1 -- 172.21.15.165:0/4271617927 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb4012020 con 0x562dd78f9160 2024-04-27T03:32:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.381+0000 7f3dafe07640 1 -- 172.21.15.165:0/4271617927 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb400e5a0 con 0x562dd78f9160 2024-04-27T03:32:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.381+0000 7f3dafe07640 1 -- 172.21.15.165:0/4271617927 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb400e900 con 0x562dd78f9160 2024-04-27T03:32:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.381+0000 7f3ec6212800 1 -- 172.21.15.165:0/4271617927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 msgr2=0x562dd79862b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.381+0000 7f3ec6212800 1 --2- 172.21.15.165:0/4271617927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 0x562dd79862b0 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4001de0 tx=0x7f3eb400b6c0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.381+0000 7f3ec6212800 1 -- 172.21.15.165:0/4271617927 shutdown_connections 2024-04-27T03:32:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.381+0000 7f3ec6212800 1 --2- 172.21.15.165:0/4271617927 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 0x562dd79862b0 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.381+0000 7f3ec6212800 1 -- 172.21.15.165:0/4271617927 >> 172.21.15.165:0/4271617927 conn(0x562dd7980320 msgr2=0x562dd7a727e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.381+0000 7f3ec6212800 1 -- 172.21.15.165:0/4271617927 shutdown_connections 2024-04-27T03:32:18.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.381+0000 7f3ec6212800 1 -- 172.21.15.165:0/4271617927 wait complete. 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.382+0000 7f3ec6212800 1 Processor -- start 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.382+0000 7f3ec6212800 1 -- start start 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.382+0000 7f3ec6212800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 0x562dd7a80be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.382+0000 7f3ec6212800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562dd798cf40 con 0x562dd78f9160 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.382+0000 7f3ec187d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 0x562dd7a80be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.382+0000 7f3ec187d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 0x562dd7a80be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60016/0 (socket says 172.21.15.165:60016) 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.382+0000 7f3ec187d640 1 -- 172.21.15.165:0/4155874288 learned_addr learned my addr 172.21.15.165:0/4155874288 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.382+0000 7f3ec187d640 1 -- 172.21.15.165:0/4155874288 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dd7a81120 con 0x562dd78f9160 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.383+0000 7f3ec187d640 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 0x562dd7a80be0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400c010 tx=0x7f3eb4000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.383+0000 7f3dae604640 1 -- 172.21.15.165:0/4155874288 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb4012030 con 0x562dd78f9160 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.383+0000 7f3dae604640 1 -- 172.21.15.165:0/4155874288 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb401a7a0 con 0x562dd78f9160 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.383+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562dd7a813b0 con 0x562dd78f9160 2024-04-27T03:32:18.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.383+0000 7f3dae604640 1 -- 172.21.15.165:0/4155874288 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb401a950 con 0x562dd78f9160 2024-04-27T03:32:18.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.383+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dd7a84060 con 0x562dd78f9160 2024-04-27T03:32:18.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.384+0000 7f3dae604640 1 -- 172.21.15.165:0/4155874288 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f3eb4015020 con 0x562dd78f9160 2024-04-27T03:32:18.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.384+0000 7f3dae604640 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3da8036950 0x7f3da8038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.384+0000 7f3dae604640 1 -- 172.21.15.165:0/4155874288 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f3eb4020030 con 0x562dd78f9160 2024-04-27T03:32:18.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.384+0000 7f3ec107c640 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3da8036950 0x7f3da8038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.384+0000 7f3ec6212800 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562dd798a0d0 0x562dd7a87e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.384+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7a88360 con 0x562dd798a0d0 2024-04-27T03:32:18.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.384+0000 7f3ec207e640 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562dd798a0d0 0x562dd7a87e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.384+0000 7f3ec107c640 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3da8036950 0x7f3da8038e10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8054000 tx=0x7f3eb804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.398 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.384+0000 7f3ec207e640 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562dd798a0d0 0x562dd7a87e20 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.385+0000 7f3ec207e640 1 -- 172.21.15.165:0/4155874288 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd798a0d0 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.385+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7a853f0 con 0x562dd798a0d0 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.385+0000 7f3ec207e640 1 -- 172.21.15.165:0/4155874288 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd798a0d0 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.385+0000 7f3ec6212800 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562dd7a8a6b0 0x562dd7a8caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.385+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7a8d190 con 0x562dd7a8a6b0 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.385+0000 7f3ec187d640 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562dd7a8a6b0 0x562dd7a8caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.385+0000 7f3ec187d640 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562dd7a8a6b0 0x562dd7a8caa0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.386+0000 7f3ec187d640 1 -- 172.21.15.165:0/4155874288 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3eb400dcd0 con 0x562dd7a8a6b0 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.386+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7a89660 con 0x562dd798a0d0 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.386+0000 7f3ec207e640 1 -- 172.21.15.165:0/4155874288 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd798a0d0 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.386+0000 7f3ec6212800 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562dd7a8e640 0x562dd7a90a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.386+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7a911f0 con 0x562dd7a8e640 2024-04-27T03:32:18.399 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.386+0000 7f3ec107c640 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562dd7a8e640 0x562dd7a90a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.386+0000 7f3ec107c640 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562dd7a8e640 0x562dd7a90a90 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.387+0000 7f3ec107c640 1 -- 172.21.15.165:0/4155874288 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f3eb8052a40 con 0x562dd7a8e640 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.387+0000 7f3ec6212800 1 Processor -- start 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.387+0000 7f3ec6212800 1 -- start start 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.387+0000 7f3ec6212800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7a9f650 0x562dd7a9fa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.387+0000 7f3ec6212800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562dd78776b0 con 0x562dd7a9f650 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.387+0000 7f3ec187d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7a9f650 0x562dd7a9fa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.387+0000 7f3ec187d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7a9f650 0x562dd7a9fa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60024/0 (socket says 172.21.15.165:60024) 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.387+0000 7f3ec187d640 1 -- 172.21.15.165:0/2355912087 learned_addr learned my addr 172.21.15.165:0/2355912087 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.388+0000 7f3ec187d640 1 -- 172.21.15.165:0/2355912087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dd7a9ffb0 con 0x562dd7a9f650 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.388+0000 7f3ec187d640 1 --2- 172.21.15.165:0/2355912087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7a9f650 0x562dd7a9fa70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3eb404aca0 tx=0x7f3eb4016b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.388+0000 7f3da77fe640 1 -- 172.21.15.165:0/2355912087 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb4040040 con 0x562dd7a9f650 2024-04-27T03:32:18.400 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.388+0000 7f3da77fe640 1 -- 172.21.15.165:0/2355912087 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb404c1a0 con 0x562dd7a9f650 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.388+0000 7f3ec6212800 1 -- 172.21.15.165:0/2355912087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562dd7aa0a20 con 0x562dd7a9f650 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.388+0000 7f3da77fe640 1 -- 172.21.15.165:0/2355912087 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb404c4c0 con 0x562dd7a9f650 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.388+0000 7f3ec6212800 1 -- 172.21.15.165:0/2355912087 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dd7aa0f20 con 0x562dd7a9f650 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.389+0000 7f3da77fe640 1 -- 172.21.15.165:0/2355912087 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f3eb4044300 con 0x562dd7a9f650 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.389+0000 7f3da77fe640 1 --2- 172.21.15.165:0/2355912087 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d9c036900 0x7f3d9c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.389+0000 7f3da77fe640 1 -- 172.21.15.165:0/2355912087 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f3eb4037bc0 con 0x562dd7a9f650 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.389+0000 7f3ec107c640 1 --2- 172.21.15.165:0/2355912087 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d9c036900 0x7f3d9c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.389+0000 7f3ec6212800 1 Processor -- start 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.389+0000 7f3ec6212800 1 -- start start 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.389+0000 7f3ec107c640 1 --2- 172.21.15.165:0/2355912087 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d9c036900 0x7f3d9c038dc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x562dd7a88c60 tx=0x7f3eb8005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.389+0000 7f3ec6212800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 0x562dd7aacc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.389+0000 7f3ec6212800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562dd7984bf0 con 0x562dd7aac850 2024-04-27T03:32:18.401 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.390+0000 7f3ec187d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 0x562dd7aacc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.390+0000 7f3ec187d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 0x562dd7aacc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60028/0 (socket says 172.21.15.165:60028) 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.390+0000 7f3ec187d640 1 -- 172.21.15.165:0/106509849 learned_addr learned my addr 172.21.15.165:0/106509849 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.390+0000 7f3ec187d640 1 -- 172.21.15.165:0/106509849 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dd7aad240 con 0x562dd7aac850 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.390+0000 7f3ec187d640 1 --2- 172.21.15.165:0/106509849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 0x562dd7aacc70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4039fb0 tx=0x7f3eb402d4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42c9af770317e881 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.390+0000 7f3da57fa640 1 -- 172.21.15.165:0/106509849 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb400ca60 con 0x562dd7aac850 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.390+0000 7f3da57fa640 1 -- 172.21.15.165:0/106509849 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb405b070 con 0x562dd7aac850 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.390+0000 7f3da57fa640 1 -- 172.21.15.165:0/106509849 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb400c670 con 0x562dd7aac850 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.391+0000 7f3ec6212800 1 -- 172.21.15.165:0/106509849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 msgr2=0x562dd7aacc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.391+0000 7f3ec6212800 1 --2- 172.21.15.165:0/106509849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 0x562dd7aacc70 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f3eb4039fb0 tx=0x7f3eb402d4b0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.391+0000 7f3ec6212800 1 -- 172.21.15.165:0/106509849 shutdown_connections 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.391+0000 7f3ec6212800 1 --2- 172.21.15.165:0/106509849 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 0x562dd7aacc70 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.391+0000 7f3ec6212800 1 -- 172.21.15.165:0/106509849 >> 172.21.15.165:0/106509849 conn(0x562dd7aa8ad0 msgr2=0x562dd7aa6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.402 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.391+0000 7f3ec6212800 1 -- 172.21.15.165:0/106509849 shutdown_connections 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.391+0000 7f3ec6212800 1 -- 172.21.15.165:0/106509849 wait complete. 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.391+0000 7f3ec6212800 1 Processor -- start 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.391+0000 7f3ec6212800 1 -- start start 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.391+0000 7f3ec6212800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 0x562dd7ab3c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.391+0000 7f3ec6212800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562dd7aad8b0 con 0x562dd7aac850 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.391+0000 7f3ec187d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 0x562dd7ab3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.392+0000 7f3ec187d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 0x562dd7ab3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60030/0 (socket says 172.21.15.165:60030) 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.392+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 learned_addr learned my addr 172.21.15.165:0/2560047106 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.392+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562dd7ab4150 con 0x562dd7aac850 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.392+0000 7f3ec187d640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 0x562dd7ab3c10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40498b0 tx=0x7f3eb4042f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.392+0000 7f3d937fe640 1 -- 172.21.15.165:0/2560047106 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb4049400 con 0x562dd7aac850 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.392+0000 7f3d937fe640 1 -- 172.21.15.165:0/2560047106 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3eb401d940 con 0x562dd7aac850 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.392+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562dd7ab4150 con 0x562dd7aac850 2024-04-27T03:32:18.403 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.392+0000 7f3d937fe640 1 -- 172.21.15.165:0/2560047106 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3eb401dc20 con 0x562dd7aac850 2024-04-27T03:32:18.404 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.392+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562dd7ab4150 con 0x562dd7aac850 2024-04-27T03:32:18.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.393+0000 7f3d937fe640 1 -- 172.21.15.165:0/2560047106 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f3eb404c920 con 0x562dd7aac850 2024-04-27T03:32:18.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.393+0000 7f3d937fe640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d8c036900 0x7f3d8c038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.393+0000 7f3d937fe640 1 -- 172.21.15.165:0/2560047106 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f3eb406cbe0 con 0x562dd7aac850 2024-04-27T03:32:18.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.393+0000 7f3ec107c640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d8c036900 0x7f3d8c038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec107c640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d8c036900 0x7f3d8c038dc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x562dd7a915f0 tx=0x7f3eb804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.417 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562dd7abf350 0x562dd7ac1810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac1f60 con 0x562dd7abf350 2024-04-27T03:32:18.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec207e640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562dd7abf350 0x562dd7ac1810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec207e640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562dd7abf350 0x562dd7ac1810 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec6212800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec6212800 20 realm 2024-04-27T03:32:18.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2370 con 0x562dd7abf350 2024-04-27T03:32:18.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec6212800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.418 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec6212800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.394+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2c40 con 0x562dd7abf350 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.395+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.395+0000 7f3ec6212800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.395+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.395+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7abf350 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.395+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.395+0000 7f3ec6212800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.395+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.395+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562dd7ac3d40 0x562dd7ac6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.395+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac6740 con 0x562dd7ac3d40 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.395+0000 7f3ec187d640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562dd7ac3d40 0x562dd7ac6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.395+0000 7f3ec187d640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562dd7ac3d40 0x562dd7ac6130 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.419 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.396+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.396+0000 7f3ec6212800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.396+0000 7f3ec6212800 20 searching for the correct realm 2024-04-27T03:32:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.396+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac8050 con 0x562dd7ac3d40 2024-04-27T03:32:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.396+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.396+0000 7f3ec187d640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3eb4044640 0x7f3eb406f1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.396+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac8050 con 0x7f3eb4044640 2024-04-27T03:32:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.396+0000 7f3ec107c640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3eb4044640 0x7f3eb406f1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.397+0000 7f3ec107c640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3eb4044640 0x7f3eb406f1c0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.397+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.397+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac8050 con 0x7f3eb4044640 2024-04-27T03:32:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.397+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.420 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.397+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac8050 con 0x562dd7abf350 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.397+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.397+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7abf350 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.397+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.397+0000 7f3ec207e640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3ebc016440 0x7f3ebc0168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.397+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3ebc016440 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.398+0000 7f3ec207e640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3ebc016440 0x7f3ebc0168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.398+0000 7f3ec207e640 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3ebc016440 0x7f3ebc0168c0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.398+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.398+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3ebc016440 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.399+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.399+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3eb4044640 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.399+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.399+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7ac3d40 2024-04-27T03:32:18.421 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.399+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.399+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7abf350 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.399+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.399+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3eb4044640 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.400+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.400+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7abf350 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.400+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.400+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3eb4044640 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.400+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.400+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7ac3d40 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.400+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.400+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7ac3d40 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.401+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.422 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.401+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3ebc016440 2024-04-27T03:32:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.401+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.401+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7ac3d40 2024-04-27T03:32:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.402+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.402+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7ac3d40 2024-04-27T03:32:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.402+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.402+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7abf350 2024-04-27T03:32:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.402+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.402+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7abf350 2024-04-27T03:32:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.402+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.402+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3ebc016440 2024-04-27T03:32:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.403+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.423 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.403+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7ac3d40 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.403+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.403+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3eb4044640 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.403+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.403+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7abf350 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.403+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.403+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3ebc016440 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.404+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.404+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7ac3d40 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.404+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.404+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3ebc016440 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.404+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.404+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3eb4044640 2024-04-27T03:32:18.424 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.404+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.404+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7ac3d40 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3ebc016440 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x562dd7abf350 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac2fe0 con 0x7f3eb4044640 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec6212800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec6212800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec6212800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec6212800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec6212800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:18.425 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec6212800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.405+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac7000 con 0x562dd7abf350 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac73f0 con 0x562dd7abf350 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac73f0 con 0x7f3eb4044640 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:18.426 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 20 zone default found 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 4 Realm: () 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 20 zonegroup default 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.406+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac84d0 con 0x7f3ebc016440 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec6212800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac9ee0 con 0x562dd7ac3d40 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.427 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec6212800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7aca3e0 con 0x562dd7ac3d40 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec6212800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec6212800 20 started sync module instance, tier type = 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec6212800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.407+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac9350 con 0x7f3eb4044640 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.408+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.408+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94754891929408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ac8fb0 con 0x7f3eb4044640 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.408+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7acb310 con 0x562dd7abf350 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.409+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94754891929408] v27'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.409+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.409+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94754891935328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7acb730 con 0x562dd7abf350 2024-04-27T03:32:18.428 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.409+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7acc0f0 con 0x562dd7ac3d40 2024-04-27T03:32:18.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.411+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94754891935328] v27'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.411+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.411+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94754891936384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7acb6a0 con 0x562dd7ac3d40 2024-04-27T03:32:18.429 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.411+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7accf60 con 0x7f3eb4044640 2024-04-27T03:32:18.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.412+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.412+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94754891944704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7acdbd0 con 0x7f3eb4044640 2024-04-27T03:32:18.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.412+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ace5f0 con 0x562dd7abf350 2024-04-27T03:32:18.435 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.412+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94754891936384] v27'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.413+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.413+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94754891948256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7acea10 con 0x562dd7abf350 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.413+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7acf430 con 0x7f3ebc016440 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.413+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94754891944704] v27'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.414+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.414+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94754891951904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7acf3f0 con 0x7f3ebc016440 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.414+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7acfee0 con 0x7f3ebc016440 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.414+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94754891948256] v27'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.415+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94754891951904] v27'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.415+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.416+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94754891945760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7acf9c0 con 0x7f3ebc016440 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.416+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ad0390 con 0x7f3ebc016440 2024-04-27T03:32:18.436 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.416+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94754891945760] v27'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.417+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'246 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.417+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94754891941008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ad0390 con 0x7f3ebc016440 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.417+0000 7f3ec6212800 20 add_watcher() i=0 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.417+0000 7f3ec6212800 20 add_watcher() i=1 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.417+0000 7f3ec6212800 20 add_watcher() i=2 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.417+0000 7f3ec6212800 20 add_watcher() i=3 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.417+0000 7f3ec6212800 20 add_watcher() i=4 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.417+0000 7f3ec6212800 20 add_watcher() i=5 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.417+0000 7f3ec6212800 20 add_watcher() i=6 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.418+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94754891941008] v27'247 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.418+0000 7f3ec6212800 20 add_watcher() i=7 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.418+0000 7f3ec6212800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:18.437 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.418+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ad3690 con 0x562dd7abf350 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.419+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.419+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94754891956304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ad0920 con 0x562dd7abf350 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.420+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94754891956304] v27'249 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.420+0000 7f3d737fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.420+0000 7f3ec6212800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.420+0000 7f3ec6212800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.420+0000 7f3ec6212800 5 note: GC not initialized 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.420+0000 7f3d727fc640 20 reqs_thread_entry: start 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.420+0000 7f3ec6212800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.420+0000 7f3ec6212800 20 Filter name: none 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.420+0000 7f3ce3fff640 20 reqs_thread_entry: start 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.420+0000 7f3ec6212800 10 cache get: name=default.rgw.meta+users.uid+system_user : miss 2024-04-27T03:32:18.438 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.420+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.420+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.1e 8:78ce4db0:users.uid::system_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7b06a60 con 0x7f3ebc016440 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 system_user [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 239+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec6212800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec6212800 10 cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x0 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec6212800 10 adding default.rgw.meta+users.uid+system_user to cache LRU end 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec6212800 10 cache get: name=default.rgw.meta+users.keys+system_akey : miss 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec6212800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:63 8.0 8:01223cbf:users.keys::system_akey:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7b05200 con 0x562dd7ac3d40 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(63 system_akey [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 197+0+0 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec6212800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec6212800 10 cache put: name=default.rgw.meta+users.keys+system_akey info.flags=0x0 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec6212800 10 adding default.rgw.meta+users.keys+system_akey to cache LRU end 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec6212800 10 cache get: name=default.rgw.meta+users.keys+system_akey : hit (negative entry) 2024-04-27T03:32:18.439 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec6212800 10 cache get: name=default.rgw.meta+users.keys+system_akey : hit (negative entry) 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.421+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:64 8.1e 8:78ce4db0:users.uid::system_user:head [delete,create,call version.set in=58b,writefull 0~349 in=349b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7b05aa0 con 0x7f3ebc016440 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(64 system_user [delete,create,call,writefull 0~349] v27'1 uv1 ondisk = 0) ==== 281+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3ec6212800 10 cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x17 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3ec6212800 10 moving default.rgw.meta+users.uid+system_user to cache LRU end 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3ec6212800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 0, obj: default.rgw.meta:users.uid:system_user, ofs0, ns] 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:65 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94754892174832 in=545b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7b0a500 con 0x7f3eb4044640 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== watch-notify(notify (1) cookie 94754891929408 notify 115964117017 ret 0) ==== 575+0+0 (crc 0 0 0) 0x7f3eb808a040 con 0x7f3eb4044640 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3da57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117017 cookie 94754891929408 notifier 4833 bl.length()=533 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(65 notify.0 [notify cookie 94754892174832 out=8b] v0'0 uv246 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3da57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x17 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3da57fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+system_user to cache LRU end 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3da57fa640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f3d84003ea0 con 0x7f3eb4044640 2024-04-27T03:32:18.440 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(66 notify.0 [notify-ack] v0'0 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== watch-notify(notify_complete (2) cookie 94754892174832 notify 115964117017 ret 0) ==== 42+0+48 (crc 0 0 0) 0x562dd7843590 con 0x7f3eb4044640 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.423+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 8.0 8:01223cbf:users.keys::system_akey:head [delete,create,writefull 0~15 in=15b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7b055f0 con 0x562dd7ac3d40 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.424+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 system_akey [delete,create,writefull 0~15] v27'1 uv1 ondisk = 0) ==== 239+0+0 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.424+0000 7f3ec6212800 10 cache put: name=default.rgw.meta+users.keys+system_akey info.flags=0x7 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.424+0000 7f3ec6212800 10 moving default.rgw.meta+users.keys+system_akey to cache LRU end 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.424+0000 7f3ec6212800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 0, obj: default.rgw.meta:users.keys:system_akey, ofs0, ns] 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.424+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [notify cookie 94754892174832 in=188b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562dd7b0a660 con 0x562dd7abf350 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.425+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== watch-notify(notify (1) cookie 94754891935328 notify 115964117040 ret 0) ==== 218+0+0 (crc 0 0 0) 0x562dd7b05e40 con 0x562dd7abf350 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.425+0000 7f3da57fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117040 cookie 94754891935328 notifier 4833 bl.length()=176 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.425+0000 7f3da57fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+system_akey info.flags=0x7 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.425+0000 7f3da57fa640 10 rgw watcher librados: moving default.rgw.meta+users.keys+system_akey to cache LRU end 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.425+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(68 notify.1 [notify cookie 94754892174832 out=8b] v0'0 uv246 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.425+0000 7f3da57fa640 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:69 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f3d84003ea0 con 0x562dd7abf350 2024-04-27T03:32:18.441 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.425+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== watch-notify(notify_complete (2) cookie 94754892174832 notify 115964117040 ret 0) ==== 42+0+48 (crc 0 0 0) 0x562dd7b064b0 con 0x562dd7abf350 2024-04-27T03:32:18.442 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.425+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(69 notify.1 [notify-ack] v0'0 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.442 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"system_user","display_name":"System User","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"system_user","access_key":"system_akey","secret_key":"system_skey","active":true,"create_date":"2024-04-27T03:32:18.422759Z"}],"swift_keys":[],"caps":[{"type":"user-info-without-keys","perm":"read"}],"op_mask":"read, write, delete","system":true,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:32:18.422747Z","tags":[],"group_ids":[]} 2024-04-27T03:32:18.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.426+0000 7f3d737fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:18.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.426+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94754891956304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7abba60 con 0x562dd7abf350 2024-04-27T03:32:18.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.427+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94754891956304] v27'250 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.427+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94754891929408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ad0050 con 0x7f3eb4044640 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.428+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94754891929408] v27'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.428+0000 7f3ec6212800 20 remove_watcher() i=0 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.428+0000 7f3ec6212800 2 removed watcher, disabling cache 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.428+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94754891935328] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ad0050 con 0x562dd7abf350 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.429+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94754891935328] v27'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.429+0000 7f3ec6212800 20 remove_watcher() i=1 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.430+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94754891936384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7acae60 con 0x562dd7ac3d40 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.431+0000 7f3ec187d640 1 -- 172.21.15.165:0/2560047106 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94754891936384] v27'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eb4071020 con 0x562dd7ac3d40 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.431+0000 7f3ec6212800 20 remove_watcher() i=2 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.431+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94754891944704] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ad0050 con 0x7f3eb4044640 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.432+0000 7f3ec107c640 1 -- 172.21.15.165:0/2560047106 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94754891944704] v27'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3eb8081070 con 0x7f3eb4044640 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.432+0000 7f3ec6212800 20 remove_watcher() i=3 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.432+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94754891948256] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7acd300 con 0x562dd7abf350 2024-04-27T03:32:18.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.433+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94754891948256] v27'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x562dd7abf350 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.433+0000 7f3ec6212800 20 remove_watcher() i=4 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.433+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94754891951904] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ad0050 con 0x7f3ebc016440 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.434+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94754891951904] v27'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.434+0000 7f3ec6212800 20 remove_watcher() i=5 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.434+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94754891945760] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ace0e0 con 0x7f3ebc016440 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.435+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94754891945760] v27'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.435+0000 7f3ec6212800 20 remove_watcher() i=6 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.435+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94754891941008] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562dd7ad0050 con 0x7f3ebc016440 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94754891941008] v27'248 uv246 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3ebc00a4a0 con 0x7f3ebc016440 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 20 remove_watcher() i=7 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3ebc016440 msgr2=0x7f3ebc0168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f3ebc016440 0x7f3ebc0168c0 crc :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562dd7abf350 msgr2=0x562dd7ac1810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562dd7abf350 0x562dd7ac1810 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3eb4044640 msgr2=0x7f3eb406f1c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f3eb4044640 0x7f3eb406f1c0 crc :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562dd7ac3d40 msgr2=0x562dd7ac6130 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562dd7ac3d40 0x562dd7ac6130 crc :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d8c036900 msgr2=0x7f3d8c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d8c036900 0x7f3d8c038dc0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x562dd7a915f0 tx=0x7f3eb804f000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec207e640 1 -- 172.21.15.165:0/2560047106 reap_dead start 2024-04-27T03:32:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 msgr2=0x562dd7ab3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 0x562dd7ab3c10 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f3eb40498b0 tx=0x7f3eb4042f70 comp rx=0 tx=0).stop 2024-04-27T03:32:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 shutdown_connections 2024-04-27T03:32:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2560047106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7aac850 0x562dd7ab3c10 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.450 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 >> 172.21.15.165:0/2560047106 conn(0x562dd7aa8ad0 msgr2=0x562dd7aa6130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 shutdown_connections 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2560047106 wait complete. 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2355912087 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d9c036900 msgr2=0x7f3d9c038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2355912087 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d9c036900 0x7f3d9c038dc0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x562dd7a88c60 tx=0x7f3eb8005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2355912087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7a9f650 msgr2=0x562dd7a9fa70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2355912087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7a9f650 0x562dd7a9fa70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f3eb404aca0 tx=0x7f3eb4016b00 comp rx=0 tx=0).stop 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2355912087 shutdown_connections 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2355912087 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3d9c036900 0x7f3d9c038dc0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 --2- 172.21.15.165:0/2355912087 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd7a9f650 0x562dd7a9fa70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2355912087 >> 172.21.15.165:0/2355912087 conn(0x562dd7a936f0 msgr2=0x562dd7a93af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2355912087 shutdown_connections 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.436+0000 7f3ec6212800 1 -- 172.21.15.165:0/2355912087 wait complete. 2024-04-27T03:32:18.451 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562dd798a0d0 msgr2=0x562dd7a87e20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562dd798a0d0 0x562dd7a87e20 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562dd7a8e640 msgr2=0x562dd7a90a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562dd7a8e640 0x562dd7a90a90 crc :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562dd7a8a6b0 msgr2=0x562dd7a8caa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562dd7a8a6b0 0x562dd7a8caa0 crc :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3da8036950 msgr2=0x7f3da8038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f3da8036950 0x7f3da8038e10 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f3eb8054000 tx=0x7f3eb804e000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 msgr2=0x562dd7a80be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 --2- 172.21.15.165:0/4155874288 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562dd78f9160 0x562dd7a80be0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f3eb400c010 tx=0x7f3eb4000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec207e640 1 -- 172.21.15.165:0/4155874288 reap_dead start 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 shutdown_connections 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 >> 172.21.15.165:0/4155874288 conn(0x562dd7980320 msgr2=0x562dd7985150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.452 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 shutdown_connections 2024-04-27T03:32:18.453 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.437+0000 7f3ec6212800 1 -- 172.21.15.165:0/4155874288 wait complete. 2024-04-27T03:32:18.453 DEBUG:tasks.util.rgw: json result: {'user_id': 'system_user', 'display_name': 'System User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'system_user', 'access_key': 'system_akey', 'secret_key': 'system_skey', 'active': True, 'create_date': '2024-04-27T03:32:18.422759Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'system': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.422747Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:18.454 ERROR:root:{'user_id': 'system_user', 'display_name': 'System User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'system_user', 'access_key': 'system_akey', 'secret_key': 'system_skey', 'active': True, 'create_date': '2024-04-27T03:32:18.422759Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'system': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.422747Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:18.454 ERROR:root:0 2024-04-27T03:32:18.454 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:18.459 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:18.459 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'system_user'] 2024-04-27T03:32:18.459 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'system_user'] 2024-04-27T03:32:18.459 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid system_user 2024-04-27T03:32:18.501 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:18.501 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:18.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.496+0000 7f079a606800 1 Processor -- start 2024-04-27T03:32:18.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.496+0000 7f079a606800 1 -- start start 2024-04-27T03:32:18.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.496+0000 7f079a606800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a7509a4480 0x55a7509a6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.497+0000 7f079a606800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55a75085ed40 con 0x55a7509a0970 2024-04-27T03:32:18.502 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.497+0000 7f079a606800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55a7508314d0 con 0x55a7509a4480 2024-04-27T03:32:18.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.497+0000 7f0797904640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55a7509a4480 0x55a7509a6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.497+0000 7f0798105640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55a7509a0970 0x55a7509a2d60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42348/0 (socket says 172.21.15.165:42348) 2024-04-27T03:32:18.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.497+0000 7f0798105640 1 -- 172.21.15.165:0/2816422015 learned_addr learned my addr 172.21.15.165:0/2816422015 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:18.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.497+0000 7f0797103640 1 -- 172.21.15.165:0/2816422015 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2574353974 0 0) 0x55a75085ed40 con 0x55a7509a0970 2024-04-27T03:32:18.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.497+0000 7f0797103640 1 -- 172.21.15.165:0/2816422015 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0780001e10 con 0x55a7509a0970 2024-04-27T03:32:18.503 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.497+0000 7f0797904640 1 -- 172.21.15.165:0/2816422015 >> v1:172.21.15.89:6789/0 conn(0x55a7509a0970 legacy=0x55a7509a2d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.504 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.497+0000 7f0797904640 1 -- 172.21.15.165:0/2816422015 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a75085d590 con 0x55a7509a4480 2024-04-27T03:32:18.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.499+0000 7f0797904640 1 --2- 172.21.15.165:0/2816422015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a4480 0x55a7509a6870 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f078c00a470 tx=0x7f078c01cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eba6f4788d601830 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.499+0000 7f0797103640 1 -- 172.21.15.165:0/2816422015 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f078c057070 con 0x55a7509a4480 2024-04-27T03:32:18.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.499+0000 7f0797103640 1 -- 172.21.15.165:0/2816422015 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078c052570 con 0x55a7509a4480 2024-04-27T03:32:18.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.499+0000 7f0797103640 1 -- 172.21.15.165:0/2816422015 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f078c052870 con 0x55a7509a4480 2024-04-27T03:32:18.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.500+0000 7f079a606800 1 -- 172.21.15.165:0/2816422015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a4480 msgr2=0x55a7509a6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.500+0000 7f079a606800 1 --2- 172.21.15.165:0/2816422015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a4480 0x55a7509a6870 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f078c00a470 tx=0x7f078c01cdb0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.500+0000 7f079a606800 1 -- 172.21.15.165:0/2816422015 shutdown_connections 2024-04-27T03:32:18.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.500+0000 7f079a606800 1 --2- 172.21.15.165:0/2816422015 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a4480 0x55a7509a6870 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.500+0000 7f079a606800 1 -- 172.21.15.165:0/2816422015 >> 172.21.15.165:0/2816422015 conn(0x55a75099a320 msgr2=0x55a75099c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.500+0000 7f079a606800 1 -- 172.21.15.165:0/2816422015 shutdown_connections 2024-04-27T03:32:18.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.500+0000 7f079a606800 1 -- 172.21.15.165:0/2816422015 wait complete. 2024-04-27T03:32:18.517 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.512+0000 7f079a606800 1 Processor -- start 2024-04-27T03:32:18.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.512+0000 7f079a606800 1 -- start start 2024-04-27T03:32:18.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.512+0000 7f079a606800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 0x55a750a90f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.512+0000 7f079a606800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a7508eb3c0 con 0x55a7509a0970 2024-04-27T03:32:18.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.512+0000 7f0798105640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 0x55a750a90f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.512+0000 7f0798105640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 0x55a750a90f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60052/0 (socket says 172.21.15.165:60052) 2024-04-27T03:32:18.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.512+0000 7f0798105640 1 -- 172.21.15.165:0/3227206632 learned_addr learned my addr 172.21.15.165:0/3227206632 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.512+0000 7f0798105640 1 -- 172.21.15.165:0/3227206632 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a750a914e0 con 0x55a7509a0970 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.513+0000 7f0798105640 1 --2- 172.21.15.165:0/3227206632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 0x55a750a90f10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0788001de0 tx=0x7f078800b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e65c21c3bc0cdb3 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.513+0000 7f068660c640 1 -- 172.21.15.165:0/3227206632 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0788012020 con 0x55a7509a0970 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.513+0000 7f068660c640 1 -- 172.21.15.165:0/3227206632 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078800e5a0 con 0x55a7509a0970 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.513+0000 7f068660c640 1 -- 172.21.15.165:0/3227206632 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f078800e900 con 0x55a7509a0970 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.513+0000 7f079a606800 1 -- 172.21.15.165:0/3227206632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 msgr2=0x55a750a90f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.513+0000 7f079a606800 1 --2- 172.21.15.165:0/3227206632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 0x55a750a90f10 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f0788001de0 tx=0x7f078800b6c0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.513+0000 7f079a606800 1 -- 172.21.15.165:0/3227206632 shutdown_connections 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.513+0000 7f079a606800 1 --2- 172.21.15.165:0/3227206632 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 0x55a750a90f10 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.513+0000 7f079a606800 1 -- 172.21.15.165:0/3227206632 >> 172.21.15.165:0/3227206632 conn(0x55a75099a320 msgr2=0x55a750a8c830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.514+0000 7f079a606800 1 -- 172.21.15.165:0/3227206632 shutdown_connections 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.514+0000 7f079a606800 1 -- 172.21.15.165:0/3227206632 wait complete. 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.514+0000 7f079a606800 1 Processor -- start 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.514+0000 7f079a606800 1 -- start start 2024-04-27T03:32:18.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.514+0000 7f079a606800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 0x55a750a98a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.514+0000 7f079a606800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a7509a90c0 con 0x55a7509a0970 2024-04-27T03:32:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.514+0000 7f0798105640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 0x55a750a98a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.514+0000 7f0798105640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 0x55a750a98a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60058/0 (socket says 172.21.15.165:60058) 2024-04-27T03:32:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.514+0000 7f0798105640 1 -- 172.21.15.165:0/1493557843 learned_addr learned my addr 172.21.15.165:0/1493557843 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.515+0000 7f0798105640 1 -- 172.21.15.165:0/1493557843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a750a98f90 con 0x55a7509a0970 2024-04-27T03:32:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.515+0000 7f0798105640 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 0x55a750a98a50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f078800c010 tx=0x7f0788000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.515+0000 7f0684e09640 1 -- 172.21.15.165:0/1493557843 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0788012030 con 0x55a7509a0970 2024-04-27T03:32:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.515+0000 7f0684e09640 1 -- 172.21.15.165:0/1493557843 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078801a7a0 con 0x55a7509a0970 2024-04-27T03:32:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.515+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a750a99220 con 0x55a7509a0970 2024-04-27T03:32:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.515+0000 7f0684e09640 1 -- 172.21.15.165:0/1493557843 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f078801a950 con 0x55a7509a0970 2024-04-27T03:32:18.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.515+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a750a9bed0 con 0x55a7509a0970 2024-04-27T03:32:18.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.516+0000 7f0684e09640 1 -- 172.21.15.165:0/1493557843 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0788015020 con 0x55a7509a0970 2024-04-27T03:32:18.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.516+0000 7f0684e09640 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0680036950 0x7f0680038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.516+0000 7f0684e09640 1 -- 172.21.15.165:0/1493557843 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0788020030 con 0x55a7509a0970 2024-04-27T03:32:18.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.516+0000 7f0797904640 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0680036950 0x7f0680038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.516+0000 7f079a606800 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a7509a4480 0x55a750a9fc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.516+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750aa01d0 con 0x55a7509a4480 2024-04-27T03:32:18.526 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.516+0000 7f0798906640 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a7509a4480 0x55a750a9fc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.516+0000 7f0797904640 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0680036950 0x7f0680038e10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f078c054000 tx=0x7f078c04e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.517+0000 7f0798906640 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a7509a4480 0x55a750a9fc90 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.517+0000 7f0798906640 1 -- 172.21.15.165:0/1493557843 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a7509a4480 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.517+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750a9d260 con 0x55a7509a4480 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.517+0000 7f0798906640 1 -- 172.21.15.165:0/1493557843 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f079000a4a0 con 0x55a7509a4480 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.517+0000 7f079a606800 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a750aa2520 0x55a750aa4910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.517+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750aa5000 con 0x55a750aa2520 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.517+0000 7f0798105640 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a750aa2520 0x55a750aa4910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.517+0000 7f0798105640 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a750aa2520 0x55a750aa4910 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.518+0000 7f0798105640 1 -- 172.21.15.165:0/1493557843 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f078800dcd0 con 0x55a750aa2520 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.518+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750aa14d0 con 0x55a7509a4480 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.518+0000 7f0798906640 1 -- 172.21.15.165:0/1493557843 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f079000a4a0 con 0x55a7509a4480 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.518+0000 7f079a606800 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a750aa64b0 0x55a750aa8900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.527 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.518+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750aa9060 con 0x55a750aa64b0 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.518+0000 7f0797904640 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a750aa64b0 0x55a750aa8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.518+0000 7f0797904640 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a750aa64b0 0x55a750aa8900 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.519+0000 7f0797904640 1 -- 172.21.15.165:0/1493557843 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f078c052a40 con 0x55a750aa64b0 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.519+0000 7f079a606800 1 Processor -- start 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.519+0000 7f079a606800 1 -- start start 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.519+0000 7f079a606800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a750ab74c0 0x55a750ab78e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.519+0000 7f079a606800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a7508916b0 con 0x55a750ab74c0 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.520+0000 7f0798105640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a750ab74c0 0x55a750ab78e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.520+0000 7f0798105640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a750ab74c0 0x55a750ab78e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60074/0 (socket says 172.21.15.165:60074) 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.520+0000 7f0798105640 1 -- 172.21.15.165:0/1285181654 learned_addr learned my addr 172.21.15.165:0/1285181654 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.520+0000 7f0798105640 1 -- 172.21.15.165:0/1285181654 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a750ab7e20 con 0x55a750ab74c0 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.520+0000 7f0798105640 1 --2- 172.21.15.165:0/1285181654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a750ab74c0 0x55a750ab78e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f078804aca0 tx=0x7f0788016b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.528 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.520+0000 7f067dffb640 1 -- 172.21.15.165:0/1285181654 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0788040040 con 0x55a750ab74c0 2024-04-27T03:32:18.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.520+0000 7f067dffb640 1 -- 172.21.15.165:0/1285181654 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078804c1a0 con 0x55a750ab74c0 2024-04-27T03:32:18.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.520+0000 7f079a606800 1 -- 172.21.15.165:0/1285181654 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a750ab8890 con 0x55a750ab74c0 2024-04-27T03:32:18.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.520+0000 7f067dffb640 1 -- 172.21.15.165:0/1285181654 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f078804c4c0 con 0x55a750ab74c0 2024-04-27T03:32:18.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.520+0000 7f079a606800 1 -- 172.21.15.165:0/1285181654 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a750ab8d90 con 0x55a750ab74c0 2024-04-27T03:32:18.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.521+0000 7f067dffb640 1 -- 172.21.15.165:0/1285181654 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0788044300 con 0x55a750ab74c0 2024-04-27T03:32:18.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.521+0000 7f067dffb640 1 --2- 172.21.15.165:0/1285181654 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0670036900 0x7f0670038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.521+0000 7f067dffb640 1 -- 172.21.15.165:0/1285181654 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0788037bc0 con 0x55a750ab74c0 2024-04-27T03:32:18.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.521+0000 7f0797904640 1 --2- 172.21.15.165:0/1285181654 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0670036900 0x7f0670038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f079a606800 1 Processor -- start 2024-04-27T03:32:18.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f079a606800 1 -- start start 2024-04-27T03:32:18.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f0797904640 1 --2- 172.21.15.165:0/1285181654 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0670036900 0x7f0670038dc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55a750aa0ad0 tx=0x7f078c005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f079a606800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 0x55a75099e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f079a606800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a750a91860 con 0x55a7509a0030 2024-04-27T03:32:18.541 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f0798105640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 0x55a75099e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f0798105640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 0x55a75099e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60090/0 (socket says 172.21.15.165:60090) 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f0798105640 1 -- 172.21.15.165:0/2086901217 learned_addr learned my addr 172.21.15.165:0/2086901217 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f0798105640 1 -- 172.21.15.165:0/2086901217 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7509a04e0 con 0x55a7509a0030 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f0798105640 1 --2- 172.21.15.165:0/2086901217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 0x55a75099e760 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f078802d4a0 tx=0x7f0788034370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b1c9ad61981f9f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f06777fe640 1 -- 172.21.15.165:0/2086901217 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0788024420 con 0x55a7509a0030 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f06777fe640 1 -- 172.21.15.165:0/2086901217 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f078805b070 con 0x55a7509a0030 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.522+0000 7f06777fe640 1 -- 172.21.15.165:0/2086901217 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f078800c540 con 0x55a7509a0030 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.523+0000 7f079a606800 1 -- 172.21.15.165:0/2086901217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 msgr2=0x55a75099e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.523+0000 7f079a606800 1 --2- 172.21.15.165:0/2086901217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 0x55a75099e760 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f078802d4a0 tx=0x7f0788034370 comp rx=0 tx=0).stop 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.523+0000 7f079a606800 1 -- 172.21.15.165:0/2086901217 shutdown_connections 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.523+0000 7f079a606800 1 --2- 172.21.15.165:0/2086901217 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 0x55a75099e760 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.523+0000 7f079a606800 1 -- 172.21.15.165:0/2086901217 >> 172.21.15.165:0/2086901217 conn(0x55a750ac08e0 msgr2=0x55a750abe940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.542 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.523+0000 7f079a606800 1 -- 172.21.15.165:0/2086901217 shutdown_connections 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.523+0000 7f079a606800 1 -- 172.21.15.165:0/2086901217 wait complete. 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.523+0000 7f079a606800 1 Processor -- start 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.523+0000 7f079a606800 1 -- start start 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.523+0000 7f079a606800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 0x55a750acdb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.523+0000 7f079a606800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55a75099eca0 con 0x55a7509a0030 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.524+0000 7f0798105640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 0x55a750acdb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.524+0000 7f0798105640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 0x55a750acdb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60102/0 (socket says 172.21.15.165:60102) 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.524+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 learned_addr learned my addr 172.21.15.165:0/2467100966 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.524+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a750ace0c0 con 0x55a7509a0030 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.524+0000 7f0798105640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 0x55a750acdb80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0788049aa0 tx=0x7f078801be90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.524+0000 7f0675ffb640 1 -- 172.21.15.165:0/2467100966 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f078806e070 con 0x55a7509a0030 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.524+0000 7f0675ffb640 1 -- 172.21.15.165:0/2467100966 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0788024420 con 0x55a7509a0030 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.524+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55a750ace0c0 con 0x55a7509a0030 2024-04-27T03:32:18.543 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.524+0000 7f0675ffb640 1 -- 172.21.15.165:0/2467100966 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f07880083e0 con 0x55a7509a0030 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.524+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a750ace0c0 con 0x55a7509a0030 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.525+0000 7f0675ffb640 1 -- 172.21.15.165:0/2467100966 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0788071070 con 0x55a7509a0030 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.525+0000 7f0675ffb640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0668036950 0x7f0668038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.525+0000 7f0675ffb640 1 -- 172.21.15.165:0/2467100966 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0788052030 con 0x55a7509a0030 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.525+0000 7f0797904640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0668036950 0x7f0668038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.526+0000 7f0797904640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0668036950 0x7f0668038e10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55a750aa9460 tx=0x7f078c04f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.526+0000 7f079a606800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.526+0000 7f079a606800 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a750ad9220 0x55a750adb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.526+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750adbe30 con 0x55a750ad9220 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.526+0000 7f0798906640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a750ad9220 0x55a750adb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.526+0000 7f0798906640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a750ad9220 0x55a750adb6e0 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.526+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.544 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.526+0000 7f079a606800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.526+0000 7f079a606800 20 realm 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.526+0000 7f079a606800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.526+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750adc240 con 0x55a750ad9220 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f079a606800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f079a606800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f079a606800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750adcb10 con 0x55a750ad9220 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f079a606800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f079a606800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750ad9220 2024-04-27T03:32:18.545 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f079a606800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f079a606800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f079a606800 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a750addc10 0x55a750ae0000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae0610 con 0x55a750addc10 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f0798105640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a750addc10 0x55a750ae0000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.527+0000 7f0798105640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a750addc10 0x55a750ae0000 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.528+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.528+0000 7f079a606800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.528+0000 7f079a606800 20 searching for the correct realm 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.528+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae1f40 con 0x55a750addc10 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.528+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.528+0000 7f0798105640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0788073060 0x7f07880490c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.546 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.528+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae1f40 con 0x7f0788073060 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.528+0000 7f0797904640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0788073060 0x7f07880490c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.529+0000 7f0797904640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0788073060 0x7f07880490c0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.529+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.529+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae1f40 con 0x7f0788073060 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.529+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.529+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae1f40 con 0x55a750ad9220 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.529+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.529+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750ad9220 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.530+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.530+0000 7f0798906640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0790016440 0x7f07900168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.530+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0790016440 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.530+0000 7f0798906640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0790016440 0x7f07900168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.530+0000 7f0798906640 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0790016440 0x7f07900168c0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.531+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.531+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0790016440 2024-04-27T03:32:18.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.531+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.531+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0788073060 2024-04-27T03:32:18.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.531+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.531+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750addc10 2024-04-27T03:32:18.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.532+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.532+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750ad9220 2024-04-27T03:32:18.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.532+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.532+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0788073060 2024-04-27T03:32:18.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.532+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.532+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750ad9220 2024-04-27T03:32:18.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.532+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.532+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0788073060 2024-04-27T03:32:18.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.532+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.532+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750addc10 2024-04-27T03:32:18.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.533+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.533+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750addc10 2024-04-27T03:32:18.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.533+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.533+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0790016440 2024-04-27T03:32:18.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.533+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.534+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750addc10 2024-04-27T03:32:18.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.534+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.534+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750addc10 2024-04-27T03:32:18.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.534+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.534+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750ad9220 2024-04-27T03:32:18.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.534+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.534+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750ad9220 2024-04-27T03:32:18.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.534+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.534+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0790016440 2024-04-27T03:32:18.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.535+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.535+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750addc10 2024-04-27T03:32:18.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.535+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.535+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0788073060 2024-04-27T03:32:18.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.535+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.535+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750ad9220 2024-04-27T03:32:18.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.535+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.535+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0790016440 2024-04-27T03:32:18.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.536+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.536+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750addc10 2024-04-27T03:32:18.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.536+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.536+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0790016440 2024-04-27T03:32:18.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.536+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.536+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0788073060 2024-04-27T03:32:18.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.536+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.559 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.536+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750addc10 2024-04-27T03:32:18.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0790016440 2024-04-27T03:32:18.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x55a750ad9220 2024-04-27T03:32:18.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55a750adceb0 con 0x7f0788073060 2024-04-27T03:32:18.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f079a606800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:18.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f079a606800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:18.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f079a606800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:18.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f079a606800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:18.560 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f079a606800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f079a606800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f079a606800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.537+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae0ed0 con 0x55a750ad9220 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae12c0 con 0x55a750ad9220 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae12c0 con 0x7f0788073060 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.561 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 20 zone default found 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 4 Realm: () 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 20 zonegroup default 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.538+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae23c0 con 0x7f0790016440 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f079a606800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f079a606800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae3e00 con 0x55a750addc10 2024-04-27T03:32:18.562 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f079a606800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f079a606800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae42f0 con 0x55a750addc10 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f079a606800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f079a606800 20 started sync module instance, tier type = 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f079a606800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.539+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae3240 con 0x7f0788073060 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.540+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.540+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94177101493808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae4c00 con 0x7f0788073060 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.540+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae5450 con 0x55a750ad9220 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.541+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94177101493808] v27'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.541+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.541+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94177101498192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae5450 con 0x55a750ad9220 2024-04-27T03:32:18.563 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.541+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae5f70 con 0x55a750addc10 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.543+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.543+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94177101498192] v27'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.543+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94177101503040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae6810 con 0x55a750addc10 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.543+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae4fe0 con 0x7f0788073060 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.544+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.544+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94177101503040] v27'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.544+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94177101508528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae79c0 con 0x7f0788073060 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.544+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae63a0 con 0x55a750ad9220 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.545+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94177101508528] v27'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.545+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.545+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94177101509648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae8460 con 0x55a750ad9220 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.545+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae8e20 con 0x7f0790016440 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.546+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.564 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.546+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94177101515120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae9240 con 0x7f0790016440 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.546+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae9ce0 con 0x7f0790016440 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.546+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94177101509648] v27'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.547+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94177101515120] v27'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.547+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.547+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94177101518768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae8d90 con 0x7f0790016440 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.547+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750aea370 con 0x7f0790016440 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.548+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94177101518768] v27'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.550+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'249 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.550+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94177101512624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750aea370 con 0x7f0790016440 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.550+0000 7f079a606800 20 add_watcher() i=0 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.550+0000 7f079a606800 20 add_watcher() i=1 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.550+0000 7f079a606800 20 add_watcher() i=2 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.550+0000 7f079a606800 20 add_watcher() i=3 2024-04-27T03:32:18.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.550+0000 7f079a606800 20 add_watcher() i=4 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.550+0000 7f079a606800 20 add_watcher() i=5 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.550+0000 7f079a606800 20 add_watcher() i=6 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.551+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94177101512624] v27'250 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.551+0000 7f079a606800 20 add_watcher() i=7 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.551+0000 7f079a606800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.551+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750aed510 con 0x55a750ad9220 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.552+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.552+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94177101520576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750aea790 con 0x55a750ad9220 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.553+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94177101520576] v27'251 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.553+0000 7f064e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.553+0000 7f079a606800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.553+0000 7f079a606800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:18.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.553+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af5eb0 con 0x55a750ad9220 2024-04-27T03:32:18.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.554+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'59 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.554+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af6500 con 0x7f0790016440 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.555+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.555+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af6960 con 0x55a750addc10 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.556+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'37 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.556+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af6960 con 0x55a750ad9220 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.557+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'252 uv172 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.557+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af6f50 con 0x7f0788073060 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.558+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'83 uv59 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.558+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af74c0 con 0x55a750addc10 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.559+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'65 uv49 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.559+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af7550 con 0x55a750ad9220 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.561+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'37 uv29 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.561+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af75e0 con 0x7f0790016440 2024-04-27T03:32:18.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.562+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'57 uv42 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.562+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af7670 con 0x7f0788073060 2024-04-27T03:32:18.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.563+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'29 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.563+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af7be0 con 0x55a750addc10 2024-04-27T03:32:18.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.564+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.564+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af7be0 con 0x7f0788073060 2024-04-27T03:32:18.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.565+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'84 uv61 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.565+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x7f0790016440 2024-04-27T03:32:18.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.566+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'35 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.566+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x7f0788073060 2024-04-27T03:32:18.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.567+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.567+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x7f0790016440 2024-04-27T03:32:18.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.568+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'69 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.568+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x55a750ad9220 2024-04-27T03:32:18.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.569+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'253 uv168 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.569+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x55a750addc10 2024-04-27T03:32:18.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.570+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'78 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.576 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.570+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x7f0788073060 2024-04-27T03:32:18.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.571+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'85 uv57 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.577 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.571+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x55a750addc10 2024-04-27T03:32:18.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.572+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'41 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.572+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x55a750ad9220 2024-04-27T03:32:18.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.573+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'37 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.573+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x55a750ad9220 2024-04-27T03:32:18.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.574+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'254 uv170 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.574+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x55a750addc10 2024-04-27T03:32:18.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.576+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'79 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.576+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x7f0788073060 2024-04-27T03:32:18.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.577+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'45 uv37 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.577+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x55a750addc10 2024-04-27T03:32:18.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.578+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'37 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.578+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x55a750addc10 2024-04-27T03:32:18.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.578+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'48 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.578+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x55a750addc10 2024-04-27T03:32:18.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.579+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'31 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.580+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x55a750addc10 2024-04-27T03:32:18.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.581+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'66 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.581+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x7f0790016440 2024-04-27T03:32:18.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.582+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'39 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.582+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x55a750addc10 2024-04-27T03:32:18.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.583+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'42 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.583+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x7f0790016440 2024-04-27T03:32:18.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.584+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'58 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.584+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x55a750addc10 2024-04-27T03:32:18.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.585+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'80 uv56 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.585+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x7f0790016440 2024-04-27T03:32:18.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.586+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'70 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.586+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af8090 con 0x7f0788073060 2024-04-27T03:32:18.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.588+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'35 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.588+0000 7f064d7e2640 20 reqs_thread_entry: start 2024-04-27T03:32:18.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.588+0000 7f079a606800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:18.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.588+0000 7f079a606800 20 Filter name: none 2024-04-27T03:32:18.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.588+0000 7f05beffd640 20 reqs_thread_entry: start 2024-04-27T03:32:18.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.588+0000 7f079a606800 10 cache get: name=default.rgw.meta+users.uid+system_user : miss 2024-04-27T03:32:18.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.588+0000 7f079a606800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.588+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:94 8.1e 8:78ce4db0:users.uid::system_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750b22f50 con 0x7f0790016440 2024-04-27T03:32:18.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.589+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(94 system_user [call out=48b,read 0~349 out=349b,getxattrs out=75b] v0'0 uv1 ondisk = 0) ==== 239+0+472 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.589+0000 7f079a606800 20 rados_obj.operate() r=0 bl.length=349 2024-04-27T03:32:18.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.589+0000 7f079a606800 10 cache put: name=default.rgw.meta+users.uid+system_user info.flags=0x13 2024-04-27T03:32:18.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.589+0000 7f079a606800 10 adding default.rgw.meta+users.uid+system_user to cache LRU end 2024-04-27T03:32:18.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.589+0000 7f079a606800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.589+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:95 8.1e 8:79f34a7d:users.uid::system_user.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750b25810 con 0x7f0790016440 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.589+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(95 system_user.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 163+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.589+0000 7f079a606800 10 removing key index: system_akey 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.589+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:96 8.0 8:01223cbf:users.keys::system_akey:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750b237f0 con 0x55a750addc10 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.590+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(96 system_akey [delete] v27'2 uv2 ondisk = 0) ==== 155+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.590+0000 7f079a606800 10 distributing notification oid=default.rgw.control:notify.1 cni=[op: 1, obj: default.rgw.meta:users.keys:system_akey, ofs0, ns] 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.590+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:97 7.0 7:05bf5b68:::notify.1:head [notify cookie 94177101759312 in=173b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750b25c30 con 0x55a750ad9220 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.590+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== watch-notify(notify (1) cookie 94177101498192 notify 115964117041 ret 0) ==== 203+0+0 (crc 0 0 0) 0x55a750af58a0 con 0x55a750ad9220 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.590+0000 7f06777fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117041 cookie 94177101498192 notifier 4840 bl.length()=161 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.590+0000 7f06777fe640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:98 7.0 7:05bf5b68:::notify.1:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f065c003d60 con 0x55a750ad9220 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.590+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(97 notify.1 [notify cookie 94177101759312 out=8b] v0'0 uv249 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.591+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(98 notify.1 [notify-ack] v0'0 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.591+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== watch-notify(notify_complete (2) cookie 94177101759312 notify 115964117041 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a750ab8890 con 0x55a750ad9220 2024-04-27T03:32:18.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.591+0000 7f079a606800 10 removing email index: 2024-04-27T03:32:18.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.591+0000 7f079a606800 10 removing user buckets index 2024-04-27T03:32:18.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.591+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:99 8.1e 8:79f34a7d:users.uid::system_user.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750b23710 con 0x7f0790016440 2024-04-27T03:32:18.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.592+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(99 system_user.buckets [delete] v27'2 uv1 ondisk = -2 ((2) No such file or directory)) ==== 163+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.592+0000 7f079a606800 10 removing user index: system_user 2024-04-27T03:32:18.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.592+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:100 8.1e 8:78ce4db0:users.uid::system_user:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750b278b0 con 0x7f0790016440 2024-04-27T03:32:18.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.593+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(100 system_user [call,call,delete] v27'3 uv3 ondisk = 0) ==== 239+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.593+0000 7f079a606800 10 removing default.rgw.meta+users.uid+system_user from cache 2024-04-27T03:32:18.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.593+0000 7f079a606800 10 distributing notification oid=default.rgw.control:notify.0 cni=[op: 1, obj: default.rgw.meta:users.uid:system_user, ofs0, ns] 2024-04-27T03:32:18.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.593+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:101 7.1f 7:f95f44c2:::notify.0:head [notify cookie 94177101771776 in=172b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55a750b24270 con 0x7f0788073060 2024-04-27T03:32:18.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.593+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== watch-notify(notify (1) cookie 94177101493808 notify 115964117018 ret 0) ==== 202+0+0 (crc 0 0 0) 0x55a750af6fa0 con 0x7f0788073060 2024-04-27T03:32:18.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.593+0000 7f0676ffd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117018 cookie 94177101493808 notifier 4840 bl.length()=160 2024-04-27T03:32:18.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.593+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(101 notify.0 [notify cookie 94177101771776 out=8b] v0'0 uv249 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.593+0000 7f0676ffd640 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:102 7.1f 7:f95f44c2:::notify.0:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f05cc0032c0 con 0x7f0788073060 2024-04-27T03:32:18.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.593+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 23 ==== osd_op_reply(102 notify.0 [notify-ack] v0'0 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.600 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.593+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 24 ==== watch-notify(notify_complete (2) cookie 94177101771776 notify 115964117018 ret 0) ==== 42+0+48 (crc 0 0 0) 0x55a750af7a10 con 0x7f0788073060 2024-04-27T03:32:18.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.594+0000 7f064e7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:18.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.594+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:103 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94177101520576] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af3870 con 0x55a750ad9220 2024-04-27T03:32:18.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.595+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(103 data_loggenerations_metadata [watch unwatch cookie 94177101520576] v27'255 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.595+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94177101493808] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750af3870 con 0x7f0788073060 2024-04-27T03:32:18.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.596+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 25 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94177101493808] v27'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.596+0000 7f079a606800 20 remove_watcher() i=0 2024-04-27T03:32:18.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.596+0000 7f079a606800 2 removed watcher, disabling cache 2024-04-27T03:32:18.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.596+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94177101498192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae3630 con 0x55a750ad9220 2024-04-27T03:32:18.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.598+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94177101498192] v27'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.598+0000 7f079a606800 20 remove_watcher() i=1 2024-04-27T03:32:18.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.598+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94177101503040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae3630 con 0x55a750addc10 2024-04-27T03:32:18.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.599+0000 7f0798105640 1 -- 172.21.15.165:0/2467100966 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94177101503040] v27'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0788070030 con 0x55a750addc10 2024-04-27T03:32:18.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.599+0000 7f079a606800 20 remove_watcher() i=2 2024-04-27T03:32:18.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.599+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94177101508528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae5a40 con 0x7f0788073060 2024-04-27T03:32:18.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.600+0000 7f0797904640 1 -- 172.21.15.165:0/2467100966 <== osd.2 v2:172.21.15.165:6816/975457786 26 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94177101508528] v27'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f078c081070 con 0x7f0788073060 2024-04-27T03:32:18.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.600+0000 7f079a606800 20 remove_watcher() i=3 2024-04-27T03:32:18.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.600+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94177101509648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae3630 con 0x55a750ad9220 2024-04-27T03:32:18.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.601+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94177101509648] v27'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x55a750ad9220 2024-04-27T03:32:18.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.601+0000 7f079a606800 20 remove_watcher() i=4 2024-04-27T03:32:18.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.601+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94177101515120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae7460 con 0x7f0790016440 2024-04-27T03:32:18.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.602+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94177101515120] v27'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.602+0000 7f079a606800 20 remove_watcher() i=5 2024-04-27T03:32:18.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.602+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94177101518768] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae5a40 con 0x7f0790016440 2024-04-27T03:32:18.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.603+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94177101518768] v27'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.603+0000 7f079a606800 20 remove_watcher() i=6 2024-04-27T03:32:18.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.603+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94177101512624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55a750ae3630 con 0x7f0790016440 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94177101512624] v27'251 uv249 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f079000a4a0 con 0x7f0790016440 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 20 remove_watcher() i=7 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0790016440 msgr2=0x7f07900168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0790016440 0x7f07900168c0 crc :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a750ad9220 msgr2=0x55a750adb6e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a750ad9220 0x55a750adb6e0 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0788073060 msgr2=0x7f07880490c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0788073060 0x7f07880490c0 crc :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a750addc10 msgr2=0x55a750ae0000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a750addc10 0x55a750ae0000 crc :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0668036950 msgr2=0x7f0668038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0668036950 0x7f0668038e10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x55a750aa9460 tx=0x7f078c04f000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 msgr2=0x55a750acdb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 --2- 172.21.15.165:0/2467100966 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0030 0x55a750acdb80 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f0788049aa0 tx=0x7f078801be90 comp rx=0 tx=0).stop 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 reap_dead start 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f0798906640 1 -- 172.21.15.165:0/2467100966 reap_dead start 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 shutdown_connections 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 >> 172.21.15.165:0/2467100966 conn(0x55a750ac08e0 msgr2=0x55a750abdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 shutdown_connections 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/2467100966 wait complete. 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/1285181654 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0670036900 msgr2=0x7f0670038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 --2- 172.21.15.165:0/1285181654 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0670036900 0x7f0670038dc0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x55a750aa0ad0 tx=0x7f078c005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/1285181654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a750ab74c0 msgr2=0x55a750ab78e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 --2- 172.21.15.165:0/1285181654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a750ab74c0 0x55a750ab78e0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f078804aca0 tx=0x7f0788016b00 comp rx=0 tx=0).stop 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/1285181654 shutdown_connections 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 --2- 172.21.15.165:0/1285181654 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0670036900 0x7f0670038dc0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 --2- 172.21.15.165:0/1285181654 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a750ab74c0 0x55a750ab78e0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.604+0000 7f079a606800 1 -- 172.21.15.165:0/1285181654 >> 172.21.15.165:0/1285181654 conn(0x55a750aab560 msgr2=0x55a750aab960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 -- 172.21.15.165:0/1285181654 shutdown_connections 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 -- 172.21.15.165:0/1285181654 wait complete. 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a7509a4480 msgr2=0x55a750a9fc90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55a7509a4480 0x55a750a9fc90 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a750aa64b0 msgr2=0x55a750aa8900 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55a750aa64b0 0x55a750aa8900 crc :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a750aa2520 msgr2=0x55a750aa4910 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55a750aa2520 0x55a750aa4910 crc :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0680036950 msgr2=0x7f0680038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0680036950 0x7f0680038e10 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f078c054000 tx=0x7f078c04e000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 msgr2=0x55a750a98a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 --2- 172.21.15.165:0/1493557843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55a7509a0970 0x55a750a98a50 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f078800c010 tx=0x7f0788000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f0798906640 1 -- 172.21.15.165:0/1493557843 reap_dead start 2024-04-27T03:32:18.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 shutdown_connections 2024-04-27T03:32:18.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 >> 172.21.15.165:0/1493557843 conn(0x55a75099a320 msgr2=0x55a750a8b030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 shutdown_connections 2024-04-27T03:32:18.615 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.605+0000 7f079a606800 1 -- 172.21.15.165:0/1493557843 wait complete. 2024-04-27T03:32:18.619 ERROR:root:None 2024-04-27T03:32:18.619 ERROR:root:0 2024-04-27T03:32:18.619 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'create', '--uid', 'a_user', '--display-name', 'True Admin User', '--access-key', 'true_admin_akey', '--secret', 'true_admin_skey', '--caps', 'user-info-without-keys=read', '--admin'] 2024-04-27T03:32:18.619 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'create', '--uid', 'a_user', '--display-name', 'True Admin User', '--access-key', 'true_admin_akey', '--secret', 'true_admin_skey', '--caps', 'user-info-without-keys=read', '--admin'] 2024-04-27T03:32:18.619 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user create --uid a_user --display-name 'True Admin User' --access-key true_admin_akey --secret true_admin_skey --caps user-info-without-keys=read --admin 2024-04-27T03:32:18.711 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:18.711 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:18.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.706+0000 7fa0612df800 1 Processor -- start 2024-04-27T03:32:18.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.707+0000 7fa0612df800 1 -- start start 2024-04-27T03:32:18.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.707+0000 7fa0612df800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x564975947360 0x564975949750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.707+0000 7fa0612df800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x564975801d40 con 0x564975943850 2024-04-27T03:32:18.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.707+0000 7fa0612df800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5649757d44d0 con 0x564975947360 2024-04-27T03:32:18.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.707+0000 7fa0565e4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x564975947360 0x564975949750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.707+0000 7fa05ede5640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x564975943850 0x564975945c40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42354/0 (socket says 172.21.15.165:42354) 2024-04-27T03:32:18.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.707+0000 7fa05ede5640 1 -- 172.21.15.165:0/1913695103 learned_addr learned my addr 172.21.15.165:0/1913695103 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:18.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.707+0000 7fa0565e4640 1 -- 172.21.15.165:0/1913695103 >> v1:172.21.15.89:6789/0 conn(0x564975943850 legacy=0x564975945c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.707+0000 7fa0565e4640 1 -- 172.21.15.165:0/1913695103 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x564975800590 con 0x564975947360 2024-04-27T03:32:18.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.707+0000 7fa05e5e4640 1 -- 172.21.15.165:0/1913695103 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1150293125 0 0) 0x564975801d40 con 0x564975943850 2024-04-27T03:32:18.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.709+0000 7fa0565e4640 1 --2- 172.21.15.165:0/1913695103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975947360 0x564975949750 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa04800a470 tx=0x7fa04801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a973af5b320a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.709+0000 7fa05e5e4640 1 -- 172.21.15.165:0/1913695103 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa048057070 con 0x564975947360 2024-04-27T03:32:18.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.709+0000 7fa05e5e4640 1 -- 172.21.15.165:0/1913695103 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa048052570 con 0x564975947360 2024-04-27T03:32:18.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.709+0000 7fa05e5e4640 1 -- 172.21.15.165:0/1913695103 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa048052870 con 0x564975947360 2024-04-27T03:32:18.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.710+0000 7fa0612df800 1 -- 172.21.15.165:0/1913695103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975947360 msgr2=0x564975949750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.710+0000 7fa0612df800 1 --2- 172.21.15.165:0/1913695103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975947360 0x564975949750 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7fa04800a470 tx=0x7fa04801ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:18.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.710+0000 7fa0612df800 1 -- 172.21.15.165:0/1913695103 shutdown_connections 2024-04-27T03:32:18.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.710+0000 7fa0612df800 1 --2- 172.21.15.165:0/1913695103 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975947360 0x564975949750 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.710+0000 7fa0612df800 1 -- 172.21.15.165:0/1913695103 >> 172.21.15.165:0/1913695103 conn(0x56497593d320 msgr2=0x56497593f740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.710+0000 7fa0612df800 1 -- 172.21.15.165:0/1913695103 shutdown_connections 2024-04-27T03:32:18.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.710+0000 7fa0612df800 1 -- 172.21.15.165:0/1913695103 wait complete. 2024-04-27T03:32:18.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.722+0000 7fa0612df800 1 Processor -- start 2024-04-27T03:32:18.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.722+0000 7fa0612df800 1 -- start start 2024-04-27T03:32:18.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.722+0000 7fa0612df800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 0x564975943190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.722+0000 7fa0612df800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x56497588e360 con 0x564975943850 2024-04-27T03:32:18.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.722+0000 7fa05ede5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 0x564975943190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.722+0000 7fa05ede5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 0x564975943190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60122/0 (socket says 172.21.15.165:60122) 2024-04-27T03:32:18.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.722+0000 7fa05ede5640 1 -- 172.21.15.165:0/1328339611 learned_addr learned my addr 172.21.15.165:0/1328339611 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.722+0000 7fa05ede5640 1 -- 172.21.15.165:0/1328339611 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564975941750 con 0x564975943850 2024-04-27T03:32:18.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.723+0000 7fa05ede5640 1 --2- 172.21.15.165:0/1328339611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 0x564975943190 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa05000b210 tx=0x7fa05000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc1010e3d54d0020 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.723+0000 7f9f4ce09640 1 -- 172.21.15.165:0/1328339611 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050008070 con 0x564975943850 2024-04-27T03:32:18.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.723+0000 7f9f4ce09640 1 -- 172.21.15.165:0/1328339611 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05000d500 con 0x564975943850 2024-04-27T03:32:18.729 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.723+0000 7f9f4ce09640 1 -- 172.21.15.165:0/1328339611 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa05000d840 con 0x564975943850 2024-04-27T03:32:18.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.723+0000 7fa0612df800 1 -- 172.21.15.165:0/1328339611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 msgr2=0x564975943190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.723+0000 7fa0612df800 1 --2- 172.21.15.165:0/1328339611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 0x564975943190 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7fa05000b210 tx=0x7fa05000b6d0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.723+0000 7fa0612df800 1 -- 172.21.15.165:0/1328339611 shutdown_connections 2024-04-27T03:32:18.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.723+0000 7fa0612df800 1 --2- 172.21.15.165:0/1328339611 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 0x564975943190 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.723+0000 7fa0612df800 1 -- 172.21.15.165:0/1328339611 >> 172.21.15.165:0/1328339611 conn(0x56497593d320 msgr2=0x564975a2f840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.723+0000 7fa0612df800 1 -- 172.21.15.165:0/1328339611 shutdown_connections 2024-04-27T03:32:18.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.724+0000 7fa0612df800 1 -- 172.21.15.165:0/1328339611 wait complete. 2024-04-27T03:32:18.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.724+0000 7fa0612df800 1 Processor -- start 2024-04-27T03:32:18.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.724+0000 7fa0612df800 1 -- start start 2024-04-27T03:32:18.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.724+0000 7fa0612df800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 0x564975a3dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.724+0000 7fa0612df800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564975949f90 con 0x564975943850 2024-04-27T03:32:18.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.724+0000 7fa05ede5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 0x564975a3dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.724+0000 7fa05ede5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 0x564975a3dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60128/0 (socket says 172.21.15.165:60128) 2024-04-27T03:32:18.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.724+0000 7fa05ede5640 1 -- 172.21.15.165:0/2649542396 learned_addr learned my addr 172.21.15.165:0/2649542396 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.724+0000 7fa05ede5640 1 -- 172.21.15.165:0/2649542396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564975a3e1d0 con 0x564975943850 2024-04-27T03:32:18.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.725+0000 7fa05ede5640 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 0x564975a3dc90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa05000b210 tx=0x7fa050000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.725+0000 7f9f4b606640 1 -- 172.21.15.165:0/2649542396 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa05000f950 con 0x564975943850 2024-04-27T03:32:18.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.725+0000 7f9f4b606640 1 -- 172.21.15.165:0/2649542396 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05000faf0 con 0x564975943850 2024-04-27T03:32:18.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.725+0000 7f9f4b606640 1 -- 172.21.15.165:0/2649542396 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa05000fdd0 con 0x564975943850 2024-04-27T03:32:18.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.725+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564975a3e460 con 0x564975943850 2024-04-27T03:32:18.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.725+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564975a41110 con 0x564975943850 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.726+0000 7f9f4b606640 1 -- 172.21.15.165:0/2649542396 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa050017040 con 0x564975943850 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.726+0000 7f9f4b606640 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f44036900 0x7f9f44038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.726+0000 7f9f4b606640 1 -- 172.21.15.165:0/2649542396 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fa050015020 con 0x564975943850 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.726+0000 7fa0565e4640 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f44036900 0x7f9f44038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.726+0000 7fa0612df800 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564975947360 0x564975a44ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.726+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a45410 con 0x564975947360 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.726+0000 7fa05f5e6640 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564975947360 0x564975a44ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.726+0000 7fa0565e4640 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f44036900 0x7f9f44038dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa048054000 tx=0x7fa04804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.726+0000 7fa05f5e6640 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564975947360 0x564975a44ed0 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.727+0000 7fa05f5e6640 1 -- 172.21.15.165:0/2649542396 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975947360 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.727+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a424a0 con 0x564975947360 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.727+0000 7fa05f5e6640 1 -- 172.21.15.165:0/2649542396 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975947360 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.727+0000 7fa0612df800 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564975a47790 0x564975a49b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.727+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a4a220 con 0x564975a47790 2024-04-27T03:32:18.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.727+0000 7fa05ede5640 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564975a47790 0x564975a49b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.727+0000 7fa05ede5640 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564975a47790 0x564975a49b80 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.728+0000 7fa05ede5640 1 -- 172.21.15.165:0/2649542396 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fa05002d070 con 0x564975a47790 2024-04-27T03:32:18.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.728+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a46740 con 0x564975947360 2024-04-27T03:32:18.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.728+0000 7fa05f5e6640 1 -- 172.21.15.165:0/2649542396 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975947360 2024-04-27T03:32:18.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.728+0000 7fa0612df800 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x564975a4b6d0 0x564975a4db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.728+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a4e280 con 0x564975a4b6d0 2024-04-27T03:32:18.748 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.728+0000 7fa0565e4640 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x564975a4b6d0 0x564975a4db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.728+0000 7fa0565e4640 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x564975a4b6d0 0x564975a4db20 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.729+0000 7fa0565e4640 1 -- 172.21.15.165:0/2649542396 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fa048052940 con 0x564975a4b6d0 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.729+0000 7fa0612df800 1 Processor -- start 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.729+0000 7fa0612df800 1 -- start start 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.729+0000 7fa0612df800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a5c690 0x564975a5cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.729+0000 7fa0612df800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5649758346b0 con 0x564975a5c690 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.730+0000 7fa05ede5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a5c690 0x564975a5cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.730+0000 7fa05ede5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a5c690 0x564975a5cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60138/0 (socket says 172.21.15.165:60138) 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.730+0000 7fa05ede5640 1 -- 172.21.15.165:0/268516989 learned_addr learned my addr 172.21.15.165:0/268516989 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.730+0000 7fa05ede5640 1 -- 172.21.15.165:0/268516989 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564975a5cff0 con 0x564975a5c690 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.730+0000 7fa05ede5640 1 --2- 172.21.15.165:0/268516989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a5c690 0x564975a5cab0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa05004b4b0 tx=0x7fa05001aaf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.730+0000 7f9f48e01640 1 -- 172.21.15.165:0/268516989 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050019470 con 0x564975a5c690 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.730+0000 7f9f48e01640 1 -- 172.21.15.165:0/268516989 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa050019610 con 0x564975a5c690 2024-04-27T03:32:18.749 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.730+0000 7f9f48e01640 1 -- 172.21.15.165:0/268516989 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0500198f0 con 0x564975a5c690 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.730+0000 7fa0612df800 1 -- 172.21.15.165:0/268516989 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564975a5da60 con 0x564975a5c690 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.730+0000 7fa0612df800 1 -- 172.21.15.165:0/268516989 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564975a5df60 con 0x564975a5c690 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.731+0000 7f9f48e01640 1 -- 172.21.15.165:0/268516989 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa05003f1b0 con 0x564975a5c690 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.731+0000 7f9f48e01640 1 --2- 172.21.15.165:0/268516989 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f38036970 0x7f9f38038e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.731+0000 7f9f48e01640 1 -- 172.21.15.165:0/268516989 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fa05003cb90 con 0x564975a5c690 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.731+0000 7fa0565e4640 1 --2- 172.21.15.165:0/268516989 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f38036970 0x7f9f38038e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.731+0000 7fa0612df800 1 Processor -- start 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.731+0000 7fa0612df800 1 -- start start 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.732+0000 7fa0565e4640 1 --2- 172.21.15.165:0/268516989 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f38036970 0x7f9f38038e30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x564975a423c0 tx=0x7fa048005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.732+0000 7fa0612df800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 0x564975a69be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.732+0000 7fa0612df800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564975941ad0 con 0x564975a697c0 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.732+0000 7fa05ede5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 0x564975a69be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.750 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.732+0000 7fa05ede5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 0x564975a69be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60146/0 (socket says 172.21.15.165:60146) 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.732+0000 7fa05ede5640 1 -- 172.21.15.165:0/2643896843 learned_addr learned my addr 172.21.15.165:0/2643896843 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.732+0000 7fa05ede5640 1 -- 172.21.15.165:0/2643896843 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564975a6a1b0 con 0x564975a697c0 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.732+0000 7fa05ede5640 1 --2- 172.21.15.165:0/2643896843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 0x564975a69be0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa0500266c0 tx=0x7fa050026430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a76195d751f8081 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.733+0000 7f9f427fc640 1 -- 172.21.15.165:0/2643896843 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa0500269d0 con 0x564975a697c0 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.733+0000 7f9f427fc640 1 -- 172.21.15.165:0/2643896843 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa050040e20 con 0x564975a697c0 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.733+0000 7f9f427fc640 1 -- 172.21.15.165:0/2643896843 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050041dd0 con 0x564975a697c0 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.733+0000 7fa0612df800 1 -- 172.21.15.165:0/2643896843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 msgr2=0x564975a69be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.733+0000 7fa0612df800 1 --2- 172.21.15.165:0/2643896843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 0x564975a69be0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fa0500266c0 tx=0x7fa050026430 comp rx=0 tx=0).stop 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.733+0000 7fa0612df800 1 -- 172.21.15.165:0/2643896843 shutdown_connections 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.733+0000 7fa0612df800 1 --2- 172.21.15.165:0/2643896843 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 0x564975a69be0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.733+0000 7fa0612df800 1 -- 172.21.15.165:0/2643896843 >> 172.21.15.165:0/2643896843 conn(0x564975a65ae0 msgr2=0x564975a63b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.733+0000 7fa0612df800 1 -- 172.21.15.165:0/2643896843 shutdown_connections 2024-04-27T03:32:18.751 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.733+0000 7fa0612df800 1 -- 172.21.15.165:0/2643896843 wait complete. 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.733+0000 7fa0612df800 1 Processor -- start 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.733+0000 7fa0612df800 1 -- start start 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.734+0000 7fa0612df800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 0x564975a70c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.734+0000 7fa0612df800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x564975a6a8d0 con 0x564975a697c0 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.734+0000 7fa05ede5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 0x564975a70c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.734+0000 7fa05ede5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 0x564975a70c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60162/0 (socket says 172.21.15.165:60162) 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.734+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 learned_addr learned my addr 172.21.15.165:0/965431310 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.734+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564975a711b0 con 0x564975a697c0 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.734+0000 7fa05ede5640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 0x564975a70c70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa05003ce00 tx=0x7fa050027210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.734+0000 7f9f40ff9640 1 -- 172.21.15.165:0/965431310 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050044830 con 0x564975a697c0 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.734+0000 7f9f40ff9640 1 -- 172.21.15.165:0/965431310 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa05005b050 con 0x564975a697c0 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.734+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x564975a711b0 con 0x564975a697c0 2024-04-27T03:32:18.752 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.734+0000 7f9f40ff9640 1 -- 172.21.15.165:0/965431310 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa050040a30 con 0x564975a697c0 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.734+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564975a711b0 con 0x564975a697c0 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.735+0000 7f9f40ff9640 1 -- 172.21.15.165:0/965431310 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa050026be0 con 0x564975a697c0 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.735+0000 7f9f40ff9640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f280368b0 0x7f9f28038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.735+0000 7f9f40ff9640 1 -- 172.21.15.165:0/965431310 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fa050069030 con 0x564975a697c0 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.735+0000 7fa0565e4640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f280368b0 0x7f9f28038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.736+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.736+0000 7fa0565e4640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f280368b0 0x7f9f28038d70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa048052fd0 tx=0x7fa04804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.736+0000 7fa0612df800 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564975a7c320 0x564975a7e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.736+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ef30 con 0x564975a7c320 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.736+0000 7fa05f5e6640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564975a7c320 0x564975a7e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.736+0000 7fa05f5e6640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564975a7c320 0x564975a7e7e0 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.736+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.736+0000 7fa0612df800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.736+0000 7fa0612df800 20 realm 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.736+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.736+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a7f340 con 0x564975a7c320 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa0612df800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa0612df800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a7fc10 con 0x564975a7c320 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa0612df800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a7c320 2024-04-27T03:32:18.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa0612df800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa0612df800 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564975a80d10 0x564975a83100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a83710 con 0x564975a80d10 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa05ede5640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564975a80d10 0x564975a83100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.737+0000 7fa05ede5640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564975a80d10 0x564975a83100 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.738+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.738+0000 7fa0612df800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.738+0000 7fa0612df800 20 searching for the correct realm 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.738+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a85040 con 0x564975a80d10 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.738+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.738+0000 7fa05ede5640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa05003c750 0x7fa050082730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.738+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a85040 con 0x7fa05003c750 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.738+0000 7fa0565e4640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa05003c750 0x7fa050082730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.739+0000 7fa0565e4640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa05003c750 0x7fa050082730 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.739+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.739+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a85040 con 0x7fa05003c750 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.739+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.739+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a85040 con 0x564975a7c320 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.739+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.739+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a7c320 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.740+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.740+0000 7fa05f5e6640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa058016440 0x7fa0580168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.740+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa058016440 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.740+0000 7fa05f5e6640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa058016440 0x7fa0580168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.740+0000 7fa05f5e6640 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa058016440 0x7fa0580168c0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.741+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.741+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa058016440 2024-04-27T03:32:18.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.741+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.741+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa05003c750 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.741+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.741+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a80d10 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.741+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.741+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a7c320 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.742+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.742+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa05003c750 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.742+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.742+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a7c320 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.742+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.742+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa05003c750 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.742+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.742+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a80d10 2024-04-27T03:32:18.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.742+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.742+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a80d10 2024-04-27T03:32:18.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.743+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.743+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa058016440 2024-04-27T03:32:18.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.743+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.743+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a80d10 2024-04-27T03:32:18.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.744+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.744+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a80d10 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.744+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.744+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a7c320 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.744+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.744+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a7c320 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.744+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.744+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa058016440 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.744+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.745+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a80d10 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.745+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.745+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa05003c750 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.745+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.745+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a7c320 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.745+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.745+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa058016440 2024-04-27T03:32:18.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.745+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.745+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a80d10 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.746+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.746+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa058016440 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.746+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.746+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa05003c750 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.746+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.746+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a80d10 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.746+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.746+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa058016440 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x564975a7c320 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x564975a7ffb0 con 0x7fa05003c750 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa0612df800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa0612df800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa0612df800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa0612df800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa0612df800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa0612df800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a83fd0 con 0x564975a7c320 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa0612df800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.747+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a843c0 con 0x564975a7c320 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a843c0 con 0x7fa05003c750 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 20 zone default found 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 4 Realm: () 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 20 zonegroup default 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.774 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a854c0 con 0x7fa058016440 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.748+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a86ea0 con 0x564975a80d10 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.749+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.749+0000 7fa0612df800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.749+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.749+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a87360 con 0x564975a80d10 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.749+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.749+0000 7fa0612df800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.749+0000 7fa0612df800 20 started sync module instance, tier type = 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.749+0000 7fa0612df800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:18.775 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.749+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a86340 con 0x7fa05003c750 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.750+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.750+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94873506572080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a85fa0 con 0x7fa05003c750 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.750+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a88180 con 0x564975a7c320 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.751+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94873506572080] v27'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.752+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.752+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94873506576320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a885a0 con 0x564975a7c320 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.752+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a88fa0 con 0x564975a80d10 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.753+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.753+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94873506581168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a89870 con 0x564975a80d10 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.753+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94873506576320] v27'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.753+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a87c40 con 0x7fa05003c750 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.754+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.754+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94873506586640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8aa20 con 0x7fa05003c750 2024-04-27T03:32:18.776 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.754+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8b4b0 con 0x564975a7c320 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.754+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94873506581168] v27'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.755+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94873506586640] v27'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.755+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.755+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94873506590592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8a470 con 0x564975a7c320 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.755+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8bec0 con 0x7fa058016440 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.756+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94873506590592] v27'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.756+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.756+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94873506583488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8bec0 con 0x7fa058016440 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.756+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8c880 con 0x7fa058016440 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.757+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94873506583488] v27'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.757+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.757+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94873506593776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8c880 con 0x7fa058016440 2024-04-27T03:32:18.777 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.757+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8d240 con 0x7fa058016440 2024-04-27T03:32:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.758+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94873506593776] v27'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.758+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'252 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.758+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94873506596272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8d240 con 0x7fa058016440 2024-04-27T03:32:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.758+0000 7fa0612df800 20 add_watcher() i=0 2024-04-27T03:32:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.758+0000 7fa0612df800 20 add_watcher() i=1 2024-04-27T03:32:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.758+0000 7fa0612df800 20 add_watcher() i=2 2024-04-27T03:32:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.758+0000 7fa0612df800 20 add_watcher() i=3 2024-04-27T03:32:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.758+0000 7fa0612df800 20 add_watcher() i=4 2024-04-27T03:32:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.758+0000 7fa0612df800 20 add_watcher() i=5 2024-04-27T03:32:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.758+0000 7fa0612df800 20 add_watcher() i=6 2024-04-27T03:32:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.759+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94873506596272] v27'253 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.778 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.759+0000 7fa0612df800 20 add_watcher() i=7 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.759+0000 7fa0612df800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.759+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975a90540 con 0x564975a7c320 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.760+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.760+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94873506598656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8d7d0 con 0x564975a7c320 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.761+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94873506598656] v27'256 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.761+0000 7f9f157e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.761+0000 7fa0612df800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.761+0000 7fa0612df800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.761+0000 7fa0612df800 5 note: GC not initialized 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.761+0000 7f9e8ffff640 20 reqs_thread_entry: start 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.761+0000 7fa0612df800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.761+0000 7fa0612df800 20 Filter name: none 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.761+0000 7f9e8dffb640 20 reqs_thread_entry: start 2024-04-27T03:32:18.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.761+0000 7fa0612df800 10 cache get: name=default.rgw.meta+users.uid+a_user : miss 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.761+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.761+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:62 8.14 8:2efb937b:users.uid::a_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975ac3910 con 0x564975a80d10 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(62 a_user [call,read 0~0,getxattrs] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 234+0+0 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa0612df800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa0612df800 10 cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x0 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa0612df800 10 adding default.rgw.meta+users.uid+a_user to cache LRU end 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa0612df800 10 cache get: name=default.rgw.meta+users.keys+true_admin_akey : miss 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa0612df800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:63 8.15 8:afe09604:users.keys::true_admin_akey:head [stat,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975ac2560 con 0x564975a80d10 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(63 true_admin_akey [stat,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 201+0+0 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa0612df800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa0612df800 10 cache put: name=default.rgw.meta+users.keys+true_admin_akey info.flags=0x0 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa0612df800 10 adding default.rgw.meta+users.keys+true_admin_akey to cache LRU end 2024-04-27T03:32:18.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa0612df800 10 cache get: name=default.rgw.meta+users.keys+true_admin_akey : hit (negative entry) 2024-04-27T03:32:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa0612df800 10 cache get: name=default.rgw.meta+users.keys+true_admin_akey : hit (negative entry) 2024-04-27T03:32:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.762+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 8.14 8:2efb937b:users.uid::a_user:head [delete,create,call version.set in=58b,writefull 0~363 in=363b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975ac2560 con 0x564975a80d10 2024-04-27T03:32:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.763+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(64 a_user [delete,create,call,writefull 0~363] v27'3 uv3 ondisk = 0) ==== 276+0+0 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.763+0000 7fa0612df800 10 cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x17 2024-04-27T03:32:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.763+0000 7fa0612df800 10 moving default.rgw.meta+users.uid+a_user to cache LRU end 2024-04-27T03:32:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.763+0000 7fa0612df800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 0, obj: default.rgw.meta:users.uid:a_user, ofs0, ns] 2024-04-27T03:32:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.763+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:65 7.15 7:a93a5511:::notify.2:head [notify cookie 94873506816480 in=554b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975ac7410 con 0x564975a80d10 2024-04-27T03:32:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.763+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== watch-notify(notify (1) cookie 94873506581168 notify 115964117001 ret 0) ==== 584+0+0 (crc 0 0 0) 0x564975ac2b80 con 0x564975a80d10 2024-04-27T03:32:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.763+0000 7f9f427fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117001 cookie 94873506581168 notifier 4847 bl.length()=542 2024-04-27T03:32:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.763+0000 7f9f427fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x17 2024-04-27T03:32:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.763+0000 7f9f427fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+a_user to cache LRU end 2024-04-27T03:32:18.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.763+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(65 notify.2 [notify cookie 94873506816480 out=8b] v0'0 uv252 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.763+0000 7f9f427fc640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:66 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f9f24003ea0 con 0x564975a80d10 2024-04-27T03:32:18.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.764+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(66 notify.2 [notify-ack] v0'0 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.764+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== watch-notify(notify_complete (2) cookie 94873506816480 notify 115964117001 ret 0) ==== 42+0+48 (crc 0 0 0) 0x564975a5da60 con 0x564975a80d10 2024-04-27T03:32:18.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.764+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 8.15 8:afe09604:users.keys::true_admin_akey:head [delete,create,writefull 0~10 in=10b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975ac21b0 con 0x564975a80d10 2024-04-27T03:32:18.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.766+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(67 true_admin_akey [delete,create,writefull 0~10] v27'3 uv3 ondisk = 0) ==== 243+0+0 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.766+0000 7fa0612df800 10 cache put: name=default.rgw.meta+users.keys+true_admin_akey info.flags=0x7 2024-04-27T03:32:18.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.766+0000 7fa0612df800 10 moving default.rgw.meta+users.keys+true_admin_akey to cache LRU end 2024-04-27T03:32:18.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.766+0000 7fa0612df800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 0, obj: default.rgw.meta:users.keys:true_admin_akey, ofs0, ns] 2024-04-27T03:32:18.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.766+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:68 7.3 7:c609908c:::notify.5:head [notify cookie 94873506816432 in=187b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x564975ac7840 con 0x7fa058016440 2024-04-27T03:32:18.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.766+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== watch-notify(notify (1) cookie 94873506583488 notify 115964117022 ret 0) ==== 217+0+0 (crc 0 0 0) 0x7fa058030040 con 0x7fa058016440 2024-04-27T03:32:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.766+0000 7f9f427fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117022 cookie 94873506583488 notifier 4847 bl.length()=175 2024-04-27T03:32:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.766+0000 7f9f427fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.keys+true_admin_akey info.flags=0x7 2024-04-27T03:32:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.766+0000 7f9f427fc640 10 rgw watcher librados: moving default.rgw.meta+users.keys+true_admin_akey to cache LRU end 2024-04-27T03:32:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.766+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(68 notify.5 [notify cookie 94873506816432 out=8b] v0'0 uv252 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.766+0000 7f9f427fc640 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f9f24003ea0 con 0x7fa058016440 2024-04-27T03:32:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.767+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(69 notify.5 [notify-ack] v0'0 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.767+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== watch-notify(notify_complete (2) cookie 94873506816432 notify 115964117022 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fa05802a510 con 0x7fa058016440 2024-04-27T03:32:18.784 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"a_user","display_name":"True Admin User","email":"","suspended":0,"max_buckets":1000,"subusers":[],"keys":[{"user":"a_user","access_key":"true_admin_akey","secret_key":"true_admin_skey","active":true,"create_date":"2024-04-27T03:32:18.763382Z"}],"swift_keys":[],"caps":[{"type":"user-info-without-keys","perm":"read"}],"op_mask":"read, write, delete","admin":true,"default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:32:18.763362Z","tags":[],"group_ids":[]} 2024-04-27T03:32:18.789 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.767+0000 7f9f157e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.767+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:70 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94873506598656] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a78a30 con 0x564975a7c320 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.768+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(70 data_loggenerations_metadata [watch unwatch cookie 94873506598656] v27'257 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.769+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:71 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94873506572080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8cf00 con 0x7fa05003c750 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.770+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(71 notify.0 [watch unwatch cookie 94873506572080] v27'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.770+0000 7fa0612df800 20 remove_watcher() i=0 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.770+0000 7fa0612df800 2 removed watcher, disabling cache 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.770+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:72 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94873506576320] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8cf00 con 0x564975a7c320 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.771+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(72 notify.1 [watch unwatch cookie 94873506576320] v27'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.771+0000 7fa0612df800 20 remove_watcher() i=1 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.771+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:73 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94873506581168] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a877c0 con 0x564975a80d10 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.772+0000 7fa05ede5640 1 -- 172.21.15.165:0/965431310 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(73 notify.2 [watch unwatch cookie 94873506581168] v27'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa050048040 con 0x564975a80d10 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.772+0000 7fa0612df800 20 remove_watcher() i=2 2024-04-27T03:32:18.790 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.772+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94873506586640] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8cf00 con 0x7fa05003c750 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.773+0000 7fa0565e4640 1 -- 172.21.15.165:0/965431310 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(74 notify.3 [watch unwatch cookie 94873506586640] v27'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa048081070 con 0x7fa05003c750 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.773+0000 7fa0612df800 20 remove_watcher() i=3 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.773+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:75 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94873506590592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a877c0 con 0x564975a7c320 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.774+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(75 notify.4 [watch unwatch cookie 94873506590592] v27'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x564975a7c320 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.774+0000 7fa0612df800 20 remove_watcher() i=4 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.774+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:76 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94873506583488] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8cf00 con 0x7fa058016440 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.776+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(76 notify.5 [watch unwatch cookie 94873506583488] v27'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.776+0000 7fa0612df800 20 remove_watcher() i=5 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.776+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:77 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94873506593776] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a8af80 con 0x7fa058016440 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.777+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(77 notify.6 [watch unwatch cookie 94873506593776] v27'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.777+0000 7fa0612df800 20 remove_watcher() i=6 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.777+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:78 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94873506596272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x564975a877c0 con 0x7fa058016440 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(78 notify.7 [watch unwatch cookie 94873506596272] v27'254 uv252 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fa05800a4a0 con 0x7fa058016440 2024-04-27T03:32:18.791 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 20 remove_watcher() i=7 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa058016440 msgr2=0x7fa0580168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fa058016440 0x7fa0580168c0 crc :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564975a7c320 msgr2=0x564975a7e7e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564975a7c320 0x564975a7e7e0 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa05003c750 msgr2=0x7fa050082730 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fa05003c750 0x7fa050082730 crc :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564975a80d10 msgr2=0x564975a83100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564975a80d10 0x564975a83100 crc :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f280368b0 msgr2=0x7f9f28038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f280368b0 0x7f9f28038d70 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7fa048052fd0 tx=0x7fa04804f000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 msgr2=0x564975a70c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 0x564975a70c70 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7fa05003ce00 tx=0x7fa050027210 comp rx=0 tx=0).stop 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa05f5e6640 1 -- 172.21.15.165:0/965431310 reap_dead start 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 shutdown_connections 2024-04-27T03:32:18.792 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 --2- 172.21.15.165:0/965431310 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a697c0 0x564975a70c70 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 >> 172.21.15.165:0/965431310 conn(0x564975a65ae0 msgr2=0x564975a63140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 shutdown_connections 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.778+0000 7fa0612df800 1 -- 172.21.15.165:0/965431310 wait complete. 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/268516989 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f38036970 msgr2=0x7f9f38038e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 --2- 172.21.15.165:0/268516989 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f38036970 0x7f9f38038e30 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x564975a423c0 tx=0x7fa048005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/268516989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a5c690 msgr2=0x564975a5cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 --2- 172.21.15.165:0/268516989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a5c690 0x564975a5cab0 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7fa05004b4b0 tx=0x7fa05001aaf0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/268516989 shutdown_connections 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 --2- 172.21.15.165:0/268516989 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f38036970 0x7f9f38038e30 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 --2- 172.21.15.165:0/268516989 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975a5c690 0x564975a5cab0 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/268516989 >> 172.21.15.165:0/268516989 conn(0x564975a50730 msgr2=0x564975a50b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/268516989 shutdown_connections 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/268516989 wait complete. 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564975947360 msgr2=0x564975a44ed0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.793 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x564975947360 0x564975a44ed0 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x564975a4b6d0 msgr2=0x564975a4db20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x564975a4b6d0 0x564975a4db20 crc :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564975a47790 msgr2=0x564975a49b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x564975a47790 0x564975a49b80 crc :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f44036900 msgr2=0x7f9f44038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f9f44036900 0x7f9f44038dc0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa048054000 tx=0x7fa04804e000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 msgr2=0x564975a3dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 --2- 172.21.15.165:0/2649542396 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x564975943850 0x564975a3dc90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7fa05000b210 tx=0x7fa050000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa05f5e6640 1 -- 172.21.15.165:0/2649542396 reap_dead start 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 shutdown_connections 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 >> 172.21.15.165:0/2649542396 conn(0x56497593d320 msgr2=0x564975942030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 shutdown_connections 2024-04-27T03:32:18.794 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.779+0000 7fa0612df800 1 -- 172.21.15.165:0/2649542396 wait complete. 2024-04-27T03:32:18.795 DEBUG:tasks.util.rgw: json result: {'user_id': 'a_user', 'display_name': 'True Admin User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'a_user', 'access_key': 'true_admin_akey', 'secret_key': 'true_admin_skey', 'active': True, 'create_date': '2024-04-27T03:32:18.763382Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'admin': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.763362Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:18.795 ERROR:root:{'user_id': 'a_user', 'display_name': 'True Admin User', 'email': '', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'a_user', 'access_key': 'true_admin_akey', 'secret_key': 'true_admin_skey', 'active': True, 'create_date': '2024-04-27T03:32:18.763382Z'}], 'swift_keys': [], 'caps': [{'type': 'user-info-without-keys', 'perm': 'read'}], 'op_mask': 'read, write, delete', 'admin': True, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.763362Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:18.796 ERROR:root:0 2024-04-27T03:32:18.796 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:18.801 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:18.801 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'rm', '--uid', 'a_user'] 2024-04-27T03:32:18.801 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'rm', '--uid', 'a_user'] 2024-04-27T03:32:18.801 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user rm --uid a_user 2024-04-27T03:32:18.844 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:18.844 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:18.845 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.839+0000 7f0d4de19800 1 Processor -- start 2024-04-27T03:32:18.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.839+0000 7f0d4de19800 1 -- start start 2024-04-27T03:32:18.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.839+0000 7f0d4de19800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x559681e4b990 0x559681e4dd80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.839+0000 7f0d4de19800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x559681d09d40 con 0x559681e4e2c0 2024-04-27T03:32:18.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.839+0000 7f0d4de19800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x559681cdc4d0 con 0x559681e4b990 2024-04-27T03:32:18.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.839+0000 7f0d4b918640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x559681e4b990 0x559681e4dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.839+0000 7f0d4b117640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x559681e4e2c0 0x559681e506b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42358/0 (socket says 172.21.15.165:42358) 2024-04-27T03:32:18.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.840+0000 7f0d4b117640 1 -- 172.21.15.165:0/1782891747 learned_addr learned my addr 172.21.15.165:0/1782891747 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:18.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.840+0000 7f0d4a916640 1 -- 172.21.15.165:0/1782891747 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4209221932 0 0) 0x559681d09d40 con 0x559681e4e2c0 2024-04-27T03:32:18.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.840+0000 7f0d4a916640 1 -- 172.21.15.165:0/1782891747 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f0d34001e10 con 0x559681e4e2c0 2024-04-27T03:32:18.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.840+0000 7f0d4b918640 1 -- 172.21.15.165:0/1782891747 >> v1:172.21.15.89:6789/0 conn(0x559681e4e2c0 legacy=0x559681e506b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.846 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.840+0000 7f0d4b918640 1 -- 172.21.15.165:0/1782891747 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x559681d08590 con 0x559681e4b990 2024-04-27T03:32:18.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.842+0000 7f0d4b918640 1 --2- 172.21.15.165:0/1782891747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681e4dd80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00a470 tx=0x7f0d3c01cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea84802297e2742a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.842+0000 7f0d4a916640 1 -- 172.21.15.165:0/1782891747 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c057070 con 0x559681e4b990 2024-04-27T03:32:18.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.842+0000 7f0d4a916640 1 -- 172.21.15.165:0/1782891747 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3c052570 con 0x559681e4b990 2024-04-27T03:32:18.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.842+0000 7f0d4a916640 1 -- 172.21.15.165:0/1782891747 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c052870 con 0x559681e4b990 2024-04-27T03:32:18.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.842+0000 7f0d4de19800 1 -- 172.21.15.165:0/1782891747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 msgr2=0x559681e4dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.842+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1782891747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681e4dd80 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00a470 tx=0x7f0d3c01cdb0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.842+0000 7f0d4de19800 1 -- 172.21.15.165:0/1782891747 shutdown_connections 2024-04-27T03:32:18.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.842+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1782891747 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681e4dd80 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.842+0000 7f0d4de19800 1 -- 172.21.15.165:0/1782891747 >> 172.21.15.165:0/1782891747 conn(0x559681e45320 msgr2=0x559681e47740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.842+0000 7f0d4de19800 1 -- 172.21.15.165:0/1782891747 shutdown_connections 2024-04-27T03:32:18.849 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.842+0000 7f0d4de19800 1 -- 172.21.15.165:0/1782891747 wait complete. 2024-04-27T03:32:18.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.854+0000 7f0d4de19800 1 Processor -- start 2024-04-27T03:32:18.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.854+0000 7f0d4de19800 1 -- start start 2024-04-27T03:32:18.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.854+0000 7f0d4de19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681f3bf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.854+0000 7f0d4de19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559681d963c0 con 0x559681e4b990 2024-04-27T03:32:18.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.855+0000 7f0d4b918640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681f3bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.855+0000 7f0d4b918640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681f3bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60190/0 (socket says 172.21.15.165:60190) 2024-04-27T03:32:18.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.855+0000 7f0d4b918640 1 -- 172.21.15.165:0/1704471794 learned_addr learned my addr 172.21.15.165:0/1704471794 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.855+0000 7f0d4b918640 1 -- 172.21.15.165:0/1704471794 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559681f3c4e0 con 0x559681e4b990 2024-04-27T03:32:18.861 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.855+0000 7f0d4b918640 1 --2- 172.21.15.165:0/1704471794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681f3bf10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c055040 tx=0x7f0d3c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8af53923a72208d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.855+0000 7f0c39e0b640 1 -- 172.21.15.165:0/1704471794 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c057070 con 0x559681e4b990 2024-04-27T03:32:18.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.855+0000 7f0c39e0b640 1 -- 172.21.15.165:0/1704471794 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3c063070 con 0x559681e4b990 2024-04-27T03:32:18.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.855+0000 7f0c39e0b640 1 -- 172.21.15.165:0/1704471794 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c052720 con 0x559681e4b990 2024-04-27T03:32:18.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.856+0000 7f0d4de19800 1 -- 172.21.15.165:0/1704471794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 msgr2=0x559681f3bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.856+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1704471794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681f3bf10 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c055040 tx=0x7f0d3c04c000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.856+0000 7f0d4de19800 1 -- 172.21.15.165:0/1704471794 shutdown_connections 2024-04-27T03:32:18.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.856+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1704471794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681f3bf10 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.856+0000 7f0d4de19800 1 -- 172.21.15.165:0/1704471794 >> 172.21.15.165:0/1704471794 conn(0x559681e45320 msgr2=0x559681f37830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.856+0000 7f0d4de19800 1 -- 172.21.15.165:0/1704471794 shutdown_connections 2024-04-27T03:32:18.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.856+0000 7f0d4de19800 1 -- 172.21.15.165:0/1704471794 wait complete. 2024-04-27T03:32:18.880 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.856+0000 7f0d4de19800 1 Processor -- start 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.856+0000 7f0d4de19800 1 -- start start 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.857+0000 7f0d4de19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681f439a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.857+0000 7f0d4de19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559681e540e0 con 0x559681e4b990 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.857+0000 7f0d4b918640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681f439a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.857+0000 7f0d4b918640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681f439a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60200/0 (socket says 172.21.15.165:60200) 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.857+0000 7f0d4b918640 1 -- 172.21.15.165:0/1216561651 learned_addr learned my addr 172.21.15.165:0/1216561651 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.857+0000 7f0d4b918640 1 -- 172.21.15.165:0/1216561651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559681f43ee0 con 0x559681e4b990 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.857+0000 7f0d4b918640 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681f439a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c060040 tx=0x7f0d3c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.858+0000 7f0c33fff640 1 -- 172.21.15.165:0/1216561651 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c052e30 con 0x559681e4b990 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.858+0000 7f0c33fff640 1 -- 172.21.15.165:0/1216561651 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3c06c070 con 0x559681e4b990 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.858+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559681f44170 con 0x559681e4b990 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.858+0000 7f0c33fff640 1 -- 172.21.15.165:0/1216561651 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c05e5f0 con 0x559681e4b990 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.858+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559681f46e20 con 0x559681e4b990 2024-04-27T03:32:18.881 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.858+0000 7f0c33fff640 1 -- 172.21.15.165:0/1216561651 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0d3c05e790 con 0x559681e4b990 2024-04-27T03:32:18.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.859+0000 7f0c33fff640 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c34036950 0x7f0c34038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.859+0000 7f0c33fff640 1 -- 172.21.15.165:0/1216561651 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0d3c08e810 con 0x559681e4b990 2024-04-27T03:32:18.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.859+0000 7f0d4b117640 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c34036950 0x7f0c34038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.859+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559681e4e2c0 0x559681f4abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.859+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f4b120 con 0x559681e4e2c0 2024-04-27T03:32:18.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.859+0000 7f0d4c119640 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559681e4e2c0 0x559681f4abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.859+0000 7f0d4b117640 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c34036950 0x7f0c34038e10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d40000f60 tx=0x7f0d400010a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.859+0000 7f0d4c119640 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559681e4e2c0 0x559681f4abe0 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.859+0000 7f0d4c119640 1 -- 172.21.15.165:0/1216561651 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681e4e2c0 2024-04-27T03:32:18.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.860+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f48190 con 0x559681e4e2c0 2024-04-27T03:32:18.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.860+0000 7f0d4c119640 1 -- 172.21.15.165:0/1216561651 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681e4e2c0 2024-04-27T03:32:18.882 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.860+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559681f4d450 0x559681f4f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.860+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f4ff30 con 0x559681f4d450 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.860+0000 7f0d4b918640 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559681f4d450 0x559681f4f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.860+0000 7f0d4b918640 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559681f4d450 0x559681f4f840 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.861+0000 7f0d4b918640 1 -- 172.21.15.165:0/1216561651 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f4d450 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.861+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f4c400 con 0x559681e4e2c0 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.861+0000 7f0d4c119640 1 -- 172.21.15.165:0/1216561651 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681e4e2c0 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.861+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x559681f513e0 0x559681f53830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.861+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f53f90 con 0x559681f513e0 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.861+0000 7f0d4b117640 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x559681f513e0 0x559681f53830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.861+0000 7f0d4b117640 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x559681f513e0 0x559681f53830 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.862+0000 7f0d4b117640 1 -- 172.21.15.165:0/1216561651 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f0d40009050 con 0x559681f513e0 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.862+0000 7f0d4de19800 1 Processor -- start 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.862+0000 7f0d4de19800 1 -- start start 2024-04-27T03:32:18.883 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.862+0000 7f0d4de19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b530 0x559681e499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.862+0000 7f0d4de19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559681d3c6b0 con 0x559681e4b530 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.862+0000 7f0d4b918640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b530 0x559681e499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.862+0000 7f0d4b918640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b530 0x559681e499a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60210/0 (socket says 172.21.15.165:60210) 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.862+0000 7f0d4b918640 1 -- 172.21.15.165:0/1493579294 learned_addr learned my addr 172.21.15.165:0/1493579294 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.863+0000 7f0d4b918640 1 -- 172.21.15.165:0/1493579294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559681e49ee0 con 0x559681e4b530 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.863+0000 7f0d4b918640 1 --2- 172.21.15.165:0/1493579294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b530 0x559681e499a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c071ca0 tx=0x7f0d3c066e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.863+0000 7f0c317fa640 1 -- 172.21.15.165:0/1493579294 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c092020 con 0x559681e4b530 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.863+0000 7f0c317fa640 1 -- 172.21.15.165:0/1493579294 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3c058040 con 0x559681e4b530 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.863+0000 7f0d4de19800 1 -- 172.21.15.165:0/1493579294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559681e4a170 con 0x559681e4b530 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.863+0000 7f0c317fa640 1 -- 172.21.15.165:0/1493579294 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c08eb70 con 0x559681e4b530 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.863+0000 7f0d4de19800 1 -- 172.21.15.165:0/1493579294 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559681e4a670 con 0x559681e4b530 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.864+0000 7f0c317fa640 1 -- 172.21.15.165:0/1493579294 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0d3c095020 con 0x559681e4b530 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.864+0000 7f0c317fa640 1 --2- 172.21.15.165:0/1493579294 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c24036950 0x7f0c24038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.884 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.864+0000 7f0c317fa640 1 -- 172.21.15.165:0/1493579294 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0d3c08b070 con 0x559681e4b530 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.864+0000 7f0d4b117640 1 --2- 172.21.15.165:0/1493579294 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c24036950 0x7f0c24038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.864+0000 7f0d4de19800 1 Processor -- start 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.864+0000 7f0d4de19800 1 -- start start 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.864+0000 7f0d4b117640 1 --2- 172.21.15.165:0/1493579294 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c24036950 0x7f0c24038e10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000bf90 tx=0x7f0d4000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.864+0000 7f0d4de19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 0x559681f71b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.864+0000 7f0d4de19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559681f3c7a0 con 0x559681f716f0 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.865+0000 7f0d4b918640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 0x559681f71b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.865+0000 7f0d4b918640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 0x559681f71b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60214/0 (socket says 172.21.15.165:60214) 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.865+0000 7f0d4b918640 1 -- 172.21.15.165:0/333911411 learned_addr learned my addr 172.21.15.165:0/333911411 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.865+0000 7f0d4b918640 1 -- 172.21.15.165:0/333911411 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559681f720e0 con 0x559681f716f0 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.865+0000 7f0d4b918640 1 --2- 172.21.15.165:0/333911411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 0x559681f71b10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0677d0 tx=0x7f0d3c09af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e26e7329fc8014 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.865+0000 7f0c2affd640 1 -- 172.21.15.165:0/333911411 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c08b3a0 con 0x559681f716f0 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.865+0000 7f0c2affd640 1 -- 172.21.15.165:0/333911411 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3c004600 con 0x559681f716f0 2024-04-27T03:32:18.885 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.865+0000 7f0c2affd640 1 -- 172.21.15.165:0/333911411 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c0669a0 con 0x559681f716f0 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4de19800 1 -- 172.21.15.165:0/333911411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 msgr2=0x559681f71b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4de19800 1 --2- 172.21.15.165:0/333911411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 0x559681f71b10 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0677d0 tx=0x7f0d3c09af30 comp rx=0 tx=0).stop 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4de19800 1 -- 172.21.15.165:0/333911411 shutdown_connections 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4de19800 1 --2- 172.21.15.165:0/333911411 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 0x559681f71b10 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4de19800 1 -- 172.21.15.165:0/333911411 >> 172.21.15.165:0/333911411 conn(0x559681f6da10 msgr2=0x559681f6ba70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4de19800 1 -- 172.21.15.165:0/333911411 shutdown_connections 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4de19800 1 -- 172.21.15.165:0/333911411 wait complete. 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4de19800 1 Processor -- start 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4de19800 1 -- start start 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4de19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 0x559681f78b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4de19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x559681f72800 con 0x559681f716f0 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4b918640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 0x559681f78b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.886 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4b918640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 0x559681f78b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60220/0 (socket says 172.21.15.165:60220) 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.866+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 learned_addr learned my addr 172.21.15.165:0/2542543933 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.867+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559681f790c0 con 0x559681f716f0 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.867+0000 7f0d4b918640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 0x559681f78b80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c09adf0 tx=0x7f0d3c079fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.867+0000 7f0c297fa640 1 -- 172.21.15.165:0/2542543933 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c067c90 con 0x559681f716f0 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.867+0000 7f0c297fa640 1 -- 172.21.15.165:0/2542543933 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3c08b600 con 0x559681f716f0 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.867+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x559681f790c0 con 0x559681f716f0 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.867+0000 7f0c297fa640 1 -- 172.21.15.165:0/2542543933 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c08b8e0 con 0x559681f716f0 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.867+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559681f790c0 con 0x559681f716f0 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.868+0000 7f0c297fa640 1 -- 172.21.15.165:0/2542543933 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0d3c0bc030 con 0x559681f716f0 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.868+0000 7f0c297fa640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c140368b0 0x7f0c14038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.868+0000 7f0c297fa640 1 -- 172.21.15.165:0/2542543933 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0d3c0ba070 con 0x559681f716f0 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.868+0000 7f0d4b117640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c140368b0 0x7f0c14038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.887 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.868+0000 7f0d4b117640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c140368b0 0x7f0c14038d70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x559681f4ba00 tx=0x7f0d40010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.868+0000 7f0d4de19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.868+0000 7f0d4de19800 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559681f84290 0x559681f86750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.868+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f86ea0 con 0x559681f84290 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.868+0000 7f0d4c119640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559681f84290 0x559681f86750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4c119640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559681f84290 0x559681f86750 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4de19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4de19800 20 realm 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4de19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f872b0 con 0x559681f84290 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4de19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.888 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4de19800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4de19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f87b80 con 0x559681f84290 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4de19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4de19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.869+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f84290 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.870+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.870+0000 7f0d4de19800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.870+0000 7f0d4de19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.870+0000 7f0d4de19800 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559681f88c80 0x559681f8b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.870+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f8b680 con 0x559681f88c80 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.870+0000 7f0d4b918640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559681f88c80 0x559681f8b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.870+0000 7f0d4b918640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559681f88c80 0x559681f8b070 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.870+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.870+0000 7f0d4de19800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.870+0000 7f0d4de19800 20 searching for the correct realm 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.870+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f8cfb0 con 0x559681f88c80 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.871+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.871+0000 7f0d4b918640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d3c097020 0x7f0d3c09a580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.871+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f8cfb0 con 0x7f0d3c097020 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.871+0000 7f0d4b117640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d3c097020 0x7f0d3c09a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.871+0000 7f0d4b117640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d3c097020 0x7f0d3c09a580 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.872+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.872+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f8cfb0 con 0x7f0d3c097020 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.872+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.872+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f8cfb0 con 0x559681f84290 2024-04-27T03:32:18.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.872+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.872+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f84290 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.872+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.872+0000 7f0d4c119640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0d44016440 0x7f0d440168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.872+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d44016440 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.873+0000 7f0d4c119640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0d44016440 0x7f0d440168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.873+0000 7f0d4c119640 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0d44016440 0x7f0d440168c0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.873+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.873+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d44016440 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.874+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.874+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d3c097020 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.874+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.874+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f88c80 2024-04-27T03:32:18.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.874+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.874+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f84290 2024-04-27T03:32:18.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.874+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.874+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d3c097020 2024-04-27T03:32:18.904 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.875+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.875+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f84290 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.875+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.875+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d3c097020 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.875+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.875+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f88c80 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.875+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.875+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f88c80 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.875+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.875+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d44016440 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.876+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.876+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f88c80 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.876+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.905 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.876+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f88c80 2024-04-27T03:32:18.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.876+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.876+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f84290 2024-04-27T03:32:18.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.876+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.876+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f84290 2024-04-27T03:32:18.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.877+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.877+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d44016440 2024-04-27T03:32:18.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.877+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.877+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f88c80 2024-04-27T03:32:18.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.877+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.877+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d3c097020 2024-04-27T03:32:18.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.877+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.877+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f84290 2024-04-27T03:32:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.878+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.878+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d44016440 2024-04-27T03:32:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.878+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.878+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f88c80 2024-04-27T03:32:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.878+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.878+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d44016440 2024-04-27T03:32:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.879+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.879+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d3c097020 2024-04-27T03:32:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.879+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.879+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f88c80 2024-04-27T03:32:18.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.879+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.879+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d44016440 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.879+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.879+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x559681f84290 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x559681f87f20 con 0x7f0d3c097020 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4de19800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4de19800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4de19800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4de19800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4de19800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4de19800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:18.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4de19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f8bf40 con 0x559681f84290 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4de19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4de19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.880+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f8c330 con 0x559681f84290 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f8c330 con 0x7f0d3c097020 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 20 zone default found 2024-04-27T03:32:18.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 4 Realm: () 2024-04-27T03:32:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 20 zonegroup default 2024-04-27T03:32:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f8d430 con 0x7f0d44016440 2024-04-27T03:32:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f8ee90 con 0x559681f88c80 2024-04-27T03:32:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.881+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f8f340 con 0x559681f88c80 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.882+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.882+0000 7f0d4de19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.882+0000 7f0d4de19800 20 started sync module instance, tier type = 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.882+0000 7f0d4de19800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.882+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f8e2b0 con 0x7f0d3c097020 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.883+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.883+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94104914028192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f8df10 con 0x7f0d3c097020 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.883+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f90140 con 0x559681f84290 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.884+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94104914028192] v27'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.884+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.884+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94104914032544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f90560 con 0x559681f84290 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.884+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f90fb0 con 0x559681f88c80 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.885+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94104914032544] v27'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.885+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.886+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94104914037408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f913a0 con 0x559681f88c80 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.886+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f91e40 con 0x7f0d3c097020 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.886+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.887+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94104914043440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f92b00 con 0x7f0d3c097020 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.887+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f93520 con 0x559681f84290 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.887+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94104914037408] v27'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.888+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.888+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94104914043440] v27'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.888+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94104914046992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f8fbc0 con 0x559681f84290 2024-04-27T03:32:18.912 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.888+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f93df0 con 0x7f0d44016440 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.889+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.889+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94104914046992] v27'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.889+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94104914049264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f93d10 con 0x7f0d44016440 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.889+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f92690 con 0x7f0d44016440 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.890+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94104914049264] v27'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.890+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.890+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94104914044560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f94800 con 0x7f0d44016440 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.890+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f951c0 con 0x7f0d44016440 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.891+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94104914044560] v27'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.891+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'255 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.891+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94104914051472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f951c0 con 0x7f0d44016440 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.891+0000 7f0d4de19800 20 add_watcher() i=0 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.891+0000 7f0d4de19800 20 add_watcher() i=1 2024-04-27T03:32:18.913 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.891+0000 7f0d4de19800 20 add_watcher() i=2 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.891+0000 7f0d4de19800 20 add_watcher() i=3 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.891+0000 7f0d4de19800 20 add_watcher() i=4 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.891+0000 7f0d4de19800 20 add_watcher() i=5 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.891+0000 7f0d4de19800 20 add_watcher() i=6 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.892+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94104914051472] v27'256 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.892+0000 7f0d4de19800 20 add_watcher() i=7 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.892+0000 7f0d4de19800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.892+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681f98500 con 0x559681f84290 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.892+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.892+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94104914067232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f95330 con 0x559681f84290 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.893+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94104914067232] v27'258 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.893+0000 7f0c01fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:18.914 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.894+0000 7f0d4de19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.894+0000 7f0d4de19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.894+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f977c0 con 0x559681f84290 2024-04-27T03:32:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.895+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'60 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.895+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa15c0 con 0x7f0d44016440 2024-04-27T03:32:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.896+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.896+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa15c0 con 0x559681f88c80 2024-04-27T03:32:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.897+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'38 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.897+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa1a70 con 0x559681f84290 2024-04-27T03:32:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.898+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'259 uv172 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.898+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa2060 con 0x7f0d3c097020 2024-04-27T03:32:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.899+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'86 uv59 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.915 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.899+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa2560 con 0x559681f88c80 2024-04-27T03:32:18.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.900+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'67 uv49 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.900+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa25f0 con 0x559681f84290 2024-04-27T03:32:18.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.901+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'38 uv29 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.901+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa25f0 con 0x7f0d44016440 2024-04-27T03:32:18.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.902+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'59 uv42 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.902+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa25f0 con 0x7f0d3c097020 2024-04-27T03:32:18.918 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.903+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'30 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.903+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa2b60 con 0x559681f88c80 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.904+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.904+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa2bf0 con 0x7f0d3c097020 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.905+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'87 uv61 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.905+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3160 con 0x7f0d44016440 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.906+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'36 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.906+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3160 con 0x7f0d3c097020 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.907+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.907+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa33f0 con 0x7f0d44016440 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.908+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'71 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.908+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa33f0 con 0x559681f84290 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.910+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'260 uv168 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.919 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.910+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3340 con 0x559681f88c80 2024-04-27T03:32:18.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.911+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'81 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.911+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3300 con 0x7f0d3c097020 2024-04-27T03:32:18.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.912+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'88 uv57 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.920 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.912+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3300 con 0x559681f88c80 2024-04-27T03:32:18.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.913+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'42 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.913+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3410 con 0x559681f84290 2024-04-27T03:32:18.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.914+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'38 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.914+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3410 con 0x559681f84290 2024-04-27T03:32:18.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.915+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'261 uv170 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.921 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.915+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3410 con 0x559681f88c80 2024-04-27T03:32:18.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.916+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'82 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.922 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.917+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3410 con 0x7f0d3c097020 2024-04-27T03:32:18.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.917+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'46 uv37 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.923 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.917+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3960 con 0x559681f88c80 2024-04-27T03:32:18.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.918+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'38 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.924 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.918+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3960 con 0x559681f88c80 2024-04-27T03:32:18.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.919+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'49 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.925 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.919+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3960 con 0x559681f88c80 2024-04-27T03:32:18.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.920+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'32 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.920+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3960 con 0x559681f88c80 2024-04-27T03:32:18.926 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.921+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'68 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.927 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.921+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3960 con 0x7f0d44016440 2024-04-27T03:32:18.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.922+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'40 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.928 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.922+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3960 con 0x559681f88c80 2024-04-27T03:32:18.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.923+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'43 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.929 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.923+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3960 con 0x7f0d44016440 2024-04-27T03:32:18.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.925+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'60 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.930 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.925+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3960 con 0x559681f88c80 2024-04-27T03:32:18.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.926+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'83 uv56 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.931 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.926+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3960 con 0x7f0d44016440 2024-04-27T03:32:18.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.927+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'72 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.932 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.927+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fa3960 con 0x7f0d3c097020 2024-04-27T03:32:18.933 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.928+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'36 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.928+0000 7f0c00fe1640 20 reqs_thread_entry: start 2024-04-27T03:32:18.934 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.928+0000 7f0d4de19800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.928+0000 7f0d4de19800 20 Filter name: none 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.928+0000 7f0b7a7fc640 20 reqs_thread_entry: start 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.928+0000 7f0d4de19800 10 cache get: name=default.rgw.meta+users.uid+a_user : miss 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.928+0000 7f0d4de19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.928+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:94 8.14 8:2efb937b:users.uid::a_user:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681fce1d0 con 0x559681f88c80 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.929+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(94 a_user [call out=48b,read 0~363 out=363b,getxattrs out=75b] v0'0 uv3 ondisk = 0) ==== 234+0+486 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.929+0000 7f0d4de19800 20 rados_obj.operate() r=0 bl.length=363 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.929+0000 7f0d4de19800 10 cache put: name=default.rgw.meta+users.uid+a_user info.flags=0x13 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.929+0000 7f0d4de19800 10 adding default.rgw.meta+users.uid+a_user to cache LRU end 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.929+0000 7f0d4de19800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.929+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:95 8.11 8:8d3ad08f:users.uid::a_user.buckets:head [call user.list_buckets in=34b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681fd0c60 con 0x559681f88c80 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.929+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(95 a_user.buckets [call] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 158+0+0 (crc 0 0 0) 0x7f0d3c071990 con 0x559681f88c80 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.929+0000 7f0d4de19800 10 removing key index: true_admin_akey 2024-04-27T03:32:18.935 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.929+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:96 8.15 8:afe09604:users.keys::true_admin_akey:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fcec40 con 0x559681f88c80 2024-04-27T03:32:18.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.930+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 29 ==== osd_op_reply(96 true_admin_akey [delete] v27'4 uv4 ondisk = 0) ==== 159+0+0 (crc 0 0 0) 0x7f0d3c0e6040 con 0x559681f88c80 2024-04-27T03:32:18.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.930+0000 7f0d4de19800 10 distributing notification oid=default.rgw.control:notify.5 cni=[op: 1, obj: default.rgw.meta:users.keys:true_admin_akey, ofs0, ns] 2024-04-27T03:32:18.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.930+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:97 7.3 7:c609908c:::notify.5:head [notify cookie 94104914294608 in=177b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681fd1190 con 0x7f0d44016440 2024-04-27T03:32:18.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.930+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== watch-notify(notify (1) cookie 94104914049264 notify 115964117023 ret 0) ==== 207+0+0 (crc 0 0 0) 0x559681fa2b60 con 0x7f0d44016440 2024-04-27T03:32:18.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.930+0000 7f0c2affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117023 cookie 94104914049264 notifier 4854 bl.length()=165 2024-04-27T03:32:18.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.930+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(97 notify.5 [notify cookie 94104914294608 out=8b] v0'0 uv255 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.930+0000 7f0c2affd640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:98 7.3 7:c609908c:::notify.5:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f0c10003d60 con 0x7f0d44016440 2024-04-27T03:32:18.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.931+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(98 notify.5 [notify-ack] v0'0 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.937 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.931+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== watch-notify(notify_complete (2) cookie 94104914294608 notify 115964117023 ret 0) ==== 42+0+48 (crc 0 0 0) 0x559681e4a170 con 0x7f0d44016440 2024-04-27T03:32:18.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.931+0000 7f0d4de19800 10 removing email index: 2024-04-27T03:32:18.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.931+0000 7f0d4de19800 10 removing user buckets index 2024-04-27T03:32:18.938 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.931+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:99 8.11 8:8d3ad08f:users.uid::a_user.buckets:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fd10f0 con 0x559681f88c80 2024-04-27T03:32:18.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.932+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 30 ==== osd_op_reply(99 a_user.buckets [delete] v27'3 uv2 ondisk = -2 ((2) No such file or directory)) ==== 158+0+0 (crc 0 0 0) 0x7f0d3c0e6040 con 0x559681f88c80 2024-04-27T03:32:18.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.932+0000 7f0d4de19800 10 removing user index: a_user 2024-04-27T03:32:18.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.932+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:100 8.14 8:2efb937b:users.uid::a_user:head [call version.check_conds in=122b,call version.set in=58b,delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fd10f0 con 0x559681f88c80 2024-04-27T03:32:18.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.933+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 31 ==== osd_op_reply(100 a_user [call,call,delete] v27'4 uv4 ondisk = 0) ==== 234+0+0 (crc 0 0 0) 0x7f0d3c0e6040 con 0x559681f88c80 2024-04-27T03:32:18.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.933+0000 7f0d4de19800 10 removing default.rgw.meta+users.uid+a_user from cache 2024-04-27T03:32:18.939 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.933+0000 7f0d4de19800 10 distributing notification oid=default.rgw.control:notify.2 cni=[op: 1, obj: default.rgw.meta:users.uid:a_user, ofs0, ns] 2024-04-27T03:32:18.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.933+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:101 7.15 7:a93a5511:::notify.2:head [notify cookie 94104914301168 in=167b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x559681fcf750 con 0x559681f88c80 2024-04-27T03:32:18.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.933+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 32 ==== watch-notify(notify (1) cookie 94104914037408 notify 115964117002 ret 0) ==== 197+0+0 (crc 0 0 0) 0x559681fa2640 con 0x559681f88c80 2024-04-27T03:32:18.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.933+0000 7f0c2affd640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117002 cookie 94104914037408 notifier 4854 bl.length()=155 2024-04-27T03:32:18.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.933+0000 7f0c2affd640 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:102 7.15 7:a93a5511:::notify.2:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f0c10003d60 con 0x559681f88c80 2024-04-27T03:32:18.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.933+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 33 ==== osd_op_reply(101 notify.2 [notify cookie 94104914301168 out=8b] v0'0 uv255 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f0d3c0e6040 con 0x559681f88c80 2024-04-27T03:32:18.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.933+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 34 ==== osd_op_reply(102 notify.2 [notify-ack] v0'0 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d3c0e6040 con 0x559681f88c80 2024-04-27T03:32:18.940 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.933+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 35 ==== watch-notify(notify_complete (2) cookie 94104914301168 notify 115964117002 ret 0) ==== 42+0+48 (crc 0 0 0) 0x559681e49ee0 con 0x559681f88c80 2024-04-27T03:32:18.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.934+0000 7f0c01fe3640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:18.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.934+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:103 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94104914067232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681fd10f0 con 0x559681f84290 2024-04-27T03:32:18.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.935+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(103 data_loggenerations_metadata [watch unwatch cookie 94104914067232] v27'262 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.941 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.935+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:104 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94104914028192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f97f20 con 0x7f0d3c097020 2024-04-27T03:32:18.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.936+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== osd_op_reply(104 notify.0 [watch unwatch cookie 94104914028192] v27'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.936+0000 7f0d4de19800 20 remove_watcher() i=0 2024-04-27T03:32:18.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.936+0000 7f0d4de19800 2 removed watcher, disabling cache 2024-04-27T03:32:18.942 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.936+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:105 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94104914032544] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f8e6a0 con 0x559681f84290 2024-04-27T03:32:18.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.937+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(105 notify.1 [watch unwatch cookie 94104914032544] v27'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.938+0000 7f0d4de19800 20 remove_watcher() i=1 2024-04-27T03:32:18.943 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.938+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:106 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94104914037408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f8e6a0 con 0x559681f88c80 2024-04-27T03:32:18.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.939+0000 7f0d4b918640 1 -- 172.21.15.165:0/2542543933 <== osd.3 v2:172.21.15.165:6808/2380918637 36 ==== osd_op_reply(106 notify.2 [watch unwatch cookie 94104914037408] v27'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d3c0e6040 con 0x559681f88c80 2024-04-27T03:32:18.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.939+0000 7f0d4de19800 20 remove_watcher() i=2 2024-04-27T03:32:18.944 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.939+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:107 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94104914043440] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f90aa0 con 0x7f0d3c097020 2024-04-27T03:32:18.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.940+0000 7f0d4b117640 1 -- 172.21.15.165:0/2542543933 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(107 notify.3 [watch unwatch cookie 94104914043440] v27'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d40009050 con 0x7f0d3c097020 2024-04-27T03:32:18.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.940+0000 7f0d4de19800 20 remove_watcher() i=3 2024-04-27T03:32:18.945 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.940+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:108 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94104914046992] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f8e6a0 con 0x559681f84290 2024-04-27T03:32:18.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.941+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(108 notify.4 [watch unwatch cookie 94104914046992] v27'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x559681f84290 2024-04-27T03:32:18.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.941+0000 7f0d4de19800 20 remove_watcher() i=4 2024-04-27T03:32:18.946 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.941+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:109 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94104914049264] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f92230 con 0x7f0d44016440 2024-04-27T03:32:18.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.942+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(109 notify.5 [watch unwatch cookie 94104914049264] v27'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.942+0000 7f0d4de19800 20 remove_watcher() i=5 2024-04-27T03:32:18.947 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.942+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:110 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94104914044560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f90aa0 con 0x7f0d44016440 2024-04-27T03:32:18.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.943+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(110 notify.6 [watch unwatch cookie 94104914044560] v27'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.943+0000 7f0d4de19800 20 remove_watcher() i=6 2024-04-27T03:32:18.948 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.943+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:111 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94104914051472] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x559681f92230 con 0x7f0d44016440 2024-04-27T03:32:18.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(111 notify.7 [watch unwatch cookie 94104914051472] v27'257 uv255 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f0d4400a4a0 con 0x7f0d44016440 2024-04-27T03:32:18.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 20 remove_watcher() i=7 2024-04-27T03:32:18.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0d44016440 msgr2=0x7f0d440168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f0d44016440 0x7f0d440168c0 crc :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559681f84290 msgr2=0x559681f86750 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.952 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559681f84290 0x559681f86750 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d3c097020 msgr2=0x7f0d3c09a580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d3c097020 0x7f0d3c09a580 crc :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559681f88c80 msgr2=0x559681f8b070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559681f88c80 0x559681f8b070 crc :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c140368b0 msgr2=0x7f0c14038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c140368b0 0x7f0c14038d70 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x559681f4ba00 tx=0x7f0d40010000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 msgr2=0x559681f78b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 --2- 172.21.15.165:0/2542543933 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681f716f0 0x559681f78b80 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c09adf0 tx=0x7f0d3c079fc0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 reap_dead start 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4c119640 1 -- 172.21.15.165:0/2542543933 reap_dead start 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 shutdown_connections 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.944+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 >> 172.21.15.165:0/2542543933 conn(0x559681f6da10 msgr2=0x559681f6b070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.953 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 shutdown_connections 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/2542543933 wait complete. 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1493579294 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c24036950 msgr2=0x7f0c24038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1493579294 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c24036950 0x7f0c24038e10 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f0d4000bf90 tx=0x7f0d4000f000 comp rx=0 tx=0).stop 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1493579294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b530 msgr2=0x559681e499a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1493579294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b530 0x559681e499a0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c071ca0 tx=0x7f0d3c066e20 comp rx=0 tx=0).stop 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1493579294 shutdown_connections 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1493579294 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c24036950 0x7f0c24038e10 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1493579294 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b530 0x559681e499a0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1493579294 >> 172.21.15.165:0/1493579294 conn(0x559681f56490 msgr2=0x559681f56890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1493579294 shutdown_connections 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1493579294 wait complete. 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559681e4e2c0 msgr2=0x559681f4abe0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x559681e4e2c0 0x559681f4abe0 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.954 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x559681f513e0 msgr2=0x559681f53830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x559681f513e0 0x559681f53830 crc :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559681f4d450 msgr2=0x559681f4f840 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x559681f4d450 0x559681f4f840 crc :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c34036950 msgr2=0x7f0c34038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0c34036950 0x7f0c34038e10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f0d40000f60 tx=0x7f0d400010a0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 msgr2=0x559681f439a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:18.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 --2- 172.21.15.165:0/1216561651 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x559681e4b990 0x559681f439a0 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c060040 tx=0x7f0d3c002fb0 comp rx=0 tx=0).stop 2024-04-27T03:32:18.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4c119640 1 -- 172.21.15.165:0/1216561651 reap_dead start 2024-04-27T03:32:18.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 shutdown_connections 2024-04-27T03:32:18.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.945+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 >> 172.21.15.165:0/1216561651 conn(0x559681e45320 msgr2=0x559681f36030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:18.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.946+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 shutdown_connections 2024-04-27T03:32:18.955 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:18.946+0000 7f0d4de19800 1 -- 172.21.15.165:0/1216561651 wait complete. 2024-04-27T03:32:18.959 ERROR:root:None 2024-04-27T03:32:18.959 ERROR:root:0 2024-04-27T03:32:18.959 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=read'] 2024-04-27T03:32:18.959 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=read'] 2024-04-27T03:32:18.960 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps user-info-without-keys=read 2024-04-27T03:32:19.044 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:19.044 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:19.045 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.039+0000 7fc4a4925800 1 Processor -- start 2024-04-27T03:32:19.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.039+0000 7fc4a4925800 1 -- start start 2024-04-27T03:32:19.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.039+0000 7fc4a4925800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5651f7c061d0 0x5651f7c085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.039+0000 7fc4a4925800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5651f7ac1d40 con 0x5651f7c01d40 2024-04-27T03:32:19.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.039+0000 7fc4a4925800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5651f7a944d0 con 0x5651f7c061d0 2024-04-27T03:32:19.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.040+0000 7fc497fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5651f7c061d0 0x5651f7c085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.040+0000 7fc4a06b7640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5651f7c01d40 0x5651f7c02160 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42366/0 (socket says 172.21.15.165:42366) 2024-04-27T03:32:19.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.040+0000 7fc4a06b7640 1 -- 172.21.15.165:0/262069838 learned_addr learned my addr 172.21.15.165:0/262069838 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:19.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.040+0000 7fc497fff640 1 -- 172.21.15.165:0/262069838 >> v1:172.21.15.89:6789/0 conn(0x5651f7c01d40 legacy=0x5651f7c02160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.040+0000 7fc49feb6640 1 -- 172.21.15.165:0/262069838 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3065703731 0 0) 0x5651f7ac1d40 con 0x5651f7c01d40 2024-04-27T03:32:19.046 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.040+0000 7fc497fff640 1 -- 172.21.15.165:0/262069838 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651f7ac0590 con 0x5651f7c061d0 2024-04-27T03:32:19.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.042+0000 7fc497fff640 1 --2- 172.21.15.165:0/262069838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c061d0 0x5651f7c085c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc48800a470 tx=0x7fc48801ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=928b42041f0b8304 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.042+0000 7fc49feb6640 1 -- 172.21.15.165:0/262069838 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc488057070 con 0x5651f7c061d0 2024-04-27T03:32:19.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.042+0000 7fc49feb6640 1 -- 172.21.15.165:0/262069838 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc488052570 con 0x5651f7c061d0 2024-04-27T03:32:19.048 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.042+0000 7fc49feb6640 1 -- 172.21.15.165:0/262069838 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc488052870 con 0x5651f7c061d0 2024-04-27T03:32:19.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.043+0000 7fc4a4925800 1 -- 172.21.15.165:0/262069838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c061d0 msgr2=0x5651f7c085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.043+0000 7fc4a4925800 1 --2- 172.21.15.165:0/262069838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c061d0 0x5651f7c085c0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fc48800a470 tx=0x7fc48801ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:19.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.043+0000 7fc4a4925800 1 -- 172.21.15.165:0/262069838 shutdown_connections 2024-04-27T03:32:19.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.043+0000 7fc4a4925800 1 --2- 172.21.15.165:0/262069838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c061d0 0x5651f7c085c0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.043+0000 7fc4a4925800 1 -- 172.21.15.165:0/262069838 >> 172.21.15.165:0/262069838 conn(0x5651f7bfd3a0 msgr2=0x5651f7bff7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.043+0000 7fc4a4925800 1 -- 172.21.15.165:0/262069838 shutdown_connections 2024-04-27T03:32:19.049 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.043+0000 7fc4a4925800 1 -- 172.21.15.165:0/262069838 wait complete. 2024-04-27T03:32:19.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.054+0000 7fc4a4925800 1 Processor -- start 2024-04-27T03:32:19.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.055+0000 7fc4a4925800 1 -- start start 2024-04-27T03:32:19.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.055+0000 7fc4a4925800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 0x5651f7b76010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.055+0000 7fc4a4925800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5651f7b4e3b0 con 0x5651f7c01d40 2024-04-27T03:32:19.061 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.055+0000 7fc4a06b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 0x5651f7b76010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.055+0000 7fc4a06b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 0x5651f7b76010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60234/0 (socket says 172.21.15.165:60234) 2024-04-27T03:32:19.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.055+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3916015762 learned_addr learned my addr 172.21.15.165:0/3916015762 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.055+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3916015762 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651f7cf6cc0 con 0x5651f7c01d40 2024-04-27T03:32:19.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.055+0000 7fc4a06b7640 1 --2- 172.21.15.165:0/3916015762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 0x5651f7b76010 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc49000b240 tx=0x7fc49000b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1f9385759eb402 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.056+0000 7fc38e60c640 1 -- 172.21.15.165:0/3916015762 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc490008070 con 0x5651f7c01d40 2024-04-27T03:32:19.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.056+0000 7fc38e60c640 1 -- 172.21.15.165:0/3916015762 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc49000d500 con 0x5651f7c01d40 2024-04-27T03:32:19.062 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.056+0000 7fc38e60c640 1 -- 172.21.15.165:0/3916015762 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc49000d860 con 0x5651f7c01d40 2024-04-27T03:32:19.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.056+0000 7fc4a4925800 1 -- 172.21.15.165:0/3916015762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 msgr2=0x5651f7b76010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.056+0000 7fc4a4925800 1 --2- 172.21.15.165:0/3916015762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 0x5651f7b76010 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fc49000b240 tx=0x7fc49000b700 comp rx=0 tx=0).stop 2024-04-27T03:32:19.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.056+0000 7fc4a4925800 1 -- 172.21.15.165:0/3916015762 shutdown_connections 2024-04-27T03:32:19.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.056+0000 7fc4a4925800 1 --2- 172.21.15.165:0/3916015762 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 0x5651f7b76010 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.056+0000 7fc4a4925800 1 -- 172.21.15.165:0/3916015762 >> 172.21.15.165:0/3916015762 conn(0x5651f7bfd3a0 msgr2=0x5651f7cee7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.056+0000 7fc4a4925800 1 -- 172.21.15.165:0/3916015762 shutdown_connections 2024-04-27T03:32:19.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.056+0000 7fc4a4925800 1 -- 172.21.15.165:0/3916015762 wait complete. 2024-04-27T03:32:19.067 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.057+0000 7fc4a4925800 1 Processor -- start 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.057+0000 7fc4a4925800 1 -- start start 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.057+0000 7fc4a4925800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 0x5651f7cf4610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.057+0000 7fc4a4925800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5651f7c08e90 con 0x5651f7c01d40 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.057+0000 7fc4a06b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 0x5651f7cf4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.057+0000 7fc4a06b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 0x5651f7cf4610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60248/0 (socket says 172.21.15.165:60248) 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.057+0000 7fc4a06b7640 1 -- 172.21.15.165:0/644293606 learned_addr learned my addr 172.21.15.165:0/644293606 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.058+0000 7fc4a06b7640 1 -- 172.21.15.165:0/644293606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651f7cf2d00 con 0x5651f7c01d40 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.058+0000 7fc4a06b7640 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 0x5651f7cf4610 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x5651f7b75640 tx=0x7fc490000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.058+0000 7fc38ce09640 1 -- 172.21.15.165:0/644293606 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc490019870 con 0x5651f7c01d40 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.058+0000 7fc38ce09640 1 -- 172.21.15.165:0/644293606 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc490019a10 con 0x5651f7c01d40 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.058+0000 7fc38ce09640 1 -- 172.21.15.165:0/644293606 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc490019d30 con 0x5651f7c01d40 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.058+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5651f7cf2f90 con 0x5651f7c01d40 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.058+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651f7cf34b0 con 0x5651f7c01d40 2024-04-27T03:32:19.068 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.059+0000 7fc38ce09640 1 -- 172.21.15.165:0/644293606 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc490017040 con 0x5651f7c01d40 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.059+0000 7fc38ce09640 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc388036950 0x7fc388038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.059+0000 7fc38ce09640 1 -- 172.21.15.165:0/644293606 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fc490015020 con 0x5651f7c01d40 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.059+0000 7fc497fff640 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc388036950 0x7fc388038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.059+0000 7fc4a4925800 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5651f7c061d0 0x5651f7d05080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.059+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d055c0 con 0x5651f7c061d0 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.059+0000 7fc4a0eb8640 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5651f7c061d0 0x5651f7d05080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.060+0000 7fc497fff640 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc388036950 0x7fc388038e10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc488054000 tx=0x7fc48804e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.060+0000 7fc4a0eb8640 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5651f7c061d0 0x5651f7d05080 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.060+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/644293606 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7c061d0 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.060+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d05a10 con 0x5651f7c061d0 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.060+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/644293606 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7c061d0 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.060+0000 7fc4a4925800 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5651f7d07820 0x5651f7d09c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.069 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.060+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d0a300 con 0x5651f7d07820 2024-04-27T03:32:19.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.060+0000 7fc4a06b7640 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5651f7d07820 0x5651f7d09c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.061+0000 7fc4a06b7640 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5651f7d07820 0x5651f7d09c10 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.061+0000 7fc4a06b7640 1 -- 172.21.15.165:0/644293606 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d07820 2024-04-27T03:32:19.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.061+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d067d0 con 0x5651f7c061d0 2024-04-27T03:32:19.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.061+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/644293606 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7c061d0 2024-04-27T03:32:19.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.061+0000 7fc4a4925800 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5651f7d0b7b0 0x5651f7d0dc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.061+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d0e360 con 0x5651f7d0b7b0 2024-04-27T03:32:19.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.061+0000 7fc497fff640 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5651f7d0b7b0 0x5651f7d0dc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.070 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.062+0000 7fc497fff640 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5651f7d0b7b0 0x5651f7d0dc00 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.062+0000 7fc497fff640 1 -- 172.21.15.165:0/644293606 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fc488052940 con 0x5651f7d0b7b0 2024-04-27T03:32:19.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.062+0000 7fc4a4925800 1 Processor -- start 2024-04-27T03:32:19.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.062+0000 7fc4a4925800 1 -- start start 2024-04-27T03:32:19.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.062+0000 7fc4a4925800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d1c7c0 0x5651f7d1cbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.084 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.062+0000 7fc4a4925800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5651f7af46b0 con 0x5651f7d1c7c0 2024-04-27T03:32:19.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.063+0000 7fc4a06b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d1c7c0 0x5651f7d1cbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.063+0000 7fc4a06b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d1c7c0 0x5651f7d1cbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60262/0 (socket says 172.21.15.165:60262) 2024-04-27T03:32:19.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.063+0000 7fc4a06b7640 1 -- 172.21.15.165:0/2659914370 learned_addr learned my addr 172.21.15.165:0/2659914370 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.063+0000 7fc4a06b7640 1 -- 172.21.15.165:0/2659914370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651f7d1d120 con 0x5651f7d1c7c0 2024-04-27T03:32:19.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.063+0000 7fc4a06b7640 1 --2- 172.21.15.165:0/2659914370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d1c7c0 0x5651f7d1cbe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc49004b480 tx=0x7fc49000ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.063+0000 7fc385ffb640 1 -- 172.21.15.165:0/2659914370 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc490044a50 con 0x5651f7d1c7c0 2024-04-27T03:32:19.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.063+0000 7fc385ffb640 1 -- 172.21.15.165:0/2659914370 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc490044bf0 con 0x5651f7d1c7c0 2024-04-27T03:32:19.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.063+0000 7fc4a4925800 1 -- 172.21.15.165:0/2659914370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5651f7d1db90 con 0x5651f7d1c7c0 2024-04-27T03:32:19.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.063+0000 7fc385ffb640 1 -- 172.21.15.165:0/2659914370 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc490044ed0 con 0x5651f7d1c7c0 2024-04-27T03:32:19.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.063+0000 7fc4a4925800 1 -- 172.21.15.165:0/2659914370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651f7d1e090 con 0x5651f7d1c7c0 2024-04-27T03:32:19.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.064+0000 7fc385ffb640 1 -- 172.21.15.165:0/2659914370 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc49003f1b0 con 0x5651f7d1c7c0 2024-04-27T03:32:19.085 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.064+0000 7fc385ffb640 1 --2- 172.21.15.165:0/2659914370 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc378036950 0x7fc378038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.064+0000 7fc385ffb640 1 -- 172.21.15.165:0/2659914370 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fc49003cb90 con 0x5651f7d1c7c0 2024-04-27T03:32:19.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.064+0000 7fc497fff640 1 --2- 172.21.15.165:0/2659914370 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc378036950 0x7fc378038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.064+0000 7fc4a4925800 1 Processor -- start 2024-04-27T03:32:19.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.064+0000 7fc4a4925800 1 -- start start 2024-04-27T03:32:19.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.065+0000 7fc497fff640 1 --2- 172.21.15.165:0/2659914370 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc378036950 0x7fc378038e10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5651f7d05dd0 tx=0x7fc488005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.065+0000 7fc4a4925800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 0x5651f7d29cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.065+0000 7fc4a4925800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5651f7cf7010 con 0x5651f7d298e0 2024-04-27T03:32:19.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.065+0000 7fc4a06b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 0x5651f7d29cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.065+0000 7fc4a06b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 0x5651f7d29cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60276/0 (socket says 172.21.15.165:60276) 2024-04-27T03:32:19.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.065+0000 7fc4a06b7640 1 -- 172.21.15.165:0/2766458997 learned_addr learned my addr 172.21.15.165:0/2766458997 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.065+0000 7fc4a06b7640 1 -- 172.21.15.165:0/2766458997 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651f7d2a290 con 0x5651f7d298e0 2024-04-27T03:32:19.086 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.065+0000 7fc4a06b7640 1 --2- 172.21.15.165:0/2766458997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 0x5651f7d29cc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc490011ff0 tx=0x7fc4900266d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=336633b528014957 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.065+0000 7fc37f7fe640 1 -- 172.21.15.165:0/2766458997 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4900267f0 con 0x5651f7d298e0 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.065+0000 7fc37f7fe640 1 -- 172.21.15.165:0/2766458997 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc490026a00 con 0x5651f7d298e0 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.065+0000 7fc37f7fe640 1 -- 172.21.15.165:0/2766458997 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc490041dd0 con 0x5651f7d298e0 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.066+0000 7fc4a4925800 1 -- 172.21.15.165:0/2766458997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 msgr2=0x5651f7d29cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.066+0000 7fc4a4925800 1 --2- 172.21.15.165:0/2766458997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 0x5651f7d29cc0 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fc490011ff0 tx=0x7fc4900266d0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.066+0000 7fc4a4925800 1 -- 172.21.15.165:0/2766458997 shutdown_connections 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.066+0000 7fc4a4925800 1 --2- 172.21.15.165:0/2766458997 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 0x5651f7d29cc0 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.066+0000 7fc4a4925800 1 -- 172.21.15.165:0/2766458997 >> 172.21.15.165:0/2766458997 conn(0x5651f7d25bb0 msgr2=0x5651f7d23c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.066+0000 7fc4a4925800 1 -- 172.21.15.165:0/2766458997 shutdown_connections 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.066+0000 7fc4a4925800 1 -- 172.21.15.165:0/2766458997 wait complete. 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.066+0000 7fc4a4925800 1 Processor -- start 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.066+0000 7fc4a4925800 1 -- start start 2024-04-27T03:32:19.087 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.066+0000 7fc4a4925800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 0x5651f7d30cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.066+0000 7fc4a4925800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5651f7d2a9a0 con 0x5651f7d298e0 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.067+0000 7fc4a06b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 0x5651f7d30cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.067+0000 7fc4a06b7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 0x5651f7d30cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60280/0 (socket says 172.21.15.165:60280) 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.067+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 learned_addr learned my addr 172.21.15.165:0/3222766279 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.067+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651f7d311f0 con 0x5651f7d298e0 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.067+0000 7fc4a06b7640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 0x5651f7d30cb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc49003cfd0 tx=0x7fc4900273f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.067+0000 7fc37dffb640 1 -- 172.21.15.165:0/3222766279 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc4900408d0 con 0x5651f7d298e0 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.067+0000 7fc37dffb640 1 -- 172.21.15.165:0/3222766279 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc49005c050 con 0x5651f7d298e0 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.067+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5651f7d311f0 con 0x5651f7d298e0 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.067+0000 7fc37dffb640 1 -- 172.21.15.165:0/3222766279 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc490027070 con 0x5651f7d298e0 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.067+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651f7d311f0 con 0x5651f7d298e0 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.068+0000 7fc37dffb640 1 -- 172.21.15.165:0/3222766279 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc49000c280 con 0x5651f7d298e0 2024-04-27T03:32:19.088 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.068+0000 7fc37dffb640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc370036950 0x7fc370038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.068+0000 7fc37dffb640 1 -- 172.21.15.165:0/3222766279 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fc49006a030 con 0x5651f7d298e0 2024-04-27T03:32:19.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.068+0000 7fc497fff640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc370036950 0x7fc370038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.069+0000 7fc497fff640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc370036950 0x7fc370038e10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc488052fd0 tx=0x7fc48804f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.069+0000 7fc4a4925800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.069+0000 7fc4a4925800 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5651f7d3c370 0x5651f7d3e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.069+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d3ef80 con 0x5651f7d3c370 2024-04-27T03:32:19.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.069+0000 7fc4a0eb8640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5651f7d3c370 0x5651f7d3e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.069+0000 7fc4a0eb8640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5651f7d3c370 0x5651f7d3e830 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.069+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.069+0000 7fc4a4925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.069+0000 7fc4a4925800 20 realm 2024-04-27T03:32:19.089 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.069+0000 7fc4a4925800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.069+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d3f390 con 0x5651f7d3c370 2024-04-27T03:32:19.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a4925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a4925800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:19.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a4925800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d3fc60 con 0x5651f7d3c370 2024-04-27T03:32:19.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a4925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a4925800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d3c370 2024-04-27T03:32:19.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.090 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a4925800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a4925800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a4925800 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5651f7d40d60 0x5651f7d43150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d43760 con 0x5651f7d40d60 2024-04-27T03:32:19.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a06b7640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5651f7d40d60 0x5651f7d43150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.070+0000 7fc4a06b7640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5651f7d40d60 0x5651f7d43150 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.071+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.071+0000 7fc4a4925800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:19.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.071+0000 7fc4a4925800 20 searching for the correct realm 2024-04-27T03:32:19.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.071+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d45090 con 0x5651f7d40d60 2024-04-27T03:32:19.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.071+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.071+0000 7fc4a06b7640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc4900444f0 0x7fc490043330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.091 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.071+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d45090 con 0x7fc4900444f0 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.071+0000 7fc497fff640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc4900444f0 0x7fc490043330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.072+0000 7fc497fff640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc4900444f0 0x7fc490043330 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.072+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.072+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d45090 con 0x7fc4900444f0 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.072+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.072+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d45090 con 0x5651f7d3c370 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.072+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.072+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d3c370 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.073+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.073+0000 7fc4a0eb8640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc498016440 0x7fc4980168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.073+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x7fc498016440 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.073+0000 7fc4a0eb8640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc498016440 0x7fc4980168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.092 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.073+0000 7fc4a0eb8640 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc498016440 0x7fc4980168c0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.074+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.074+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x7fc498016440 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.074+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.074+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x7fc4900444f0 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.074+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.074+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d40d60 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.074+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.074+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d3c370 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.075+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.075+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x7fc4900444f0 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.075+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.075+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d3c370 2024-04-27T03:32:19.093 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.075+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.075+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x7fc4900444f0 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.075+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.075+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d40d60 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.076+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.076+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d40d60 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.076+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.076+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x7fc498016440 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.076+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.076+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d40d60 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.077+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.077+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d40d60 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.077+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.094 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.077+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d3c370 2024-04-27T03:32:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.077+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.077+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d3c370 2024-04-27T03:32:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.077+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.077+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x7fc498016440 2024-04-27T03:32:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.078+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.078+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d40d60 2024-04-27T03:32:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.078+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.078+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x7fc4900444f0 2024-04-27T03:32:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.078+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.078+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d3c370 2024-04-27T03:32:19.095 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.078+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.078+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x7fc498016440 2024-04-27T03:32:19.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.079+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.096 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.079+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d40d60 2024-04-27T03:32:19.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.079+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.079+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x7fc498016440 2024-04-27T03:32:19.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.079+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.079+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x7fc4900444f0 2024-04-27T03:32:19.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.080+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.080+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d40000 con 0x5651f7d40d60 2024-04-27T03:32:19.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.080+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.080+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc490091040 con 0x7fc498016440 2024-04-27T03:32:19.106 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.080+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.080+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc490091040 con 0x5651f7d3c370 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.080+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.080+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7fc490091040 con 0x7fc4900444f0 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d44020 con 0x5651f7d3c370 2024-04-27T03:32:19.107 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d44410 con 0x5651f7d3c370 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.081+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d44410 con 0x7fc4900444f0 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 20 zone default found 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 4 Realm: () 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:19.108 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 20 zonegroup default 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d45510 con 0x7fc498016440 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d46ef0 con 0x5651f7d40d60 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.082+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d47330 con 0x5651f7d40d60 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.083+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.083+0000 7fc4a4925800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.083+0000 7fc4a4925800 20 started sync module instance, tier type = 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.083+0000 7fc4a4925800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:19.109 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.083+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d46390 con 0x7fc4900444f0 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.084+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.084+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94910050232192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d45ff0 con 0x7fc4900444f0 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.084+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d48260 con 0x5651f7d3c370 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.085+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94910050232192] v27'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.085+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.085+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94910050237872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d481d0 con 0x5651f7d3c370 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.085+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d49120 con 0x5651f7d40d60 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.086+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.086+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94910050241648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d499c0 con 0x5651f7d40d60 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.086+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d4a4b0 con 0x7fc4900444f0 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.086+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94910050237872] v27'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.087+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94910050241648] v27'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.110 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.087+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.087+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94910050246528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d4a8a0 con 0x7fc4900444f0 2024-04-27T03:32:19.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.087+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d4b130 con 0x5651f7d3c370 2024-04-27T03:32:19.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.088+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.088+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94910050249792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d4b9d0 con 0x5651f7d3c370 2024-04-27T03:32:19.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.088+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d4c480 con 0x7fc498016440 2024-04-27T03:32:19.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.088+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94910050246528] v27'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.089+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.089+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94910050240080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d4c440 con 0x7fc498016440 2024-04-27T03:32:19.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.089+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94910050249792] v27'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.089+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d4c9e0 con 0x7fc498016440 2024-04-27T03:32:19.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.090+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94910050240080] v27'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.111 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.091+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.091+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94910050252064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d4cbc0 con 0x7fc498016440 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.091+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d4d410 con 0x7fc498016440 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.092+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(58 notify.7 [create] v27'258 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.092+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94910050258336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d4d730 con 0x7fc498016440 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.092+0000 7fc4a4925800 20 add_watcher() i=0 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.092+0000 7fc4a4925800 20 add_watcher() i=1 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.092+0000 7fc4a4925800 20 add_watcher() i=2 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.092+0000 7fc4a4925800 20 add_watcher() i=3 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.092+0000 7fc4a4925800 20 add_watcher() i=4 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.092+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94910050252064] v27'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.092+0000 7fc4a4925800 20 add_watcher() i=5 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.092+0000 7fc4a4925800 20 add_watcher() i=6 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.093+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94910050258336] v27'259 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.093+0000 7fc4a4925800 20 add_watcher() i=7 2024-04-27T03:32:19.112 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.093+0000 7fc4a4925800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.093+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d4bf90 con 0x5651f7d3c370 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.094+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.094+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94910050259456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d4bf90 con 0x5651f7d3c370 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.095+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94910050259456] v27'263 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.095+0000 7fc3567e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.095+0000 7fc4a4925800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.095+0000 7fc4a4925800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.095+0000 7fc4a4925800 5 note: GC not initialized 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.095+0000 7fc3557e2640 20 reqs_thread_entry: start 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.095+0000 7fc4a4925800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.095+0000 7fc4a4925800 20 Filter name: none 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.095+0000 7fc2c6ffd640 20 reqs_thread_entry: start 2024-04-27T03:32:19.113 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.095+0000 7fc4a4925800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-27T03:32:19.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.095+0000 7fc4a4925800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.095+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d83a60 con 0x7fc498016440 2024-04-27T03:32:19.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.096+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 ada [call out=48b,read 0~460 out=460b,getxattrs out=75b] v0'0 uv5 ondisk = 0) ==== 231+0+583 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.096+0000 7fc4a4925800 20 rados_obj.operate() r=0 bl.length=460 2024-04-27T03:32:19.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.096+0000 7fc4a4925800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-27T03:32:19.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.096+0000 7fc4a4925800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.096+0000 7fc4a4925800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:19.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.096+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~490 in=490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d8a090 con 0x7fc498016440 2024-04-27T03:32:19.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.097+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~490] v27'6 uv6 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.097+0000 7fc4a4925800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.097+0000 7fc4a4925800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.114 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.097+0000 7fc4a4925800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-27T03:32:19.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.097+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94910050507520 in=678b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d85780 con 0x5651f7d3c370 2024-04-27T03:32:19.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.098+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== watch-notify(notify (1) cookie 94910050249792 notify 115964117042 ret 0) ==== 708+0+0 (crc 0 0 0) 0x7fc498030040 con 0x5651f7d3c370 2024-04-27T03:32:19.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.098+0000 7fc37f7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117042 cookie 94910050249792 notifier 4861 bl.length()=666 2024-04-27T03:32:19.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.098+0000 7fc37f7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.098+0000 7fc37f7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.098+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 notify.4 [notify cookie 94910050507520 out=8b] v0'0 uv258 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.098+0000 7fc37f7fe640 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fc364003ea0 con 0x5651f7d3c370 2024-04-27T03:32:19.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.098+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.115 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.098+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify_complete (2) cookie 94910050507520 notify 115964117042 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fc49802a910 con 0x5651f7d3c370 2024-04-27T03:32:19.116 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-27T03:32:18.064161Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"read"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:32:18.064141Z","tags":[],"group_ids":[]} 2024-04-27T03:32:19.120 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.099+0000 7fc3567e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.099+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94910050259456] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d821c0 con 0x5651f7d3c370 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.100+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94910050259456] v27'264 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.100+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94910050232192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d85aa0 con 0x7fc4900444f0 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.101+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94910050232192] v27'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.101+0000 7fc4a4925800 20 remove_watcher() i=0 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.101+0000 7fc4a4925800 2 removed watcher, disabling cache 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.101+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94910050237872] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d46780 con 0x5651f7d3c370 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.102+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94910050237872] v27'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.102+0000 7fc4a4925800 20 remove_watcher() i=1 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.102+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94910050241648] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d46780 con 0x5651f7d40d60 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.105+0000 7fc4a06b7640 1 -- 172.21.15.165:0/3222766279 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94910050241648] v27'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49002d070 con 0x5651f7d40d60 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.105+0000 7fc4a4925800 20 remove_watcher() i=2 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.105+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94910050246528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d48c70 con 0x7fc4900444f0 2024-04-27T03:32:19.121 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.106+0000 7fc497fff640 1 -- 172.21.15.165:0/3222766279 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94910050246528] v27'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc488081070 con 0x7fc4900444f0 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.106+0000 7fc4a4925800 20 remove_watcher() i=3 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.106+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94910050249792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d46780 con 0x5651f7d3c370 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.107+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94910050249792] v27'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x5651f7d3c370 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.107+0000 7fc4a4925800 20 remove_watcher() i=4 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.107+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94910050240080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d49f80 con 0x7fc498016440 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.108+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94910050240080] v27'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.108+0000 7fc4a4925800 20 remove_watcher() i=5 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.108+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94910050252064] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d48c70 con 0x7fc498016440 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.109+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94910050252064] v27'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.109+0000 7fc4a4925800 20 remove_watcher() i=6 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.109+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94910050258336] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5651f7d46780 con 0x7fc498016440 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.110+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94910050258336] v27'260 uv258 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fc49800a4a0 con 0x7fc498016440 2024-04-27T03:32:19.122 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.110+0000 7fc4a4925800 20 remove_watcher() i=7 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.110+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc498016440 msgr2=0x7fc4980168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.110+0000 7fc4a4925800 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fc498016440 0x7fc4980168c0 crc :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.110+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5651f7d3c370 msgr2=0x5651f7d3e830 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.110+0000 7fc4a4925800 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5651f7d3c370 0x5651f7d3e830 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.110+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc4900444f0 msgr2=0x7fc490043330 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.110+0000 7fc4a4925800 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fc4900444f0 0x7fc490043330 crc :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.110+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5651f7d40d60 msgr2=0x5651f7d43150 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.110+0000 7fc4a4925800 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5651f7d40d60 0x5651f7d43150 crc :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.110+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc370036950 msgr2=0x7fc370038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.110+0000 7fc4a4925800 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc370036950 0x7fc370038e10 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fc488052fd0 tx=0x7fc48804f000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 msgr2=0x5651f7d30cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 --2- 172.21.15.165:0/3222766279 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d298e0 0x5651f7d30cb0 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fc49003cfd0 tx=0x7fc4900273f0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/3222766279 reap_dead start 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 shutdown_connections 2024-04-27T03:32:19.123 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 >> 172.21.15.165:0/3222766279 conn(0x5651f7d25bb0 msgr2=0x5651f7d23210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 shutdown_connections 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/3222766279 wait complete. 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/2659914370 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc378036950 msgr2=0x7fc378038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 --2- 172.21.15.165:0/2659914370 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc378036950 0x7fc378038e10 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x5651f7d05dd0 tx=0x7fc488005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/2659914370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d1c7c0 msgr2=0x5651f7d1cbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 --2- 172.21.15.165:0/2659914370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d1c7c0 0x5651f7d1cbe0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fc49004b480 tx=0x7fc49000ed40 comp rx=0 tx=0).stop 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/2659914370 shutdown_connections 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 --2- 172.21.15.165:0/2659914370 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc378036950 0x7fc378038e10 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 --2- 172.21.15.165:0/2659914370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7d1c7c0 0x5651f7d1cbe0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/2659914370 >> 172.21.15.165:0/2659914370 conn(0x5651f7d10860 msgr2=0x5651f7d10c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/2659914370 shutdown_connections 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/2659914370 wait complete. 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5651f7c061d0 msgr2=0x5651f7d05080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.124 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5651f7c061d0 0x5651f7d05080 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5651f7d0b7b0 msgr2=0x5651f7d0dc00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5651f7d0b7b0 0x5651f7d0dc00 crc :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5651f7d07820 msgr2=0x5651f7d09c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5651f7d07820 0x5651f7d09c10 crc :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc388036950 msgr2=0x7fc388038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc388036950 0x7fc388038e10 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fc488054000 tx=0x7fc48804e000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 msgr2=0x5651f7cf4610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.111+0000 7fc4a4925800 1 --2- 172.21.15.165:0/644293606 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5651f7c01d40 0x5651f7cf4610 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x5651f7b75640 tx=0x7fc490000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.112+0000 7fc4a0eb8640 1 -- 172.21.15.165:0/644293606 reap_dead start 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.112+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 shutdown_connections 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.112+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 >> 172.21.15.165:0/644293606 conn(0x5651f7bfd3a0 msgr2=0x5651f7ced070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.112+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 shutdown_connections 2024-04-27T03:32:19.125 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.112+0000 7fc4a4925800 1 -- 172.21.15.165:0/644293606 wait complete. 2024-04-27T03:32:19.126 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'read'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.126 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'read'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.127 ERROR:root:0 2024-04-27T03:32:19.127 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:19.131 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:19.131 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=*'] 2024-04-27T03:32:19.132 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=*'] 2024-04-27T03:32:19.132 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'user-info-without-keys=*' 2024-04-27T03:32:19.176 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:19.176 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:19.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.171+0000 7f2d7bd19800 1 Processor -- start 2024-04-27T03:32:19.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.171+0000 7f2d7bd19800 1 -- start start 2024-04-27T03:32:19.177 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.171+0000 7f2d7bd19800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55e1a66de350 0x55e1a66e0740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.171+0000 7f2d7bd19800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e1a6598d40 con 0x55e1a66d9fb0 2024-04-27T03:32:19.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.171+0000 7f2d7bd19800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55e1a656b4d0 con 0x55e1a66de350 2024-04-27T03:32:19.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.171+0000 7f2d71017640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55e1a66de350 0x55e1a66e0740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.171+0000 7f2d79818640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55e1a66d9fb0 0x55e1a66da3d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42378/0 (socket says 172.21.15.165:42378) 2024-04-27T03:32:19.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.172+0000 7f2d79818640 1 -- 172.21.15.165:0/3294306001 learned_addr learned my addr 172.21.15.165:0/3294306001 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:19.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.172+0000 7f2d71017640 1 -- 172.21.15.165:0/3294306001 >> v1:172.21.15.89:6789/0 conn(0x55e1a66d9fb0 legacy=0x55e1a66da3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.172+0000 7f2d79017640 1 -- 172.21.15.165:0/3294306001 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1996393908 0 0) 0x55e1a6598d40 con 0x55e1a66d9fb0 2024-04-27T03:32:19.178 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.172+0000 7f2d71017640 1 -- 172.21.15.165:0/3294306001 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a6597590 con 0x55e1a66de350 2024-04-27T03:32:19.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.174+0000 7f2d71017640 1 --2- 172.21.15.165:0/3294306001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66de350 0x55e1a66e0740 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400a470 tx=0x7f2d6401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66bd5ecb9a413af4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.174+0000 7f2d79017640 1 -- 172.21.15.165:0/3294306001 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d64057070 con 0x55e1a66de350 2024-04-27T03:32:19.179 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.174+0000 7f2d79017640 1 -- 172.21.15.165:0/3294306001 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d64052570 con 0x55e1a66de350 2024-04-27T03:32:19.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.174+0000 7f2d79017640 1 -- 172.21.15.165:0/3294306001 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d64052870 con 0x55e1a66de350 2024-04-27T03:32:19.180 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.174+0000 7f2d7bd19800 1 -- 172.21.15.165:0/3294306001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66de350 msgr2=0x55e1a66e0740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.175+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/3294306001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66de350 0x55e1a66e0740 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f2d6400a470 tx=0x7f2d6401ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:19.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.175+0000 7f2d7bd19800 1 -- 172.21.15.165:0/3294306001 shutdown_connections 2024-04-27T03:32:19.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.175+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/3294306001 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66de350 0x55e1a66e0740 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.175+0000 7f2d7bd19800 1 -- 172.21.15.165:0/3294306001 >> 172.21.15.165:0/3294306001 conn(0x55e1a66d5550 msgr2=0x55e1a66d7970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.175+0000 7f2d7bd19800 1 -- 172.21.15.165:0/3294306001 shutdown_connections 2024-04-27T03:32:19.181 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.175+0000 7f2d7bd19800 1 -- 172.21.15.165:0/3294306001 wait complete. 2024-04-27T03:32:19.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.186+0000 7f2d7bd19800 1 Processor -- start 2024-04-27T03:32:19.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.186+0000 7f2d7bd19800 1 -- start start 2024-04-27T03:32:19.192 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.186+0000 7f2d7bd19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 0x55e1a664f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.187+0000 7f2d7bd19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e1a66253b0 con 0x55e1a66d9fb0 2024-04-27T03:32:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.187+0000 7f2d79818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 0x55e1a664f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.187+0000 7f2d79818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 0x55e1a664f9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60298/0 (socket says 172.21.15.165:60298) 2024-04-27T03:32:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.187+0000 7f2d79818640 1 -- 172.21.15.165:0/488278913 learned_addr learned my addr 172.21.15.165:0/488278913 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.193 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.187+0000 7f2d79818640 1 -- 172.21.15.165:0/488278913 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a664df80 con 0x55e1a66d9fb0 2024-04-27T03:32:19.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.187+0000 7f2d79818640 1 --2- 172.21.15.165:0/488278913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 0x55e1a664f9c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00b240 tx=0x7f2d6c00b700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c2c217e79eac8d server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.187+0000 7f2c67e07640 1 -- 172.21.15.165:0/488278913 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d6c008070 con 0x55e1a66d9fb0 2024-04-27T03:32:19.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.187+0000 7f2c67e07640 1 -- 172.21.15.165:0/488278913 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c00d500 con 0x55e1a66d9fb0 2024-04-27T03:32:19.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.187+0000 7f2c67e07640 1 -- 172.21.15.165:0/488278913 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d6c00d860 con 0x55e1a66d9fb0 2024-04-27T03:32:19.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.188+0000 7f2d7bd19800 1 -- 172.21.15.165:0/488278913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 msgr2=0x55e1a664f9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.188+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/488278913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 0x55e1a664f9c0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00b240 tx=0x7f2d6c00b700 comp rx=0 tx=0).stop 2024-04-27T03:32:19.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.188+0000 7f2d7bd19800 1 -- 172.21.15.165:0/488278913 shutdown_connections 2024-04-27T03:32:19.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.188+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/488278913 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 0x55e1a664f9c0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.188+0000 7f2d7bd19800 1 -- 172.21.15.165:0/488278913 >> 172.21.15.165:0/488278913 conn(0x55e1a66d5550 msgr2=0x55e1a67c68e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.188+0000 7f2d7bd19800 1 -- 172.21.15.165:0/488278913 shutdown_connections 2024-04-27T03:32:19.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.188+0000 7f2d7bd19800 1 -- 172.21.15.165:0/488278913 wait complete. 2024-04-27T03:32:19.197 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.188+0000 7f2d7bd19800 1 Processor -- start 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.189+0000 7f2d7bd19800 1 -- start start 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.189+0000 7f2d7bd19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 0x55e1a67d4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.189+0000 7f2d7bd19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e1a66e1010 con 0x55e1a66d9fb0 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.189+0000 7f2d79818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 0x55e1a67d4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.189+0000 7f2d79818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 0x55e1a67d4d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60300/0 (socket says 172.21.15.165:60300) 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.189+0000 7f2d79818640 1 -- 172.21.15.165:0/1862286996 learned_addr learned my addr 172.21.15.165:0/1862286996 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.189+0000 7f2d79818640 1 -- 172.21.15.165:0/1862286996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a67d52d0 con 0x55e1a66d9fb0 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.189+0000 7f2d79818640 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 0x55e1a67d4d90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55e1a664eff0 tx=0x7f2d6c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.190+0000 7f2c66604640 1 -- 172.21.15.165:0/1862286996 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d6c019870 con 0x55e1a66d9fb0 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.190+0000 7f2c66604640 1 -- 172.21.15.165:0/1862286996 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c019a10 con 0x55e1a66d9fb0 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.190+0000 7f2c66604640 1 -- 172.21.15.165:0/1862286996 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d6c019d30 con 0x55e1a66d9fb0 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.190+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e1a67d5560 con 0x55e1a66d9fb0 2024-04-27T03:32:19.198 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.190+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a67d8210 con 0x55e1a66d9fb0 2024-04-27T03:32:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.190+0000 7f2c66604640 1 -- 172.21.15.165:0/1862286996 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2d6c017040 con 0x55e1a66d9fb0 2024-04-27T03:32:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.191+0000 7f2c66604640 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c60036950 0x7f2c60038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.191+0000 7f2c66604640 1 -- 172.21.15.165:0/1862286996 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f2d6c015020 con 0x55e1a66d9fb0 2024-04-27T03:32:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.191+0000 7f2d71017640 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c60036950 0x7f2c60038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.191+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e1a66de350 0x55e1a67dbfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.191+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a67dc560 con 0x55e1a66de350 2024-04-27T03:32:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.191+0000 7f2d7a019640 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e1a66de350 0x55e1a67dbfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.191+0000 7f2d71017640 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c60036950 0x7f2c60038e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2d64054000 tx=0x7f2d6404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.191+0000 7f2d7a019640 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e1a66de350 0x55e1a67dbfd0 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.191+0000 7f2d7a019640 1 -- 172.21.15.165:0/1862286996 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a66de350 2024-04-27T03:32:19.199 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.191+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a67d95a0 con 0x55e1a66de350 2024-04-27T03:32:19.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.192+0000 7f2d7a019640 1 -- 172.21.15.165:0/1862286996 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a66de350 2024-04-27T03:32:19.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.192+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e1a67de890 0x55e1a67e0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.192+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a67e1320 con 0x55e1a67de890 2024-04-27T03:32:19.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.192+0000 7f2d79818640 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e1a67de890 0x55e1a67e0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.192+0000 7f2d79818640 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e1a67de890 0x55e1a67e0c80 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.192+0000 7f2d79818640 1 -- 172.21.15.165:0/1862286996 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f2d6c02d070 con 0x55e1a67de890 2024-04-27T03:32:19.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.192+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a67dd840 con 0x55e1a66de350 2024-04-27T03:32:19.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.193+0000 7f2d7a019640 1 -- 172.21.15.165:0/1862286996 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a66de350 2024-04-27T03:32:19.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.193+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e1a67e27d0 0x55e1a67e4c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.193+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a67e5380 con 0x55e1a67e27d0 2024-04-27T03:32:19.214 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.193+0000 7f2d71017640 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e1a67e27d0 0x55e1a67e4c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.193+0000 7f2d71017640 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e1a67e27d0 0x55e1a67e4c20 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.193+0000 7f2d71017640 1 -- 172.21.15.165:0/1862286996 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f2d64052940 con 0x55e1a67e27d0 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.194+0000 7f2d7bd19800 1 Processor -- start 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.194+0000 7f2d7bd19800 1 -- start start 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.194+0000 7f2d7bd19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a67f37e0 0x55e1a67f3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.194+0000 7f2d7bd19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e1a65cb6b0 con 0x55e1a67f37e0 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.194+0000 7f2d79818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a67f37e0 0x55e1a67f3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.194+0000 7f2d79818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a67f37e0 0x55e1a67f3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60306/0 (socket says 172.21.15.165:60306) 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.194+0000 7f2d79818640 1 -- 172.21.15.165:0/115375803 learned_addr learned my addr 172.21.15.165:0/115375803 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.194+0000 7f2d79818640 1 -- 172.21.15.165:0/115375803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a67f4140 con 0x55e1a67f37e0 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.194+0000 7f2d79818640 1 --2- 172.21.15.165:0/115375803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a67f37e0 0x55e1a67f3c00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c04b480 tx=0x7f2d6c00ed40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.195+0000 7f2c5f7fe640 1 -- 172.21.15.165:0/115375803 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d6c044a50 con 0x55e1a67f37e0 2024-04-27T03:32:19.215 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.195+0000 7f2c5f7fe640 1 -- 172.21.15.165:0/115375803 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c044bf0 con 0x55e1a67f37e0 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.195+0000 7f2d7bd19800 1 -- 172.21.15.165:0/115375803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e1a67f4bb0 con 0x55e1a67f37e0 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.195+0000 7f2c5f7fe640 1 -- 172.21.15.165:0/115375803 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d6c044ed0 con 0x55e1a67f37e0 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.195+0000 7f2d7bd19800 1 -- 172.21.15.165:0/115375803 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a67f50b0 con 0x55e1a67f37e0 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.195+0000 7f2c5f7fe640 1 -- 172.21.15.165:0/115375803 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2d6c03f1b0 con 0x55e1a67f37e0 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.195+0000 7f2c5f7fe640 1 --2- 172.21.15.165:0/115375803 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c54036950 0x7f2c54038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.195+0000 7f2c5f7fe640 1 -- 172.21.15.165:0/115375803 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f2d6c03cbf0 con 0x55e1a67f37e0 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.196+0000 7f2d71017640 1 --2- 172.21.15.165:0/115375803 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c54036950 0x7f2c54038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.196+0000 7f2d7bd19800 1 Processor -- start 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.196+0000 7f2d7bd19800 1 -- start start 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.196+0000 7f2d71017640 1 --2- 172.21.15.165:0/115375803 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c54036950 0x7f2c54038e10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55e1a67d94c0 tx=0x7f2d64005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.196+0000 7f2d7bd19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 0x55e1a6800ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.196+0000 7f2d7bd19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e1a664e300 con 0x55e1a6800aa0 2024-04-27T03:32:19.216 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.196+0000 7f2d79818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 0x55e1a6800ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.196+0000 7f2d79818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 0x55e1a6800ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60322/0 (socket says 172.21.15.165:60322) 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.196+0000 7f2d79818640 1 -- 172.21.15.165:0/1638500370 learned_addr learned my addr 172.21.15.165:0/1638500370 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.196+0000 7f2d79818640 1 -- 172.21.15.165:0/1638500370 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a6801490 con 0x55e1a6800aa0 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.197+0000 7f2d79818640 1 --2- 172.21.15.165:0/1638500370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 0x55e1a6800ec0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0264e0 tx=0x7f2d6c0267f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba94fed4ad7b7526 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.197+0000 7f2c5d7fa640 1 -- 172.21.15.165:0/1638500370 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d6c0408d0 con 0x55e1a6800aa0 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.197+0000 7f2c5d7fa640 1 -- 172.21.15.165:0/1638500370 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c05e020 con 0x55e1a6800aa0 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.197+0000 7f2c5d7fa640 1 -- 172.21.15.165:0/1638500370 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d6c01f140 con 0x55e1a6800aa0 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.197+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1638500370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 msgr2=0x55e1a6800ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.197+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/1638500370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 0x55e1a6800ec0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c0264e0 tx=0x7f2d6c0267f0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.197+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1638500370 shutdown_connections 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.197+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/1638500370 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 0x55e1a6800ec0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.197+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1638500370 >> 172.21.15.165:0/1638500370 conn(0x55e1a67fce00 msgr2=0x55e1a67fae60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.197+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1638500370 shutdown_connections 2024-04-27T03:32:19.217 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.197+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1638500370 wait complete. 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.198+0000 7f2d7bd19800 1 Processor -- start 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.198+0000 7f2d7bd19800 1 -- start start 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.198+0000 7f2d7bd19800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 0x55e1a6807ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.198+0000 7f2d7bd19800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e1a6801b40 con 0x55e1a6800aa0 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.198+0000 7f2d79818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 0x55e1a6807ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.198+0000 7f2d79818640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 0x55e1a6807ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60338/0 (socket says 172.21.15.165:60338) 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.198+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 learned_addr learned my addr 172.21.15.165:0/2104221055 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.198+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e1a6808400 con 0x55e1a6800aa0 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.198+0000 7f2d79818640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 0x55e1a6807ec0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c027660 tx=0x7f2d6c01af70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.199+0000 7f2c4b7fe640 1 -- 172.21.15.165:0/2104221055 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d6c05d070 con 0x55e1a6800aa0 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.199+0000 7f2c4b7fe640 1 -- 172.21.15.165:0/2104221055 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2d6c03f5d0 con 0x55e1a6800aa0 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.199+0000 7f2c4b7fe640 1 -- 172.21.15.165:0/2104221055 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2d6c01c740 con 0x55e1a6800aa0 2024-04-27T03:32:19.218 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.199+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e1a6808400 con 0x55e1a6800aa0 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.199+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e1a6808400 con 0x55e1a6800aa0 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.199+0000 7f2c4b7fe640 1 -- 172.21.15.165:0/2104221055 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2d6c027070 con 0x55e1a6800aa0 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.199+0000 7f2c4b7fe640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c44036950 0x7f2c44038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.200+0000 7f2c4b7fe640 1 -- 172.21.15.165:0/2104221055 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f2d6c06c030 con 0x55e1a6800aa0 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.200+0000 7f2d71017640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c44036950 0x7f2c44038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.200+0000 7f2d71017640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c44036950 0x7f2c44038e10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2d64052fd0 tx=0x7f2d6404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.200+0000 7f2d7bd19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.200+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e1a6813640 0x55e1a6815b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.200+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6816250 con 0x55e1a6813640 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.200+0000 7f2d7a019640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e1a6813640 0x55e1a6815b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7a019640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e1a6813640 0x55e1a6815b00 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7bd19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.219 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7bd19800 20 realm 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7bd19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6816660 con 0x55e1a6813640 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7bd19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7bd19800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7bd19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6816f30 con 0x55e1a6813640 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7bd19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7bd19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6813640 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.201+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.202+0000 7f2d7bd19800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.220 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.202+0000 7f2d7bd19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.202+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e1a6818030 0x55e1a681a420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.202+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681aa30 con 0x55e1a6818030 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.202+0000 7f2d79818640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e1a6818030 0x55e1a681a420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.202+0000 7f2d79818640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e1a6818030 0x55e1a681a420 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.202+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.202+0000 7f2d7bd19800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.202+0000 7f2d7bd19800 20 searching for the correct realm 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.202+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681c360 con 0x55e1a6818030 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.203+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.203+0000 7f2d79818640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2d6c043110 0x7f2d6c059450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.203+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681c360 con 0x7f2d6c043110 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.203+0000 7f2d71017640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2d6c043110 0x7f2d6c059450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.203+0000 7f2d71017640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2d6c043110 0x7f2d6c059450 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.221 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.203+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.203+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681c360 con 0x7f2d6c043110 2024-04-27T03:32:19.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.204+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.204+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681c360 con 0x55e1a6813640 2024-04-27T03:32:19.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.204+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.204+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6813640 2024-04-27T03:32:19.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.204+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.204+0000 7f2d7a019640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2d74016440 0x7f2d740168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.204+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d74016440 2024-04-27T03:32:19.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.204+0000 7f2d7a019640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2d74016440 0x7f2d740168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.205+0000 7f2d7a019640 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2d74016440 0x7f2d740168c0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.205+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.222 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.205+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d74016440 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.205+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.205+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d6c043110 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.205+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.205+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6818030 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.206+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.206+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6813640 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.206+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.206+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d6c043110 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.206+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.206+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6813640 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.206+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.206+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d6c043110 2024-04-27T03:32:19.223 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.207+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.207+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6818030 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.207+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.207+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6818030 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.207+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.207+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d74016440 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.207+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.207+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6818030 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.208+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.208+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6818030 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.208+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.208+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6813640 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.208+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.208+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6813640 2024-04-27T03:32:19.224 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.208+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.225 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.208+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d74016440 2024-04-27T03:32:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.209+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.209+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6818030 2024-04-27T03:32:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.209+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.209+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d6c043110 2024-04-27T03:32:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.209+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.209+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6813640 2024-04-27T03:32:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.209+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.236 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.209+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d74016440 2024-04-27T03:32:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.210+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.210+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6818030 2024-04-27T03:32:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.210+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.210+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d74016440 2024-04-27T03:32:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.210+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.210+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d6c043110 2024-04-27T03:32:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.211+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.211+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6818030 2024-04-27T03:32:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.211+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.211+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d74016440 2024-04-27T03:32:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.211+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.237 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.211+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x55e1a6813640 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68172d0 con 0x7f2d6c043110 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681b2f0 con 0x55e1a6813640 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.238 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681b6e0 con 0x55e1a6813640 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.212+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681b6e0 con 0x7f2d6c043110 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7bd19800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7bd19800 20 zone default found 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7bd19800 4 Realm: () 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7bd19800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7bd19800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7bd19800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:19.239 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7bd19800 20 zonegroup default 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7bd19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681c7e0 con 0x7f2d74016440 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7bd19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7bd19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.213+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681e1c0 con 0x55e1a6818030 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.214+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.214+0000 7f2d7bd19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.214+0000 7f2d7bd19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.214+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681e620 con 0x55e1a6818030 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.214+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.214+0000 7f2d7bd19800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.214+0000 7f2d7bd19800 20 started sync module instance, tier type = 2024-04-27T03:32:19.240 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.214+0000 7f2d7bd19800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.214+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681d660 con 0x7f2d6c043110 2024-04-27T03:32:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.215+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.215+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94427649530448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681d2c0 con 0x7f2d6c043110 2024-04-27T03:32:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.215+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681f460 con 0x55e1a6813640 2024-04-27T03:32:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.216+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94427649530448] v27'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.216+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.216+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94427649534592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681f850 con 0x55e1a6813640 2024-04-27T03:32:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.216+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6820290 con 0x55e1a6818030 2024-04-27T03:32:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.218+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.218+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94427649534592] v27'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.218+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94427649539424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6820b60 con 0x55e1a6818030 2024-04-27T03:32:19.241 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.218+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a681eea0 con 0x7f2d6c043110 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.219+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94427649539424] v27'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.219+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.219+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94427649542080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6821600 con 0x7f2d6c043110 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.219+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6821fc0 con 0x55e1a6813640 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.220+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.220+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94427649547504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6822740 con 0x55e1a6813640 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.220+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6823260 con 0x7f2d74016440 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.220+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94427649542080] v27'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.221+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94427649547504] v27'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.221+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.221+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94427649551664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6823220 con 0x7f2d74016440 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.221+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6823b00 con 0x7f2d74016440 2024-04-27T03:32:19.242 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.222+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94427649551664] v27'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.222+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.222+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94427649544528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68236d0 con 0x7f2d74016440 2024-04-27T03:32:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.222+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6824180 con 0x7f2d74016440 2024-04-27T03:32:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.223+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(58 notify.7 [create] v27'261 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.223+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(57 notify.6 [watch watch cookie 94427649544528] v27'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.223+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94427649555552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6824630 con 0x7f2d74016440 2024-04-27T03:32:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.223+0000 7f2d7bd19800 20 add_watcher() i=0 2024-04-27T03:32:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.223+0000 7f2d7bd19800 20 add_watcher() i=1 2024-04-27T03:32:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.223+0000 7f2d7bd19800 20 add_watcher() i=2 2024-04-27T03:32:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.223+0000 7f2d7bd19800 20 add_watcher() i=3 2024-04-27T03:32:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.223+0000 7f2d7bd19800 20 add_watcher() i=4 2024-04-27T03:32:19.243 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.223+0000 7f2d7bd19800 20 add_watcher() i=5 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.223+0000 7f2d7bd19800 20 add_watcher() i=6 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.224+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94427649555552] v27'262 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.224+0000 7f2d7bd19800 20 add_watcher() i=7 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.224+0000 7f2d7bd19800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.224+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6824630 con 0x55e1a6813640 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.225+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.225+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94427649556864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6824eb0 con 0x55e1a6813640 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.226+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94427649556864] v27'265 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.226+0000 7f2c2b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.226+0000 7f2d7bd19800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.226+0000 7f2d7bd19800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.226+0000 7f2d7bd19800 5 note: GC not initialized 2024-04-27T03:32:19.244 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.226+0000 7f2c2a7fc640 20 reqs_thread_entry: start 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.226+0000 7f2d7bd19800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.226+0000 7f2d7bd19800 20 Filter name: none 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.227+0000 7f2b9bfff640 20 reqs_thread_entry: start 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.227+0000 7f2d7bd19800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.227+0000 7f2d7bd19800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.227+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a685a8c0 con 0x7f2d74016440 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.227+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 ada [call out=48b,read 0~490 out=490b,getxattrs out=75b] v0'0 uv6 ondisk = 0) ==== 231+0+613 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.227+0000 7f2d7bd19800 20 rados_obj.operate() r=0 bl.length=490 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.227+0000 7f2d7bd19800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.227+0000 7f2d7bd19800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.227+0000 7f2d7bd19800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.227+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~490 in=490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6861230 con 0x7f2d74016440 2024-04-27T03:32:19.245 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~490] v27'7 uv7 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2d7bd19800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2d7bd19800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2d7bd19800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-27T03:32:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94427649805472 in=678b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e1a685c900 con 0x55e1a6813640 2024-04-27T03:32:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== watch-notify(notify (1) cookie 94427649547504 notify 115964117043 ret 0) ==== 708+0+0 (crc 0 0 0) 0x7f2d74030040 con 0x55e1a6813640 2024-04-27T03:32:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2c5d7fa640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117043 cookie 94427649547504 notifier 4868 bl.length()=666 2024-04-27T03:32:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2c5d7fa640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2c5d7fa640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2c5d7fa640 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f2c3c003ea0 con 0x55e1a6813640 2024-04-27T03:32:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 notify.4 [notify cookie 94427649805472 out=8b] v0'0 uv261 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.246 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.229+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify_complete (2) cookie 94427649805472 notify 115964117043 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f2d7402a910 con 0x55e1a6813640 2024-04-27T03:32:19.247 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-27T03:32:18.064161Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:32:18.064141Z","tags":[],"group_ids":[]} 2024-04-27T03:32:19.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.230+0000 7f2c2b7fe640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.230+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94427649556864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a685f8d0 con 0x55e1a6813640 2024-04-27T03:32:19.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.231+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94427649556864] v27'266 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.231+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94427649530448] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6824180 con 0x7f2d6c043110 2024-04-27T03:32:19.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.232+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94427649530448] v27'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.252 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.232+0000 7f2d7bd19800 20 remove_watcher() i=0 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.232+0000 7f2d7bd19800 2 removed watcher, disabling cache 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.232+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94427649534592] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6824180 con 0x55e1a6813640 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.233+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94427649534592] v27'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.233+0000 7f2d7bd19800 20 remove_watcher() i=1 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.233+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94427649539424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6824180 con 0x55e1a6818030 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.234+0000 7f2d79818640 1 -- 172.21.15.165:0/2104221055 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94427649539424] v27'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d6c053040 con 0x55e1a6818030 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.234+0000 7f2d7bd19800 20 remove_watcher() i=2 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.234+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94427649542080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6824180 con 0x7f2d6c043110 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.235+0000 7f2d71017640 1 -- 172.21.15.165:0/2104221055 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94427649542080] v27'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d64081070 con 0x7f2d6c043110 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.235+0000 7f2d7bd19800 20 remove_watcher() i=3 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.235+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94427649547504] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68207c0 con 0x55e1a6813640 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.236+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94427649547504] v27'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x55e1a6813640 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.236+0000 7f2d7bd19800 20 remove_watcher() i=4 2024-04-27T03:32:19.253 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.236+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94427649551664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6824180 con 0x7f2d74016440 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.237+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94427649551664] v27'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.237+0000 7f2d7bd19800 20 remove_watcher() i=5 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.237+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94427649544528] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a6821cf0 con 0x7f2d74016440 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.238+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94427649544528] v27'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.238+0000 7f2d7bd19800 20 remove_watcher() i=6 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.238+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94427649555552] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e1a68207c0 con 0x7f2d74016440 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94427649555552] v27'263 uv261 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f2d7400a4a0 con 0x7f2d74016440 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7bd19800 20 remove_watcher() i=7 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2d74016440 msgr2=0x7f2d740168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f2d74016440 0x7f2d740168c0 crc :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e1a6813640 msgr2=0x55e1a6815b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e1a6813640 0x55e1a6815b00 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.254 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2d6c043110 msgr2=0x7f2d6c059450 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f2d6c043110 0x7f2d6c059450 crc :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e1a6818030 msgr2=0x55e1a681a420 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e1a6818030 0x55e1a681a420 crc :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c44036950 msgr2=0x7f2c44038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c44036950 0x7f2c44038e10 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f2d64052fd0 tx=0x7f2d6404f000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 msgr2=0x55e1a6807ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/2104221055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a6800aa0 0x55e1a6807ec0 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c027660 tx=0x7f2d6c01af70 comp rx=0 tx=0).stop 2024-04-27T03:32:19.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.239+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 reap_dead start 2024-04-27T03:32:19.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7a019640 1 -- 172.21.15.165:0/2104221055 reap_dead start 2024-04-27T03:32:19.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 shutdown_connections 2024-04-27T03:32:19.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 >> 172.21.15.165:0/2104221055 conn(0x55e1a67fce00 msgr2=0x55e1a67fa460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.255 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 shutdown_connections 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/2104221055 wait complete. 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/115375803 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c54036950 msgr2=0x7f2c54038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/115375803 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c54036950 0x7f2c54038e10 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x55e1a67d94c0 tx=0x7f2d64005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/115375803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a67f37e0 msgr2=0x55e1a67f3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/115375803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a67f37e0 0x55e1a67f3c00 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c04b480 tx=0x7f2d6c00ed40 comp rx=0 tx=0).stop 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/115375803 shutdown_connections 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/115375803 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c54036950 0x7f2c54038e10 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/115375803 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a67f37e0 0x55e1a67f3c00 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/115375803 >> 172.21.15.165:0/115375803 conn(0x55e1a67e7880 msgr2=0x55e1a67e7c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/115375803 shutdown_connections 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/115375803 wait complete. 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e1a66de350 msgr2=0x55e1a67dbfd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e1a66de350 0x55e1a67dbfd0 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.256 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e1a67e27d0 msgr2=0x55e1a67e4c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e1a67e27d0 0x55e1a67e4c20 crc :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e1a67de890 msgr2=0x55e1a67e0c80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e1a67de890 0x55e1a67e0c80 crc :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c60036950 msgr2=0x7f2c60038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2c60036950 0x7f2c60038e10 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f2d64054000 tx=0x7f2d6404e000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 msgr2=0x55e1a67d4d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7bd19800 1 --2- 172.21.15.165:0/1862286996 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e1a66d9fb0 0x55e1a67d4d90 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x55e1a664eff0 tx=0x7f2d6c000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.240+0000 7f2d7a019640 1 -- 172.21.15.165:0/1862286996 reap_dead start 2024-04-27T03:32:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.241+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 shutdown_connections 2024-04-27T03:32:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.241+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 >> 172.21.15.165:0/1862286996 conn(0x55e1a66d5550 msgr2=0x55e1a664e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.241+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 shutdown_connections 2024-04-27T03:32:19.257 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.241+0000 7f2d7bd19800 1 -- 172.21.15.165:0/1862286996 wait complete. 2024-04-27T03:32:19.261 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.262 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.262 ERROR:root:0 2024-04-27T03:32:19.262 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:19.266 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:19.267 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-27T03:32:19.267 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-27T03:32:19.267 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'users=read, write' 2024-04-27T03:32:19.311 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:19.312 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:19.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.307+0000 7ffa63430800 1 Processor -- start 2024-04-27T03:32:19.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.307+0000 7ffa63430800 1 -- start start 2024-04-27T03:32:19.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.307+0000 7ffa63430800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x563bb07794c0 0x563bb077b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.307+0000 7ffa63430800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x563bb0633d40 con 0x563bb07759b0 2024-04-27T03:32:19.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.307+0000 7ffa63430800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x563bb06064d0 con 0x563bb07794c0 2024-04-27T03:32:19.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.307+0000 7ffa5de8a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x563bb07794c0 0x563bb077b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.307+0000 7ffa5e68b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x563bb07759b0 0x563bb0777da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42382/0 (socket says 172.21.15.165:42382) 2024-04-27T03:32:19.313 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.307+0000 7ffa5e68b640 1 -- 172.21.15.165:0/1630111140 learned_addr learned my addr 172.21.15.165:0/1630111140 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:19.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.308+0000 7ffa5de8a640 1 -- 172.21.15.165:0/1630111140 >> v1:172.21.15.89:6789/0 conn(0x563bb07759b0 legacy=0x563bb0777da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.308+0000 7ffa5d689640 1 -- 172.21.15.165:0/1630111140 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1692517161 0 0) 0x563bb0633d40 con 0x563bb07759b0 2024-04-27T03:32:19.314 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.308+0000 7ffa5de8a640 1 -- 172.21.15.165:0/1630111140 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb0632590 con 0x563bb07794c0 2024-04-27T03:32:19.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.309+0000 7ffa5de8a640 1 --2- 172.21.15.165:0/1630111140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07794c0 0x563bb077b8b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400a470 tx=0x7ffa5401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62503c3e7a0712a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.310+0000 7ffa5d689640 1 -- 172.21.15.165:0/1630111140 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa54057070 con 0x563bb07794c0 2024-04-27T03:32:19.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.310+0000 7ffa5d689640 1 -- 172.21.15.165:0/1630111140 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa54052570 con 0x563bb07794c0 2024-04-27T03:32:19.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.310+0000 7ffa5d689640 1 -- 172.21.15.165:0/1630111140 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa54052870 con 0x563bb07794c0 2024-04-27T03:32:19.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.310+0000 7ffa63430800 1 -- 172.21.15.165:0/1630111140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07794c0 msgr2=0x563bb077b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.310+0000 7ffa63430800 1 --2- 172.21.15.165:0/1630111140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07794c0 0x563bb077b8b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffa5400a470 tx=0x7ffa5401ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:19.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.310+0000 7ffa63430800 1 -- 172.21.15.165:0/1630111140 shutdown_connections 2024-04-27T03:32:19.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.310+0000 7ffa63430800 1 --2- 172.21.15.165:0/1630111140 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07794c0 0x563bb077b8b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.310+0000 7ffa63430800 1 -- 172.21.15.165:0/1630111140 >> 172.21.15.165:0/1630111140 conn(0x563bb076f340 msgr2=0x563bb0771760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.310+0000 7ffa63430800 1 -- 172.21.15.165:0/1630111140 shutdown_connections 2024-04-27T03:32:19.316 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.310+0000 7ffa63430800 1 -- 172.21.15.165:0/1630111140 wait complete. 2024-04-27T03:32:19.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.322+0000 7ffa63430800 1 Processor -- start 2024-04-27T03:32:19.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.322+0000 7ffa63430800 1 -- start start 2024-04-27T03:32:19.328 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.322+0000 7ffa63430800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 0x563bb0865fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.322+0000 7ffa63430800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563bb06c03b0 con 0x563bb07759b0 2024-04-27T03:32:19.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.322+0000 7ffa5e68b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 0x563bb0865fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.322+0000 7ffa5e68b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 0x563bb0865fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60354/0 (socket says 172.21.15.165:60354) 2024-04-27T03:32:19.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.322+0000 7ffa5e68b640 1 -- 172.21.15.165:0/3001768961 learned_addr learned my addr 172.21.15.165:0/3001768961 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.323+0000 7ffa5e68b640 1 -- 172.21.15.165:0/3001768961 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb08665a0 con 0x563bb07759b0 2024-04-27T03:32:19.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.323+0000 7ffa5e68b640 1 --2- 172.21.15.165:0/3001768961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 0x563bb0865fd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000b210 tx=0x7ffa5000b6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ff97070a2a61f2e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.323+0000 7ff94c608640 1 -- 172.21.15.165:0/3001768961 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa50008070 con 0x563bb07759b0 2024-04-27T03:32:19.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.323+0000 7ff94c608640 1 -- 172.21.15.165:0/3001768961 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5000d500 con 0x563bb07759b0 2024-04-27T03:32:19.329 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.323+0000 7ff94c608640 1 -- 172.21.15.165:0/3001768961 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa5000d840 con 0x563bb07759b0 2024-04-27T03:32:19.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa63430800 1 -- 172.21.15.165:0/3001768961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 msgr2=0x563bb0865fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.332 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa63430800 1 --2- 172.21.15.165:0/3001768961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 0x563bb0865fd0 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000b210 tx=0x7ffa5000b6d0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa63430800 1 -- 172.21.15.165:0/3001768961 shutdown_connections 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa63430800 1 --2- 172.21.15.165:0/3001768961 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 0x563bb0865fd0 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa63430800 1 -- 172.21.15.165:0/3001768961 >> 172.21.15.165:0/3001768961 conn(0x563bb076f340 msgr2=0x563bb08618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa63430800 1 -- 172.21.15.165:0/3001768961 shutdown_connections 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa63430800 1 -- 172.21.15.165:0/3001768961 wait complete. 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa63430800 1 Processor -- start 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa63430800 1 -- start start 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa63430800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 0x563bb086db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa63430800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563bb077e100 con 0x563bb07759b0 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa5e68b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 0x563bb086db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa5e68b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 0x563bb086db50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60360/0 (socket says 172.21.15.165:60360) 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.324+0000 7ffa5e68b640 1 -- 172.21.15.165:0/3478486690 learned_addr learned my addr 172.21.15.165:0/3478486690 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.333 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.325+0000 7ffa5e68b640 1 -- 172.21.15.165:0/3478486690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb086e090 con 0x563bb07759b0 2024-04-27T03:32:19.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.325+0000 7ffa5e68b640 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 0x563bb086db50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000b210 tx=0x7ffa50000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.325+0000 7ff94ae05640 1 -- 172.21.15.165:0/3478486690 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa5000f950 con 0x563bb07759b0 2024-04-27T03:32:19.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.325+0000 7ff94ae05640 1 -- 172.21.15.165:0/3478486690 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5000faf0 con 0x563bb07759b0 2024-04-27T03:32:19.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.325+0000 7ff94ae05640 1 -- 172.21.15.165:0/3478486690 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa5000fdd0 con 0x563bb07759b0 2024-04-27T03:32:19.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.325+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563bb086e320 con 0x563bb07759b0 2024-04-27T03:32:19.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.325+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb0870fd0 con 0x563bb07759b0 2024-04-27T03:32:19.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.326+0000 7ff94ae05640 1 -- 172.21.15.165:0/3478486690 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffa50017040 con 0x563bb07759b0 2024-04-27T03:32:19.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.326+0000 7ff94ae05640 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9440368b0 0x7ff944038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.326+0000 7ff94ae05640 1 -- 172.21.15.165:0/3478486690 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7ffa50015020 con 0x563bb07759b0 2024-04-27T03:32:19.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.326+0000 7ffa5de8a640 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9440368b0 0x7ff944038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.326+0000 7ffa63430800 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563bb07794c0 0x563bb0874d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.334 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.326+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb0875310 con 0x563bb07794c0 2024-04-27T03:32:19.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.327+0000 7ffa5ee8c640 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563bb07794c0 0x563bb0874d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.327+0000 7ffa5de8a640 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9440368b0 0x7ff944038d70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffa54054000 tx=0x7ffa5404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.327+0000 7ffa5ee8c640 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563bb07794c0 0x563bb0874d90 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.327+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/3478486690 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb07794c0 2024-04-27T03:32:19.335 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.327+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb0872360 con 0x563bb07794c0 2024-04-27T03:32:19.345 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.327+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/3478486690 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb07794c0 2024-04-27T03:32:19.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.327+0000 7ffa63430800 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563bb0877640 0x563bb0879a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.327+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb087a0d0 con 0x563bb0877640 2024-04-27T03:32:19.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.328+0000 7ffa5e68b640 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563bb0877640 0x563bb0879a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.328+0000 7ffa5e68b640 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563bb0877640 0x563bb0879a30 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.328+0000 7ffa5e68b640 1 -- 172.21.15.165:0/3478486690 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7ffa5002d070 con 0x563bb0877640 2024-04-27T03:32:19.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.328+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08765f0 con 0x563bb07794c0 2024-04-27T03:32:19.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.328+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/3478486690 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb07794c0 2024-04-27T03:32:19.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.328+0000 7ffa63430800 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x563bb087b580 0x563bb087d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.328+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb087e130 con 0x563bb087b580 2024-04-27T03:32:19.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.328+0000 7ffa5de8a640 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x563bb087b580 0x563bb087d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.329+0000 7ffa5de8a640 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x563bb087b580 0x563bb087d9d0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.346 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.329+0000 7ffa5de8a640 1 -- 172.21.15.165:0/3478486690 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7ffa54052940 con 0x563bb087b580 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.329+0000 7ffa63430800 1 Processor -- start 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.329+0000 7ffa63430800 1 -- start start 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.330+0000 7ffa63430800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb0775550 0x563bb07739c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.330+0000 7ffa63430800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563bb06666b0 con 0x563bb0775550 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.330+0000 7ffa5e68b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb0775550 0x563bb07739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.330+0000 7ffa5e68b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb0775550 0x563bb07739c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60368/0 (socket says 172.21.15.165:60368) 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.330+0000 7ffa5e68b640 1 -- 172.21.15.165:0/3944768808 learned_addr learned my addr 172.21.15.165:0/3944768808 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.330+0000 7ffa5e68b640 1 -- 172.21.15.165:0/3944768808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb0773f00 con 0x563bb0775550 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.330+0000 7ffa5e68b640 1 --2- 172.21.15.165:0/3944768808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb0775550 0x563bb07739c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000b710 tx=0x7ffa50041d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.330+0000 7ff943fff640 1 -- 172.21.15.165:0/3944768808 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa500195a0 con 0x563bb0775550 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.330+0000 7ff943fff640 1 -- 172.21.15.165:0/3944768808 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa50019740 con 0x563bb0775550 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.330+0000 7ff943fff640 1 -- 172.21.15.165:0/3944768808 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa50044af0 con 0x563bb0775550 2024-04-27T03:32:19.347 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.330+0000 7ffa63430800 1 -- 172.21.15.165:0/3944768808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563bb0774190 con 0x563bb0775550 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.330+0000 7ffa63430800 1 -- 172.21.15.165:0/3944768808 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb0774690 con 0x563bb0775550 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.331+0000 7ff943fff640 1 -- 172.21.15.165:0/3944768808 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffa50044c90 con 0x563bb0775550 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.331+0000 7ff943fff640 1 --2- 172.21.15.165:0/3944768808 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff938036950 0x7ff938038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.331+0000 7ff943fff640 1 -- 172.21.15.165:0/3944768808 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7ffa5003bbd0 con 0x563bb0775550 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.331+0000 7ffa5de8a640 1 --2- 172.21.15.165:0/3944768808 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff938036950 0x7ff938038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.332+0000 7ffa63430800 1 Processor -- start 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.332+0000 7ffa63430800 1 -- start start 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.332+0000 7ffa5de8a640 1 --2- 172.21.15.165:0/3944768808 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff938036950 0x7ff938038e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x563bb0872280 tx=0x7ffa54005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.332+0000 7ffa63430800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 0x563bb089bef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.332+0000 7ffa63430800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563bb0866920 con 0x563bb089bad0 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.332+0000 7ffa5e68b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 0x563bb089bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.332+0000 7ffa5e68b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 0x563bb089bef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60374/0 (socket says 172.21.15.165:60374) 2024-04-27T03:32:19.348 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.332+0000 7ffa5e68b640 1 -- 172.21.15.165:0/1043896235 learned_addr learned my addr 172.21.15.165:0/1043896235 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.332+0000 7ffa5e68b640 1 -- 172.21.15.165:0/1043896235 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb089c4c0 con 0x563bb089bad0 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.332+0000 7ffa5e68b640 1 --2- 172.21.15.165:0/1043896235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 0x563bb089bef0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffa50018fd0 tx=0x7ffa5000c570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a08eb6437087aa3f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.333+0000 7ff941ffb640 1 -- 172.21.15.165:0/1043896235 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa5001cc10 con 0x563bb089bad0 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.333+0000 7ff941ffb640 1 -- 172.21.15.165:0/1043896235 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa50030d50 con 0x563bb089bad0 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.333+0000 7ff941ffb640 1 -- 172.21.15.165:0/1043896235 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa50018980 con 0x563bb089bad0 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.333+0000 7ffa63430800 1 -- 172.21.15.165:0/1043896235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 msgr2=0x563bb089bef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.333+0000 7ffa63430800 1 --2- 172.21.15.165:0/1043896235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 0x563bb089bef0 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ffa50018fd0 tx=0x7ffa5000c570 comp rx=0 tx=0).stop 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.333+0000 7ffa63430800 1 -- 172.21.15.165:0/1043896235 shutdown_connections 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.333+0000 7ffa63430800 1 --2- 172.21.15.165:0/1043896235 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 0x563bb089bef0 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.333+0000 7ffa63430800 1 -- 172.21.15.165:0/1043896235 >> 172.21.15.165:0/1043896235 conn(0x563bb0897da0 msgr2=0x563bb0895e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.333+0000 7ffa63430800 1 -- 172.21.15.165:0/1043896235 shutdown_connections 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.333+0000 7ffa63430800 1 -- 172.21.15.165:0/1043896235 wait complete. 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.333+0000 7ffa63430800 1 Processor -- start 2024-04-27T03:32:19.349 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.333+0000 7ffa63430800 1 -- start start 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.334+0000 7ffa63430800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 0x563bb08a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.334+0000 7ffa63430800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x563bb089c9a0 con 0x563bb089bad0 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.334+0000 7ffa5e68b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 0x563bb08a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.334+0000 7ffa5e68b640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 0x563bb08a2c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60378/0 (socket says 172.21.15.165:60378) 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.334+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 learned_addr learned my addr 172.21.15.165:0/2496423828 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.334+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563bb08a31d0 con 0x563bb089bad0 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.334+0000 7ffa5e68b640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 0x563bb08a2c90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000c5b0 tx=0x7ffa50026f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.334+0000 7ff92ffff640 1 -- 172.21.15.165:0/2496423828 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa5006e070 con 0x563bb089bad0 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.334+0000 7ff92ffff640 1 -- 172.21.15.165:0/2496423828 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7ffa5003f200 con 0x563bb089bad0 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.334+0000 7ff92ffff640 1 -- 172.21.15.165:0/2496423828 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7ffa50018980 con 0x563bb089bad0 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.334+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x563bb08a31d0 con 0x563bb089bad0 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.334+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563bb08a31d0 con 0x563bb089bad0 2024-04-27T03:32:19.350 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.335+0000 7ff92ffff640 1 -- 172.21.15.165:0/2496423828 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7ffa50071070 con 0x563bb089bad0 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.335+0000 7ff92ffff640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff928036900 0x7ff928038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.335+0000 7ff92ffff640 1 -- 172.21.15.165:0/2496423828 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7ffa50066070 con 0x563bb089bad0 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.335+0000 7ffa5de8a640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff928036900 0x7ff928038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.336+0000 7ffa5de8a640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff928036900 0x7ff928038dc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffa54052fd0 tx=0x7ffa5404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.336+0000 7ffa63430800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.336+0000 7ffa63430800 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563bb08ae260 0x563bb08b0720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.336+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b0e70 con 0x563bb08ae260 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.336+0000 7ffa5ee8c640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563bb08ae260 0x563bb08b0720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.336+0000 7ffa5ee8c640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563bb08ae260 0x563bb08b0720 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.336+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.336+0000 7ffa63430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.336+0000 7ffa63430800 20 realm 2024-04-27T03:32:19.351 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.336+0000 7ffa63430800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.336+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1280 con 0x563bb08ae260 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa63430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa63430800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa63430800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1b50 con 0x563bb08ae260 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa63430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa63430800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08ae260 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa63430800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa63430800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.352 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa63430800 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563bb08b2c50 0x563bb08b5040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b5650 con 0x563bb08b2c50 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa5e68b640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563bb08b2c50 0x563bb08b5040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.337+0000 7ffa5e68b640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563bb08b2c50 0x563bb08b5040 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.338+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.338+0000 7ffa63430800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.338+0000 7ffa63430800 20 searching for the correct realm 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.338+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b6f80 con 0x563bb08b2c50 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.338+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.338+0000 7ffa5e68b640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffa50073060 0x7ffa500305f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.338+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b6f80 con 0x7ffa50073060 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.338+0000 7ffa5de8a640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffa50073060 0x7ffa500305f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.339+0000 7ffa5de8a640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffa50073060 0x7ffa500305f0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.353 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.339+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.339+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b6f80 con 0x7ffa50073060 2024-04-27T03:32:19.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.339+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.339+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b6f80 con 0x563bb08ae260 2024-04-27T03:32:19.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.339+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.339+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08ae260 2024-04-27T03:32:19.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.340+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.340+0000 7ffa5ee8c640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffa58016440 0x7ffa580168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.340+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa58016440 2024-04-27T03:32:19.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.340+0000 7ffa5ee8c640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffa58016440 0x7ffa580168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.354 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.340+0000 7ffa5ee8c640 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffa58016440 0x7ffa580168c0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.340+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.341+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa58016440 2024-04-27T03:32:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.341+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.341+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa50073060 2024-04-27T03:32:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.341+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.341+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08b2c50 2024-04-27T03:32:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.341+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.364 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.341+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08ae260 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.341+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.341+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa50073060 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.342+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.342+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08ae260 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.342+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.342+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa50073060 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.342+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.342+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08b2c50 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.342+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.342+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08b2c50 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.343+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.343+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa58016440 2024-04-27T03:32:19.365 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.343+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.343+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08b2c50 2024-04-27T03:32:19.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.343+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.343+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08b2c50 2024-04-27T03:32:19.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.343+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.343+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08ae260 2024-04-27T03:32:19.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.344+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.344+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08ae260 2024-04-27T03:32:19.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.344+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.344+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa58016440 2024-04-27T03:32:19.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.344+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.344+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08b2c50 2024-04-27T03:32:19.366 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.344+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.344+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa50073060 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.345+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.345+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08ae260 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.345+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.345+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa58016440 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.345+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.345+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08b2c50 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.345+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.346+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa58016440 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.346+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.346+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa50073060 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.346+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.346+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08b2c50 2024-04-27T03:32:19.367 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.346+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.346+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa58016440 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.346+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.346+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x563bb08ae260 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b1ef0 con 0x7ffa50073060 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.368 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b5f10 con 0x563bb08ae260 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b6300 con 0x563bb08ae260 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.347+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b6300 con 0x7ffa50073060 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 20 zone default found 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 4 Realm: () 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:19.369 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 20 zonegroup default 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b7400 con 0x7ffa58016440 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b8de0 con 0x563bb08b2c50 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.348+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b92c0 con 0x563bb08b2c50 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.349+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.370 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.349+0000 7ffa63430800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.349+0000 7ffa63430800 20 started sync module instance, tier type = 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.349+0000 7ffa63430800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.349+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b8280 con 0x7ffa50073060 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.350+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.350+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94814364993136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08b7ee0 con 0x7ffa50073060 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.350+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08ba0c0 con 0x563bb08ae260 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.351+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94814364993136] v27'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.351+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.351+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94814364997408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08ba4b0 con 0x563bb08ae260 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.351+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08baee0 con 0x563bb08b2c50 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.352+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94814364997408] v27'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.352+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.371 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.352+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94814364998464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bb300 con 0x563bb08b2c50 2024-04-27T03:32:19.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.352+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bbd40 con 0x7ffa50073060 2024-04-27T03:32:19.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.355+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94814364998464] v27'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.355+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.355+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94814365002224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bc5e0 con 0x7ffa50073060 2024-04-27T03:32:19.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.355+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bcfa0 con 0x563bb08ae260 2024-04-27T03:32:19.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.356+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94814365002224] v27'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.356+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.356+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94814365010672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bd3c0 con 0x563bb08ae260 2024-04-27T03:32:19.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.357+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bdd80 con 0x7ffa58016440 2024-04-27T03:32:19.372 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.357+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94814365010672] v27'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.359+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.359+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94814365012208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bdd80 con 0x7ffa58016440 2024-04-27T03:32:19.376 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.359+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08be7a0 con 0x7ffa58016440 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.360+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94814365012208] v27'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.360+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.360+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94814365016720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08be980 con 0x7ffa58016440 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.360+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bf1d0 con 0x7ffa58016440 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.361+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94814365016720] v27'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.361+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'264 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.361+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94814365018496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bf250 con 0x7ffa58016440 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.361+0000 7ffa63430800 20 add_watcher() i=0 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.361+0000 7ffa63430800 20 add_watcher() i=1 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.361+0000 7ffa63430800 20 add_watcher() i=2 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.361+0000 7ffa63430800 20 add_watcher() i=3 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.361+0000 7ffa63430800 20 add_watcher() i=4 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.361+0000 7ffa63430800 20 add_watcher() i=5 2024-04-27T03:32:19.377 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.361+0000 7ffa63430800 20 add_watcher() i=6 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.362+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94814365018496] v27'265 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.362+0000 7ffa63430800 20 add_watcher() i=7 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.362+0000 7ffa63430800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.362+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08c2570 con 0x563bb08ae260 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.363+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.363+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94814365019920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bf7e0 con 0x563bb08ae260 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.364+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94814365019920] v27'267 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.364+0000 7ff914fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.364+0000 7ffa63430800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.364+0000 7ffa63430800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.364+0000 7ffa63430800 5 note: GC not initialized 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.364+0000 7ff88f7fe640 20 reqs_thread_entry: start 2024-04-27T03:32:19.378 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.364+0000 7ffa63430800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.364+0000 7ffa63430800 20 Filter name: none 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.365+0000 7ff88d7fa640 20 reqs_thread_entry: start 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.365+0000 7ffa63430800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.365+0000 7ffa63430800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.365+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08f5940 con 0x7ffa58016440 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.365+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 ada [call out=48b,read 0~490 out=490b,getxattrs out=75b] v0'0 uv7 ondisk = 0) ==== 231+0+613 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.365+0000 7ffa63430800 20 rados_obj.operate() r=0 bl.length=490 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.365+0000 7ffa63430800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.365+0000 7ffa63430800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.365+0000 7ffa63430800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.365+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~503 in=503b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08fc2a0 con 0x7ffa58016440 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.366+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~503] v27'8 uv8 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.379 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.366+0000 7ffa63430800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.366+0000 7ffa63430800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.366+0000 7ffa63430800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-27T03:32:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.366+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94814365269264 in=691b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x563bb08f7970 con 0x563bb08ae260 2024-04-27T03:32:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.366+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== watch-notify(notify (1) cookie 94814365010672 notify 115964117044 ret 0) ==== 721+0+0 (crc 0 0 0) 0x7ffa58030040 con 0x563bb08ae260 2024-04-27T03:32:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.367+0000 7ff941ffb640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117044 cookie 94814365010672 notifier 4875 bl.length()=679 2024-04-27T03:32:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.367+0000 7ff941ffb640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.367+0000 7ff941ffb640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.367+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 notify.4 [notify cookie 94814365269264 out=8b] v0'0 uv264 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.367+0000 7ff941ffb640 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7ff924003ea0 con 0x563bb08ae260 2024-04-27T03:32:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.367+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.380 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.367+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify_complete (2) cookie 94814365269264 notify 115964117044 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7ffa5802a5e0 con 0x563bb08ae260 2024-04-27T03:32:19.381 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-27T03:32:18.064161Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:32:18.064141Z","tags":[],"group_ids":[]} 2024-04-27T03:32:19.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.368+0000 7ff914fe1640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.368+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94814365019920] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08c1790 con 0x563bb08ae260 2024-04-27T03:32:19.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.369+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94814365019920] v27'268 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.369+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94814364993136] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bef10 con 0x7ffa50073060 2024-04-27T03:32:19.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.370+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94814364993136] v27'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.370+0000 7ffa63430800 20 remove_watcher() i=0 2024-04-27T03:32:19.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.370+0000 7ffa63430800 2 removed watcher, disabling cache 2024-04-27T03:32:19.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.370+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94814364997408] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bef10 con 0x563bb08ae260 2024-04-27T03:32:19.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.371+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94814364997408] v27'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.371+0000 7ffa63430800 20 remove_watcher() i=1 2024-04-27T03:32:19.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.371+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94814364998464] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bef10 con 0x563bb08b2c50 2024-04-27T03:32:19.386 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.372+0000 7ffa5e68b640 1 -- 172.21.15.165:0/2496423828 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94814364998464] v27'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa50070030 con 0x563bb08b2c50 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.372+0000 7ffa63430800 20 remove_watcher() i=2 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.372+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94814365002224] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bef10 con 0x7ffa50073060 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.373+0000 7ffa5de8a640 1 -- 172.21.15.165:0/2496423828 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94814365002224] v27'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa54081070 con 0x7ffa50073060 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.373+0000 7ffa63430800 20 remove_watcher() i=3 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.374+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94814365010672] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bef10 con 0x563bb08ae260 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.374+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94814365010672] v27'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x563bb08ae260 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.375+0000 7ffa63430800 20 remove_watcher() i=4 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.375+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94814365012208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bcaf0 con 0x7ffa58016440 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.375+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94814365012208] v27'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.375+0000 7ffa63430800 20 remove_watcher() i=5 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.376+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94814365016720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bd0f0 con 0x7ffa58016440 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.376+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94814365016720] v27'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.376+0000 7ffa63430800 20 remove_watcher() i=6 2024-04-27T03:32:19.387 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.376+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94814365018496] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x563bb08bcaf0 con 0x7ffa58016440 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94814365018496] v27'266 uv264 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7ffa5800a4a0 con 0x7ffa58016440 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 20 remove_watcher() i=7 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffa58016440 msgr2=0x7ffa580168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7ffa58016440 0x7ffa580168c0 crc :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563bb08ae260 msgr2=0x563bb08b0720 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563bb08ae260 0x563bb08b0720 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffa50073060 msgr2=0x7ffa500305f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7ffa50073060 0x7ffa500305f0 crc :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563bb08b2c50 msgr2=0x563bb08b5040 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563bb08b2c50 0x563bb08b5040 crc :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff928036900 msgr2=0x7ff928038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff928036900 0x7ff928038dc0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7ffa54052fd0 tx=0x7ffa5404f000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.388 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 msgr2=0x563bb08a2c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 --2- 172.21.15.165:0/2496423828 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb089bad0 0x563bb08a2c90 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000c5b0 tx=0x7ffa50026f00 comp rx=0 tx=0).stop 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 reap_dead start 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/2496423828 reap_dead start 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 shutdown_connections 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 >> 172.21.15.165:0/2496423828 conn(0x563bb0897da0 msgr2=0x563bb0895400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 shutdown_connections 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/2496423828 wait complete. 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/3944768808 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff938036950 msgr2=0x7ff938038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 --2- 172.21.15.165:0/3944768808 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff938036950 0x7ff938038e10 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x563bb0872280 tx=0x7ffa54005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/3944768808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb0775550 msgr2=0x563bb07739c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 --2- 172.21.15.165:0/3944768808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb0775550 0x563bb07739c0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000b710 tx=0x7ffa50041d50 comp rx=0 tx=0).stop 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/3944768808 shutdown_connections 2024-04-27T03:32:19.389 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 --2- 172.21.15.165:0/3944768808 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff938036950 0x7ff938038e10 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 --2- 172.21.15.165:0/3944768808 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb0775550 0x563bb07739c0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/3944768808 >> 172.21.15.165:0/3944768808 conn(0x563bb0880630 msgr2=0x563bb0880a30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/3944768808 shutdown_connections 2024-04-27T03:32:19.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.378+0000 7ffa63430800 1 -- 172.21.15.165:0/3944768808 wait complete. 2024-04-27T03:32:19.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563bb07794c0 msgr2=0x563bb0874d90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x563bb07794c0 0x563bb0874d90 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x563bb087b580 msgr2=0x563bb087d9d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x563bb087b580 0x563bb087d9d0 crc :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563bb0877640 msgr2=0x563bb0879a30 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x563bb0877640 0x563bb0879a30 crc :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9440368b0 msgr2=0x7ff944038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.390 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7ff9440368b0 0x7ff944038d70 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7ffa54054000 tx=0x7ffa5404e000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 msgr2=0x563bb086db50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 --2- 172.21.15.165:0/3478486690 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x563bb07759b0 0x563bb086db50 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffa5000b210 tx=0x7ffa50000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa5ee8c640 1 -- 172.21.15.165:0/3478486690 reap_dead start 2024-04-27T03:32:19.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 shutdown_connections 2024-04-27T03:32:19.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 >> 172.21.15.165:0/3478486690 conn(0x563bb076f340 msgr2=0x563bb0860130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 shutdown_connections 2024-04-27T03:32:19.391 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.379+0000 7ffa63430800 1 -- 172.21.15.165:0/3478486690 wait complete. 2024-04-27T03:32:19.393 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.393 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.393 ERROR:root:0 2024-04-27T03:32:19.393 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:19.398 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:19.398 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write; user-info-without-keys=*'] 2024-04-27T03:32:19.398 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'ada', '--caps', 'users=read, write; user-info-without-keys=*'] 2024-04-27T03:32:19.398 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid ada --caps 'users=read, write; user-info-without-keys=*' 2024-04-27T03:32:19.444 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:19.444 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:19.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.439+0000 7fe47ef2f800 1 Processor -- start 2024-04-27T03:32:19.445 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.439+0000 7fe47ef2f800 1 -- start start 2024-04-27T03:32:19.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.439+0000 7fe47ef2f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d68e5247a0 0x55d68e524bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.439+0000 7fe47ef2f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55d68e3e4d40 con 0x55d68e525100 2024-04-27T03:32:19.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.439+0000 7fe47ef2f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55d68e3b74d0 con 0x55d68e5247a0 2024-04-27T03:32:19.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.439+0000 7fe47acf2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55d68e5247a0 0x55d68e524bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.439+0000 7fe47a4f1640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55d68e525100 0x55d68e5255b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42396/0 (socket says 172.21.15.165:42396) 2024-04-27T03:32:19.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.439+0000 7fe47a4f1640 1 -- 172.21.15.165:0/2368426384 learned_addr learned my addr 172.21.15.165:0/2368426384 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:19.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.440+0000 7fe479cf0640 1 -- 172.21.15.165:0/2368426384 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3911873198 0 0) 0x55d68e3e4d40 con 0x55d68e525100 2024-04-27T03:32:19.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.440+0000 7fe479cf0640 1 -- 172.21.15.165:0/2368426384 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7fe464001e10 con 0x55d68e525100 2024-04-27T03:32:19.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.440+0000 7fe47acf2640 1 -- 172.21.15.165:0/2368426384 >> v1:172.21.15.89:6789/0 conn(0x55d68e525100 legacy=0x55d68e5255b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.446 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.440+0000 7fe47acf2640 1 -- 172.21.15.165:0/2368426384 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d68e3e3590 con 0x55d68e5247a0 2024-04-27T03:32:19.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.442+0000 7fe47acf2640 1 --2- 172.21.15.165:0/2368426384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e524bc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00a470 tx=0x7fe46c01cdb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e33ddd4a5e83e server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.442+0000 7fe479cf0640 1 -- 172.21.15.165:0/2368426384 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c057070 con 0x55d68e5247a0 2024-04-27T03:32:19.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.442+0000 7fe479cf0640 1 -- 172.21.15.165:0/2368426384 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe46c04acd0 con 0x55d68e5247a0 2024-04-27T03:32:19.447 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.442+0000 7fe479cf0640 1 -- 172.21.15.165:0/2368426384 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c052440 con 0x55d68e5247a0 2024-04-27T03:32:19.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.442+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2368426384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 msgr2=0x55d68e524bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.448 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.442+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/2368426384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e524bc0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7fe46c00a470 tx=0x7fe46c01cdb0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.442+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2368426384 shutdown_connections 2024-04-27T03:32:19.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.442+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/2368426384 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e524bc0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.442+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2368426384 >> 172.21.15.165:0/2368426384 conn(0x55d68e520320 msgr2=0x55d68e522740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.442+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2368426384 shutdown_connections 2024-04-27T03:32:19.449 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.442+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2368426384 wait complete. 2024-04-27T03:32:19.460 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.454+0000 7fe47ef2f800 1 Processor -- start 2024-04-27T03:32:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.454+0000 7fe47ef2f800 1 -- start start 2024-04-27T03:32:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.454+0000 7fe47ef2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e619210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.454+0000 7fe47ef2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d68e4713b0 con 0x55d68e5247a0 2024-04-27T03:32:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.454+0000 7fe47acf2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e619210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.454+0000 7fe47acf2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e619210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60396/0 (socket says 172.21.15.165:60396) 2024-04-27T03:32:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.454+0000 7fe47acf2640 1 -- 172.21.15.165:0/1526262862 learned_addr learned my addr 172.21.15.165:0/1526262862 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.455+0000 7fe47acf2640 1 -- 172.21.15.165:0/1526262862 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d68e6197e0 con 0x55d68e5247a0 2024-04-27T03:32:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.455+0000 7fe47acf2640 1 --2- 172.21.15.165:0/1526262862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e619210 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe46c055040 tx=0x7fe46c04c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68424acd8f727b17 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.455+0000 7fe368e09640 1 -- 172.21.15.165:0/1526262862 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c057040 con 0x55d68e5247a0 2024-04-27T03:32:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.455+0000 7fe368e09640 1 -- 172.21.15.165:0/1526262862 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe46c063070 con 0x55d68e5247a0 2024-04-27T03:32:19.461 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.455+0000 7fe368e09640 1 -- 172.21.15.165:0/1526262862 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c00a2a0 con 0x55d68e5247a0 2024-04-27T03:32:19.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47ef2f800 1 -- 172.21.15.165:0/1526262862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 msgr2=0x55d68e619210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/1526262862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e619210 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7fe46c055040 tx=0x7fe46c04c000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47ef2f800 1 -- 172.21.15.165:0/1526262862 shutdown_connections 2024-04-27T03:32:19.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/1526262862 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e619210 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47ef2f800 1 -- 172.21.15.165:0/1526262862 >> 172.21.15.165:0/1526262862 conn(0x55d68e520320 msgr2=0x55d68e614b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47ef2f800 1 -- 172.21.15.165:0/1526262862 shutdown_connections 2024-04-27T03:32:19.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47ef2f800 1 -- 172.21.15.165:0/1526262862 wait complete. 2024-04-27T03:32:19.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47ef2f800 1 Processor -- start 2024-04-27T03:32:19.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47ef2f800 1 -- start start 2024-04-27T03:32:19.466 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47ef2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e620d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47ef2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d68e525f60 con 0x55d68e5247a0 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47acf2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e620d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.456+0000 7fe47acf2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e620d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60412/0 (socket says 172.21.15.165:60412) 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.457+0000 7fe47acf2640 1 -- 172.21.15.165:0/2709573970 learned_addr learned my addr 172.21.15.165:0/2709573970 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.457+0000 7fe47acf2640 1 -- 172.21.15.165:0/2709573970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d68e621290 con 0x55d68e5247a0 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.457+0000 7fe47acf2640 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e620d50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe46c060040 tx=0x7fe46c002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.457+0000 7fe367606640 1 -- 172.21.15.165:0/2709573970 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c057040 con 0x55d68e5247a0 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.457+0000 7fe367606640 1 -- 172.21.15.165:0/2709573970 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe46c05db00 con 0x55d68e5247a0 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.457+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d68e621520 con 0x55d68e5247a0 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.457+0000 7fe367606640 1 -- 172.21.15.165:0/2709573970 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c05de00 con 0x55d68e5247a0 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.457+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d68e624130 con 0x55d68e5247a0 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.458+0000 7fe367606640 1 -- 172.21.15.165:0/2709573970 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fe46c076060 con 0x55d68e5247a0 2024-04-27T03:32:19.467 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.458+0000 7fe367606640 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe360036950 0x7fe360038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.458+0000 7fe367606640 1 -- 172.21.15.165:0/2709573970 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fe46c08daa0 con 0x55d68e5247a0 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.458+0000 7fe47a4f1640 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe360036950 0x7fe360038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.458+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d68e525100 0x55d68e627ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.458+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6284c0 con 0x55d68e525100 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.458+0000 7fe47b4f3640 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d68e525100 0x55d68e627ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.459+0000 7fe47a4f1640 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe360036950 0x7fe360038e10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe470000f60 tx=0x7fe4700010a0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.459+0000 7fe47b4f3640 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d68e525100 0x55d68e627ef0 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.459+0000 7fe47b4f3640 1 -- 172.21.15.165:0/2709573970 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e525100 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.459+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6254a0 con 0x55d68e525100 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.459+0000 7fe47b4f3640 1 -- 172.21.15.165:0/2709573970 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e525100 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.459+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d68e62a7d0 0x55d68e62cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.459+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e62d260 con 0x55d68e62a7d0 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.459+0000 7fe47acf2640 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d68e62a7d0 0x55d68e62cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.468 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.460+0000 7fe47acf2640 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d68e62a7d0 0x55d68e62cbc0 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.460+0000 7fe47acf2640 1 -- 172.21.15.165:0/2709573970 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fe46c05a040 con 0x55d68e62a7d0 2024-04-27T03:32:19.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.460+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e629780 con 0x55d68e525100 2024-04-27T03:32:19.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.460+0000 7fe47b4f3640 1 -- 172.21.15.165:0/2709573970 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e525100 2024-04-27T03:32:19.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.460+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55d68e62e710 0x55d68e630b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.460+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6312c0 con 0x55d68e62e710 2024-04-27T03:32:19.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.460+0000 7fe47a4f1640 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55d68e62e710 0x55d68e630b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.469 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.461+0000 7fe47a4f1640 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55d68e62e710 0x55d68e630b60 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.461+0000 7fe47a4f1640 1 -- 172.21.15.165:0/2709573970 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fe470009050 con 0x55d68e62e710 2024-04-27T03:32:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.461+0000 7fe47ef2f800 1 Processor -- start 2024-04-27T03:32:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.461+0000 7fe47ef2f800 1 -- start start 2024-04-27T03:32:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.461+0000 7fe47ef2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e63f720 0x55d68e63fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.461+0000 7fe47ef2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d68e4176b0 con 0x55d68e63f720 2024-04-27T03:32:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.462+0000 7fe47acf2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e63f720 0x55d68e63fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.462+0000 7fe47acf2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e63f720 0x55d68e63fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60418/0 (socket says 172.21.15.165:60418) 2024-04-27T03:32:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.462+0000 7fe47acf2640 1 -- 172.21.15.165:0/218860052 learned_addr learned my addr 172.21.15.165:0/218860052 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.462+0000 7fe47acf2640 1 -- 172.21.15.165:0/218860052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d68e640080 con 0x55d68e63f720 2024-04-27T03:32:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.462+0000 7fe47acf2640 1 --2- 172.21.15.165:0/218860052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e63f720 0x55d68e63fb40 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe46c08beb0 tx=0x7fe46c054cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.484 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.462+0000 7fe364e01640 1 -- 172.21.15.165:0/218860052 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c054e20 con 0x55d68e63f720 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.462+0000 7fe364e01640 1 -- 172.21.15.165:0/218860052 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe46c06e3d0 con 0x55d68e63f720 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.462+0000 7fe47ef2f800 1 -- 172.21.15.165:0/218860052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d68e640af0 con 0x55d68e63f720 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.462+0000 7fe364e01640 1 -- 172.21.15.165:0/218860052 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c06e710 con 0x55d68e63f720 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.463+0000 7fe47ef2f800 1 -- 172.21.15.165:0/218860052 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d68e640ff0 con 0x55d68e63f720 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.463+0000 7fe364e01640 1 -- 172.21.15.165:0/218860052 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fe46c067850 con 0x55d68e63f720 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.463+0000 7fe364e01640 1 --2- 172.21.15.165:0/218860052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe354036950 0x7fe354038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.463+0000 7fe364e01640 1 -- 172.21.15.165:0/218860052 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fe46c08de00 con 0x55d68e63f720 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.463+0000 7fe47a4f1640 1 --2- 172.21.15.165:0/218860052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe354036950 0x7fe354038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.464+0000 7fe47ef2f800 1 Processor -- start 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.464+0000 7fe47ef2f800 1 -- start start 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.464+0000 7fe47a4f1640 1 --2- 172.21.15.165:0/218860052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe354036950 0x7fe354038e10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe47000bf90 tx=0x7fe47000f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.464+0000 7fe47ef2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 0x55d68e64ccf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.485 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.464+0000 7fe47ef2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d68e619aa0 con 0x55d68e64c8d0 2024-04-27T03:32:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.464+0000 7fe47acf2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 0x55d68e64ccf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.464+0000 7fe47acf2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 0x55d68e64ccf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60434/0 (socket says 172.21.15.165:60434) 2024-04-27T03:32:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.464+0000 7fe47acf2640 1 -- 172.21.15.165:0/706640493 learned_addr learned my addr 172.21.15.165:0/706640493 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.464+0000 7fe47acf2640 1 -- 172.21.15.165:0/706640493 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d68e64d2c0 con 0x55d68e64c8d0 2024-04-27T03:32:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.465+0000 7fe47acf2640 1 --2- 172.21.15.165:0/706640493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 0x55d68e64ccf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe46c09a000 tx=0x7fe46c082750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa72f6f8b9a0a203 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.465+0000 7fe35e7fc640 1 -- 172.21.15.165:0/706640493 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c08a4b0 con 0x55d68e64c8d0 2024-04-27T03:32:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.465+0000 7fe35e7fc640 1 -- 172.21.15.165:0/706640493 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe46c054080 con 0x55d68e64c8d0 2024-04-27T03:32:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.465+0000 7fe35e7fc640 1 -- 172.21.15.165:0/706640493 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c05b5c0 con 0x55d68e64c8d0 2024-04-27T03:32:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.465+0000 7fe47ef2f800 1 -- 172.21.15.165:0/706640493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 msgr2=0x55d68e64ccf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.465+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/706640493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 0x55d68e64ccf0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7fe46c09a000 tx=0x7fe46c082750 comp rx=0 tx=0).stop 2024-04-27T03:32:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.465+0000 7fe47ef2f800 1 -- 172.21.15.165:0/706640493 shutdown_connections 2024-04-27T03:32:19.486 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.465+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/706640493 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 0x55d68e64ccf0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.465+0000 7fe47ef2f800 1 -- 172.21.15.165:0/706640493 >> 172.21.15.165:0/706640493 conn(0x55d68e648ba0 msgr2=0x55d68e646c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.465+0000 7fe47ef2f800 1 -- 172.21.15.165:0/706640493 shutdown_connections 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.465+0000 7fe47ef2f800 1 -- 172.21.15.165:0/706640493 wait complete. 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.465+0000 7fe47ef2f800 1 Processor -- start 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe47ef2f800 1 -- start start 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe47ef2f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 0x55d68e653cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe47ef2f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55d68e64d920 con 0x55d68e64c8d0 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe47acf2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 0x55d68e653cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe47acf2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 0x55d68e653cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60442/0 (socket says 172.21.15.165:60442) 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 learned_addr learned my addr 172.21.15.165:0/3888765639 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d68e654200 con 0x55d68e64c8d0 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe47acf2640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 0x55d68e653cc0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe46c05af60 tx=0x7fe46c05aef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.487 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe35cff9640 1 -- 172.21.15.165:0/3888765639 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c08a2d0 con 0x55d68e64c8d0 2024-04-27T03:32:19.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe35cff9640 1 -- 172.21.15.165:0/3888765639 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fe46c078970 con 0x55d68e64c8d0 2024-04-27T03:32:19.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55d68e654200 con 0x55d68e64c8d0 2024-04-27T03:32:19.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe35cff9640 1 -- 172.21.15.165:0/3888765639 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe46c078580 con 0x55d68e64c8d0 2024-04-27T03:32:19.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.466+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d68e654200 con 0x55d68e64c8d0 2024-04-27T03:32:19.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.467+0000 7fe35cff9640 1 -- 172.21.15.165:0/3888765639 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fe46c099100 con 0x55d68e64c8d0 2024-04-27T03:32:19.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.467+0000 7fe35cff9640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe344036950 0x7fe344038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.467+0000 7fe35cff9640 1 -- 172.21.15.165:0/3888765639 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fe46c0b1070 con 0x55d68e64c8d0 2024-04-27T03:32:19.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.467+0000 7fe47a4f1640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe344036950 0x7fe344038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.468+0000 7fe47a4f1640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe344036950 0x7fe344038e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55d68e6253e0 tx=0x7fe470010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.468+0000 7fe47ef2f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.468+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d68e65f400 0x55d68e6618c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.488 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.468+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e662010 con 0x55d68e65f400 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.468+0000 7fe47b4f3640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d68e65f400 0x55d68e6618c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.468+0000 7fe47b4f3640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d68e65f400 0x55d68e6618c0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 20 realm 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e662420 con 0x55d68e65f400 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e662cf0 con 0x55d68e65f400 2024-04-27T03:32:19.489 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e65f400 2024-04-27T03:32:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d68e663df0 0x55d68e6661e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6667f0 con 0x55d68e663df0 2024-04-27T03:32:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.469+0000 7fe47acf2640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d68e663df0 0x55d68e6661e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.470+0000 7fe47acf2640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d68e663df0 0x55d68e6661e0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.490 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.470+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.470+0000 7fe47ef2f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.470+0000 7fe47ef2f800 20 searching for the correct realm 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.470+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e668120 con 0x55d68e663df0 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.471+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.471+0000 7fe47acf2640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe46c0b4030 0x7fe46c0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.471+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e668120 con 0x7fe46c0b4030 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.471+0000 7fe47a4f1640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe46c0b4030 0x7fe46c0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.471+0000 7fe47a4f1640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe46c0b4030 0x7fe46c0ab8d0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.471+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.471+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e668120 con 0x7fe46c0b4030 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.471+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.471+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e668120 con 0x55d68e65f400 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.472+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.491 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.472+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e65f400 2024-04-27T03:32:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.472+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.472+0000 7fe47b4f3640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe474016440 0x7fe4740168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.472+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe474016440 2024-04-27T03:32:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.472+0000 7fe47b4f3640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe474016440 0x7fe4740168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.472+0000 7fe47b4f3640 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe474016440 0x7fe4740168c0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.473+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.473+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe474016440 2024-04-27T03:32:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.473+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.473+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe46c0b4030 2024-04-27T03:32:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.473+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.473+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e663df0 2024-04-27T03:32:19.492 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.473+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.473+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e65f400 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.474+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.474+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe46c0b4030 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.474+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.474+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e65f400 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.474+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.474+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe46c0b4030 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.474+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.474+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e663df0 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.475+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.475+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e663df0 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.475+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.475+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe474016440 2024-04-27T03:32:19.493 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.475+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.475+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e663df0 2024-04-27T03:32:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.475+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.475+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e663df0 2024-04-27T03:32:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.476+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.476+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e65f400 2024-04-27T03:32:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.476+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.476+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e65f400 2024-04-27T03:32:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.476+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.476+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe474016440 2024-04-27T03:32:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.476+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.476+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e663df0 2024-04-27T03:32:19.494 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.477+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.477+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe46c0b4030 2024-04-27T03:32:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.477+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.477+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e65f400 2024-04-27T03:32:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.477+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.477+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe474016440 2024-04-27T03:32:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.477+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.477+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e663df0 2024-04-27T03:32:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.478+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.478+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe474016440 2024-04-27T03:32:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.478+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.478+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe46c0b4030 2024-04-27T03:32:19.495 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.478+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.478+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e663df0 2024-04-27T03:32:19.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.478+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.496 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.478+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe474016440 2024-04-27T03:32:19.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x55d68e65f400 2024-04-27T03:32:19.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55d68e663090 con 0x7fe46c0b4030 2024-04-27T03:32:19.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47ef2f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:19.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47ef2f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:19.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47ef2f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:19.505 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47ef2f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47ef2f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47ef2f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47ef2f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6670b0 con 0x55d68e65f400 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47ef2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47ef2f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.479+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6674a0 con 0x55d68e65f400 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6674a0 con 0x7fe46c0b4030 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.506 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 20 zone default found 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 4 Realm: () 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 20 zonegroup default 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6685a0 con 0x7fe474016440 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e669fe0 con 0x55d68e663df0 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.480+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.507 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.481+0000 7fe47ef2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.481+0000 7fe47ef2f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.481+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66a420 con 0x55d68e663df0 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.481+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.481+0000 7fe47ef2f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.481+0000 7fe47ef2f800 20 started sync module instance, tier type = 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.481+0000 7fe47ef2f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.481+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e669420 con 0x7fe46c0b4030 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.482+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.482+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94380000450624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e669080 con 0x7fe46c0b4030 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.482+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66b350 con 0x55d68e65f400 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.483+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94380000450624] v27'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.483+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.483+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94380000456352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66b2c0 con 0x55d68e65f400 2024-04-27T03:32:19.508 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.483+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66c210 con 0x55d68e663df0 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.484+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94380000456352] v27'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.484+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.484+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94380000460128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66c5b0 con 0x55d68e663df0 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.484+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66cbd0 con 0x7fe46c0b4030 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.485+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.485+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94380000464832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66d890 con 0x7fe46c0b4030 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.485+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94380000460128] v27'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.485+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66e2e0 con 0x55d68e65f400 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.487+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.487+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94380000458560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66e750 con 0x55d68e65f400 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.487+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94380000464832] v27'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.487+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66f110 con 0x7fe474016440 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.488+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.488+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94380000465888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66f110 con 0x7fe474016440 2024-04-27T03:32:19.509 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.488+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66fb60 con 0x7fe474016440 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.488+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94380000458560] v27'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.489+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94380000465888] v27'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.489+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.489+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94380000472160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66f670 con 0x7fe474016440 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.489+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e670030 con 0x7fe474016440 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.490+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94380000472160] v27'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.490+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'267 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.490+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94380000469664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e670030 con 0x7fe474016440 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.490+0000 7fe47ef2f800 20 add_watcher() i=0 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.490+0000 7fe47ef2f800 20 add_watcher() i=1 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.490+0000 7fe47ef2f800 20 add_watcher() i=2 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.490+0000 7fe47ef2f800 20 add_watcher() i=3 2024-04-27T03:32:19.510 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.490+0000 7fe47ef2f800 20 add_watcher() i=4 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.490+0000 7fe47ef2f800 20 add_watcher() i=5 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.490+0000 7fe47ef2f800 20 add_watcher() i=6 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.491+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94380000469664] v27'268 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.491+0000 7fe47ef2f800 20 add_watcher() i=7 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.491+0000 7fe47ef2f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.491+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e673390 con 0x55d68e65f400 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.492+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.492+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94380000476400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6705c0 con 0x55d68e65f400 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.492+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94380000476400] v27'269 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.493+0000 7fe3317e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.493+0000 7fe47ef2f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.493+0000 7fe47ef2f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.493+0000 7fe47ef2f800 5 note: GC not initialized 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.493+0000 7fe2abfff640 20 reqs_thread_entry: start 2024-04-27T03:32:19.511 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.493+0000 7fe47ef2f800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.493+0000 7fe47ef2f800 20 Filter name: none 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.493+0000 7fe2a9ffb640 20 reqs_thread_entry: start 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.493+0000 7fe47ef2f800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.493+0000 7fe47ef2f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.493+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6a67a0 con 0x7fe474016440 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.494+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 ada [call out=48b,read 0~503 out=503b,getxattrs out=75b] v0'0 uv8 ondisk = 0) ==== 231+0+626 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.494+0000 7fe47ef2f800 20 rados_obj.operate() r=0 bl.length=503 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.494+0000 7fe47ef2f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.494+0000 7fe47ef2f800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.494+0000 7fe47ef2f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.494+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~460 in=460b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6ad170 con 0x7fe474016440 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.495+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~460] v27'9 uv9 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.495+0000 7fe47ef2f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.495+0000 7fe47ef2f800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.495+0000 7fe47ef2f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-27T03:32:19.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.495+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94380000725984 in=648b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6a8880 con 0x55d68e65f400 2024-04-27T03:32:19.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.496+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== watch-notify(notify (1) cookie 94380000458560 notify 115964117045 ret 0) ==== 678+0+0 (crc 0 0 0) 0x7fe474030040 con 0x55d68e65f400 2024-04-27T03:32:19.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.496+0000 7fe35e7fc640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117045 cookie 94380000458560 notifier 4882 bl.length()=636 2024-04-27T03:32:19.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.496+0000 7fe35e7fc640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.496+0000 7fe35e7fc640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.496+0000 7fe35e7fc640 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fe340003ea0 con 0x55d68e65f400 2024-04-27T03:32:19.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.496+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 notify.4 [notify cookie 94380000725984 out=8b] v0'0 uv267 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.496+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.496+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify_complete (2) cookie 94380000725984 notify 115964117045 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fe47402a8f0 con 0x55d68e65f400 2024-04-27T03:32:19.513 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-27T03:32:18.064161Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:32:18.064141Z","tags":[],"group_ids":[]} 2024-04-27T03:32:19.518 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.497+0000 7fe3317e2640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.497+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94380000476400] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e6a8c90 con 0x55d68e65f400 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.498+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94380000476400] v27'270 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.498+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94380000450624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66fcf0 con 0x7fe46c0b4030 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.499+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94380000450624] v27'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.499+0000 7fe47ef2f800 20 remove_watcher() i=0 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.499+0000 7fe47ef2f800 2 removed watcher, disabling cache 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.499+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94380000456352] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66fcf0 con 0x55d68e65f400 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.500+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94380000456352] v27'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.500+0000 7fe47ef2f800 20 remove_watcher() i=1 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.500+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94380000460128] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66aea0 con 0x55d68e663df0 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.501+0000 7fe47acf2640 1 -- 172.21.15.165:0/3888765639 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94380000460128] v27'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe46c0ba020 con 0x55d68e663df0 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.501+0000 7fe47ef2f800 20 remove_watcher() i=2 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.501+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94380000464832] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66fcf0 con 0x7fe46c0b4030 2024-04-27T03:32:19.519 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.502+0000 7fe47a4f1640 1 -- 172.21.15.165:0/3888765639 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94380000464832] v27'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe470009050 con 0x7fe46c0b4030 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.502+0000 7fe47ef2f800 20 remove_watcher() i=3 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.502+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94380000458560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66aea0 con 0x55d68e65f400 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.503+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94380000458560] v27'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x55d68e65f400 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.503+0000 7fe47ef2f800 20 remove_watcher() i=4 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.503+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94380000465888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66cfc0 con 0x7fe474016440 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.504+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94380000465888] v27'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.504+0000 7fe47ef2f800 20 remove_watcher() i=5 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.504+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94380000472160] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66fcf0 con 0x7fe474016440 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.505+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94380000472160] v27'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.505+0000 7fe47ef2f800 20 remove_watcher() i=6 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.505+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94380000469664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55d68e66d470 con 0x7fe474016440 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94380000469664] v27'269 uv267 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fe47400a4a0 con 0x7fe474016440 2024-04-27T03:32:19.520 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 20 remove_watcher() i=7 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe474016440 msgr2=0x7fe4740168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fe474016440 0x7fe4740168c0 crc :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d68e65f400 msgr2=0x55d68e6618c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d68e65f400 0x55d68e6618c0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe46c0b4030 msgr2=0x7fe46c0ab8d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fe46c0b4030 0x7fe46c0ab8d0 crc :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d68e663df0 msgr2=0x55d68e6661e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d68e663df0 0x55d68e6661e0 crc :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe344036950 msgr2=0x7fe344038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe344036950 0x7fe344038e10 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x55d68e6253e0 tx=0x7fe470010000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 msgr2=0x55d68e653cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/3888765639 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e64c8d0 0x55d68e653cc0 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7fe46c05af60 tx=0x7fe46c05aef0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.521 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 reap_dead start 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47b4f3640 1 -- 172.21.15.165:0/3888765639 reap_dead start 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 shutdown_connections 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 >> 172.21.15.165:0/3888765639 conn(0x55d68e648ba0 msgr2=0x55d68e646200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.506+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 shutdown_connections 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/3888765639 wait complete. 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/218860052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe354036950 msgr2=0x7fe354038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/218860052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe354036950 0x7fe354038e10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fe47000bf90 tx=0x7fe47000f000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/218860052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e63f720 msgr2=0x55d68e63fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/218860052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e63f720 0x55d68e63fb40 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7fe46c08beb0 tx=0x7fe46c054cf0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/218860052 shutdown_connections 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/218860052 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe354036950 0x7fe354038e10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/218860052 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e63f720 0x55d68e63fb40 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.522 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/218860052 >> 172.21.15.165:0/218860052 conn(0x55d68e6337c0 msgr2=0x55d68e633bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/218860052 shutdown_connections 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/218860052 wait complete. 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d68e525100 msgr2=0x55d68e627ef0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55d68e525100 0x55d68e627ef0 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55d68e62e710 msgr2=0x55d68e630b60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55d68e62e710 0x55d68e630b60 crc :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d68e62a7d0 msgr2=0x55d68e62cbc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55d68e62a7d0 0x55d68e62cbc0 crc :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe360036950 msgr2=0x7fe360038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fe360036950 0x7fe360038e10 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fe470000f60 tx=0x7fe4700010a0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 msgr2=0x55d68e620d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 --2- 172.21.15.165:0/2709573970 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55d68e5247a0 0x55d68e620d50 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7fe46c060040 tx=0x7fe46c002fb0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47b4f3640 1 -- 172.21.15.165:0/2709573970 reap_dead start 2024-04-27T03:32:19.523 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 shutdown_connections 2024-04-27T03:32:19.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 >> 172.21.15.165:0/2709573970 conn(0x55d68e520320 msgr2=0x55d68e613330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 shutdown_connections 2024-04-27T03:32:19.524 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.507+0000 7fe47ef2f800 1 -- 172.21.15.165:0/2709573970 wait complete. 2024-04-27T03:32:19.525 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.525 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.525 ERROR:root:0 2024-04-27T03:32:19.525 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-04-27T03:32:19.525 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-04-27T03:32:19.525 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps user-info-without-keys=write 2024-04-27T03:32:19.570 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:19.570 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:19.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.565+0000 7fcc9ea3f800 1 Processor -- start 2024-04-27T03:32:19.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.565+0000 7fcc9ea3f800 1 -- start start 2024-04-27T03:32:19.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.565+0000 7fcc9ea3f800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5602a6be4d40 0x5602a6be5160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.565+0000 7fcc9ea3f800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x5602a6aa4d40 con 0x5602a6be5fa0 2024-04-27T03:32:19.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.565+0000 7fcc9ea3f800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x5602a6a774d0 con 0x5602a6be4d40 2024-04-27T03:32:19.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.566+0000 7fcc9c545640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x5602a6be4d40 0x5602a6be5160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.566+0000 7fcc9bd44640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x5602a6be5fa0 0x5602a6be6420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42402/0 (socket says 172.21.15.165:42402) 2024-04-27T03:32:19.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.566+0000 7fcc9bd44640 1 -- 172.21.15.165:0/202948197 learned_addr learned my addr 172.21.15.165:0/202948197 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:19.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.566+0000 7fcc9c545640 1 -- 172.21.15.165:0/202948197 >> v1:172.21.15.89:6789/0 conn(0x5602a6be5fa0 legacy=0x5602a6be6420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.566+0000 7fcc9c545640 1 -- 172.21.15.165:0/202948197 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602a6aa3590 con 0x5602a6be4d40 2024-04-27T03:32:19.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.566+0000 7fcc9b543640 1 -- 172.21.15.165:0/202948197 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1823612148 0 0) 0x5602a6aa4d40 con 0x5602a6be5fa0 2024-04-27T03:32:19.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.568+0000 7fcc9c545640 1 --2- 172.21.15.165:0/202948197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6be5160 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000a470 tx=0x7fcc9001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79afb12d14750312 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.568+0000 7fcc9b543640 1 -- 172.21.15.165:0/202948197 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90057070 con 0x5602a6be4d40 2024-04-27T03:32:19.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.568+0000 7fcc9b543640 1 -- 172.21.15.165:0/202948197 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc90052570 con 0x5602a6be4d40 2024-04-27T03:32:19.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.568+0000 7fcc9b543640 1 -- 172.21.15.165:0/202948197 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90052870 con 0x5602a6be4d40 2024-04-27T03:32:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.569+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/202948197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 msgr2=0x5602a6be5160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.569+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/202948197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6be5160 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7fcc9000a470 tx=0x7fcc9001ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.569+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/202948197 shutdown_connections 2024-04-27T03:32:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.569+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/202948197 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6be5160 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.569+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/202948197 >> 172.21.15.165:0/202948197 conn(0x5602a6be03a0 msgr2=0x5602a6be27c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.569+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/202948197 shutdown_connections 2024-04-27T03:32:19.575 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.569+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/202948197 wait complete. 2024-04-27T03:32:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.580+0000 7fcc9ea3f800 1 Processor -- start 2024-04-27T03:32:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.580+0000 7fcc9ea3f800 1 -- start start 2024-04-27T03:32:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.581+0000 7fcc9ea3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6b59010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.581+0000 7fcc9ea3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5602a6b313b0 con 0x5602a6be4d40 2024-04-27T03:32:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.581+0000 7fcc9c545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6b59010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.581+0000 7fcc9c545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6b59010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60462/0 (socket says 172.21.15.165:60462) 2024-04-27T03:32:19.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.581+0000 7fcc9c545640 1 -- 172.21.15.165:0/487549356 learned_addr learned my addr 172.21.15.165:0/487549356 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.581+0000 7fcc9c545640 1 -- 172.21.15.165:0/487549356 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602a6cda060 con 0x5602a6be4d40 2024-04-27T03:32:19.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.581+0000 7fcc9c545640 1 --2- 172.21.15.165:0/487549356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6b59010 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcc90055040 tx=0x7fcc9004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1e6d3116abe6fc4 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.582+0000 7fcb8a60c640 1 -- 172.21.15.165:0/487549356 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90063070 con 0x5602a6be4d40 2024-04-27T03:32:19.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.582+0000 7fcb8a60c640 1 -- 172.21.15.165:0/487549356 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc90058040 con 0x5602a6be4d40 2024-04-27T03:32:19.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.582+0000 7fcb8a60c640 1 -- 172.21.15.165:0/487549356 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90057070 con 0x5602a6be4d40 2024-04-27T03:32:19.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.582+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/487549356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 msgr2=0x5602a6b59010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.582+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/487549356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6b59010 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7fcc90055040 tx=0x7fcc9004c000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.582+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/487549356 shutdown_connections 2024-04-27T03:32:19.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.582+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/487549356 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6b59010 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.582+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/487549356 >> 172.21.15.165:0/487549356 conn(0x5602a6be03a0 msgr2=0x5602a6cd1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.582+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/487549356 shutdown_connections 2024-04-27T03:32:19.591 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.582+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/487549356 wait complete. 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.583+0000 7fcc9ea3f800 1 Processor -- start 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.583+0000 7fcc9ea3f800 1 -- start start 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.583+0000 7fcc9ea3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6cdee20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.583+0000 7fcc9ea3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5602a6bee440 con 0x5602a6be4d40 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.583+0000 7fcc9c545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6cdee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.583+0000 7fcc9c545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6cdee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60476/0 (socket says 172.21.15.165:60476) 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.583+0000 7fcc9c545640 1 -- 172.21.15.165:0/3235599988 learned_addr learned my addr 172.21.15.165:0/3235599988 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.583+0000 7fcc9c545640 1 -- 172.21.15.165:0/3235599988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602a6cdf360 con 0x5602a6be4d40 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.584+0000 7fcc9c545640 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6cdee20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcc90061040 tx=0x7fcc90002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.584+0000 7fcb88e09640 1 -- 172.21.15.165:0/3235599988 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90063030 con 0x5602a6be4d40 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.584+0000 7fcb88e09640 1 -- 172.21.15.165:0/3235599988 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc90052a60 con 0x5602a6be4d40 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.584+0000 7fcb88e09640 1 -- 172.21.15.165:0/3235599988 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90052d40 con 0x5602a6be4d40 2024-04-27T03:32:19.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.584+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5602a6cdf5f0 con 0x5602a6be4d40 2024-04-27T03:32:19.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.584+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602a6ce22a0 con 0x5602a6be4d40 2024-04-27T03:32:19.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.585+0000 7fcb88e09640 1 -- 172.21.15.165:0/3235599988 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fcc9005e480 con 0x5602a6be4d40 2024-04-27T03:32:19.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.585+0000 7fcb88e09640 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb84036900 0x7fcb84038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.585+0000 7fcb88e09640 1 -- 172.21.15.165:0/3235599988 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fcc90064070 con 0x5602a6be4d40 2024-04-27T03:32:19.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.585+0000 7fcc9bd44640 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb84036900 0x7fcb84038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.585+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5602a6be5fa0 0x5602a6ce6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.585+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6ce65a0 con 0x5602a6be5fa0 2024-04-27T03:32:19.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.585+0000 7fcc9cd46640 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5602a6be5fa0 0x5602a6ce6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.585+0000 7fcc9bd44640 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb84036900 0x7fcb84038dc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c000c00 tx=0x7fcc8c0013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.586+0000 7fcc9cd46640 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5602a6be5fa0 0x5602a6ce6060 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.586+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3235599988 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6be5fa0 2024-04-27T03:32:19.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.586+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6ce3610 con 0x5602a6be5fa0 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.586+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3235599988 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6be5fa0 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.586+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5602a6ce8890 0x5602a6ceac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.586+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6ceb350 con 0x5602a6ce8890 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.586+0000 7fcc9c545640 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5602a6ce8890 0x5602a6ceac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.587+0000 7fcc9c545640 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5602a6ce8890 0x5602a6ceac80 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.587+0000 7fcc9c545640 1 -- 172.21.15.165:0/3235599988 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fcc90070020 con 0x5602a6ce8890 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.587+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6ce7810 con 0x5602a6be5fa0 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.587+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3235599988 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6be5fa0 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.587+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5602a6cec790 0x5602a6ceec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.587+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6cef3b0 con 0x5602a6cec790 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.587+0000 7fcc9bd44640 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5602a6cec790 0x5602a6ceec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.587+0000 7fcc9bd44640 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5602a6cec790 0x5602a6ceec20 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.588+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3235599988 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fcc8c00f670 con 0x5602a6cec790 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.588+0000 7fcc9ea3f800 1 Processor -- start 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.588+0000 7fcc9ea3f800 1 -- start start 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.588+0000 7fcc9ea3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6cd7c10 0x5602a6cd6080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.588+0000 7fcc9ea3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5602a6ad76b0 con 0x5602a6cd7c10 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.589+0000 7fcc9c545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6cd7c10 0x5602a6cd6080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.589+0000 7fcc9c545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6cd7c10 0x5602a6cd6080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60482/0 (socket says 172.21.15.165:60482) 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.589+0000 7fcc9c545640 1 -- 172.21.15.165:0/724347289 learned_addr learned my addr 172.21.15.165:0/724347289 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.589+0000 7fcc9c545640 1 -- 172.21.15.165:0/724347289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602a6cd65c0 con 0x5602a6cd7c10 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.589+0000 7fcc9c545640 1 --2- 172.21.15.165:0/724347289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6cd7c10 0x5602a6cd6080 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcc9008cab0 tx=0x7fcc90069df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.589+0000 7fcb81ffb640 1 -- 172.21.15.165:0/724347289 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc9008e050 con 0x5602a6cd7c10 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.589+0000 7fcb81ffb640 1 -- 172.21.15.165:0/724347289 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc9008e1f0 con 0x5602a6cd7c10 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.589+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/724347289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5602a6cd6850 con 0x5602a6cd7c10 2024-04-27T03:32:19.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.589+0000 7fcb81ffb640 1 -- 172.21.15.165:0/724347289 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90073a80 con 0x5602a6cd7c10 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.589+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/724347289 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602a6cd6850 con 0x5602a6cd7c10 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.590+0000 7fcb81ffb640 1 -- 172.21.15.165:0/724347289 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fcc90073c20 con 0x5602a6cd7c10 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.590+0000 7fcb81ffb640 1 --2- 172.21.15.165:0/724347289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb74036950 0x7fcb74038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.590+0000 7fcb81ffb640 1 -- 172.21.15.165:0/724347289 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fcc9007f050 con 0x5602a6cd7c10 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.590+0000 7fcc9bd44640 1 --2- 172.21.15.165:0/724347289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb74036950 0x7fcb74038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.590+0000 7fcc9ea3f800 1 Processor -- start 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.590+0000 7fcc9ea3f800 1 -- start start 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.590+0000 7fcc9bd44640 1 --2- 172.21.15.165:0/724347289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb74036950 0x7fcb74038e10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x5602a6ce6a30 tx=0x7fcc8c006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.591+0000 7fcc9ea3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 0x5602a6d0cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.591+0000 7fcc9ea3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5602a6cda2f0 con 0x5602a6d0c970 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.591+0000 7fcc9c545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 0x5602a6d0cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.591+0000 7fcc9c545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 0x5602a6d0cd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60484/0 (socket says 172.21.15.165:60484) 2024-04-27T03:32:19.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.591+0000 7fcc9c545640 1 -- 172.21.15.165:0/327087350 learned_addr learned my addr 172.21.15.165:0/327087350 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.591+0000 7fcc9c545640 1 -- 172.21.15.165:0/327087350 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602a6d0d360 con 0x5602a6d0c970 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.591+0000 7fcc9c545640 1 --2- 172.21.15.165:0/327087350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 0x5602a6d0cd90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009f10 tx=0x7fcc9005e9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a7eec71dbb3056 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.591+0000 7fcb7b7fe640 1 -- 172.21.15.165:0/327087350 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90009740 con 0x5602a6d0c970 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.591+0000 7fcb7b7fe640 1 -- 172.21.15.165:0/327087350 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc900a6070 con 0x5602a6d0c970 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.591+0000 7fcb7b7fe640 1 -- 172.21.15.165:0/327087350 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc9007e560 con 0x5602a6d0c970 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.592+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/327087350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 msgr2=0x5602a6d0cd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.592+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/327087350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 0x5602a6d0cd90 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009f10 tx=0x7fcc9005e9f0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.592+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/327087350 shutdown_connections 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.592+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/327087350 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 0x5602a6d0cd90 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.592+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/327087350 >> 172.21.15.165:0/327087350 conn(0x5602a6d08c80 msgr2=0x5602a6d06ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.592+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/327087350 shutdown_connections 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.592+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/327087350 wait complete. 2024-04-27T03:32:19.608 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.592+0000 7fcc9ea3f800 1 Processor -- start 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.592+0000 7fcc9ea3f800 1 -- start start 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.592+0000 7fcc9ea3f800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 0x5602a6d13d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.592+0000 7fcc9ea3f800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x5602a6d0d9f0 con 0x5602a6d0c970 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.593+0000 7fcc9c545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 0x5602a6d13d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.593+0000 7fcc9c545640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 0x5602a6d13d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60488/0 (socket says 172.21.15.165:60488) 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.593+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 learned_addr learned my addr 172.21.15.165:0/3660692953 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.593+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5602a6d14240 con 0x5602a6d0c970 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.593+0000 7fcc9c545640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 0x5602a6d13d00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009f10 tx=0x7fcc9008b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.593+0000 7fcb79ffb640 1 -- 172.21.15.165:0/3660692953 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc9004bed0 con 0x5602a6d0c970 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.593+0000 7fcb79ffb640 1 -- 172.21.15.165:0/3660692953 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fcc900b1070 con 0x5602a6d0c970 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.593+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x5602a6d14240 con 0x5602a6d0c970 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.593+0000 7fcb79ffb640 1 -- 172.21.15.165:0/3660692953 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcc90097c10 con 0x5602a6d0c970 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.593+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5602a6d14240 con 0x5602a6d0c970 2024-04-27T03:32:19.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.594+0000 7fcb79ffb640 1 -- 172.21.15.165:0/3660692953 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fcc900a0070 con 0x5602a6d0c970 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.594+0000 7fcb79ffb640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb6c036950 0x7fcb6c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.594+0000 7fcb79ffb640 1 -- 172.21.15.165:0/3660692953 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fcc900b7c10 con 0x5602a6d0c970 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.594+0000 7fcc9bd44640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb6c036950 0x7fcb6c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.595+0000 7fcc9ea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.595+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5602a6d1f2d0 0x5602a6d21790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.595+0000 7fcc9bd44640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb6c036950 0x7fcb6c038e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00ffd0 tx=0x7fcc8c023040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.595+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d21ee0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.595+0000 7fcc9cd46640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5602a6d1f2d0 0x5602a6d21790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.595+0000 7fcc9cd46640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5602a6d1f2d0 0x5602a6d21790 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.595+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.595+0000 7fcc9ea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.595+0000 7fcc9ea3f800 20 realm 2024-04-27T03:32:19.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.595+0000 7fcc9ea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.595+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d222f0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9ea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9ea3f800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9ea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22bc0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9ea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9ea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d1f2d0 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9ea3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9ea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5602a6d23cc0 0x5602a6d260b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d26670 con 0x5602a6d23cc0 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.596+0000 7fcc9c545640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5602a6d23cc0 0x5602a6d260b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.597+0000 7fcc9c545640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5602a6d23cc0 0x5602a6d260b0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.597+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.597+0000 7fcc9ea3f800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.597+0000 7fcc9ea3f800 20 searching for the correct realm 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.597+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d27fa0 con 0x5602a6d23cc0 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.597+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.597+0000 7fcc9c545640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fcc900ad040 0x7fcc900aa870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.597+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d27fa0 con 0x7fcc900ad040 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.598+0000 7fcc9bd44640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fcc900ad040 0x7fcc900aa870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.598+0000 7fcc9bd44640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fcc900ad040 0x7fcc900aa870 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.598+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.612 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.598+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d27fa0 con 0x7fcc900ad040 2024-04-27T03:32:19.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.598+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.598+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d27fa0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.599+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.599+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d1f2d0 2024-04-27T03:32:19.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.599+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.599+0000 7fcc9cd46640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fcc94016440 0x7fcc940168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.599+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc94016440 2024-04-27T03:32:19.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.599+0000 7fcc9cd46640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fcc94016440 0x7fcc940168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.599+0000 7fcc9cd46640 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fcc94016440 0x7fcc940168c0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.600+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.600+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc94016440 2024-04-27T03:32:19.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.600+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.600+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc900ad040 2024-04-27T03:32:19.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.600+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.600+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d23cc0 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.600+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.600+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d1f2d0 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.601+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.601+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc900ad040 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.601+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.601+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d1f2d0 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.601+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.601+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc900ad040 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.601+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.601+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d23cc0 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.602+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.602+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d23cc0 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.602+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.602+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc94016440 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.602+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.602+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d23cc0 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.602+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.602+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d23cc0 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.602+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.602+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d1f2d0 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.603+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.603+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d1f2d0 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.603+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.603+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc94016440 2024-04-27T03:32:19.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.603+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.603+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d23cc0 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.603+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.604+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc900ad040 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.604+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.604+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d1f2d0 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.604+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.604+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc94016440 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.604+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.604+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d23cc0 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.605+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.605+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc94016440 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.605+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.605+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc900ad040 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.605+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.605+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d23cc0 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.605+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.605+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc94016440 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.605+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x5602a6d1f2d0 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d22f60 con 0x7fcc900ad040 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:19.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d26f30 con 0x5602a6d1f2d0 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d27320 con 0x5602a6d1f2d0 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.606+0000 7fcc9ea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d27320 con 0x7fcc900ad040 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 20 zone default found 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 4 Realm: () 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 20 zonegroup default 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d28420 con 0x7fcc94016440 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d27870 con 0x5602a6d23cc0 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.607+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2a380 con 0x5602a6d23cc0 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.608+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.608+0000 7fcc9ea3f800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.608+0000 7fcc9ea3f800 20 started sync module instance, tier type = 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.608+0000 7fcc9ea3f800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.608+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d29740 con 0x7fcc900ad040 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.609+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.609+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94569388743520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2ac90 con 0x7fcc900ad040 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.609+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2b4e0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.610+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94569388743520] v27'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.610+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.610+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94569388746720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2b540 con 0x5602a6d1f2d0 2024-04-27T03:32:19.633 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.610+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2c050 con 0x5602a6d23cc0 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.611+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.611+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94569388751616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2c010 con 0x5602a6d23cc0 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.612+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2d000 con 0x7fcc900ad040 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.612+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94569388746720] v27'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.613+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.613+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94569388755728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2d420 con 0x7fcc900ad040 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.613+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2de60 con 0x5602a6d1f2d0 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.613+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94569388751616] v27'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.614+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.614+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94569388759344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2e250 con 0x5602a6d1f2d0 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.614+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2ec90 con 0x7fcc94016440 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.614+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94569388755728] v27'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.615+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94569388759344] v27'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.615+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.615+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94569388753984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2ec50 con 0x7fcc94016440 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.615+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2f530 con 0x7fcc94016440 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.616+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94569388753984] v27'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.616+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.616+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94569388762976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2ec50 con 0x7fcc94016440 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.616+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2fb90 con 0x7fcc94016440 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.617+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94569388762976] v27'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.617+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'270 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.617+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94569388765232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2fb50 con 0x7fcc94016440 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.617+0000 7fcc9ea3f800 20 add_watcher() i=0 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.617+0000 7fcc9ea3f800 20 add_watcher() i=1 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.617+0000 7fcc9ea3f800 20 add_watcher() i=2 2024-04-27T03:32:19.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.617+0000 7fcc9ea3f800 20 add_watcher() i=3 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.617+0000 7fcc9ea3f800 20 add_watcher() i=4 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.617+0000 7fcc9ea3f800 20 add_watcher() i=5 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.617+0000 7fcc9ea3f800 20 add_watcher() i=6 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.618+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94569388765232] v27'271 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.618+0000 7fcc9ea3f800 20 add_watcher() i=7 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.618+0000 7fcc9ea3f800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.618+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d32f80 con 0x5602a6d1f2d0 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.618+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.619+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94569388767344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d304a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.619+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94569388767344] v27'271 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fcc9400a4a0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.620+0000 7fcb527e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.620+0000 7fcc9ea3f800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.620+0000 7fcc9ea3f800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:19.636 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.620+0000 7fcc9ea3f800 5 note: GC not initialized 2024-04-27T03:32:19.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.620+0000 7fcb517e2640 20 reqs_thread_entry: start 2024-04-27T03:32:19.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.620+0000 7fcc9ea3f800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:19.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.620+0000 7fcc9ea3f800 20 Filter name: none 2024-04-27T03:32:19.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.620+0000 7fcac2ffd640 20 reqs_thread_entry: start 2024-04-27T03:32:19.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.620+0000 7fcc9ea3f800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-27T03:32:19.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.620+0000 7fcc9ea3f800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.620+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d662c0 con 0x7fcc94016440 2024-04-27T03:32:19.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.621+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 ada [call out=48b,read 0~460 out=460b,getxattrs out=75b] v0'0 uv9 ondisk = 0) ==== 231+0+583 (crc 0 0 0) 0x7fcc9400a4a0 con 0x7fcc94016440 2024-04-27T03:32:19.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.621+0000 7fcc9ea3f800 20 rados_obj.operate() r=0 bl.length=460 2024-04-27T03:32:19.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.621+0000 7fcc9ea3f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-27T03:32:19.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.621+0000 7fcc9ea3f800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.621+0000 7fcc9ea3f800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:19.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.621+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~490 in=490b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d6c8d0 con 0x7fcc94016440 2024-04-27T03:32:19.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.622+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~490] v27'10 uv10 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fcc9404a040 con 0x7fcc94016440 2024-04-27T03:32:19.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.622+0000 7fcc9ea3f800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.622+0000 7fcc9ea3f800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.622+0000 7fcc9ea3f800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-27T03:32:19.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.622+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94569389015872 in=678b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d67fe0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.623+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== watch-notify(notify (1) cookie 94569388759344 notify 115964117046 ret 0) ==== 708+0+0 (crc 0 0 0) 0x7fcc94030040 con 0x5602a6d1f2d0 2024-04-27T03:32:19.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.623+0000 7fcb7b7fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117046 cookie 94569388759344 notifier 4889 bl.length()=666 2024-04-27T03:32:19.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.623+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 notify.4 [notify cookie 94569389015872 out=8b] v0'0 uv270 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fcc9404a040 con 0x5602a6d1f2d0 2024-04-27T03:32:19.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.623+0000 7fcb7b7fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.623+0000 7fcb7b7fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.623+0000 7fcb7b7fe640 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fcb60003ea0 con 0x5602a6d1f2d0 2024-04-27T03:32:19.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.623+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9404a040 con 0x5602a6d1f2d0 2024-04-27T03:32:19.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.623+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify_complete (2) cookie 94569389015872 notify 115964117046 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fcc9402a910 con 0x5602a6d1f2d0 2024-04-27T03:32:19.640 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-27T03:32:18.064161Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"user-info-without-keys","perm":"write"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:32:18.064141Z","tags":[],"group_ids":[]} 2024-04-27T03:32:19.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.624+0000 7fcb527e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.624+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94569388767344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d32670 con 0x5602a6d1f2d0 2024-04-27T03:32:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.625+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94569388767344] v27'272 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fcc9404a040 con 0x5602a6d1f2d0 2024-04-27T03:32:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.625+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94569388743520] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2f840 con 0x7fcc900ad040 2024-04-27T03:32:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.626+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94569388743520] v27'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.626+0000 7fcc9ea3f800 20 remove_watcher() i=0 2024-04-27T03:32:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.626+0000 7fcc9ea3f800 2 removed watcher, disabling cache 2024-04-27T03:32:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.626+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94569388746720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d29b60 con 0x5602a6d1f2d0 2024-04-27T03:32:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.627+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94569388746720] v27'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9404a040 con 0x5602a6d1f2d0 2024-04-27T03:32:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.627+0000 7fcc9ea3f800 20 remove_watcher() i=1 2024-04-27T03:32:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.627+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94569388751616] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2a7e0 con 0x5602a6d23cc0 2024-04-27T03:32:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.628+0000 7fcc9c545640 1 -- 172.21.15.165:0/3660692953 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94569388751616] v27'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc90094020 con 0x5602a6d23cc0 2024-04-27T03:32:19.646 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.628+0000 7fcc9ea3f800 20 remove_watcher() i=2 2024-04-27T03:32:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.628+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94569388755728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2a7e0 con 0x7fcc900ad040 2024-04-27T03:32:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.629+0000 7fcc9bd44640 1 -- 172.21.15.165:0/3660692953 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94569388755728] v27'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc8c037040 con 0x7fcc900ad040 2024-04-27T03:32:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.629+0000 7fcc9ea3f800 20 remove_watcher() i=3 2024-04-27T03:32:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.629+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94569388759344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2cb10 con 0x5602a6d1f2d0 2024-04-27T03:32:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.630+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94569388759344] v27'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9404a040 con 0x5602a6d1f2d0 2024-04-27T03:32:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.630+0000 7fcc9ea3f800 20 remove_watcher() i=4 2024-04-27T03:32:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.630+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94569388753984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2cb10 con 0x7fcc94016440 2024-04-27T03:32:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.631+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94569388753984] v27'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9404a040 con 0x7fcc94016440 2024-04-27T03:32:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.631+0000 7fcc9ea3f800 20 remove_watcher() i=5 2024-04-27T03:32:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.631+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94569388762976] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2cb10 con 0x7fcc94016440 2024-04-27T03:32:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.632+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94569388762976] v27'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9404a040 con 0x7fcc94016440 2024-04-27T03:32:19.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.632+0000 7fcc9ea3f800 20 remove_watcher() i=6 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.632+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94569388765232] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x5602a6d2e760 con 0x7fcc94016440 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.633+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94569388765232] v27'272 uv270 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fcc9404a040 con 0x7fcc94016440 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.633+0000 7fcc9ea3f800 20 remove_watcher() i=7 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.633+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fcc94016440 msgr2=0x7fcc940168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.633+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fcc94016440 0x7fcc940168c0 crc :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.633+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5602a6d1f2d0 msgr2=0x5602a6d21790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.633+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5602a6d1f2d0 0x5602a6d21790 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.633+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fcc900ad040 msgr2=0x7fcc900aa870 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.633+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fcc900ad040 0x7fcc900aa870 crc :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.633+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5602a6d23cc0 msgr2=0x5602a6d260b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.633+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5602a6d23cc0 0x5602a6d260b0 crc :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.633+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb6c036950 msgr2=0x7fcb6c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.648 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.633+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb6c036950 0x7fcb6c038e10 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c00ffd0 tx=0x7fcc8c023040 comp rx=0 tx=0).stop 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 msgr2=0x5602a6d13d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3660692953 reap_dead start 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 0x5602a6d13d00 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7fcc90009f10 tx=0x7fcc9008b760 comp rx=0 tx=0).stop 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 shutdown_connections 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3660692953 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6d0c970 0x5602a6d13d00 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 >> 172.21.15.165:0/3660692953 conn(0x5602a6d08c80 msgr2=0x5602a6d062e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 shutdown_connections 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3660692953 wait complete. 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/724347289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb74036950 msgr2=0x7fcb74038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/724347289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb74036950 0x7fcb74038e10 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x5602a6ce6a30 tx=0x7fcc8c006000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/724347289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6cd7c10 msgr2=0x5602a6cd6080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/724347289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6cd7c10 0x5602a6cd6080 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7fcc9008cab0 tx=0x7fcc90069df0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.649 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/724347289 shutdown_connections 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/724347289 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb74036950 0x7fcb74038e10 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/724347289 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6cd7c10 0x5602a6cd6080 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/724347289 >> 172.21.15.165:0/724347289 conn(0x5602a6cf18b0 msgr2=0x5602a6cf1cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/724347289 shutdown_connections 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/724347289 wait complete. 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5602a6be5fa0 msgr2=0x5602a6ce6060 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x5602a6be5fa0 0x5602a6ce6060 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5602a6cec790 msgr2=0x5602a6ceec20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x5602a6cec790 0x5602a6ceec20 crc :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5602a6ce8890 msgr2=0x5602a6ceac80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x5602a6ce8890 0x5602a6ceac80 crc :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb84036900 msgr2=0x7fcb84038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.650 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fcb84036900 0x7fcb84038dc0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c000c00 tx=0x7fcc8c0013b0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 msgr2=0x5602a6cdee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9ea3f800 1 --2- 172.21.15.165:0/3235599988 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x5602a6be4d40 0x5602a6cdee20 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7fcc90061040 tx=0x7fcc90002fb0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.634+0000 7fcc9cd46640 1 -- 172.21.15.165:0/3235599988 reap_dead start 2024-04-27T03:32:19.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.635+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 shutdown_connections 2024-04-27T03:32:19.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.635+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 >> 172.21.15.165:0/3235599988 conn(0x5602a6be03a0 msgr2=0x5602a6cd03d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.635+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 shutdown_connections 2024-04-27T03:32:19.651 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.635+0000 7fcc9ea3f800 1 -- 172.21.15.165:0/3235599988 wait complete. 2024-04-27T03:32:19.652 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'write'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.652 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'user-info-without-keys', 'perm': 'write'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.652 ERROR:root:0 2024-04-27T03:32:19.652 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:19.657 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'AccessDenied', 'Message': '', 'RequestId': 'tx000004cc27f3c3755d0ac-00662c71c3-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:32:19.657 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'rm', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-04-27T03:32:19.658 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'rm', '--uid', 'ada', '--caps', 'user-info-without-keys=write'] 2024-04-27T03:32:19.658 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps rm --uid ada --caps user-info-without-keys=write 2024-04-27T03:32:19.702 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:19.703 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:19.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.698+0000 7f456cd33800 1 Processor -- start 2024-04-27T03:32:19.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.698+0000 7f456cd33800 1 -- start start 2024-04-27T03:32:19.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.698+0000 7f456cd33800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55e153a823e0 0x55e153a847d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.698+0000 7f456cd33800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x55e15393cd40 con 0x55e153a7e8d0 2024-04-27T03:32:19.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.698+0000 7f456cd33800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x55e15390f4d0 con 0x55e153a823e0 2024-04-27T03:32:19.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.698+0000 7f4563fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x55e153a823e0 0x55e153a847d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.698+0000 7f45689f5640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x55e153a7e8d0 0x55e153a80cc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42416/0 (socket says 172.21.15.165:42416) 2024-04-27T03:32:19.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.698+0000 7f45689f5640 1 -- 172.21.15.165:0/712756153 learned_addr learned my addr 172.21.15.165:0/712756153 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:19.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.699+0000 7f4563fff640 1 -- 172.21.15.165:0/712756153 >> v1:172.21.15.89:6789/0 conn(0x55e153a7e8d0 legacy=0x55e153a80cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.699+0000 7f4563fff640 1 -- 172.21.15.165:0/712756153 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e15393b590 con 0x55e153a823e0 2024-04-27T03:32:19.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.700+0000 7f4563fff640 1 --2- 172.21.15.165:0/712756153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a823e0 0x55e153a847d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f455400a470 tx=0x7f455401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66509a6552500163 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.701+0000 7f45637fe640 1 -- 172.21.15.165:0/712756153 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4554057070 con 0x55e153a823e0 2024-04-27T03:32:19.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.701+0000 7f45637fe640 1 -- 172.21.15.165:0/712756153 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4554052570 con 0x55e153a823e0 2024-04-27T03:32:19.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.701+0000 7f45637fe640 1 -- 172.21.15.165:0/712756153 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4554052870 con 0x55e153a823e0 2024-04-27T03:32:19.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.701+0000 7f456cd33800 1 -- 172.21.15.165:0/712756153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a823e0 msgr2=0x55e153a847d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.701+0000 7f456cd33800 1 --2- 172.21.15.165:0/712756153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a823e0 0x55e153a847d0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7f455400a470 tx=0x7f455401ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:19.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.701+0000 7f456cd33800 1 -- 172.21.15.165:0/712756153 shutdown_connections 2024-04-27T03:32:19.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.701+0000 7f456cd33800 1 --2- 172.21.15.165:0/712756153 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a823e0 0x55e153a847d0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.701+0000 7f456cd33800 1 -- 172.21.15.165:0/712756153 >> 172.21.15.165:0/712756153 conn(0x55e153a783a0 msgr2=0x55e153a7a7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.701+0000 7f456cd33800 1 -- 172.21.15.165:0/712756153 shutdown_connections 2024-04-27T03:32:19.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.701+0000 7f456cd33800 1 -- 172.21.15.165:0/712756153 wait complete. 2024-04-27T03:32:19.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.713+0000 7f456cd33800 1 Processor -- start 2024-04-27T03:32:19.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.713+0000 7f456cd33800 1 -- start start 2024-04-27T03:32:19.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.713+0000 7f456cd33800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 0x55e153b6f170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.713+0000 7f456cd33800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e1539c93b0 con 0x55e153a7e8d0 2024-04-27T03:32:19.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.713+0000 7f45689f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 0x55e153b6f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.713+0000 7f45689f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 0x55e153b6f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60502/0 (socket says 172.21.15.165:60502) 2024-04-27T03:32:19.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.713+0000 7f45689f5640 1 -- 172.21.15.165:0/2887979273 learned_addr learned my addr 172.21.15.165:0/2887979273 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.714+0000 7f45689f5640 1 -- 172.21.15.165:0/2887979273 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e153b6f740 con 0x55e153a7e8d0 2024-04-27T03:32:19.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.714+0000 7f45689f5640 1 --2- 172.21.15.165:0/2887979273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 0x55e153b6f170 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f455c001de0 tx=0x7f455c00b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e23b59f1c438f96 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.714+0000 7f4456e05640 1 -- 172.21.15.165:0/2887979273 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455c012020 con 0x55e153a7e8d0 2024-04-27T03:32:19.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.714+0000 7f4456e05640 1 -- 172.21.15.165:0/2887979273 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455c00e5a0 con 0x55e153a7e8d0 2024-04-27T03:32:19.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.714+0000 7f4456e05640 1 -- 172.21.15.165:0/2887979273 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455c00e8c0 con 0x55e153a7e8d0 2024-04-27T03:32:19.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.715+0000 7f456cd33800 1 -- 172.21.15.165:0/2887979273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 msgr2=0x55e153b6f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.715+0000 7f456cd33800 1 --2- 172.21.15.165:0/2887979273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 0x55e153b6f170 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f455c001de0 tx=0x7f455c00b6c0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.715+0000 7f456cd33800 1 -- 172.21.15.165:0/2887979273 shutdown_connections 2024-04-27T03:32:19.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.715+0000 7f456cd33800 1 --2- 172.21.15.165:0/2887979273 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 0x55e153b6f170 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.722 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.715+0000 7f456cd33800 1 -- 172.21.15.165:0/2887979273 >> 172.21.15.165:0/2887979273 conn(0x55e153a783a0 msgr2=0x55e153b6aa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.715+0000 7f456cd33800 1 -- 172.21.15.165:0/2887979273 shutdown_connections 2024-04-27T03:32:19.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.715+0000 7f456cd33800 1 -- 172.21.15.165:0/2887979273 wait complete. 2024-04-27T03:32:19.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.715+0000 7f456cd33800 1 Processor -- start 2024-04-27T03:32:19.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.715+0000 7f456cd33800 1 -- start start 2024-04-27T03:32:19.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.715+0000 7f456cd33800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 0x55e153b76da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.715+0000 7f456cd33800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e153a85010 con 0x55e153a7e8d0 2024-04-27T03:32:19.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.716+0000 7f45689f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 0x55e153b76da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.716+0000 7f45689f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 0x55e153b76da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60518/0 (socket says 172.21.15.165:60518) 2024-04-27T03:32:19.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.716+0000 7f45689f5640 1 -- 172.21.15.165:0/3819311631 learned_addr learned my addr 172.21.15.165:0/3819311631 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.716+0000 7f45689f5640 1 -- 172.21.15.165:0/3819311631 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e153b772e0 con 0x55e153a7e8d0 2024-04-27T03:32:19.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.716+0000 7f45689f5640 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 0x55e153b76da0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f455c00c010 tx=0x7f455c000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.723 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.716+0000 7f4455602640 1 -- 172.21.15.165:0/3819311631 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455c012070 con 0x55e153a7e8d0 2024-04-27T03:32:19.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.716+0000 7f4455602640 1 -- 172.21.15.165:0/3819311631 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455c01a7a0 con 0x55e153a7e8d0 2024-04-27T03:32:19.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.716+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e153b77570 con 0x55e153a7e8d0 2024-04-27T03:32:19.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.716+0000 7f4455602640 1 -- 172.21.15.165:0/3819311631 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455c01aa80 con 0x55e153a7e8d0 2024-04-27T03:32:19.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.716+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e153b7a180 con 0x55e153a7e8d0 2024-04-27T03:32:19.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.717+0000 7f4455602640 1 -- 172.21.15.165:0/3819311631 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f455c015020 con 0x55e153a7e8d0 2024-04-27T03:32:19.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.717+0000 7f4455602640 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4450036950 0x7f4450038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.717+0000 7f4455602640 1 -- 172.21.15.165:0/3819311631 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f455c020070 con 0x55e153a7e8d0 2024-04-27T03:32:19.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.717+0000 7f4563fff640 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4450036950 0x7f4450038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.717+0000 7f456cd33800 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e153a823e0 0x55e153b7df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.717+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153b7e4c0 con 0x55e153a823e0 2024-04-27T03:32:19.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.718+0000 7f45691f6640 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e153a823e0 0x55e153b7df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.732 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.718+0000 7f4563fff640 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4450036950 0x7f4450038e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4554054000 tx=0x7f455404e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.718+0000 7f45691f6640 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e153a823e0 0x55e153b7df40 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.718+0000 7f45691f6640 1 -- 172.21.15.165:0/3819311631 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153a823e0 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.718+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153b7b510 con 0x55e153a823e0 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.719+0000 7f45691f6640 1 -- 172.21.15.165:0/3819311631 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153a823e0 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.719+0000 7f456cd33800 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e153b80820 0x55e153b82c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.719+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153b832b0 con 0x55e153b80820 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.719+0000 7f45689f5640 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e153b80820 0x55e153b82c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.719+0000 7f45689f5640 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e153b80820 0x55e153b82c10 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.719+0000 7f45689f5640 1 -- 172.21.15.165:0/3819311631 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f455c02b070 con 0x55e153b80820 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.719+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153b7f7d0 con 0x55e153a823e0 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.720+0000 7f45691f6640 1 -- 172.21.15.165:0/3819311631 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153a823e0 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.720+0000 7f456cd33800 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e153b84760 0x55e153b86bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.733 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.720+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153b87310 con 0x55e153b84760 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.720+0000 7f4563fff640 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e153b84760 0x55e153b86bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.720+0000 7f4563fff640 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e153b84760 0x55e153b86bb0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.720+0000 7f4563fff640 1 -- 172.21.15.165:0/3819311631 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f4554052a40 con 0x55e153b84760 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.721+0000 7f456cd33800 1 Processor -- start 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.721+0000 7f456cd33800 1 -- start start 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.721+0000 7f456cd33800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153b95660 0x55e153b95a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.721+0000 7f456cd33800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e15396f6b0 con 0x55e153b95660 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.721+0000 7f45689f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153b95660 0x55e153b95a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.721+0000 7f45689f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153b95660 0x55e153b95a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60520/0 (socket says 172.21.15.165:60520) 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.721+0000 7f45689f5640 1 -- 172.21.15.165:0/2610815272 learned_addr learned my addr 172.21.15.165:0/2610815272 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.721+0000 7f45689f5640 1 -- 172.21.15.165:0/2610815272 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e153b95fc0 con 0x55e153b95660 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.721+0000 7f45689f5640 1 --2- 172.21.15.165:0/2610815272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153b95660 0x55e153b95a80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f455c04aa30 tx=0x7f455c04ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.734 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.722+0000 7f444e7fc640 1 -- 172.21.15.165:0/2610815272 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455c0490e0 con 0x55e153b95660 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.722+0000 7f444e7fc640 1 -- 172.21.15.165:0/2610815272 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455c049280 con 0x55e153b95660 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.722+0000 7f456cd33800 1 -- 172.21.15.165:0/2610815272 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e153b96a30 con 0x55e153b95660 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.722+0000 7f444e7fc640 1 -- 172.21.15.165:0/2610815272 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455c049560 con 0x55e153b95660 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.722+0000 7f456cd33800 1 -- 172.21.15.165:0/2610815272 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e153b96a30 con 0x55e153b95660 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.722+0000 7f444e7fc640 1 -- 172.21.15.165:0/2610815272 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f455c044000 con 0x55e153b95660 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.722+0000 7f444e7fc640 1 --2- 172.21.15.165:0/2610815272 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4440036950 0x7f4440038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.722+0000 7f444e7fc640 1 -- 172.21.15.165:0/2610815272 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f455c04dde0 con 0x55e153b95660 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.723+0000 7f4563fff640 1 --2- 172.21.15.165:0/2610815272 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4440036950 0x7f4440038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.723+0000 7f456cd33800 1 Processor -- start 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.723+0000 7f456cd33800 1 -- start start 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.723+0000 7f4563fff640 1 --2- 172.21.15.165:0/2610815272 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4440036950 0x7f4440038e10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55e153b7b430 tx=0x7f4554005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.723+0000 7f456cd33800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 0x55e153a7c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.735 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.723+0000 7f456cd33800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e153b6fac0 con 0x55e153a7dfd0 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.723+0000 7f45689f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 0x55e153a7c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.723+0000 7f45689f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 0x55e153a7c6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60534/0 (socket says 172.21.15.165:60534) 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.723+0000 7f45689f5640 1 -- 172.21.15.165:0/3273747231 learned_addr learned my addr 172.21.15.165:0/3273747231 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.723+0000 7f45689f5640 1 -- 172.21.15.165:0/3273747231 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e153a7e480 con 0x55e153a7dfd0 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.724+0000 7f45689f5640 1 --2- 172.21.15.165:0/3273747231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 0x55e153a7c6c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f455c04efb0 tx=0x7f455c02e570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1e40017b7f764aa server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.724+0000 7f443ffff640 1 -- 172.21.15.165:0/3273747231 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455c052040 con 0x55e153a7dfd0 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.724+0000 7f443ffff640 1 -- 172.21.15.165:0/3273747231 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455c00a770 con 0x55e153a7dfd0 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.724+0000 7f443ffff640 1 -- 172.21.15.165:0/3273747231 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455c00a410 con 0x55e153a7dfd0 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.724+0000 7f456cd33800 1 -- 172.21.15.165:0/3273747231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 msgr2=0x55e153a7c6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.724+0000 7f456cd33800 1 --2- 172.21.15.165:0/3273747231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 0x55e153a7c6c0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f455c04efb0 tx=0x7f455c02e570 comp rx=0 tx=0).stop 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.724+0000 7f456cd33800 1 -- 172.21.15.165:0/3273747231 shutdown_connections 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.724+0000 7f456cd33800 1 --2- 172.21.15.165:0/3273747231 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 0x55e153a7c6c0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.724+0000 7f456cd33800 1 -- 172.21.15.165:0/3273747231 >> 172.21.15.165:0/3273747231 conn(0x55e153b9e930 msgr2=0x55e153b9c990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.736 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.724+0000 7f456cd33800 1 -- 172.21.15.165:0/3273747231 shutdown_connections 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.724+0000 7f456cd33800 1 -- 172.21.15.165:0/3273747231 wait complete. 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.724+0000 7f456cd33800 1 Processor -- start 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.725+0000 7f456cd33800 1 -- start start 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.725+0000 7f456cd33800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 0x55e153babb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.725+0000 7f456cd33800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55e153a7cc00 con 0x55e153a7dfd0 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.725+0000 7f45689f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 0x55e153babb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.725+0000 7f45689f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 0x55e153babb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60536/0 (socket says 172.21.15.165:60536) 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.725+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 learned_addr learned my addr 172.21.15.165:0/2046969228 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.725+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e153bac0c0 con 0x55e153a7dfd0 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.725+0000 7f45689f5640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 0x55e153babb80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f455c04efb0 tx=0x7f455c00a5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.726+0000 7f443e7fc640 1 -- 172.21.15.165:0/2046969228 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455c067070 con 0x55e153a7dfd0 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.726+0000 7f443e7fc640 1 -- 172.21.15.165:0/2046969228 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f455c0225a0 con 0x55e153a7dfd0 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.726+0000 7f443e7fc640 1 -- 172.21.15.165:0/2046969228 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f455c022e80 con 0x55e153a7dfd0 2024-04-27T03:32:19.737 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.726+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55e153bac0c0 con 0x55e153a7dfd0 2024-04-27T03:32:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.726+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e153bac0c0 con 0x55e153a7dfd0 2024-04-27T03:32:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.726+0000 7f443e7fc640 1 -- 172.21.15.165:0/2046969228 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f455c06a070 con 0x55e153a7dfd0 2024-04-27T03:32:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.726+0000 7f443e7fc640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4434036950 0x7f4434038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.727+0000 7f443e7fc640 1 -- 172.21.15.165:0/2046969228 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f455c05c020 con 0x55e153a7dfd0 2024-04-27T03:32:19.738 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.727+0000 7f4563fff640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4434036950 0x7f4434038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.727+0000 7f456cd33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.727+0000 7f4563fff640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4434036950 0x7f4434038e10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4554052fd0 tx=0x7f455404f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.727+0000 7f456cd33800 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e153bb72b0 0x55e153bb9770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.727+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bb9ec0 con 0x55e153bb72b0 2024-04-27T03:32:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.727+0000 7f45691f6640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e153bb72b0 0x55e153bb9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.727+0000 7f45691f6640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e153bb72b0 0x55e153bb9770 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 20 realm 2024-04-27T03:32:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bba2d0 con 0x55e153bb72b0 2024-04-27T03:32:19.753 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaba0 con 0x55e153bb72b0 2024-04-27T03:32:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaf40 con 0x55e153bb72b0 2024-04-27T03:32:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.754 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e153bbbca0 0x55e153bbe090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbe6a0 con 0x55e153bbbca0 2024-04-27T03:32:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.728+0000 7f45689f5640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e153bbbca0 0x55e153bbe090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.729+0000 7f45689f5640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e153bbbca0 0x55e153bbe090 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.729+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.729+0000 7f456cd33800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.729+0000 7f456cd33800 20 searching for the correct realm 2024-04-27T03:32:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.729+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbffd0 con 0x55e153bbbca0 2024-04-27T03:32:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.729+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.730+0000 7f45689f5640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f455c06c070 0x7f455c03e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.730+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbffd0 con 0x7f455c06c070 2024-04-27T03:32:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.730+0000 7f4563fff640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f455c06c070 0x7f455c03e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.755 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.730+0000 7f4563fff640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f455c06c070 0x7f455c03e260 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.730+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.730+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbffd0 con 0x7f455c06c070 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.730+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.730+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbffd0 con 0x55e153bb72b0 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.731+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.731+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaf40 con 0x55e153bb72b0 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.731+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.731+0000 7f45691f6640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4564016440 0x7f45640168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.731+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaf40 con 0x7f4564016440 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.731+0000 7f45691f6640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4564016440 0x7f45640168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.732+0000 7f45691f6640 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4564016440 0x7f45640168c0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.732+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.756 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.732+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaf40 con 0x7f4564016440 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.732+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.732+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaf40 con 0x7f455c06c070 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.732+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.732+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaf40 con 0x55e153bbbca0 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.733+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.733+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaf40 con 0x55e153bb72b0 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.733+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.733+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaf40 con 0x7f455c06c070 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.733+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.733+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaf40 con 0x55e153bb72b0 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.733+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.733+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaf40 con 0x7f455c06c070 2024-04-27T03:32:19.757 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.734+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.734+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaf40 con 0x55e153bbbca0 2024-04-27T03:32:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.734+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.734+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbaf40 con 0x55e153bbbca0 2024-04-27T03:32:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.734+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.734+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x7f4564016440 2024-04-27T03:32:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.734+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.734+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x55e153bbbca0 2024-04-27T03:32:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.735+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.735+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x55e153bbbca0 2024-04-27T03:32:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.735+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.735+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x55e153bb72b0 2024-04-27T03:32:19.758 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.735+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.735+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x55e153bb72b0 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.735+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.735+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x7f4564016440 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.736+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.736+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x55e153bbbca0 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.736+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.736+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x7f455c06c070 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.736+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.736+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x55e153bb72b0 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.736+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.736+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x7f4564016440 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.737+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.737+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x55e153bbbca0 2024-04-27T03:32:19.759 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.737+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.737+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x7f4564016440 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.737+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.737+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x7f455c06c070 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.738+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.738+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x55e153bbbca0 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.738+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.738+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x7f4564016440 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.738+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.738+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x55e153bb72b0 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.738+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.738+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x7f455c084040 con 0x7f455c06c070 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:19.760 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbef60 con 0x55e153bb72b0 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbf350 con 0x55e153bb72b0 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.761 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bbf350 con 0x7f455c06c070 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 zone default found 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 4 Realm: () 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 zonegroup default 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.739+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc0450 con 0x7f4564016440 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f456cd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f456cd33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.762 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc1e30 con 0x55e153bbbca0 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f456cd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f456cd33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc2330 con 0x55e153bbbca0 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f456cd33800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f456cd33800 20 started sync module instance, tier type = 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f456cd33800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.740+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc12d0 con 0x7f455c06c070 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.741+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.741+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94426260838080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc0f30 con 0x7f455c06c070 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.741+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc30f0 con 0x55e153bb72b0 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.742+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94426260838080] v27'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.763 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.743+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.743+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94426260842864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc34e0 con 0x55e153bb72b0 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.743+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc3f20 con 0x55e153bbbca0 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.744+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94426260842864] v27'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.744+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.744+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94426260847088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc3ee0 con 0x55e153bbbca0 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.744+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc49b0 con 0x7f455c06c070 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.745+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94426260847088] v27'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.745+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.745+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94426260849888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc4da0 con 0x7f455c06c070 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.745+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc57a0 con 0x55e153bb72b0 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.746+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94426260849888] v27'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.746+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.746+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94426260844016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc5b90 con 0x55e153bb72b0 2024-04-27T03:32:19.764 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.746+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc65c0 con 0x7f4564016440 2024-04-27T03:32:19.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.747+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94426260844016] v27'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.747+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.747+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94426260857040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc65c0 con 0x7f4564016440 2024-04-27T03:32:19.765 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.747+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc7080 con 0x7f4564016440 2024-04-27T03:32:19.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.748+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94426260857040] v27'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.748+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.748+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94426260859728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc7040 con 0x7f4564016440 2024-04-27T03:32:19.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.748+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc7a60 con 0x7f4564016440 2024-04-27T03:32:19.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.750+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94426260859728] v27'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.750+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'273 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.750+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94426260862288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc7a20 con 0x7f4564016440 2024-04-27T03:32:19.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.750+0000 7f456cd33800 20 add_watcher() i=0 2024-04-27T03:32:19.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.750+0000 7f456cd33800 20 add_watcher() i=1 2024-04-27T03:32:19.769 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.750+0000 7f456cd33800 20 add_watcher() i=2 2024-04-27T03:32:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.750+0000 7f456cd33800 20 add_watcher() i=3 2024-04-27T03:32:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.750+0000 7f456cd33800 20 add_watcher() i=4 2024-04-27T03:32:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.750+0000 7f456cd33800 20 add_watcher() i=5 2024-04-27T03:32:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.750+0000 7f456cd33800 20 add_watcher() i=6 2024-04-27T03:32:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.751+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94426260862288] v27'274 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.751+0000 7f456cd33800 20 add_watcher() i=7 2024-04-27T03:32:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.751+0000 7f456cd33800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.751+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bcae80 con 0x55e153bb72b0 2024-04-27T03:32:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.752+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.752+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94426260853424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc83d0 con 0x55e153bb72b0 2024-04-27T03:32:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.753+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94426260853424] v27'273 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.770 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.753+0000 7f441efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.753+0000 7f456cd33800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.753+0000 7f456cd33800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.753+0000 7f456cd33800 5 note: GC not initialized 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.753+0000 7f441dfe3640 20 reqs_thread_entry: start 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.753+0000 7f456cd33800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.753+0000 7f456cd33800 20 Filter name: none 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.753+0000 7f438f7fe640 20 reqs_thread_entry: start 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.753+0000 7f456cd33800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.753+0000 7f456cd33800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.753+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153bfe230 con 0x7f4564016440 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.756+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 ada [call out=48b,read 0~490 out=490b,getxattrs out=75b] v0'0 uv10 ondisk = 0) ==== 231+0+613 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.756+0000 7f456cd33800 20 rados_obj.operate() r=0 bl.length=490 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.756+0000 7f456cd33800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-27T03:32:19.771 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.756+0000 7f456cd33800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.756+0000 7f456cd33800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:19.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.757+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~460 in=460b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153c04b30 con 0x7f4564016440 2024-04-27T03:32:19.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.758+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~460] v27'11 uv11 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.758+0000 7f456cd33800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.758+0000 7f456cd33800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.758+0000 7f456cd33800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-27T03:32:19.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.758+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94426261112224 in=648b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55e153c00220 con 0x55e153bb72b0 2024-04-27T03:32:19.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.758+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== watch-notify(notify (1) cookie 94426260844016 notify 115964117047 ret 0) ==== 678+0+0 (crc 0 0 0) 0x7f4564030040 con 0x55e153bb72b0 2024-04-27T03:32:19.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.758+0000 7f443ffff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117047 cookie 94426260844016 notifier 4896 bl.length()=636 2024-04-27T03:32:19.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.758+0000 7f443ffff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.758+0000 7f443ffff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.772 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.758+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 notify.4 [notify cookie 94426261112224 out=8b] v0'0 uv273 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.758+0000 7f443ffff640 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f442c003ea0 con 0x55e153bb72b0 2024-04-27T03:32:19.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.759+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.773 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.759+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify_complete (2) cookie 94426261112224 notify 115964117047 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f456402a8f0 con 0x55e153bb72b0 2024-04-27T03:32:19.773 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-27T03:32:18.064161Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:32:18.064141Z","tags":[],"group_ids":[]} 2024-04-27T03:32:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.759+0000 7f441efe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.760+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94426260853424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153c00630 con 0x55e153bb72b0 2024-04-27T03:32:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.761+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94426260853424] v27'274 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.761+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94426260838080] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc52b0 con 0x7f455c06c070 2024-04-27T03:32:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.762+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94426260838080] v27'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.762+0000 7f456cd33800 20 remove_watcher() i=0 2024-04-27T03:32:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.762+0000 7f456cd33800 2 removed watcher, disabling cache 2024-04-27T03:32:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.762+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94426260842864] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc16c0 con 0x55e153bb72b0 2024-04-27T03:32:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.763+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94426260842864] v27'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.779 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.763+0000 7f456cd33800 20 remove_watcher() i=1 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.763+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94426260847088] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc16c0 con 0x55e153bbbca0 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.764+0000 7f45689f5640 1 -- 172.21.15.165:0/2046969228 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94426260847088] v27'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f455c073070 con 0x55e153bbbca0 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.764+0000 7f456cd33800 20 remove_watcher() i=2 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.764+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94426260849888] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc16c0 con 0x7f455c06c070 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.765+0000 7f4563fff640 1 -- 172.21.15.165:0/2046969228 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94426260849888] v27'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4554081070 con 0x7f455c06c070 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.765+0000 7f456cd33800 20 remove_watcher() i=3 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.765+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94426260844016] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc2970 con 0x55e153bb72b0 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.766+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94426260844016] v27'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x55e153bb72b0 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.766+0000 7f456cd33800 20 remove_watcher() i=4 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.766+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94426260857040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc2970 con 0x7f4564016440 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.767+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94426260857040] v27'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.767+0000 7f456cd33800 20 remove_watcher() i=5 2024-04-27T03:32:19.780 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.767+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94426260859728] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc2970 con 0x7f4564016440 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.768+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94426260859728] v27'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.768+0000 7f456cd33800 20 remove_watcher() i=6 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.768+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94426260862288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55e153bc2970 con 0x7f4564016440 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94426260862288] v27'275 uv273 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f456400a4a0 con 0x7f4564016440 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 20 remove_watcher() i=7 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4564016440 msgr2=0x7f45640168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4564016440 0x7f45640168c0 crc :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e153bb72b0 msgr2=0x55e153bb9770 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e153bb72b0 0x55e153bb9770 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f455c06c070 msgr2=0x7f455c03e260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f455c06c070 0x7f455c03e260 crc :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e153bbbca0 msgr2=0x55e153bbe090 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.781 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e153bbbca0 0x55e153bbe090 crc :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4434036950 msgr2=0x7f4434038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4434036950 0x7f4434038e10 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4554052fd0 tx=0x7f455404f000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 msgr2=0x55e153babb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 --2- 172.21.15.165:0/2046969228 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7dfd0 0x55e153babb80 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f455c04efb0 tx=0x7f455c00a5c0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 reap_dead start 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f45691f6640 1 -- 172.21.15.165:0/2046969228 reap_dead start 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 shutdown_connections 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 >> 172.21.15.165:0/2046969228 conn(0x55e153b9e930 msgr2=0x55e153b9bf90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 shutdown_connections 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.769+0000 7f456cd33800 1 -- 172.21.15.165:0/2046969228 wait complete. 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/2610815272 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4440036950 msgr2=0x7f4440038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 --2- 172.21.15.165:0/2610815272 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4440036950 0x7f4440038e10 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x55e153b7b430 tx=0x7f4554005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:19.782 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/2610815272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153b95660 msgr2=0x55e153b95a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 --2- 172.21.15.165:0/2610815272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153b95660 0x55e153b95a80 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7f455c04aa30 tx=0x7f455c04ac30 comp rx=0 tx=0).stop 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/2610815272 shutdown_connections 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 --2- 172.21.15.165:0/2610815272 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4440036950 0x7f4440038e10 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 --2- 172.21.15.165:0/2610815272 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153b95660 0x55e153b95a80 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/2610815272 >> 172.21.15.165:0/2610815272 conn(0x55e153b89700 msgr2=0x55e153b89b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/2610815272 shutdown_connections 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/2610815272 wait complete. 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e153a823e0 msgr2=0x55e153b7df40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55e153a823e0 0x55e153b7df40 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e153b84760 msgr2=0x55e153b86bb0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55e153b84760 0x55e153b86bb0 crc :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e153b80820 msgr2=0x55e153b82c10 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55e153b80820 0x55e153b82c10 crc :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4450036950 msgr2=0x7f4450038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.783 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4450036950 0x7f4450038e10 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f4554054000 tx=0x7f455404e000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 msgr2=0x55e153b76da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 --2- 172.21.15.165:0/3819311631 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55e153a7e8d0 0x55e153b76da0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f455c00c010 tx=0x7f455c000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f45691f6640 1 -- 172.21.15.165:0/3819311631 reap_dead start 2024-04-27T03:32:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 shutdown_connections 2024-04-27T03:32:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 >> 172.21.15.165:0/3819311631 conn(0x55e153a783a0 msgr2=0x55e153b692f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 shutdown_connections 2024-04-27T03:32:19.784 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.770+0000 7f456cd33800 1 -- 172.21.15.165:0/3819311631 wait complete. 2024-04-27T03:32:19.785 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.785 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.785 ERROR:root:0 2024-04-27T03:32:19.785 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-27T03:32:19.785 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-27T03:32:19.786 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'users=read, write' 2024-04-27T03:32:19.830 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:19.830 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:19.831 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.825+0000 7f3926946800 1 Processor -- start 2024-04-27T03:32:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.825+0000 7f3926946800 1 -- start start 2024-04-27T03:32:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.825+0000 7f3926946800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562f261a6d80 0x562f261a71a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.825+0000 7f3926946800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x562f26066d40 con 0x562f261a7fe0 2024-04-27T03:32:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.825+0000 7f3926946800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x562f260394d0 con 0x562f261a6d80 2024-04-27T03:32:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.826+0000 7f3924445640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x562f261a6d80 0x562f261a71a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.826+0000 7f3923c44640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x562f261a7fe0 0x562f261a8460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42430/0 (socket says 172.21.15.165:42430) 2024-04-27T03:32:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.826+0000 7f3923c44640 1 -- 172.21.15.165:0/131959841 learned_addr learned my addr 172.21.15.165:0/131959841 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.826+0000 7f3924445640 1 -- 172.21.15.165:0/131959841 >> v1:172.21.15.89:6789/0 conn(0x562f261a7fe0 legacy=0x562f261a8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.826+0000 7f3924445640 1 -- 172.21.15.165:0/131959841 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f26065590 con 0x562f261a6d80 2024-04-27T03:32:19.832 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.826+0000 7f3923443640 1 -- 172.21.15.165:0/131959841 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 987853037 0 0) 0x562f26066d40 con 0x562f261a7fe0 2024-04-27T03:32:19.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.828+0000 7f3924445640 1 --2- 172.21.15.165:0/131959841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f261a71a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f391400a470 tx=0x7f391401ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b652c3ee0e323501 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.828+0000 7f3923443640 1 -- 172.21.15.165:0/131959841 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3914057070 con 0x562f261a6d80 2024-04-27T03:32:19.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.828+0000 7f3923443640 1 -- 172.21.15.165:0/131959841 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3914052570 con 0x562f261a6d80 2024-04-27T03:32:19.834 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.828+0000 7f3923443640 1 -- 172.21.15.165:0/131959841 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3914052870 con 0x562f261a6d80 2024-04-27T03:32:19.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.829+0000 7f3926946800 1 -- 172.21.15.165:0/131959841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 msgr2=0x562f261a71a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.829+0000 7f3926946800 1 --2- 172.21.15.165:0/131959841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f261a71a0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f391400a470 tx=0x7f391401ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:19.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.829+0000 7f3926946800 1 -- 172.21.15.165:0/131959841 shutdown_connections 2024-04-27T03:32:19.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.829+0000 7f3926946800 1 --2- 172.21.15.165:0/131959841 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f261a71a0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.829+0000 7f3926946800 1 -- 172.21.15.165:0/131959841 >> 172.21.15.165:0/131959841 conn(0x562f261a2340 msgr2=0x562f261a4760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.829+0000 7f3926946800 1 -- 172.21.15.165:0/131959841 shutdown_connections 2024-04-27T03:32:19.835 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.829+0000 7f3926946800 1 -- 172.21.15.165:0/131959841 wait complete. 2024-04-27T03:32:19.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.840+0000 7f3926946800 1 Processor -- start 2024-04-27T03:32:19.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.840+0000 7f3926946800 1 -- start start 2024-04-27T03:32:19.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.841+0000 7f3926946800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f2629b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.841+0000 7f3926946800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562f260f33b0 con 0x562f261a6d80 2024-04-27T03:32:19.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.841+0000 7f3924445640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f2629b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.841+0000 7f3924445640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f2629b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60552/0 (socket says 172.21.15.165:60552) 2024-04-27T03:32:19.847 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.841+0000 7f3924445640 1 -- 172.21.15.165:0/3460126011 learned_addr learned my addr 172.21.15.165:0/3460126011 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.841+0000 7f3924445640 1 -- 172.21.15.165:0/3460126011 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f2629bbd0 con 0x562f261a6d80 2024-04-27T03:32:19.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.841+0000 7f3924445640 1 --2- 172.21.15.165:0/3460126011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f2629b600 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3914055040 tx=0x7f391404c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e31d076901329a73 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.841+0000 7f381260c640 1 -- 172.21.15.165:0/3460126011 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3914057070 con 0x562f261a6d80 2024-04-27T03:32:19.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.841+0000 7f381260c640 1 -- 172.21.15.165:0/3460126011 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3914063070 con 0x562f261a6d80 2024-04-27T03:32:19.848 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.841+0000 7f381260c640 1 -- 172.21.15.165:0/3460126011 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3914052720 con 0x562f261a6d80 2024-04-27T03:32:19.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.842+0000 7f3926946800 1 -- 172.21.15.165:0/3460126011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 msgr2=0x562f2629b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.842+0000 7f3926946800 1 --2- 172.21.15.165:0/3460126011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f2629b600 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f3914055040 tx=0x7f391404c000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.842+0000 7f3926946800 1 -- 172.21.15.165:0/3460126011 shutdown_connections 2024-04-27T03:32:19.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.842+0000 7f3926946800 1 --2- 172.21.15.165:0/3460126011 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f2629b600 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.842+0000 7f3926946800 1 -- 172.21.15.165:0/3460126011 >> 172.21.15.165:0/3460126011 conn(0x562f261a2340 msgr2=0x562f26292b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.842+0000 7f3926946800 1 -- 172.21.15.165:0/3460126011 shutdown_connections 2024-04-27T03:32:19.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.842+0000 7f3926946800 1 -- 172.21.15.165:0/3460126011 wait complete. 2024-04-27T03:32:19.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.842+0000 7f3926946800 1 Processor -- start 2024-04-27T03:32:19.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.843+0000 7f3926946800 1 -- start start 2024-04-27T03:32:19.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.843+0000 7f3926946800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f262a3050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.843+0000 7f3926946800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562f2611afd0 con 0x562f261a6d80 2024-04-27T03:32:19.852 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.843+0000 7f3924445640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f262a3050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.843+0000 7f3924445640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f262a3050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60556/0 (socket says 172.21.15.165:60556) 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.843+0000 7f3924445640 1 -- 172.21.15.165:0/665303797 learned_addr learned my addr 172.21.15.165:0/665303797 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.843+0000 7f3924445640 1 -- 172.21.15.165:0/665303797 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f262a3590 con 0x562f261a6d80 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.843+0000 7f3924445640 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f262a3050 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3914060040 tx=0x7f3914002fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.844+0000 7f3810e09640 1 -- 172.21.15.165:0/665303797 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3914052c70 con 0x562f261a6d80 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.844+0000 7f3810e09640 1 -- 172.21.15.165:0/665303797 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f391406c070 con 0x562f261a6d80 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.844+0000 7f3810e09640 1 -- 172.21.15.165:0/665303797 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f391405e480 con 0x562f261a6d80 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.844+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562f262a3820 con 0x562f261a6d80 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.844+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f262a64d0 con 0x562f261a6d80 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.844+0000 7f3810e09640 1 -- 172.21.15.165:0/665303797 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f391405e620 con 0x562f261a6d80 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.845+0000 7f3810e09640 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f380c036950 0x7f380c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.845+0000 7f3810e09640 1 -- 172.21.15.165:0/665303797 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f391408e660 con 0x562f261a6d80 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.845+0000 7f3923c44640 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f380c036950 0x7f380c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.853 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.845+0000 7f3926946800 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562f261a7fe0 0x562f262aa290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.845+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262aa820 con 0x562f261a7fe0 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.845+0000 7f3924c46640 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562f261a7fe0 0x562f262aa290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.845+0000 7f3923c44640 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f380c036950 0x7f380c038e10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3918000c00 tx=0x7f39180013b0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.845+0000 7f3924c46640 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562f261a7fe0 0x562f262aa290 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.845+0000 7f3924c46640 1 -- 172.21.15.165:0/665303797 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f261a7fe0 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.845+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262a7820 con 0x562f261a7fe0 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.846+0000 7f3924c46640 1 -- 172.21.15.165:0/665303797 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f261a7fe0 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.846+0000 7f3926946800 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562f262acb50 0x562f262aef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.846+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262af610 con 0x562f262acb50 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.846+0000 7f3924445640 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562f262acb50 0x562f262aef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.846+0000 7f3924445640 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562f262acb50 0x562f262aef40 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.846+0000 7f3924445640 1 -- 172.21.15.165:0/665303797 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f3914009d10 con 0x562f262acb50 2024-04-27T03:32:19.854 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.846+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262abb00 con 0x562f261a7fe0 2024-04-27T03:32:19.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.847+0000 7f3924c46640 1 -- 172.21.15.165:0/665303797 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f261a7fe0 2024-04-27T03:32:19.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.847+0000 7f3926946800 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562f262b0a50 0x562f262b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.847+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262b3650 con 0x562f262b0a50 2024-04-27T03:32:19.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.847+0000 7f3923c44640 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562f262b0a50 0x562f262b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.847+0000 7f3923c44640 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562f262b0a50 0x562f262b2ec0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.847+0000 7f3923c44640 1 -- 172.21.15.165:0/665303797 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f391800f670 con 0x562f262b0a50 2024-04-27T03:32:19.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.848+0000 7f3926946800 1 Processor -- start 2024-04-27T03:32:19.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.848+0000 7f3926946800 1 -- start start 2024-04-27T03:32:19.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.848+0000 7f3926946800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262c1ab0 0x562f262c1ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.848+0000 7f3926946800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562f260996b0 con 0x562f262c1ab0 2024-04-27T03:32:19.867 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.848+0000 7f3924445640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262c1ab0 0x562f262c1ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.848+0000 7f3924445640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262c1ab0 0x562f262c1ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60562/0 (socket says 172.21.15.165:60562) 2024-04-27T03:32:19.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.848+0000 7f3924445640 1 -- 172.21.15.165:0/2864122060 learned_addr learned my addr 172.21.15.165:0/2864122060 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.848+0000 7f3924445640 1 -- 172.21.15.165:0/2864122060 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f262c2410 con 0x562f262c1ab0 2024-04-27T03:32:19.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.848+0000 7f3924445640 1 --2- 172.21.15.165:0/2864122060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262c1ab0 0x562f262c1ed0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f391408cab0 tx=0x7f3914067df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.849+0000 7f3809ffb640 1 -- 172.21.15.165:0/2864122060 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f3914093070 con 0x562f262c1ab0 2024-04-27T03:32:19.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.849+0000 7f3809ffb640 1 -- 172.21.15.165:0/2864122060 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3914058040 con 0x562f262c1ab0 2024-04-27T03:32:19.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.849+0000 7f3926946800 1 -- 172.21.15.165:0/2864122060 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562f262c2e80 con 0x562f262c1ab0 2024-04-27T03:32:19.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.849+0000 7f3809ffb640 1 -- 172.21.15.165:0/2864122060 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f391408ce60 con 0x562f262c1ab0 2024-04-27T03:32:19.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.849+0000 7f3926946800 1 -- 172.21.15.165:0/2864122060 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f262c3380 con 0x562f262c1ab0 2024-04-27T03:32:19.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.849+0000 7f3809ffb640 1 -- 172.21.15.165:0/2864122060 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f3914096070 con 0x562f262c1ab0 2024-04-27T03:32:19.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.849+0000 7f3809ffb640 1 --2- 172.21.15.165:0/2864122060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37fc036950 0x7f37fc038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.868 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.850+0000 7f3809ffb640 1 -- 172.21.15.165:0/2864122060 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f3914099be0 con 0x562f262c1ab0 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.850+0000 7f3923c44640 1 --2- 172.21.15.165:0/2864122060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37fc036950 0x7f37fc038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.850+0000 7f3926946800 1 Processor -- start 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.850+0000 7f3926946800 1 -- start start 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.850+0000 7f3923c44640 1 --2- 172.21.15.165:0/2864122060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37fc036950 0x7f37fc038e10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x562f262ab100 tx=0x7f3918006000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.850+0000 7f3926946800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 0x562f262cef70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.850+0000 7f3926946800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562f2629be90 con 0x562f262ceb50 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.850+0000 7f3924445640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 0x562f262cef70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.850+0000 7f3924445640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 0x562f262cef70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60570/0 (socket says 172.21.15.165:60570) 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.850+0000 7f3924445640 1 -- 172.21.15.165:0/4280311547 learned_addr learned my addr 172.21.15.165:0/4280311547 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.851+0000 7f3924445640 1 -- 172.21.15.165:0/4280311547 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f262cf540 con 0x562f262ceb50 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.851+0000 7f3924445640 1 --2- 172.21.15.165:0/4280311547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 0x562f262cef70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3914086fd0 tx=0x7f3914099f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8656216b680200ea server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.851+0000 7f38037fe640 1 -- 172.21.15.165:0/4280311547 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f391404b100 con 0x562f262ceb50 2024-04-27T03:32:19.869 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.851+0000 7f38037fe640 1 -- 172.21.15.165:0/4280311547 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f3914067420 con 0x562f262ceb50 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.851+0000 7f38037fe640 1 -- 172.21.15.165:0/4280311547 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f391404b2a0 con 0x562f262ceb50 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.851+0000 7f3926946800 1 -- 172.21.15.165:0/4280311547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 msgr2=0x562f262cef70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.851+0000 7f3926946800 1 --2- 172.21.15.165:0/4280311547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 0x562f262cef70 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f3914086fd0 tx=0x7f3914099f40 comp rx=0 tx=0).stop 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.851+0000 7f3926946800 1 -- 172.21.15.165:0/4280311547 shutdown_connections 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.851+0000 7f3926946800 1 --2- 172.21.15.165:0/4280311547 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 0x562f262cef70 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.851+0000 7f3926946800 1 -- 172.21.15.165:0/4280311547 >> 172.21.15.165:0/4280311547 conn(0x562f262cae60 msgr2=0x562f262c8ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.851+0000 7f3926946800 1 -- 172.21.15.165:0/4280311547 shutdown_connections 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.851+0000 7f3926946800 1 -- 172.21.15.165:0/4280311547 wait complete. 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.852+0000 7f3926946800 1 Processor -- start 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.852+0000 7f3926946800 1 -- start start 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.852+0000 7f3926946800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 0x562f262d5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.852+0000 7f3926946800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x562f262cfbd0 con 0x562f262ceb50 2024-04-27T03:32:19.870 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.852+0000 7f3924445640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 0x562f262d5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.852+0000 7f3924445640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 0x562f262d5e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60584/0 (socket says 172.21.15.165:60584) 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.852+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 learned_addr learned my addr 172.21.15.165:0/3170482494 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.853+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562f262d63b0 con 0x562f262ceb50 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.853+0000 7f3924445640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 0x562f262d5e70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3914009cc0 tx=0x7f3914086ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.853+0000 7f3801ffb640 1 -- 172.21.15.165:0/3170482494 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39140679e0 con 0x562f262ceb50 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.853+0000 7f3801ffb640 1 -- 172.21.15.165:0/3170482494 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f39140095f0 con 0x562f262ceb50 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.853+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x562f262d63b0 con 0x562f262ceb50 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.853+0000 7f3801ffb640 1 -- 172.21.15.165:0/3170482494 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f39140098d0 con 0x562f262ceb50 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.853+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562f262d63b0 con 0x562f262ceb50 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.854+0000 7f3801ffb640 1 -- 172.21.15.165:0/3170482494 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f39140667b0 con 0x562f262ceb50 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.854+0000 7f3801ffb640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37f40368b0 0x7f37f4038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.854+0000 7f3801ffb640 1 -- 172.21.15.165:0/3170482494 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f39140a3070 con 0x562f262ceb50 2024-04-27T03:32:19.871 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.854+0000 7f3923c44640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37f40368b0 0x7f37f4038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.854+0000 7f3923c44640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37f40368b0 0x7f37f4038d70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f391800ffd0 tx=0x7f3918023040 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.854+0000 7f3926946800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.854+0000 7f3926946800 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562f262e1540 0x562f262e3a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.854+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262e4150 con 0x562f262e1540 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3924c46640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562f262e1540 0x562f262e3a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3924c46640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562f262e1540 0x562f262e3a00 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3926946800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3926946800 20 realm 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3926946800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262e4560 con 0x562f262e1540 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.872 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3926946800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3926946800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3926946800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262e4e30 con 0x562f262e1540 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3926946800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3926946800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.855+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e1540 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.856+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.856+0000 7f3926946800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.856+0000 7f3926946800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.856+0000 7f3926946800 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562f262e5f30 0x562f262e8320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.856+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262e88e0 con 0x562f262e5f30 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.856+0000 7f3924445640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562f262e5f30 0x562f262e8320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.873 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.856+0000 7f3924445640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562f262e5f30 0x562f262e8320 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.856+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.856+0000 7f3926946800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.856+0000 7f3926946800 20 searching for the correct realm 2024-04-27T03:32:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.856+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262ea210 con 0x562f262e5f30 2024-04-27T03:32:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.857+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.857+0000 7f3924445640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f39140b1040 0x7f391406f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.857+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262ea210 con 0x7f39140b1040 2024-04-27T03:32:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.857+0000 7f3923c44640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f39140b1040 0x7f391406f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.857+0000 7f3923c44640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f39140b1040 0x7f391406f590 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.858+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.874 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.858+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262ea210 con 0x7f39140b1040 2024-04-27T03:32:19.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.858+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.858+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262ea210 con 0x562f262e1540 2024-04-27T03:32:19.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.858+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.858+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e1540 2024-04-27T03:32:19.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.858+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.858+0000 7f3924c46640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f391c016440 0x7f391c0168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.858+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f391c016440 2024-04-27T03:32:19.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.858+0000 7f3924c46640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f391c016440 0x7f391c0168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.859+0000 7f3924c46640 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f391c016440 0x7f391c0168c0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.859+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.859+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f391c016440 2024-04-27T03:32:19.875 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.859+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.859+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f39140b1040 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.859+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.859+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e5f30 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.860+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.860+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e1540 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.860+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.860+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f39140b1040 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.860+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.860+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e1540 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.860+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.860+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f39140b1040 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.861+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.876 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.861+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e5f30 2024-04-27T03:32:19.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.861+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.861+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e5f30 2024-04-27T03:32:19.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.861+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.877 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.861+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f391c016440 2024-04-27T03:32:19.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.862+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.862+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e5f30 2024-04-27T03:32:19.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.862+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.862+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e5f30 2024-04-27T03:32:19.889 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.862+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.862+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e1540 2024-04-27T03:32:19.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.862+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.862+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e1540 2024-04-27T03:32:19.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.863+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.863+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f391c016440 2024-04-27T03:32:19.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.863+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.863+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e5f30 2024-04-27T03:32:19.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.863+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.863+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f39140b1040 2024-04-27T03:32:19.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.863+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.863+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e1540 2024-04-27T03:32:19.890 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.864+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.864+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f391c016440 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.864+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.864+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e5f30 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.864+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.864+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f391c016440 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.864+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.864+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f39140b1040 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.865+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.865+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e5f30 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.865+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.865+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f391c016440 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.865+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.891 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.865+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x562f262e1540 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.865+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.865+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x562f262e51d0 con 0x7f39140b1040 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262e91a0 con 0x562f262e1540 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.892 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262e9590 con 0x562f262e1540 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262e9590 con 0x7f39140b1040 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 zone default found 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 4 Realm: () 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:19.893 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 zonegroup default 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.866+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262ea690 con 0x7f391c016440 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3926946800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3926946800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262e9b40 con 0x562f262e5f30 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3926946800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3926946800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262ec5f0 con 0x562f262e5f30 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.894 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3926946800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3926946800 20 started sync module instance, tier type = 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3926946800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.867+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262eb9b0 con 0x7f39140b1040 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.868+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.868+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94760504049056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262ecf00 con 0x7f39140b1040 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.868+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262ed750 con 0x562f262e1540 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.869+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94760504049056] v27'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.870+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.870+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94760504052304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262ed790 con 0x562f262e1540 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.870+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262ee270 con 0x562f262e5f30 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.871+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.871+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94760504057120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262eeb10 con 0x562f262e5f30 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.871+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262ef5d0 con 0x7f39140b1040 2024-04-27T03:32:19.895 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.871+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94760504052304] v27'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.872+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94760504057120] v27'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.872+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.872+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94760504062208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262ef9f0 con 0x7f39140b1040 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.872+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262f0240 con 0x562f262e1540 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.873+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94760504062208] v27'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.873+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.873+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94760504065424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262f01b0 con 0x562f262e1540 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.873+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262f1100 con 0x7f391c016440 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.874+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.874+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94760504069200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262f1520 con 0x7f391c016440 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.874+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94760504065424] v27'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.874+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262f1fa0 con 0x7f391c016440 2024-04-27T03:32:19.896 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.876+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94760504069200] v27'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.876+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.876+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94760504072848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262f1520 con 0x7f391c016440 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.876+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262f2410 con 0x7f391c016440 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.877+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94760504072848] v27'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.877+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'276 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.877+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94760504074560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262f1520 con 0x7f391c016440 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.877+0000 7f3926946800 20 add_watcher() i=0 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.877+0000 7f3926946800 20 add_watcher() i=1 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.877+0000 7f3926946800 20 add_watcher() i=2 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.877+0000 7f3926946800 20 add_watcher() i=3 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.877+0000 7f3926946800 20 add_watcher() i=4 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.877+0000 7f3926946800 20 add_watcher() i=5 2024-04-27T03:32:19.897 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.877+0000 7f3926946800 20 add_watcher() i=6 2024-04-27T03:32:19.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.878+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94760504074560] v27'277 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.878+0000 7f3926946800 20 add_watcher() i=7 2024-04-27T03:32:19.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.878+0000 7f3926946800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:19.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.878+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f262f5860 con 0x562f262e1540 2024-04-27T03:32:19.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.878+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.878+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94760504070720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262f2d30 con 0x562f262e1540 2024-04-27T03:32:19.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.879+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94760504070720] v27'275 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f391c00a4a0 con 0x562f262e1540 2024-04-27T03:32:19.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.879+0000 7f37da7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.879+0000 7f3926946800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:19.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.879+0000 7f3926946800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:19.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.879+0000 7f3926946800 5 note: GC not initialized 2024-04-27T03:32:19.898 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.880+0000 7f37d97e2640 20 reqs_thread_entry: start 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.880+0000 7f3926946800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.880+0000 7f3926946800 20 Filter name: none 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.880+0000 7f374affd640 20 reqs_thread_entry: start 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.880+0000 7f3926946800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.880+0000 7f3926946800 20 rados->read ofs=0 len=0 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.880+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f26328bd0 con 0x7f391c016440 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.881+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 ada [call out=48b,read 0~460 out=460b,getxattrs out=75b] v0'0 uv11 ondisk = 0) ==== 231+0+583 (crc 0 0 0) 0x7f391c00a4a0 con 0x7f391c016440 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.881+0000 7f3926946800 20 rados_obj.operate() r=0 bl.length=460 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.881+0000 7f3926946800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.881+0000 7f3926946800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.881+0000 7f3926946800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.881+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~473 in=473b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f2632f140 con 0x7f391c016440 2024-04-27T03:32:19.899 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.882+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~473] v27'12 uv12 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7f391c04a040 con 0x7f391c016440 2024-04-27T03:32:19.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.882+0000 7f3926946800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.882+0000 7f3926946800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.882+0000 7f3926946800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-27T03:32:19.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.882+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94760504322992 in=661b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x562f2632a850 con 0x562f262e1540 2024-04-27T03:32:19.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.882+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== watch-notify(notify (1) cookie 94760504065424 notify 115964117048 ret 0) ==== 691+0+0 (crc 0 0 0) 0x7f391c030040 con 0x562f262e1540 2024-04-27T03:32:19.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.882+0000 7f38037fe640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117048 cookie 94760504065424 notifier 4903 bl.length()=649 2024-04-27T03:32:19.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.882+0000 7f38037fe640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:19.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.882+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 notify.4 [notify cookie 94760504322992 out=8b] v0'0 uv276 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7f391c04a040 con 0x562f262e1540 2024-04-27T03:32:19.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.882+0000 7f38037fe640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:19.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.882+0000 7f38037fe640 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7f37e8003ea0 con 0x562f262e1540 2024-04-27T03:32:19.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.882+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c04a040 con 0x562f262e1540 2024-04-27T03:32:19.900 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.883+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify_complete (2) cookie 94760504322992 notify 115964117048 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7f391c02a900 con 0x562f262e1540 2024-04-27T03:32:19.901 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-27T03:32:18.064161Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:32:18.064141Z","tags":[],"group_ids":[]} 2024-04-27T03:32:19.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.883+0000 7f37da7e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:19.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.883+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94760504070720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f2632b010 con 0x562f262e1540 2024-04-27T03:32:19.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.884+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94760504070720] v27'276 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f391c04a040 con 0x562f262e1540 2024-04-27T03:32:19.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.884+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94760504049056] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f2632b010 con 0x7f39140b1040 2024-04-27T03:32:19.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.885+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94760504049056] v27'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.885+0000 7f3926946800 20 remove_watcher() i=0 2024-04-27T03:32:19.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.885+0000 7f3926946800 2 removed watcher, disabling cache 2024-04-27T03:32:19.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.885+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94760504052304] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262ebda0 con 0x562f262e1540 2024-04-27T03:32:19.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.887+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94760504052304] v27'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c04a040 con 0x562f262e1540 2024-04-27T03:32:19.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.887+0000 7f3926946800 20 remove_watcher() i=1 2024-04-27T03:32:19.906 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.887+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94760504057120] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262ebda0 con 0x562f262e5f30 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.888+0000 7f3924445640 1 -- 172.21.15.165:0/3170482494 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94760504057120] v27'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3914095030 con 0x562f262e5f30 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.888+0000 7f3926946800 20 remove_watcher() i=2 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.888+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94760504062208] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262ebda0 con 0x7f39140b1040 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.889+0000 7f3923c44640 1 -- 172.21.15.165:0/3170482494 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94760504062208] v27'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f3918037040 con 0x7f39140b1040 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.889+0000 7f3926946800 20 remove_watcher() i=3 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.889+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94760504065424] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262ef100 con 0x562f262e1540 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.890+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94760504065424] v27'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c04a040 con 0x562f262e1540 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.890+0000 7f3926946800 20 remove_watcher() i=4 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.890+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94760504069200] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262ebda0 con 0x7f391c016440 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.891+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94760504069200] v27'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c04a040 con 0x7f391c016440 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.891+0000 7f3926946800 20 remove_watcher() i=5 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.891+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94760504072848] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262efd90 con 0x7f391c016440 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.893+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94760504072848] v27'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c04a040 con 0x7f391c016440 2024-04-27T03:32:19.907 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.894+0000 7f3926946800 20 remove_watcher() i=6 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.894+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94760504074560] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x562f262f0c50 con 0x7f391c016440 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94760504074560] v27'278 uv276 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f391c04a040 con 0x7f391c016440 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 20 remove_watcher() i=7 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f391c016440 msgr2=0x7f391c0168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f391c016440 0x7f391c0168c0 crc :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562f262e1540 msgr2=0x562f262e3a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562f262e1540 0x562f262e3a00 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f39140b1040 msgr2=0x7f391406f590 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f39140b1040 0x7f391406f590 crc :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562f262e5f30 msgr2=0x562f262e8320 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562f262e5f30 0x562f262e8320 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37f40368b0 msgr2=0x7f37f4038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37f40368b0 0x7f37f4038d70 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f391800ffd0 tx=0x7f3918023040 comp rx=0 tx=0).stop 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 msgr2=0x562f262d5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 --2- 172.21.15.165:0/3170482494 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262ceb50 0x562f262d5e70 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f3914009cc0 tx=0x7f3914086ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.908 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 reap_dead start 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3924c46640 1 -- 172.21.15.165:0/3170482494 reap_dead start 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 shutdown_connections 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 >> 172.21.15.165:0/3170482494 conn(0x562f262cae60 msgr2=0x562f262c84c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 shutdown_connections 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.895+0000 7f3926946800 1 -- 172.21.15.165:0/3170482494 wait complete. 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/2864122060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37fc036950 msgr2=0x7f37fc038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 --2- 172.21.15.165:0/2864122060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37fc036950 0x7f37fc038e10 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x562f262ab100 tx=0x7f3918006000 comp rx=0 tx=0).stop 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/2864122060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262c1ab0 msgr2=0x562f262c1ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 --2- 172.21.15.165:0/2864122060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262c1ab0 0x562f262c1ed0 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f391408cab0 tx=0x7f3914067df0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/2864122060 shutdown_connections 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 --2- 172.21.15.165:0/2864122060 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f37fc036950 0x7f37fc038e10 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 --2- 172.21.15.165:0/2864122060 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f262c1ab0 0x562f262c1ed0 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/2864122060 >> 172.21.15.165:0/2864122060 conn(0x562f262b5b50 msgr2=0x562f262b5f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/2864122060 shutdown_connections 2024-04-27T03:32:19.909 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/2864122060 wait complete. 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562f261a7fe0 msgr2=0x562f262aa290 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x562f261a7fe0 0x562f262aa290 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562f262b0a50 msgr2=0x562f262b2ec0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x562f262b0a50 0x562f262b2ec0 crc :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562f262acb50 msgr2=0x562f262aef40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x562f262acb50 0x562f262aef40 crc :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f380c036950 msgr2=0x7f380c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f380c036950 0x7f380c038e10 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f3918000c00 tx=0x7f39180013b0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 msgr2=0x562f262a3050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 --2- 172.21.15.165:0/665303797 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x562f261a6d80 0x562f262a3050 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f3914060040 tx=0x7f3914002fb0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3924c46640 1 -- 172.21.15.165:0/665303797 reap_dead start 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 shutdown_connections 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 >> 172.21.15.165:0/665303797 conn(0x562f261a2340 msgr2=0x562f26291310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.910 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 shutdown_connections 2024-04-27T03:32:19.911 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.896+0000 7f3926946800 1 -- 172.21.15.165:0/665303797 wait complete. 2024-04-27T03:32:19.911 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.912 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:19.912 ERROR:root:0 2024-04-27T03:32:19.912 INFO:tasks.util.rgw:rgwadmin: client.0 : ['caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-27T03:32:19.912 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'caps', 'add', '--uid', 'ada', '--caps', 'users=read, write'] 2024-04-27T03:32:19.912 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph caps add --uid ada --caps 'users=read, write' 2024-04-27T03:32:19.956 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:32:19.956 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:32:19.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.951+0000 7fbf96d46800 1 Processor -- start 2024-04-27T03:32:19.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.951+0000 7fbf96d46800 1 -- start start 2024-04-27T03:32:19.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.952+0000 7fbf96d46800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x558799edb780 0x558799f6fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.952+0000 7fbf96d46800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x558799e25d40 con 0x558799edcac0 2024-04-27T03:32:19.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.952+0000 7fbf96d46800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x558799df84d0 con 0x558799edb780 2024-04-27T03:32:19.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.952+0000 7fbf8ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x558799edb780 0x558799f6fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.952+0000 7fbf94845640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x558799edcac0 0x558799edb070 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:42442/0 (socket says 172.21.15.165:42442) 2024-04-27T03:32:19.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.952+0000 7fbf94845640 1 -- 172.21.15.165:0/1107361376 learned_addr learned my addr 172.21.15.165:0/1107361376 (peer_addr_for_me v1:172.21.15.165:0/0) 2024-04-27T03:32:19.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.952+0000 7fbf8ffff640 1 --2- 172.21.15.165:0/1107361376 >> v2:172.21.15.89:3300/0 conn(0x558799edb780 0x558799f6fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60594/0 (socket says 172.21.15.165:60594) 2024-04-27T03:32:19.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.952+0000 7fbf8ffff640 1 -- 172.21.15.165:0/1107361376 >> v1:172.21.15.89:6789/0 conn(0x558799edcac0 legacy=0x558799edb070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.958 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.952+0000 7fbf8f7fe640 1 -- 172.21.15.165:0/1107361376 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4250981355 0 0) 0x558799e25d40 con 0x558799edcac0 2024-04-27T03:32:19.959 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.952+0000 7fbf8ffff640 1 -- 172.21.15.165:0/1107361376 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x558799e24590 con 0x558799edb780 2024-04-27T03:32:19.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.954+0000 7fbf8ffff640 1 --2- 172.21.15.165:0/1107361376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edb780 0x558799f6fe70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000a470 tx=0x7fbf8001ce90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c872b389a30aa058 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.955+0000 7fbf8f7fe640 1 -- 172.21.15.165:0/1107361376 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf80057070 con 0x558799edb780 2024-04-27T03:32:19.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.955+0000 7fbf8f7fe640 1 -- 172.21.15.165:0/1107361376 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf80052570 con 0x558799edb780 2024-04-27T03:32:19.960 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.955+0000 7fbf8f7fe640 1 -- 172.21.15.165:0/1107361376 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf80052870 con 0x558799edb780 2024-04-27T03:32:19.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.955+0000 7fbf96d46800 1 -- 172.21.15.165:0/1107361376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edb780 msgr2=0x558799f6fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.955+0000 7fbf96d46800 1 --2- 172.21.15.165:0/1107361376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edb780 0x558799f6fe70 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000a470 tx=0x7fbf8001ce90 comp rx=0 tx=0).stop 2024-04-27T03:32:19.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.955+0000 7fbf96d46800 1 -- 172.21.15.165:0/1107361376 shutdown_connections 2024-04-27T03:32:19.961 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.955+0000 7fbf96d46800 1 --2- 172.21.15.165:0/1107361376 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edb780 0x558799f6fe70 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.955+0000 7fbf96d46800 1 -- 172.21.15.165:0/1107361376 >> 172.21.15.165:0/1107361376 conn(0x558799f62550 msgr2=0x558799f64970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.955+0000 7fbf96d46800 1 -- 172.21.15.165:0/1107361376 shutdown_connections 2024-04-27T03:32:19.962 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.955+0000 7fbf96d46800 1 -- 172.21.15.165:0/1107361376 wait complete. 2024-04-27T03:32:19.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.967+0000 7fbf96d46800 1 Processor -- start 2024-04-27T03:32:19.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.967+0000 7fbf96d46800 1 -- start start 2024-04-27T03:32:19.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.967+0000 7fbf96d46800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 0x55879a05a650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.967+0000 7fbf96d46800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558799eb23b0 con 0x558799edcac0 2024-04-27T03:32:19.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.967+0000 7fbf94845640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 0x55879a05a650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.967+0000 7fbf94845640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 0x55879a05a650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60608/0 (socket says 172.21.15.165:60608) 2024-04-27T03:32:19.973 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.967+0000 7fbf94845640 1 -- 172.21.15.165:0/1643218856 learned_addr learned my addr 172.21.15.165:0/1643218856 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.968+0000 7fbf94845640 1 -- 172.21.15.165:0/1643218856 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55879a05ac20 con 0x558799edcac0 2024-04-27T03:32:19.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.968+0000 7fbf94845640 1 --2- 172.21.15.165:0/1643218856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 0x55879a05a650 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbf88001e00 tx=0x7fbf8800b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e1c9d33d8c9e45 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.968+0000 7fbe82e05640 1 -- 172.21.15.165:0/1643218856 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf88012020 con 0x558799edcac0 2024-04-27T03:32:19.976 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.968+0000 7fbe82e05640 1 -- 172.21.15.165:0/1643218856 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8800e5c0 con 0x558799edcac0 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.968+0000 7fbe82e05640 1 -- 172.21.15.165:0/1643218856 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf8800e8e0 con 0x558799edcac0 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.969+0000 7fbf96d46800 1 -- 172.21.15.165:0/1643218856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 msgr2=0x55879a05a650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.969+0000 7fbf96d46800 1 --2- 172.21.15.165:0/1643218856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 0x55879a05a650 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fbf88001e00 tx=0x7fbf8800b780 comp rx=0 tx=0).stop 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.969+0000 7fbf96d46800 1 -- 172.21.15.165:0/1643218856 shutdown_connections 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.969+0000 7fbf96d46800 1 --2- 172.21.15.165:0/1643218856 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 0x55879a05a650 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.969+0000 7fbf96d46800 1 -- 172.21.15.165:0/1643218856 >> 172.21.15.165:0/1643218856 conn(0x558799f62550 msgr2=0x55879a055ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.969+0000 7fbf96d46800 1 -- 172.21.15.165:0/1643218856 shutdown_connections 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.969+0000 7fbf96d46800 1 -- 172.21.15.165:0/1643218856 wait complete. 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.969+0000 7fbf96d46800 1 Processor -- start 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.969+0000 7fbf96d46800 1 -- start start 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.970+0000 7fbf96d46800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 0x55879a0621f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.970+0000 7fbf96d46800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558799f726c0 con 0x558799edcac0 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.970+0000 7fbf94845640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 0x55879a0621f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.977 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.970+0000 7fbf94845640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 0x55879a0621f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60610/0 (socket says 172.21.15.165:60610) 2024-04-27T03:32:19.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.970+0000 7fbf94845640 1 -- 172.21.15.165:0/3320781521 learned_addr learned my addr 172.21.15.165:0/3320781521 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.970+0000 7fbf94845640 1 -- 172.21.15.165:0/3320781521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55879a062730 con 0x558799edcac0 2024-04-27T03:32:19.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.970+0000 7fbf94845640 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 0x55879a0621f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55879a05b3e0 tx=0x7fbf88000ed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.971+0000 7fbe81602640 1 -- 172.21.15.165:0/3320781521 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf88012030 con 0x558799edcac0 2024-04-27T03:32:19.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.971+0000 7fbe81602640 1 -- 172.21.15.165:0/3320781521 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf8801a700 con 0x558799edcac0 2024-04-27T03:32:19.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.971+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55879a0629c0 con 0x558799edcac0 2024-04-27T03:32:19.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.971+0000 7fbe81602640 1 -- 172.21.15.165:0/3320781521 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf8801a9e0 con 0x558799edcac0 2024-04-27T03:32:19.978 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.971+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55879a065670 con 0x558799edcac0 2024-04-27T03:32:19.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.971+0000 7fbe81602640 1 -- 172.21.15.165:0/3320781521 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fbf88015020 con 0x558799edcac0 2024-04-27T03:32:19.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.972+0000 7fbe81602640 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe7c036950 0x7fbe7c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.972+0000 7fbe81602640 1 -- 172.21.15.165:0/3320781521 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fbf88020030 con 0x558799edcac0 2024-04-27T03:32:19.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.972+0000 7fbf8ffff640 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe7c036950 0x7fbe7c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.972+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55879a067420 0x55879a069810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.986 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.972+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a069d50 con 0x55879a067420 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.972+0000 7fbf95046640 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55879a067420 0x55879a069810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.972+0000 7fbf8ffff640 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe7c036950 0x7fbe7c038e10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000a470 tx=0x7fbf8004e000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.972+0000 7fbf95046640 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55879a067420 0x55879a069810 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.972+0000 7fbf95046640 1 -- 172.21.15.165:0/3320781521 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a067420 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.972+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0669d0 con 0x55879a067420 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.973+0000 7fbf95046640 1 -- 172.21.15.165:0/3320781521 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a067420 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.973+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55879a06bff0 0x55879a06e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.973+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a06ead0 con 0x55879a06bff0 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.973+0000 7fbf94845640 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55879a06bff0 0x55879a06e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.973+0000 7fbf94845640 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55879a06bff0 0x55879a06e3e0 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.973+0000 7fbf94845640 1 -- 172.21.15.165:0/3320781521 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7fbf8800dcd0 con 0x55879a06bff0 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.973+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a06afd0 con 0x55879a067420 2024-04-27T03:32:19.987 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.974+0000 7fbf95046640 1 -- 172.21.15.165:0/3320781521 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a067420 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.974+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55879a06ff30 0x55879a0723a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.974+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a072b00 con 0x55879a06ff30 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.974+0000 7fbf8ffff640 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55879a06ff30 0x55879a0723a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.974+0000 7fbf8ffff640 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55879a06ff30 0x55879a0723a0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.974+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3320781521 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7fbf8005a040 con 0x55879a06ff30 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.975+0000 7fbf96d46800 1 Processor -- start 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.975+0000 7fbf96d46800 1 -- start start 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.975+0000 7fbf96d46800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a080f10 0x55879a081330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.975+0000 7fbf96d46800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x558799e586b0 con 0x55879a080f10 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.975+0000 7fbf94845640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a080f10 0x55879a081330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.975+0000 7fbf94845640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a080f10 0x55879a081330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60616/0 (socket says 172.21.15.165:60616) 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.975+0000 7fbf94845640 1 -- 172.21.15.165:0/1295591605 learned_addr learned my addr 172.21.15.165:0/1295591605 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.988 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.975+0000 7fbf94845640 1 -- 172.21.15.165:0/1295591605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55879a081870 con 0x55879a080f10 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.976+0000 7fbf94845640 1 --2- 172.21.15.165:0/1295591605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a080f10 0x55879a081330 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800bd20 tx=0x7fbf88017f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.976+0000 7fbe7a7fc640 1 -- 172.21.15.165:0/1295591605 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf88040040 con 0x55879a080f10 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.976+0000 7fbe7a7fc640 1 -- 172.21.15.165:0/1295591605 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf88027d70 con 0x55879a080f10 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.976+0000 7fbf96d46800 1 -- 172.21.15.165:0/1295591605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55879a0822e0 con 0x55879a080f10 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.976+0000 7fbe7a7fc640 1 -- 172.21.15.165:0/1295591605 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf88045ba0 con 0x55879a080f10 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.976+0000 7fbf96d46800 1 -- 172.21.15.165:0/1295591605 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55879a0827e0 con 0x55879a080f10 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.976+0000 7fbe7a7fc640 1 -- 172.21.15.165:0/1295591605 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fbf8804c070 con 0x55879a080f10 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.977+0000 7fbe7a7fc640 1 --2- 172.21.15.165:0/1295591605 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe6c036950 0x7fbe6c038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.977+0000 7fbe7a7fc640 1 -- 172.21.15.165:0/1295591605 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fbf88037bc0 con 0x55879a080f10 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.977+0000 7fbf8ffff640 1 --2- 172.21.15.165:0/1295591605 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe6c036950 0x7fbe6c038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.977+0000 7fbf96d46800 1 Processor -- start 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.977+0000 7fbf96d46800 1 -- start start 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.977+0000 7fbf8ffff640 1 --2- 172.21.15.165:0/1295591605 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe6c036950 0x7fbe6c038e10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000a440 tx=0x7fbf80005a00 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.989 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.977+0000 7fbf96d46800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 0x55879a08e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.977+0000 7fbf96d46800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55879a05afa0 con 0x55879a08e2a0 2024-04-27T03:32:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.977+0000 7fbf94845640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 0x55879a08e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.977+0000 7fbf94845640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 0x55879a08e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60618/0 (socket says 172.21.15.165:60618) 2024-04-27T03:32:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.977+0000 7fbf94845640 1 -- 172.21.15.165:0/802299678 learned_addr learned my addr 172.21.15.165:0/802299678 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.978+0000 7fbf94845640 1 -- 172.21.15.165:0/802299678 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55879a08ec90 con 0x55879a08e2a0 2024-04-27T03:32:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.978+0000 7fbf94845640 1 --2- 172.21.15.165:0/802299678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 0x55879a08e6c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbf88023e90 tx=0x7fbf88037f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5543e1e3e23855f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.978+0000 7fbe6bfff640 1 -- 172.21.15.165:0/802299678 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf8805d070 con 0x55879a08e2a0 2024-04-27T03:32:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.978+0000 7fbe6bfff640 1 -- 172.21.15.165:0/802299678 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf88022320 con 0x55879a08e2a0 2024-04-27T03:32:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.978+0000 7fbe6bfff640 1 -- 172.21.15.165:0/802299678 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf88052040 con 0x55879a08e2a0 2024-04-27T03:32:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.978+0000 7fbf96d46800 1 -- 172.21.15.165:0/802299678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 msgr2=0x55879a08e6c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.978+0000 7fbf96d46800 1 --2- 172.21.15.165:0/802299678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 0x55879a08e6c0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7fbf88023e90 tx=0x7fbf88037f20 comp rx=0 tx=0).stop 2024-04-27T03:32:19.990 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.978+0000 7fbf96d46800 1 -- 172.21.15.165:0/802299678 shutdown_connections 2024-04-27T03:32:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.978+0000 7fbf96d46800 1 --2- 172.21.15.165:0/802299678 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 0x55879a08e6c0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.978+0000 7fbf96d46800 1 -- 172.21.15.165:0/802299678 >> 172.21.15.165:0/802299678 conn(0x55879a08a570 msgr2=0x55879a0885d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.978+0000 7fbf96d46800 1 -- 172.21.15.165:0/802299678 shutdown_connections 2024-04-27T03:32:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.978+0000 7fbf96d46800 1 -- 172.21.15.165:0/802299678 wait complete. 2024-04-27T03:32:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.979+0000 7fbf96d46800 1 Processor -- start 2024-04-27T03:32:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.979+0000 7fbf96d46800 1 -- start start 2024-04-27T03:32:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.979+0000 7fbf96d46800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 0x55879a095600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.979+0000 7fbf96d46800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x55879a08f2f0 con 0x55879a08e2a0 2024-04-27T03:32:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.979+0000 7fbf94845640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 0x55879a095600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.979+0000 7fbf94845640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 0x55879a095600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:60626/0 (socket says 172.21.15.165:60626) 2024-04-27T03:32:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.979+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 learned_addr learned my addr 172.21.15.165:0/3060906046 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:32:19.991 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.979+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55879a095b40 con 0x55879a08e2a0 2024-04-27T03:32:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.980+0000 7fbf94845640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 0x55879a095600 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbf8805b040 tx=0x7fbf8804ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.980+0000 7fbe6a7fc640 1 -- 172.21.15.165:0/3060906046 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf88068020 con 0x55879a08e2a0 2024-04-27T03:32:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.980+0000 7fbe6a7fc640 1 -- 172.21.15.165:0/3060906046 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fbf88044300 con 0x55879a08e2a0 2024-04-27T03:32:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.980+0000 7fbe6a7fc640 1 -- 172.21.15.165:0/3060906046 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fbf8805d030 con 0x55879a08e2a0 2024-04-27T03:32:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.980+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x55879a095b40 con 0x55879a08e2a0 2024-04-27T03:32:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.980+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55879a095b40 con 0x55879a08e2a0 2024-04-27T03:32:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.980+0000 7fbe6a7fc640 1 -- 172.21.15.165:0/3060906046 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fbf8806b020 con 0x55879a08e2a0 2024-04-27T03:32:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.981+0000 7fbe6a7fc640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe600368b0 0x7fbe60038d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.981+0000 7fbe6a7fc640 1 -- 172.21.15.165:0/3060906046 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fbf8805e070 con 0x55879a08e2a0 2024-04-27T03:32:19.992 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.981+0000 7fbf8ffff640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe600368b0 0x7fbe60038d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:20.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.981+0000 7fbf96d46800 20 rados->read ofs=0 len=0 2024-04-27T03:32:20.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.981+0000 7fbf8ffff640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe600368b0 0x7fbe60038d70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbf800447c0 tx=0x7fbf8004f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:20.006 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.981+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55879a0a0da0 0x55879a0a3260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:20.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.981+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a39b0 con 0x55879a0a0da0 2024-04-27T03:32:20.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.981+0000 7fbf95046640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55879a0a0da0 0x55879a0a3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:20.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.981+0000 7fbf95046640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55879a0a0da0 0x55879a0a3260 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:20.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:20.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 20 realm 2024-04-27T03:32:20.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 20 rados->read ofs=0 len=0 2024-04-27T03:32:20.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a3dc0 con 0x55879a0a0da0 2024-04-27T03:32:20.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:20.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:32:20.007 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 20 rados->read ofs=0 len=0 2024-04-27T03:32:20.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4690 con 0x55879a0a0da0 2024-04-27T03:32:20.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:20.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 20 rados->read ofs=0 len=0 2024-04-27T03:32:20.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a0da0 2024-04-27T03:32:20.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:20.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 20 rados->read ofs=0 len=0 2024-04-27T03:32:20.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55879a0a5790 0x55879a0a7b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:20.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.982+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a8140 con 0x55879a0a5790 2024-04-27T03:32:20.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.983+0000 7fbf94845640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55879a0a5790 0x55879a0a7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:20.008 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.983+0000 7fbf94845640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55879a0a5790 0x55879a0a7b80 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:20.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.983+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.983+0000 7fbf96d46800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:32:20.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.983+0000 7fbf96d46800 20 searching for the correct realm 2024-04-27T03:32:20.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.983+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a9a70 con 0x55879a0a5790 2024-04-27T03:32:20.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.984+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.984+0000 7fbf94845640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fbf8806d020 0x7fbf8800d5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:20.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.984+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a9a70 con 0x7fbf8806d020 2024-04-27T03:32:20.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.984+0000 7fbf8ffff640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fbf8806d020 0x7fbf8800d5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:20.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.984+0000 7fbf8ffff640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fbf8806d020 0x7fbf8800d5f0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:20.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.984+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.984+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a9a70 con 0x7fbf8806d020 2024-04-27T03:32:20.009 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.984+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.985+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a9a70 con 0x55879a0a0da0 2024-04-27T03:32:20.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.985+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.985+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a0da0 2024-04-27T03:32:20.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.985+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.985+0000 7fbf95046640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fbf90016440 0x7fbf900168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:32:20.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.985+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf90016440 2024-04-27T03:32:20.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.985+0000 7fbf95046640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fbf90016440 0x7fbf900168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:32:20.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.986+0000 7fbf95046640 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fbf90016440 0x7fbf900168c0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:32:20.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.986+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.986+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf90016440 2024-04-27T03:32:20.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.986+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.010 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.986+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf8806d020 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.986+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.986+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a5790 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.987+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.987+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a0da0 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.987+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.987+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf8806d020 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.987+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.987+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a0da0 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.987+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.987+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf8806d020 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.988+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.988+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a5790 2024-04-27T03:32:20.011 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.988+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.988+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a5790 2024-04-27T03:32:20.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.988+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.988+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf90016440 2024-04-27T03:32:20.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.989+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.989+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a5790 2024-04-27T03:32:20.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.989+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.989+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a5790 2024-04-27T03:32:20.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.989+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.989+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a0da0 2024-04-27T03:32:20.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.989+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.989+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a0da0 2024-04-27T03:32:20.012 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.990+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.990+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf90016440 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.990+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.990+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a5790 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.990+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.990+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf8806d020 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.990+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.990+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a0da0 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.991+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.991+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf90016440 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.991+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.991+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a5790 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.991+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.991+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf90016440 2024-04-27T03:32:20.013 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.991+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.992+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf8806d020 2024-04-27T03:32:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.992+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.992+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a5790 2024-04-27T03:32:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.992+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.992+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf90016440 2024-04-27T03:32:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.992+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.992+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x55879a0a0da0 2024-04-27T03:32:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a4a30 con 0x7fbf8806d020 2024-04-27T03:32:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:32:20.014 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 20 rados->read ofs=0 len=0 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a8a00 con 0x55879a0a0da0 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 20 rados->read ofs=0 len=0 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a8df0 con 0x55879a0a0da0 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:32:20.015 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 20 rados->read ofs=0 len=0 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.993+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a8df0 con 0x7fbf8806d020 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 20 zone default found 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 4 Realm: () 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 20 zonegroup default 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 20 rados->read ofs=0 len=0 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0a9ef0 con 0x7fbf90016440 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 165+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 20 rados->read ofs=0 len=0 2024-04-27T03:32:20.016 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0ab8d0 con 0x55879a0a5790 2024-04-27T03:32:20.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:20.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 20 rados->read ofs=0 len=0 2024-04-27T03:32:20.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0abd30 con 0x55879a0a5790 2024-04-27T03:32:20.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:32:20.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 20 started sync module instance, tier type = 2024-04-27T03:32:20.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:32:20.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.994+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0aad70 con 0x7fbf8806d020 2024-04-27T03:32:20.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.995+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.996+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94040893337952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0aa9d0 con 0x7fbf8806d020 2024-04-27T03:32:20.017 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.996+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0acc60 con 0x55879a0a0da0 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.997+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94040893337952] v27'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.997+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.997+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94040893343664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0ad080 con 0x55879a0a0da0 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.997+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0ada40 con 0x55879a0a5790 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.998+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94040893343664] v27'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.998+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.998+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94040893344720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0acff0 con 0x55879a0a5790 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.998+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0ae8b0 con 0x7fbf8806d020 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.999+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94040893344720] v27'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.999+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.999+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94040893350912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0aeca0 con 0x7fbf8806d020 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:19.999+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0af660 con 0x55879a0a0da0 2024-04-27T03:32:20.018 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.000+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94040893350912] v27'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.000+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.000+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94040893354416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0af5d0 con 0x55879a0a0da0 2024-04-27T03:32:20.019 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.000+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0b0520 con 0x7fbf90016440 2024-04-27T03:32:20.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.001+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.001+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94040893358192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0b0940 con 0x7fbf90016440 2024-04-27T03:32:20.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.001+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0b1420 con 0x7fbf90016440 2024-04-27T03:32:20.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.001+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94040893354416] v27'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.023 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.002+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94040893358192] v27'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.002+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.002+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94040893361968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0b0490 con 0x7fbf90016440 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.002+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0b1cf0 con 0x7fbf90016440 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.003+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94040893361968] v27'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.003+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'279 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.003+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94040893356624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0b0490 con 0x7fbf90016440 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.003+0000 7fbf96d46800 20 add_watcher() i=0 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.003+0000 7fbf96d46800 20 add_watcher() i=1 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.003+0000 7fbf96d46800 20 add_watcher() i=2 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.003+0000 7fbf96d46800 20 add_watcher() i=3 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.003+0000 7fbf96d46800 20 add_watcher() i=4 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.003+0000 7fbf96d46800 20 add_watcher() i=5 2024-04-27T03:32:20.024 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.003+0000 7fbf96d46800 20 add_watcher() i=6 2024-04-27T03:32:20.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.004+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94040893356624] v27'280 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.004+0000 7fbf96d46800 20 add_watcher() i=7 2024-04-27T03:32:20.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.004+0000 7fbf96d46800 2 all 8 watchers are set, enabling cache 2024-04-27T03:32:20.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.004+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0b4d20 con 0x55879a0a0da0 2024-04-27T03:32:20.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.005+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.005+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94040893364288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0b2220 con 0x55879a0a0da0 2024-04-27T03:32:20.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.006+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94040893364288] v27'277 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x55879a0a0da0 2024-04-27T03:32:20.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.006+0000 7fbe4afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:20.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.006+0000 7fbf96d46800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:32:20.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.006+0000 7fbf96d46800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:32:20.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.006+0000 7fbf96d46800 5 note: GC not initialized 2024-04-27T03:32:20.025 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.006+0000 7fbe49fe3640 20 reqs_thread_entry: start 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.006+0000 7fbf96d46800 20 init_complete bucket index max shards: 11 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.006+0000 7fbf96d46800 20 Filter name: none 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.007+0000 7fbdbb7fe640 20 reqs_thread_entry: start 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.007+0000 7fbf96d46800 10 cache get: name=default.rgw.meta+users.uid+ada : miss 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.007+0000 7fbf96d46800 20 rados->read ofs=0 len=0 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.007+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:62 8.7 8:e1d59a8c:users.uid::ada:head [call version.read in=11b,read 0~0,getxattrs] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0e8040 con 0x7fbf90016440 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.010+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(62 ada [call out=48b,read 0~473 out=473b,getxattrs out=75b] v0'0 uv12 ondisk = 0) ==== 231+0+596 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.010+0000 7fbf96d46800 20 rados_obj.operate() r=0 bl.length=473 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.010+0000 7fbf96d46800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x13 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.010+0000 7fbf96d46800 10 adding default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.010+0000 7fbf96d46800 10 updating xattr: name=ceph.objclass.version bl.length()=42 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.010+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 8.7 8:e1d59a8c:users.uid::ada:head [delete,create,call version.check_conds in=122b,call version.set in=58b,writefull 0~473 in=473b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0ee960 con 0x7fbf90016440 2024-04-27T03:32:20.026 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.012+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(63 ada [delete,create,call,call,writefull 0~473] v27'13 uv13 ondisk = 0) ==== 315+0+0 (crc 0 0 0) 0x7fbf9000a4a0 con 0x7fbf90016440 2024-04-27T03:32:20.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.012+0000 7fbf96d46800 10 cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:20.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.012+0000 7fbf96d46800 10 moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:20.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.012+0000 7fbf96d46800 10 distributing notification oid=default.rgw.control:notify.4 cni=[op: 0, obj: default.rgw.meta:users.uid:ada, ofs0, ns] 2024-04-27T03:32:20.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.012+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:64 7.d 7:b4812045:::notify.4:head [notify cookie 94040893613008 in=661b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x55879a0ea050 con 0x55879a0a0da0 2024-04-27T03:32:20.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.012+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== watch-notify(notify (1) cookie 94040893354416 notify 115964117049 ret 0) ==== 691+0+0 (crc 0 0 0) 0x7fbf90030040 con 0x55879a0a0da0 2024-04-27T03:32:20.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.012+0000 7fbe6bfff640 10 rgw watcher librados: RGWWatcher::handle_notify() notify_id 115964117049 cookie 94040893354416 notifier 4910 bl.length()=649 2024-04-27T03:32:20.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.012+0000 7fbe6bfff640 10 rgw watcher librados: cache put: name=default.rgw.meta+users.uid+ada info.flags=0x17 2024-04-27T03:32:20.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.012+0000 7fbe6bfff640 10 rgw watcher librados: moving default.rgw.meta+users.uid+ada to cache LRU end 2024-04-27T03:32:20.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.012+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(64 notify.4 [notify cookie 94040893613008 out=8b] v0'0 uv279 ondisk = 0) ==== 152+0+8 (crc 0 0 0) 0x7fbf90049020 con 0x55879a0a0da0 2024-04-27T03:32:20.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.012+0000 7fbe6bfff640 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 7.d 7:b4812045:::notify.4:head [notify-ack in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x7fbe58003ea0 con 0x55879a0a0da0 2024-04-27T03:32:20.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.013+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(65 notify.4 [notify-ack] v0'0 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf90049020 con 0x55879a0a0da0 2024-04-27T03:32:20.027 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.013+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== watch-notify(notify_complete (2) cookie 94040893613008 notify 115964117049 ret 0) ==== 42+0+48 (crc 0 0 0) 0x7fbf9002a900 con 0x55879a0a0da0 2024-04-27T03:32:20.028 INFO:teuthology.orchestra.run.smithi165.stdout:{"user_id":"ada","display_name":"Ms. Admin User","email":"","suspended":0,"max_buckets":0,"subusers":[],"keys":[{"user":"ada","access_key":"MH1WC2XQ1S8UISFDZC8W","secret_key":"dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG","active":true,"create_date":"2024-04-27T03:32:18.064161Z"}],"swift_keys":[],"caps":[{"type":"buckets","perm":"*"},{"type":"info","perm":"read"},{"type":"ratelimit","perm":"*"},{"type":"usage","perm":"*"},{"type":"users","perm":"*"},{"type":"zone","perm":"*"}],"op_mask":"read, write, delete","default_placement":"","default_storage_class":"","placement_tags":[],"bucket_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"user_quota":{"enabled":false,"check_on_raw":false,"max_size":-1,"max_size_kb":0,"max_objects":-1},"temp_url_keys":[],"type":"rgw","mfa_ids":[],"account_id":"","path":"/","create_date":"2024-04-27T03:32:18.064141Z","tags":[],"group_ids":[]} 2024-04-27T03:32:20.033 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.013+0000 7fbe4afe5640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:32:20.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.014+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:66 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94040893364288] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0ea810 con 0x55879a0a0da0 2024-04-27T03:32:20.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.014+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(66 data_loggenerations_metadata [watch unwatch cookie 94040893364288] v27'278 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7fbf90049020 con 0x55879a0a0da0 2024-04-27T03:32:20.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.015+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:67 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94040893337952] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0ea810 con 0x7fbf8806d020 2024-04-27T03:32:20.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.016+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(67 notify.0 [watch unwatch cookie 94040893337952] v27'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.016+0000 7fbf96d46800 20 remove_watcher() i=0 2024-04-27T03:32:20.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.016+0000 7fbf96d46800 2 removed watcher, disabling cache 2024-04-27T03:32:20.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.016+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94040893343664] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0ea810 con 0x55879a0a0da0 2024-04-27T03:32:20.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.017+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(68 notify.1 [watch unwatch cookie 94040893343664] v27'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf90049020 con 0x55879a0a0da0 2024-04-27T03:32:20.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.017+0000 7fbf96d46800 20 remove_watcher() i=1 2024-04-27T03:32:20.034 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.017+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:69 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94040893344720] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0ea810 con 0x55879a0a5790 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.018+0000 7fbf94845640 1 -- 172.21.15.165:0/3060906046 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(69 notify.2 [watch unwatch cookie 94040893344720] v27'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf88055040 con 0x55879a0a5790 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.018+0000 7fbf96d46800 20 remove_watcher() i=2 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.018+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94040893350912] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0ab160 con 0x7fbf8806d020 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.019+0000 7fbf8ffff640 1 -- 172.21.15.165:0/3060906046 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 notify.3 [watch unwatch cookie 94040893350912] v27'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf80070030 con 0x7fbf8806d020 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.019+0000 7fbf96d46800 20 remove_watcher() i=3 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.019+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:71 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94040893354416] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0ab160 con 0x55879a0a0da0 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.020+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(71 notify.4 [watch unwatch cookie 94040893354416] v27'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf90049020 con 0x55879a0a0da0 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.020+0000 7fbf96d46800 20 remove_watcher() i=4 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.020+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:72 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94040893358192] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0af1b0 con 0x7fbf90016440 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.021+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(72 notify.5 [watch unwatch cookie 94040893358192] v27'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf90049020 con 0x7fbf90016440 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.021+0000 7fbf96d46800 20 remove_watcher() i=5 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.021+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94040893361968] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0b0070 con 0x7fbf90016440 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.022+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(73 notify.6 [watch unwatch cookie 94040893361968] v27'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf90049020 con 0x7fbf90016440 2024-04-27T03:32:20.035 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.022+0000 7fbf96d46800 20 remove_watcher() i=6 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.022+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:74 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94040893356624] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x55879a0af1b0 con 0x7fbf90016440 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.023+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(74 notify.7 [watch unwatch cookie 94040893356624] v27'281 uv279 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7fbf90049020 con 0x7fbf90016440 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.023+0000 7fbf96d46800 20 remove_watcher() i=7 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.023+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fbf90016440 msgr2=0x7fbf900168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fbf90016440 0x7fbf900168c0 crc :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55879a0a0da0 msgr2=0x55879a0a3260 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55879a0a0da0 0x55879a0a3260 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fbf8806d020 msgr2=0x7fbf8800d5f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7fbf8806d020 0x7fbf8800d5f0 crc :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55879a0a5790 msgr2=0x55879a0a7b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55879a0a5790 0x55879a0a7b80 crc :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe600368b0 msgr2=0x7fbe60038d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe600368b0 0x7fbe60038d70 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fbf800447c0 tx=0x7fbf8004f000 comp rx=0 tx=0).stop 2024-04-27T03:32:20.036 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 msgr2=0x55879a095600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 0x55879a095600 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7fbf8805b040 tx=0x7fbf8804ef40 comp rx=0 tx=0).stop 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf95046640 1 -- 172.21.15.165:0/3060906046 reap_dead start 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 shutdown_connections 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3060906046 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a08e2a0 0x55879a095600 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 >> 172.21.15.165:0/3060906046 conn(0x55879a08a570 msgr2=0x55879a087bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 shutdown_connections 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/3060906046 wait complete. 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/1295591605 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe6c036950 msgr2=0x7fbe6c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 --2- 172.21.15.165:0/1295591605 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe6c036950 0x7fbe6c038e10 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000a440 tx=0x7fbf80005a00 comp rx=0 tx=0).stop 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/1295591605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a080f10 msgr2=0x55879a081330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 --2- 172.21.15.165:0/1295591605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a080f10 0x55879a081330 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7fbf8800bd20 tx=0x7fbf88017f70 comp rx=0 tx=0).stop 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/1295591605 shutdown_connections 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 --2- 172.21.15.165:0/1295591605 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe6c036950 0x7fbe6c038e10 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:20.037 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 --2- 172.21.15.165:0/1295591605 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x55879a080f10 0x55879a081330 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/1295591605 >> 172.21.15.165:0/1295591605 conn(0x55879a074fb0 msgr2=0x55879a0753b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/1295591605 shutdown_connections 2024-04-27T03:32:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/1295591605 wait complete. 2024-04-27T03:32:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55879a067420 msgr2=0x55879a069810 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.024+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x55879a067420 0x55879a069810 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55879a06ff30 msgr2=0x55879a0723a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x55879a06ff30 0x55879a0723a0 crc :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55879a06bff0 msgr2=0x55879a06e3e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x55879a06bff0 0x55879a06e3e0 crc :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:32:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe7c036950 msgr2=0x7fbe7c038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fbe7c036950 0x7fbe7c038e10 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fbf8000a470 tx=0x7fbf8004e000 comp rx=0 tx=0).stop 2024-04-27T03:32:20.038 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 msgr2=0x55879a0621f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:32:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf96d46800 1 --2- 172.21.15.165:0/3320781521 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x558799edcac0 0x55879a0621f0 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x55879a05b3e0 tx=0x7fbf88000ed0 comp rx=0 tx=0).stop 2024-04-27T03:32:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf95046640 1 -- 172.21.15.165:0/3320781521 reap_dead start 2024-04-27T03:32:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 shutdown_connections 2024-04-27T03:32:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 >> 172.21.15.165:0/3320781521 conn(0x558799f62550 msgr2=0x55879a0546d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:32:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 shutdown_connections 2024-04-27T03:32:20.039 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:32:20.025+0000 7fbf96d46800 1 -- 172.21.15.165:0/3320781521 wait complete. 2024-04-27T03:32:20.040 DEBUG:tasks.util.rgw: json result: {'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:20.040 ERROR:root:{'user_id': 'ada', 'display_name': 'Ms. Admin User', 'email': '', 'suspended': 0, 'max_buckets': 0, 'subusers': [], 'keys': [{'user': 'ada', 'access_key': 'MH1WC2XQ1S8UISFDZC8W', 'secret_key': 'dQyrTPA0s248YeN5bBv4ukvKU0kh54LWWywkrpoG', 'active': True, 'create_date': '2024-04-27T03:32:18.064161Z'}], 'swift_keys': [], 'caps': [{'type': 'buckets', 'perm': '*'}, {'type': 'info', 'perm': 'read'}, {'type': 'ratelimit', 'perm': '*'}, {'type': 'usage', 'perm': '*'}, {'type': 'users', 'perm': '*'}, {'type': 'zone', 'perm': '*'}], 'op_mask': 'read, write, delete', 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': [], 'account_id': '', 'path': '/', 'create_date': '2024-04-27T03:32:18.064141Z', 'tags': [], 'group_ids': []} 2024-04-27T03:32:20.040 ERROR:root:0 2024-04-27T03:32:20.040 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-04-27T03:32:20.047 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:20.048 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:20.051 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:20.052 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-04-27T03:32:20.057 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:20.058 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:20.061 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:20.062 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'create'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh'} 2024-04-27T03:32:20.069 INFO:tasks.radosgw_admin_rest: json result: [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True}] 2024-04-27T03:32:20.070 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:20.073 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}, {'user': 'foo', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:20.073 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'uid': 'foo', 'access-key': 'p5YnriCv1nAtykxBrupQ'} 2024-04-27T03:32:20.081 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:32:20.082 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:20.086 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:20.086 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo1', 'secret-key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'key-type': 'swift'} 2024-04-27T03:32:20.093 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}] 2024-04-27T03:32:20.094 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:20.098 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:20.098 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'create'] {'subuser': 'foo:foo2', 'secret-key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'key-type': 'swift'} 2024-04-27T03:32:20.105 INFO:tasks.radosgw_admin_rest: json result: [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}] 2024-04-27T03:32:20.106 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:20.109 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo1', 'secret_key': 'gpS2G9RREMrnbqlp29PP2D36kgPR1tm72n5fPYfL', 'active': True}, {'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:20.110 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['key', 'rm'] {'subuser': 'foo:foo1', 'key-type': 'swift'} 2024-04-27T03:32:20.116 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:32:20.117 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:20.120 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo1', 'permissions': ''}, {'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:20.121 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo1'} 2024-04-27T03:32:20.126 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:32:20.127 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:20.130 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [{'id': 'foo:foo2', 'permissions': ''}], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [{'user': 'foo:foo2', 'secret_key': 'ri2VJQcKSYATOY6uaDUX7pxgkW+W1YmC6OCxPHwy', 'active': True}], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:20.130 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['subuser', 'rm'] {'subuser': 'foo:foo2', 'key-type': 'swift', '{purge-keys': True} 2024-04-27T03:32:20.137 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:32:20.137 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:32:20.141 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:20.141 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-04-27T03:32:20.145 INFO:tasks.radosgw_admin_rest: json result: [] 2024-04-27T03:32:20.146 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-04-27T03:32:20.150 INFO:tasks.radosgw_admin_rest: json result: [] 2024-04-27T03:32:20.162 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo'} 2024-04-27T03:32:20.167 INFO:tasks.radosgw_admin_rest: json result: ['myfoo'] 2024-04-27T03:32:20.167 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-04-27T03:32:20.172 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'fd642a41-253a-48d4-9473-6eca68174247', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11', 'marker': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T03:32:20.153004Z', 'creation_time': '2024-04-27T03:32:20.149884Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T03:32:20.172 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'uid': 'foo', 'stats': True} 2024-04-27T03:32:20.177 INFO:tasks.radosgw_admin_rest: json result: [{'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'fd642a41-253a-48d4-9473-6eca68174247', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11', 'marker': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#1,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T03:32:20.153004Z', 'creation_time': '2024-04-27T03:32:20.149884Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}}] 2024-04-27T03:32:20.181 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-04-27T03:32:20.186 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'fd642a41-253a-48d4-9473-6eca68174247', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11', 'marker': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#2,6#1,7#1,8#1,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T03:32:20.153004Z', 'creation_time': '2024-04-27T03:32:20.149884Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 3, 'size_actual': 4096, 'size_utilized': 3, 'size_kb': 1, 'size_kb_actual': 4, 'size_kb_utilized': 1, 'num_objects': 1}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T03:32:20.186 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'doesnotexist'} 2024-04-27T03:32:20.190 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchBucket', 'Message': '', 'RequestId': 'tx00000ff9b89747a50317f-00662c71c4-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:32:20.194 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'unlink'] {'uid': 'foo', 'bucket': 'myfoo'} 2024-04-27T03:32:20.201 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:32:20.202 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'fud', 'display-name': 'Fud', 'access-key': 'p5YnriCv1nAtykxBrupQ', 'secret-key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'max-buckets': '1'} 2024-04-27T03:32:20.210 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'fud', 'display_name': 'Fud', 'email': '', 'suspended': 0, 'max_buckets': 1, 'subusers': [], 'keys': [{'user': 'fud', 'access_key': 'p5YnriCv1nAtykxBrupQ', 'secret_key': 'Q8Tk6Q/27hfbFSYdSkPtUqhqx1GgzvpXa4WARozh', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:32:20.218 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'fud', 'bucket': 'myfoo', 'bucket-id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11'} 2024-04-27T03:32:20.227 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:32:20.230 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'link'] {'uid': 'foo', 'bucket': 'myfoo', 'bucket-id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11'} 2024-04-27T03:32:20.240 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:32:20.240 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'fud'} 2024-04-27T03:32:20.249 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:32:20.254 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['object', 'rm'] {'bucket': 'myfoo', 'object': 'four'} 2024-04-27T03:32:20.261 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:32:20.262 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'info'] {'bucket': 'myfoo', 'stats': True} 2024-04-27T03:32:20.266 INFO:tasks.radosgw_admin_rest: json result: {'bucket': 'myfoo', 'tenant': '', 'versioning': 'off', 'zonegroup': 'fd642a41-253a-48d4-9473-6eca68174247', 'placement_rule': 'default-placement', 'explicit_placement': {'data_pool': '', 'data_extra_pool': '', 'index_pool': ''}, 'id': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11', 'marker': '708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11', 'index_type': 'Normal', 'index_generation': 0, 'num_shards': 11, 'object_lock_enabled': False, 'mfa_enabled': False, 'owner': 'foo', 'ver': '0#1,1#1,2#1,3#1,4#1,5#3,6#3,7#1,8#3,9#1,10#1', 'master_ver': '0#0,1#0,2#0,3#0,4#0,5#0,6#0,7#0,8#0,9#0,10#0', 'mtime': '2024-04-27T03:32:20.231014Z', 'creation_time': '2024-04-27T03:32:20.149884Z', 'max_marker': '0#,1#,2#,3#,4#,5#,6#,7#,8#,9#,10#', 'usage': {'rgw.main': {'size': 0, 'size_actual': 0, 'size_utilized': 0, 'size_kb': 0, 'size_kb_actual': 0, 'size_kb_utilized': 0, 'num_objects': 0}}, 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}} 2024-04-27T03:33:20.314 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'categories': 'delete_obj'} 2024-04-27T03:33:20.335 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'anonymous', 'categories': []}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'bar', 'categories': []}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'bud', 'categories': []}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'fud', 'categories': []}, {'bucket': 'myfoo7', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'fud', 'categories': []}]}], 'summary': [{'user': 'anonymous', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'bar', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'bud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}, {'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}}, {'user': 'fud', 'categories': [], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 0, 'successful_ops': 0}}]} 2024-04-27T03:33:20.335 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] None 2024-04-27T03:33:20.350 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'anonymous', 'buckets': [{'bucket': '', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'bar', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'bud', 'buckets': [{'bucket': 'myfoo6', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}]}, {'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}, {'user': 'fud', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'fud', 'categories': [{'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 0, 'ops': 1, 'successful_ops': 0}]}, {'bucket': 'myfoo7', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'anonymous', 'categories': [{'category': 'list_buckets', 'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 204, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}, {'user': 'bar', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 8, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 8, 'ops': 2, 'successful_ops': 2}}, {'user': 'bud', 'categories': [{'category': 'get_obj', 'bytes_sent': 16, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 16, 'bytes_received': 0, 'ops': 4, 'successful_ops': 4}}, {'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10}}, {'user': 'fud', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'get_obj', 'bytes_sent': 8, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 234, 'bytes_received': 8, 'ops': 2, 'successful_ops': 1}, {'category': 'stat_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 242, 'bytes_received': 8, 'ops': 5, 'successful_ops': 4}}]} 2024-04-27T03:33:20.351 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-04-27T03:33:20.357 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}, {'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}, {'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}, {'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 10, 'successful_ops': 10}}]} 2024-04-27T03:33:20.358 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'create_bucket'} 2024-04-27T03:33:20.363 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'create_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}}]} 2024-04-27T03:33:20.363 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'put_obj'} 2024-04-27T03:33:20.367 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 10, 'ops': 3, 'successful_ops': 3}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 14, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'put_obj', 'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}], 'total': {'bytes_sent': 0, 'bytes_received': 24, 'ops': 4, 'successful_ops': 4}}]} 2024-04-27T03:33:20.368 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_obj'} 2024-04-27T03:33:20.372 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 2, 'successful_ops': 2}]}, {'bucket': 'useless-bucket', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_obj', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 3, 'successful_ops': 3}}]} 2024-04-27T03:33:20.373 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo', 'categories': 'delete_bucket'} 2024-04-27T03:33:20.377 INFO:tasks.radosgw_admin_rest: json result: {'entries': [{'user': 'foo', 'buckets': [{'bucket': 'myfoo', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': []}, {'bucket': 'useless-bucket', 'time': '2024-04-27T03:00:00.000000Z', 'epoch': 1714186800, 'owner': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}]}]}], 'summary': [{'user': 'foo', 'categories': [{'category': 'delete_bucket', 'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}], 'total': {'bytes_sent': 0, 'bytes_received': 0, 'ops': 1, 'successful_ops': 1}}]} 2024-04-27T03:33:20.377 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'trim'] {'uid': 'foo'} 2024-04-27T03:33:20.384 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:33:20.385 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['usage', 'show'] {'uid': 'foo'} 2024-04-27T03:33:20.389 INFO:tasks.radosgw_admin_rest: json result: {'entries': [], 'summary': []} 2024-04-27T03:33:20.390 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': True} 2024-04-27T03:33:20.399 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 1, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:33:20.403 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'modify'] {'uid': 'foo', 'suspended': 'false'} 2024-04-27T03:33:20.413 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'foo', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 4, 'subusers': [], 'keys': [{'user': 'foo', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:33:20.667 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-04-27T03:33:20.672 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx00000328ef10e42acd0d9-00662c7200-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:33:20.778 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-04-27T03:33:20.783 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [], 'grant_map': [{'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'name': 'Foo', 'permission': {'flags': 15}}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-04-27T03:33:20.788 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['policy', 'show'] {'bucket': 'myfoo', 'object': b'bb3aec0fdcdbc2974890f805c585d432'} 2024-04-27T03:33:20.792 INFO:tasks.radosgw_admin_rest: json result: {'acl': {'acl_user_map': [{'user': 'foo', 'acl': 15}], 'acl_group_map': [{'group': 1, 'acl': 1}], 'grant_map': [{'id': '', 'grant': {'type': {'type': 2}, 'group': 1, 'permission': {'flags': 1}}}, {'id': 'foo', 'grant': {'type': {'type': 0}, 'id': 'foo', 'name': 'Foo', 'permission': {'flags': 15}}}]}, 'owner': {'id': 'foo', 'display_name': 'Foo'}} 2024-04-27T03:33:20.915 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['bucket', 'rm'] {'bucket': 'myfoo', 'purge-objects': True} 2024-04-27T03:33:20.950 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:33:20.950 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'add'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-04-27T03:33:20.956 INFO:tasks.radosgw_admin_rest: json result: [{'type': 'usage', 'perm': 'read'}] 2024-04-27T03:33:20.956 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['caps', 'rm'] {'uid': 'foo', 'user-caps': 'usage=read'} 2024-04-27T03:33:20.962 INFO:tasks.radosgw_admin_rest: json result: [] 2024-04-27T03:33:20.977 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo'} 2024-04-27T03:33:20.981 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'BucketAlreadyExists', 'Message': '', 'RequestId': 'tx00000bc0e8338495aa5eb-00662c7200-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:33:21.000 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'rm'] {'uid': 'foo', 'purge-data': True} 2024-04-27T03:33:21.034 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:33:21.034 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'info'] {'uid': 'foo'} 2024-04-27T03:33:21.038 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchUser', 'Message': '', 'RequestId': 'tx0000034b657af8b94e098-00662c7201-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:33:21.039 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['info', ''] None 2024-04-27T03:33:21.042 INFO:tasks.radosgw_admin_rest: json result: {'info': {'storage_backends': [{'name': 'rados', 'cluster_id': '7a1794da-6463-461c-a75b-269448ecfc86'}]}} 2024-04-27T03:33:21.043 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['user', 'create'] {'uid': 'ratelimit_user', 'display-name': 'Foo', 'email': 'foo@foo.com', 'access-key': '9te6NH5mcdcq0Tc5i8i1', 'secret-key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'max-buckets': '1000'} 2024-04-27T03:33:21.053 INFO:tasks.radosgw_admin_rest: json result: {'tenant': '', 'user_id': 'ratelimit_user', 'display_name': 'Foo', 'email': 'foo@foo.com', 'suspended': 0, 'max_buckets': 1000, 'subusers': [], 'keys': [{'user': 'ratelimit_user', 'access_key': '9te6NH5mcdcq0Tc5i8i1', 'secret_key': 'Ny4IOauQoL18Gp2zM7lC1vLmoawgqcYP/YGcWfXu', 'active': True}], 'swift_keys': [], 'caps': [], 'op_mask': 'read, write, delete', 'system': False, 'admin': False, 'default_placement': '', 'default_storage_class': '', 'placement_tags': [], 'bucket_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'user_quota': {'enabled': False, 'check_on_raw': False, 'max_size': -1, 'max_size_kb': 0, 'max_objects': -1}, 'temp_url_keys': [], 'type': 'rgw', 'mfa_ids': []} 2024-04-27T03:33:21.053 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-04-27T03:33:21.058 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-04-27T03:33:21.058 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring'} 2024-04-27T03:33:21.063 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000a80444aac061d250-00662c7201-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:33:21.064 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user'} 2024-04-27T03:33:21.069 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx000004e4fe4978512f137-00662c7201-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:33:21.082 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-04-27T03:33:21.087 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-04-27T03:33:21.087 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring'} 2024-04-27T03:33:21.093 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx0000092217b37dfbda79f-00662c7201-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:33:21.093 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket'} 2024-04-27T03:33:21.098 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx00000877bff179d6319ce-00662c7201-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:33:21.099 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'global': 'true'} 2024-04-27T03:33:21.103 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}, 'anonymous_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 0, 'enabled': False}} 2024-04-27T03:33:21.104 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_userstring', 'enabled': 'true'} 2024-04-27T03:33:21.107 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx000005ddbceb21c507df8-00662c7201-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:33:21.108 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user'} 2024-04-27T03:33:21.111 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx0000003f9103f194f34ef-00662c7201-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:33:21.112 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucketstring', 'enabled': 'true'} 2024-04-27T03:33:21.115 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'NoSuchKey', 'Message': '', 'RequestId': 'tx00000971a6974b19bc15b-00662c7201-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:33:21.116 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'enabled': 'true'} 2024-04-27T03:33:21.119 INFO:tasks.radosgw_admin_rest: json result: {'Code': 'InvalidArgument', 'Message': '', 'RequestId': 'tx0000014fa701f37c4dec6-00662c7201-4217-default', 'HostId': '4217-default-default'} 2024-04-27T03:33:21.120 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user', 'enabled': 'true', 'max-read-bytes': '2'} 2024-04-27T03:33:21.131 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:33:21.131 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'user', 'uid': 'ratelimit_user'} 2024-04-27T03:33:21.136 INFO:tasks.radosgw_admin_rest: json result: {'user_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 2, 'max_write_bytes': 0, 'enabled': True}} 2024-04-27T03:33:21.136 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket', 'enabled': 'true', 'max-write-bytes': '2'} 2024-04-27T03:33:21.142 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:33:21.142 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'info'] {'ratelimit-scope': 'bucket', 'bucket': 'ratelimitbucket'} 2024-04-27T03:33:21.146 INFO:tasks.radosgw_admin_rest: json result: {'bucket_ratelimit': {'max_read_ops': 0, 'max_write_ops': 0, 'max_read_bytes': 0, 'max_write_bytes': 2, 'enabled': True}} 2024-04-27T03:33:21.147 INFO:tasks.radosgw_admin_rest:radosgw-admin-rest: ['ratelimit', 'modify'] {'ratelimit-scope': 'bucket', 'global': 'true', 'enabled': 'true'} 2024-04-27T03:33:21.153 INFO:tasks.radosgw_admin_rest: empty result 2024-04-27T03:33:21.157 DEBUG:teuthology.run_tasks:Unwinding manager rgw 2024-04-27T03:33:21.167 DEBUG:tasks.rgw.client.0:waiting for process to exit 2024-04-27T03:33:21.168 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T03:33:21.169 INFO:tasks.rgw.client.0.smithi165.stdout:2024-04-27T03:33:21.163+0000 7f9201412640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper term radosgw --rgw-frontends beast port=80 -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground (PID: 34911) UID: 0 2024-04-27T03:33:21.169 INFO:tasks.rgw.client.0.smithi165.stdout:2024-04-27T03:33:21.163+0000 7f9204c6b880 -1 shutting down 2024-04-27T03:33:26.355 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 is failed for ~0s 2024-04-27T03:33:27.270 INFO:tasks.rgw.client.0:Stopped 2024-04-27T03:33:27.270 DEBUG:teuthology.orchestra.run.smithi165:> rm -f /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock 2024-04-27T03:33:27.289 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /etc/ceph/vault-root-token 2024-04-27T03:33:27.357 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f /home/ubuntu/cephtest/url_file 2024-04-27T03:33:27.423 INFO:tasks.util.rgw:rgwadmin: client.0 : ['gc', 'process', '--include-all'] 2024-04-27T03:33:27.424 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'gc', 'process', '--include-all'] 2024-04-27T03:33:27.424 DEBUG:teuthology.orchestra.run.smithi165:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph gc process --include-all 2024-04-27T03:33:27.510 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setuser ceph since I am not root 2024-04-27T03:33:27.511 INFO:teuthology.orchestra.run.smithi165.stderr:ignoring --setgroup ceph since I am not root 2024-04-27T03:33:27.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.506+0000 7f4434933800 1 Processor -- start 2024-04-27T03:33:27.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.506+0000 7f4434933800 1 -- start start 2024-04-27T03:33:27.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.506+0000 7f4434933800 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560dedb46780 0x560dedb46ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.506+0000 7f4434933800 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 26 bytes epoch 0) -- 0x560deda06d40 con 0x560dedb470e0 2024-04-27T03:33:27.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.506+0000 7f4434933800 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x560ded9d94d0 con 0x560dedb46780 2024-04-27T03:33:27.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.506+0000 7f44305d1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560dedb46780 0x560dedb46ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.506+0000 7f44305d1640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x560dedb46780 0x560dedb46ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:37384/0 (socket says 172.21.15.165:37384) 2024-04-27T03:33:27.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.506+0000 7f442fdd0640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x560dedb470e0 0x560dedb47590 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.165:34196/0 (socket says 172.21.15.165:34196) 2024-04-27T03:33:27.512 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.506+0000 7f44305d1640 1 -- 172.21.15.165:0/2684418106 learned_addr learned my addr 172.21.15.165:0/2684418106 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:33:27.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.507+0000 7f44305d1640 1 -- 172.21.15.165:0/2684418106 >> v1:172.21.15.89:6789/0 conn(0x560dedb470e0 legacy=0x560dedb47590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.513 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.507+0000 7f44305d1640 1 -- 172.21.15.165:0/2684418106 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x560deda05590 con 0x560dedb46780 2024-04-27T03:33:27.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.508+0000 7f44305d1640 1 --2- 172.21.15.165:0/2684418106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedb46ba0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f442000a520 tx=0x7f442001d010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cd8007cdeb07a5f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.509+0000 7f442f5cf640 1 -- 172.21.15.165:0/2684418106 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4420057070 con 0x560dedb46780 2024-04-27T03:33:27.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.509+0000 7f442f5cf640 1 -- 172.21.15.165:0/2684418106 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420052570 con 0x560dedb46780 2024-04-27T03:33:27.514 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.509+0000 7f442f5cf640 1 -- 172.21.15.165:0/2684418106 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4420052870 con 0x560dedb46780 2024-04-27T03:33:27.515 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.509+0000 7f4434933800 1 -- 172.21.15.165:0/2684418106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 msgr2=0x560dedb46ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.509+0000 7f4434933800 1 --2- 172.21.15.165:0/2684418106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedb46ba0 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f442000a520 tx=0x7f442001d010 comp rx=0 tx=0).stop 2024-04-27T03:33:27.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.509+0000 7f4434933800 1 -- 172.21.15.165:0/2684418106 shutdown_connections 2024-04-27T03:33:27.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.509+0000 7f4434933800 1 --2- 172.21.15.165:0/2684418106 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedb46ba0 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.509+0000 7f4434933800 1 -- 172.21.15.165:0/2684418106 >> 172.21.15.165:0/2684418106 conn(0x560dedb42300 msgr2=0x560dedb44720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:27.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.509+0000 7f4434933800 1 -- 172.21.15.165:0/2684418106 shutdown_connections 2024-04-27T03:33:27.516 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.509+0000 7f4434933800 1 -- 172.21.15.165:0/2684418106 wait complete. 2024-04-27T03:33:27.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.522+0000 7f4434933800 1 Processor -- start 2024-04-27T03:33:27.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.522+0000 7f4434933800 1 -- start start 2024-04-27T03:33:27.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.523+0000 7f4434933800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedc3b390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.523+0000 7f4434933800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560deda933c0 con 0x560dedb46780 2024-04-27T03:33:27.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.523+0000 7f44305d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedc3b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.523+0000 7f44305d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedc3b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:37396/0 (socket says 172.21.15.165:37396) 2024-04-27T03:33:27.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.523+0000 7f44305d1640 1 -- 172.21.15.165:0/1706231452 learned_addr learned my addr 172.21.15.165:0/1706231452 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:33:27.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.523+0000 7f44305d1640 1 -- 172.21.15.165:0/1706231452 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dedc3b960 con 0x560dedb46780 2024-04-27T03:33:27.529 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.523+0000 7f44305d1640 1 --2- 172.21.15.165:0/1706231452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedc3b390 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f442001d050 tx=0x7f442004c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6d0c92c4f59cc6 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.523+0000 7f431e60c640 1 -- 172.21.15.165:0/1706231452 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4420057040 con 0x560dedb46780 2024-04-27T03:33:27.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.524+0000 7f431e60c640 1 -- 172.21.15.165:0/1706231452 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420052850 con 0x560dedb46780 2024-04-27T03:33:27.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.524+0000 7f431e60c640 1 -- 172.21.15.165:0/1706231452 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4420052b70 con 0x560dedb46780 2024-04-27T03:33:27.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.524+0000 7f4434933800 1 -- 172.21.15.165:0/1706231452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 msgr2=0x560dedc3b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.524+0000 7f4434933800 1 --2- 172.21.15.165:0/1706231452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedc3b390 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f442001d050 tx=0x7f442004c000 comp rx=0 tx=0).stop 2024-04-27T03:33:27.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.524+0000 7f4434933800 1 -- 172.21.15.165:0/1706231452 shutdown_connections 2024-04-27T03:33:27.534 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.524+0000 7f4434933800 1 --2- 172.21.15.165:0/1706231452 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedc3b390 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.524+0000 7f4434933800 1 -- 172.21.15.165:0/1706231452 >> 172.21.15.165:0/1706231452 conn(0x560dedb42300 msgr2=0x560dedc36cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.524+0000 7f4434933800 1 -- 172.21.15.165:0/1706231452 shutdown_connections 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.524+0000 7f4434933800 1 -- 172.21.15.165:0/1706231452 wait complete. 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.525+0000 7f4434933800 1 Processor -- start 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.525+0000 7f4434933800 1 -- start start 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.525+0000 7f4434933800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedc42ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.525+0000 7f4434933800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560dedb47f40 con 0x560dedb46780 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.525+0000 7f44305d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedc42ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.525+0000 7f44305d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedc42ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:37404/0 (socket says 172.21.15.165:37404) 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.525+0000 7f44305d1640 1 -- 172.21.15.165:0/964831903 learned_addr learned my addr 172.21.15.165:0/964831903 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.526+0000 7f44305d1640 1 -- 172.21.15.165:0/964831903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dedc43410 con 0x560dedb46780 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.526+0000 7f44305d1640 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedc42ed0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f442001cfe0 tx=0x7f4420003020 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.535 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.526+0000 7f431ce09640 1 -- 172.21.15.165:0/964831903 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4420073070 con 0x560dedb46780 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.526+0000 7f431ce09640 1 -- 172.21.15.165:0/964831903 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f442005d650 con 0x560dedb46780 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.526+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560dedc436a0 con 0x560dedb46780 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.526+0000 7f431ce09640 1 -- 172.21.15.165:0/964831903 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4420057070 con 0x560dedb46780 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.526+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dedc462b0 con 0x560dedb46780 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.527+0000 7f431ce09640 1 -- 172.21.15.165:0/964831903 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f4420076070 con 0x560dedb46780 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.527+0000 7f431ce09640 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4318036950 0x7f4318038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.527+0000 7f431ce09640 1 -- 172.21.15.165:0/964831903 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f442008ddb0 con 0x560dedb46780 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.527+0000 7f442fdd0640 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4318036950 0x7f4318038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.527+0000 7f4434933800 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560dedb470e0 0x560dedc4a070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.527+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc4a640 con 0x560dedb470e0 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.527+0000 7f4430dd2640 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560dedb470e0 0x560dedc4a070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.528+0000 7f442fdd0640 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4318036950 0x7f4318038e10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4424000f60 tx=0x7f4424001370 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.536 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.528+0000 7f4430dd2640 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560dedb470e0 0x560dedc4a070 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.528+0000 7f4430dd2640 1 -- 172.21.15.165:0/964831903 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedb470e0 2024-04-27T03:33:27.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.528+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc47620 con 0x560dedb470e0 2024-04-27T03:33:27.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.528+0000 7f4430dd2640 1 -- 172.21.15.165:0/964831903 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedb470e0 2024-04-27T03:33:27.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.528+0000 7f4434933800 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560dedc4c9a0 0x560dedc4ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.528+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:3 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc4f480 con 0x560dedc4c9a0 2024-04-27T03:33:27.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.528+0000 7f44305d1640 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560dedc4c9a0 0x560dedc4ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.537 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.529+0000 7f44305d1640 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560dedc4c9a0 0x560dedc4ed90 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.529+0000 7f44305d1640 1 -- 172.21.15.165:0/964831903 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(3 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [call out=48b,read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 232+0+1041 (crc 0 0 0) 0x7f4420075030 con 0x560dedc4c9a0 2024-04-27T03:33:27.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.529+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc4b950 con 0x560dedb470e0 2024-04-27T03:33:27.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.529+0000 7f4430dd2640 1 -- 172.21.15.165:0/964831903 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(4 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedb470e0 2024-04-27T03:33:27.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.529+0000 7f4434933800 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x560dedc50930 0x560dedc52d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.529+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:5 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc534e0 con 0x560dedc50930 2024-04-27T03:33:27.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.529+0000 7f442fdd0640 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x560dedc50930 0x560dedc52d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.530+0000 7f442fdd0640 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x560dedc50930 0x560dedc52d80 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.530+0000 7f442fdd0640 1 -- 172.21.15.165:0/964831903 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(5 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [call out=48b,read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 237+0+484 (crc 0 0 0) 0x7f442400f690 con 0x560dedc50930 2024-04-27T03:33:27.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.530+0000 7f4434933800 1 Processor -- start 2024-04-27T03:33:27.547 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.530+0000 7f4434933800 1 -- start start 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.531+0000 7f4434933800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc61940 0x560dedc61d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.531+0000 7f4434933800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560deda396b0 con 0x560dedc61940 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.531+0000 7f44305d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc61940 0x560dedc61d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.531+0000 7f44305d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc61940 0x560dedc61d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:37412/0 (socket says 172.21.15.165:37412) 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.531+0000 7f44305d1640 1 -- 172.21.15.165:0/755672737 learned_addr learned my addr 172.21.15.165:0/755672737 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.531+0000 7f44305d1640 1 -- 172.21.15.165:0/755672737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dedc622a0 con 0x560dedc61940 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.531+0000 7f44305d1640 1 --2- 172.21.15.165:0/755672737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc61940 0x560dedc61d60 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4420066560 tx=0x7f4420006650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.532+0000 7f4315ffb640 1 -- 172.21.15.165:0/755672737 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44200774e0 con 0x560dedc61940 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.532+0000 7f4315ffb640 1 -- 172.21.15.165:0/755672737 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4420091020 con 0x560dedc61940 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.532+0000 7f4434933800 1 -- 172.21.15.165:0/755672737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560dedc62d10 con 0x560dedc61940 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.532+0000 7f4315ffb640 1 -- 172.21.15.165:0/755672737 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44200777d0 con 0x560dedc61940 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.532+0000 7f4434933800 1 -- 172.21.15.165:0/755672737 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dedc63210 con 0x560dedc61940 2024-04-27T03:33:27.548 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.533+0000 7f4315ffb640 1 -- 172.21.15.165:0/755672737 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f4420094030 con 0x560dedc61940 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.533+0000 7f4315ffb640 1 --2- 172.21.15.165:0/755672737 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4308036950 0x7f4308038e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.533+0000 7f4315ffb640 1 -- 172.21.15.165:0/755672737 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f4420092070 con 0x560dedc61940 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.533+0000 7f442fdd0640 1 --2- 172.21.15.165:0/755672737 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4308036950 0x7f4308038e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.533+0000 7f4434933800 1 Processor -- start 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.533+0000 7f4434933800 1 -- start start 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.533+0000 7f442fdd0640 1 --2- 172.21.15.165:0/755672737 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4308036950 0x7f4308038e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x560dedc4af50 tx=0x7f442400f000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.533+0000 7f4434933800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 0x560dedc6eef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.533+0000 7f4434933800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560dedc3bc20 con 0x560dedc6ead0 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.534+0000 7f44305d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 0x560dedc6eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.534+0000 7f44305d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 0x560dedc6eef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:37426/0 (socket says 172.21.15.165:37426) 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.534+0000 7f44305d1640 1 -- 172.21.15.165:0/2918074635 learned_addr learned my addr 172.21.15.165:0/2918074635 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.534+0000 7f44305d1640 1 -- 172.21.15.165:0/2918074635 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dedc6f4c0 con 0x560dedc6ead0 2024-04-27T03:33:27.549 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.534+0000 7f44305d1640 1 --2- 172.21.15.165:0/2918074635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 0x560dedc6eef0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f44200866c0 tx=0x7f4420086430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d5967f35409c918b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.534+0000 7f430f7fe640 1 -- 172.21.15.165:0/2918074635 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f442005b730 con 0x560dedc6ead0 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.534+0000 7f430f7fe640 1 -- 172.21.15.165:0/2918074635 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f44200a6030 con 0x560dedc6ead0 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.534+0000 7f430f7fe640 1 -- 172.21.15.165:0/2918074635 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4420078e90 con 0x560dedc6ead0 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.535+0000 7f4434933800 1 -- 172.21.15.165:0/2918074635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 msgr2=0x560dedc6eef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.535+0000 7f4434933800 1 --2- 172.21.15.165:0/2918074635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 0x560dedc6eef0 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7f44200866c0 tx=0x7f4420086430 comp rx=0 tx=0).stop 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.535+0000 7f4434933800 1 -- 172.21.15.165:0/2918074635 shutdown_connections 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.535+0000 7f4434933800 1 --2- 172.21.15.165:0/2918074635 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 0x560dedc6eef0 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.535+0000 7f4434933800 1 -- 172.21.15.165:0/2918074635 >> 172.21.15.165:0/2918074635 conn(0x560dedc6ade0 msgr2=0x560dedc68e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.535+0000 7f4434933800 1 -- 172.21.15.165:0/2918074635 shutdown_connections 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.535+0000 7f4434933800 1 -- 172.21.15.165:0/2918074635 wait complete. 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.535+0000 7f4434933800 1 Processor -- start 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.535+0000 7f4434933800 1 -- start start 2024-04-27T03:33:27.550 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.535+0000 7f4434933800 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 0x560dedc75f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.535+0000 7f4434933800 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x560dedc6fb50 con 0x560dedc6ead0 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.536+0000 7f44305d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 0x560dedc75f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.536+0000 7f44305d1640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 0x560dedc75f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.165:37432/0 (socket says 172.21.15.165:37432) 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.536+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 learned_addr learned my addr 172.21.15.165:0/3080026603 (peer_addr_for_me v2:172.21.15.165:0/0) 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.536+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560dedc76460 con 0x560dedc6ead0 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.536+0000 7f44305d1640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 0x560dedc75f20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f44200524f0 tx=0x7f442004fea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.536+0000 7f430dffb640 1 -- 172.21.15.165:0/3080026603 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44200bb070 con 0x560dedc6ead0 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.536+0000 7f430dffb640 1 -- 172.21.15.165:0/3080026603 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f442008a070 con 0x560dedc6ead0 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.536+0000 7f430dffb640 1 -- 172.21.15.165:0/3080026603 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4420079ac0 con 0x560dedc6ead0 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.536+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x560dedc76460 con 0x560dedc6ead0 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.536+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560dedc76460 con 0x560dedc6ead0 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.537+0000 7f430dffb640 1 -- 172.21.15.165:0/3080026603 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f44200be070 con 0x560dedc6ead0 2024-04-27T03:33:27.551 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.537+0000 7f430dffb640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4300036900 0x7f4300038dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.537+0000 7f430dffb640 1 -- 172.21.15.165:0/3080026603 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f442007f040 con 0x560dedc6ead0 2024-04-27T03:33:27.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.538+0000 7f442fdd0640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4300036900 0x7f4300038dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.538+0000 7f4434933800 20 rados->read ofs=0 len=0 2024-04-27T03:33:27.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.538+0000 7f4434933800 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560dedc81590 0x560dedc83a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.538+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:1 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc841a0 con 0x560dedc81590 2024-04-27T03:33:27.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.538+0000 7f442fdd0640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4300036900 0x7f4300038dc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x560dedc538e0 tx=0x7f4424010000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.538+0000 7f4430dd2640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560dedc81590 0x560dedc83a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.538+0000 7f4430dd2640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560dedc81590 0x560dedc83a50 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== osd_op_reply(1 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:33:27.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 20 realm 2024-04-27T03:33:27.552 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 20 rados->read ofs=0 len=0 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:2 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc845b0 con 0x560dedc81590 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== osd_op_reply(2 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 4 RGWPeriod::init failed to init realm id : (2) No such file or directory 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 20 rados->read ofs=0 len=0 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:3 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc84e80 con 0x560dedc81590 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 3 ==== osd_op_reply(3 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 20 rados->read ofs=0 len=0 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:4 5.f 5:f43b6ece:::zone_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc81590 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 4 ==== osd_op_reply(4 zone_names.default [read 0~46 out=46b] v0'0 uv3 ondisk = 0) ==== 162+0+46 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:33:27.553 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 20 rados->read ofs=0 len=0 2024-04-27T03:33:27.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560dedc85f80 0x560dedc88370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:5 5.1 5:812cf48a:::zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc88980 con 0x560dedc85f80 2024-04-27T03:33:27.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.539+0000 7f44305d1640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560dedc85f80 0x560dedc88370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.540+0000 7f44305d1640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560dedc85f80 0x560dedc88370 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.540+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== osd_op_reply(5 zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa [read 0~993 out=993b] v0'0 uv4 ondisk = 0) ==== 190+0+993 (crc 0 0 0) 0x7f44200bd030 con 0x560dedc85f80 2024-04-27T03:33:27.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.540+0000 7f4434933800 20 rados_obj.operate() r=0 bl.length=993 2024-04-27T03:33:27.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.540+0000 7f4434933800 20 searching for the correct realm 2024-04-27T03:33:27.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.540+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:6 5.0 5:00000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8a2b0 con 0x560dedc85f80 2024-04-27T03:33:27.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.541+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== osd_op_reply(6 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f44200bd030 con 0x560dedc85f80 2024-04-27T03:33:27.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.541+0000 7f44305d1640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f44200c0060 0x7f44200ab790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.541+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:7 5.10 5:08000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8a2b0 con 0x7f44200c0060 2024-04-27T03:33:27.554 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.541+0000 7f442fdd0640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f44200c0060 0x7f44200ab790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.541+0000 7f442fdd0640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f44200c0060 0x7f44200ab790 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.541+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== osd_op_reply(7 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.541+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:8 5.8 5:10000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8a2b0 con 0x7f44200c0060 2024-04-27T03:33:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.541+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== osd_op_reply(8 [pgnls start_epoch 27 out=112b] v18'1 uv1 ondisk = 1) ==== 144+0+112 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.555 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.541+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:9 5.18 5:18000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8a2b0 con 0x560dedc81590 2024-04-27T03:33:27.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.542+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 5 ==== osd_op_reply(9 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.542+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:10 5.4 5:20000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc81590 2024-04-27T03:33:27.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.542+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 6 ==== osd_op_reply(10 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.542+0000 7f4430dd2640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4428016440 0x7f44280168c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.542+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:11 5.14 5:28000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f4428016440 2024-04-27T03:33:27.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.542+0000 7f4430dd2640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4428016440 0x7f44280168c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.565 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.543+0000 7f4430dd2640 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4428016440 0x7f44280168c0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.543+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== osd_op_reply(11 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f442800a4a0 con 0x7f4428016440 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.543+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:12 5.c 5:30000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f4428016440 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.544+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== osd_op_reply(12 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f442800a4a0 con 0x7f4428016440 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.544+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:13 5.1c 5:38000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f44200c0060 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.544+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 3 ==== osd_op_reply(13 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.544+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:14 5.2 5:40000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc85f80 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.544+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 3 ==== osd_op_reply(14 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.544+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:15 5.12 5:48000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc81590 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.544+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 7 ==== osd_op_reply(15 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.544+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:16 5.a 5:50000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f44200c0060 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.545+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 4 ==== osd_op_reply(16 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.545+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:17 5.1a 5:58000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc81590 2024-04-27T03:33:27.566 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.545+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 8 ==== osd_op_reply(17 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.545+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:18 5.6 5:60000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f44200c0060 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.545+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 5 ==== osd_op_reply(18 [pgnls start_epoch 27 out=79b] v18'1 uv1 ondisk = 1) ==== 144+0+79 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.545+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:19 5.16 5:68000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc85f80 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.545+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 4 ==== osd_op_reply(19 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.545+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:20 5.e 5:70000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc85f80 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.546+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 5 ==== osd_op_reply(20 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.546+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:21 5.1e 5:78000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f4428016440 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.546+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 3 ==== osd_op_reply(21 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f442800a4a0 con 0x7f4428016440 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.546+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:22 5.1 5:80000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc85f80 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.546+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 6 ==== osd_op_reply(22 [pgnls start_epoch 27 out=107b] v27'4 uv4 ondisk = 1) ==== 144+0+107 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.546+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:23 5.11 5:88000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc85f80 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.546+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 7 ==== osd_op_reply(23 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.567 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.546+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:24 5.9 5:90000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc81590 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.547+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 9 ==== osd_op_reply(24 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.547+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:25 5.19 5:98000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc81590 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.547+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 10 ==== osd_op_reply(25 [pgnls start_epoch 27 out=74b] v27'2 uv1 ondisk = 1) ==== 144+0+74 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.547+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:26 5.5 5:a0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f4428016440 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.547+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 4 ==== osd_op_reply(26 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f442800a4a0 con 0x7f4428016440 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.547+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:27 5.15 5:a8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc85f80 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.548+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 8 ==== osd_op_reply(27 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.548+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:28 5.d 5:b0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f44200c0060 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.548+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 6 ==== osd_op_reply(28 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.548+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:29 5.1d 5:b8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc81590 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.548+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 11 ==== osd_op_reply(29 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.548+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:30 5.3 5:c0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f4428016440 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.548+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 5 ==== osd_op_reply(30 [pgnls start_epoch 27 out=82b] v27'1 uv1 ondisk = 1) ==== 144+0+82 (crc 0 0 0) 0x7f442800a4a0 con 0x7f4428016440 2024-04-27T03:33:27.568 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.548+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:31 5.13 5:c8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc85f80 2024-04-27T03:33:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.549+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 9 ==== osd_op_reply(31 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.549+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:32 5.b 5:d0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f4428016440 2024-04-27T03:33:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.549+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 6 ==== osd_op_reply(32 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f442800a4a0 con 0x7f4428016440 2024-04-27T03:33:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.549+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:33 5.1b 5:d8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f44200c0060 2024-04-27T03:33:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.549+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 7 ==== osd_op_reply(33 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.549+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:34 5.7 5:e0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc85f80 2024-04-27T03:33:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 10 ==== osd_op_reply(34 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:35 5.17 5:e8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f4428016440 2024-04-27T03:33:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 7 ==== osd_op_reply(35 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f442800a4a0 con 0x7f4428016440 2024-04-27T03:33:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:36 5.f 5:f0000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x560dedc81590 2024-04-27T03:33:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 12 ==== osd_op_reply(36 [pgnls start_epoch 27 out=115b] v27'3 uv3 ondisk = 1) ==== 144+0+115 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.569 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:37 5.1f 5:f8000000::::head [pgnls start_epoch 27 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e27) -- 0x560dedc85220 con 0x7f44200c0060 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 8 ==== osd_op_reply(37 [pgnls start_epoch 27 out=49b] v0'0 uv0 ondisk = 1) ==== 144+0+49 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4434933800 20 RGWRados::pool_iterate: got zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4434933800 20 RGWRados::pool_iterate: got default.zonegroup. 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4434933800 20 RGWRados::pool_iterate: got zone_info.708f831a-84ca-40fe-a0c7-f3b916a7f9aa 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4434933800 20 RGWRados::pool_iterate: got default.zone. 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4434933800 20 RGWRados::pool_iterate: got period_config.default 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4434933800 20 RGWRados::pool_iterate: got zone_names.default 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4434933800 20 RGWRados::pool_iterate: got zonegroups_names.default 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4434933800 20 rados->read ofs=0 len=0 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.550+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:38 5.12 5:49953fa1:::default.realm:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc89240 con 0x560dedc81590 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 13 ==== osd_op_reply(38 default.realm [read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 157+0+0 (crc 0 0 0) 0x7f442800a4a0 con 0x560dedc81590 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:33:27.570 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 20 rados->read ofs=0 len=0 2024-04-27T03:33:27.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:39 5.f 5:f4c53578:::zonegroups_names.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc89630 con 0x560dedc81590 2024-04-27T03:33:27.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 14 ==== osd_op_reply(39 zonegroups_names.default [read 0~46 out=46b] v0'0 uv2 ondisk = 0) ==== 168+0+46 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 20 rados_obj.operate() r=0 bl.length=46 2024-04-27T03:33:27.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 20 rados->read ofs=0 len=0 2024-04-27T03:33:27.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:40 5.8 5:13d819a8:::zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc89630 con 0x7f44200c0060 2024-04-27T03:33:27.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 9 ==== osd_op_reply(40 zonegroup_info.fd642a41-253a-48d4-9473-6eca68174247 [read 0~436 out=436b] v0'0 uv1 ondisk = 0) ==== 195+0+436 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 20 rados_obj.operate() r=0 bl.length=436 2024-04-27T03:33:27.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 20 zone default found 2024-04-27T03:33:27.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 4 Realm: () 2024-04-27T03:33:27.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 4 ZoneGroup: default (fd642a41-253a-48d4-9473-6eca68174247) 2024-04-27T03:33:27.571 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 4 Zone: default (708f831a-84ca-40fe-a0c7-f3b916a7f9aa) 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 10 cannot find current period zonegroup using local zonegroup configuration 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 20 zonegroup default 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 20 rados->read ofs=0 len=0 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.551+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:41 5.3 5:c52100b6:::period_config.default:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8a730 con 0x7f4428016440 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.552+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 8 ==== osd_op_reply(41 period_config.default [read 0~187 out=187b] v0'0 uv1 ondisk = 0) ==== 165+0+187 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.552+0000 7f4434933800 20 rados_obj.operate() r=0 bl.length=187 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.552+0000 7f4434933800 20 rados->read ofs=0 len=0 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.552+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:42 6.2 6:4347d321:::bucket.sync-source-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8c5c0 con 0x560dedc85f80 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.552+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 11 ==== osd_op_reply(42 bucket.sync-source-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.552+0000 7f4434933800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.552+0000 7f4434933800 20 rados->read ofs=0 len=0 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.552+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:43 6.b 6:d467b91b:::bucket.sync-target-hints.:head [call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8c960 con 0x560dedc85f80 2024-04-27T03:33:27.572 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.553+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 12 ==== osd_op_reply(43 bucket.sync-target-hints. [call,read 0~0] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) ==== 211+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.553+0000 7f4434933800 20 rados_obj.operate() r=-2 bl.length=0 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.553+0000 7f4434933800 20 started sync module instance, tier type = 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.553+0000 7f4434933800 20 started zone id=708f831a-84ca-40fe-a0c7-f3b916a7f9aa (name=default) with tier type = 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.553+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:44 7.1f 7:f95f44c2:::notify.0:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8ba00 con 0x7f44200c0060 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.554+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 10 ==== osd_op_reply(44 notify.0 [create] v27'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.554+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:45 7.1f 7:f95f44c2:::notify.0:head [watch watch cookie 94617823917392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8cd00 con 0x7f44200c0060 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.554+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:46 7.0 7:05bf5b68:::notify.1:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8d5b0 con 0x560dedc81590 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.555+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 11 ==== osd_op_reply(45 notify.0 [watch watch cookie 94617823917392] v27'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.555+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 15 ==== osd_op_reply(46 notify.1 [create] v27'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.555+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:47 7.0 7:05bf5b68:::notify.1:head [watch watch cookie 94617823924384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8d9a0 con 0x560dedc81590 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.555+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:48 7.15 7:a93a5511:::notify.2:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8e360 con 0x560dedc85f80 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.557+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 16 ==== osd_op_reply(47 notify.1 [watch watch cookie 94617823924384] v27'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.573 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.557+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 13 ==== osd_op_reply(48 notify.2 [create] v27'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.557+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:49 7.15 7:a93a5511:::notify.2:head [watch watch cookie 94617823927984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8e750 con 0x560dedc85f80 2024-04-27T03:33:27.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.557+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:50 7.e 7:7759931f:::notify.3:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8f170 con 0x7f44200c0060 2024-04-27T03:33:27.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.558+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 12 ==== osd_op_reply(50 notify.3 [create] v27'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.558+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:51 7.e 7:7759931f:::notify.3:head [watch watch cookie 94617823933792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8fe30 con 0x7f44200c0060 2024-04-27T03:33:27.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.558+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:52 7.d 7:b4812045:::notify.4:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc90850 con 0x560dedc81590 2024-04-27T03:33:27.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.558+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 14 ==== osd_op_reply(49 notify.2 [watch watch cookie 94617823927984] v27'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.559+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 17 ==== osd_op_reply(52 notify.4 [create] v27'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.559+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:53 7.d 7:b4812045:::notify.4:head [watch watch cookie 94617823937344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc90c70 con 0x560dedc81590 2024-04-27T03:33:27.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.559+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:54 7.3 7:c609908c:::notify.5:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc916c0 con 0x7f4428016440 2024-04-27T03:33:27.574 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.559+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 13 ==== osd_op_reply(51 notify.3 [watch watch cookie 94617823933792] v27'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.560+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 18 ==== osd_op_reply(53 notify.4 [watch watch cookie 94617823937344] v27'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.560+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 9 ==== osd_op_reply(54 notify.5 [create] v27'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.560+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:55 7.3 7:c609908c:::notify.5:head [watch watch cookie 94617823919600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc91680 con 0x7f4428016440 2024-04-27T03:33:27.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.560+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:56 7.14 7:2b04a3e9:::notify.6:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc91f60 con 0x7f4428016440 2024-04-27T03:33:27.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.561+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 10 ==== osd_op_reply(55 notify.5 [watch watch cookie 94617823919600] v27'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.562+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 11 ==== osd_op_reply(56 notify.6 [create] v27'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.562+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:57 7.14 7:2b04a3e9:::notify.6:head [watch watch cookie 94617823941040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc91b30 con 0x7f4428016440 2024-04-27T03:33:27.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.562+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:58 7.9 7:93e5b521:::notify.7:head [create] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc925d0 con 0x7f4428016440 2024-04-27T03:33:27.578 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.563+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 12 ==== osd_op_reply(57 notify.6 [watch watch cookie 94617823941040] v27'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.563+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 13 ==== osd_op_reply(58 notify.7 [create] v27'283 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.563+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:59 7.9 7:93e5b521:::notify.7:head [watch watch cookie 94617823944896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc92590 con 0x7f4428016440 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.563+0000 7f4434933800 20 add_watcher() i=0 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.563+0000 7f4434933800 20 add_watcher() i=1 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.563+0000 7f4434933800 20 add_watcher() i=2 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.563+0000 7f4434933800 20 add_watcher() i=3 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.563+0000 7f4434933800 20 add_watcher() i=4 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.563+0000 7f4434933800 20 add_watcher() i=5 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.563+0000 7f4434933800 20 add_watcher() i=6 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.564+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 14 ==== osd_op_reply(59 notify.7 [watch watch cookie 94617823944896] v27'284 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.564+0000 7f4434933800 20 add_watcher() i=7 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.564+0000 7f4434933800 2 all 8 watchers are set, enabling cache 2024-04-27T03:33:27.579 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.564+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:60 6.4 6:22d26bf9:::data_loggenerations_metadata:head [call version.check_conds in=74b,call version.read in=11b,read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedc95ab0 con 0x560dedc81590 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.564+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 19 ==== osd_op_reply(60 data_loggenerations_metadata [call,call out=48b,read 0~28 out=28b] v0'0 uv1 ondisk = 0) ==== 256+0+76 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.564+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:61 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch watch cookie 94617823942272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc92ea0 con 0x560dedc81590 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.565+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 20 ==== osd_op_reply(61 data_loggenerations_metadata [watch watch cookie 94617823942272] v27'280 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.565+0000 7f42e67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.566+0000 7f4434933800 20 rgw_check_secure_mon_conn(): auth registy supported: methods=[2] modes=[2,1] 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.566+0000 7f4434933800 20 rgw_check_secure_mon_conn(): mode 1 is insecure 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.566+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:62 6.19 6:9a6dd8c0:gc::gc.0:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc9e250 con 0x560dedc81590 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.567+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 21 ==== osd_op_reply(62 gc.0 [create,call,call,call] v27'61 uv51 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.567+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:63 6.18 6:1e85421f:gc::gc.1:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc9e8a0 con 0x7f4428016440 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.568+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 15 ==== osd_op_reply(63 gc.1 [create,call,call,call] v27'39 uv38 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.568+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:64 6.b 6:d242335b:gc::gc.2:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc9ed00 con 0x560dedc85f80 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.569+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 15 ==== osd_op_reply(64 gc.2 [create,call,call,call] v27'40 uv28 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.580 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.569+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:65 6.4 6:2166d8e1:gc::gc.3:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc9f0a0 con 0x560dedc81590 2024-04-27T03:33:27.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.570+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 22 ==== osd_op_reply(65 gc.3 [create,call,call,call] v27'281 uv172 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.570+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:66 6.3 6:c60da066:gc::gc.4:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc9f660 con 0x7f44200c0060 2024-04-27T03:33:27.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.571+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 14 ==== osd_op_reply(66 gc.4 [create,call,call,call] v27'90 uv59 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.571+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:67 6.9 6:954be8d2:gc::gc.5:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc9fbd0 con 0x560dedc85f80 2024-04-27T03:33:27.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.572+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 16 ==== osd_op_reply(67 gc.5 [create,call,call,call] v27'69 uv49 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.581 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.572+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:68 6.e 6:748722ff:gc::gc.6:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc9fbd0 con 0x560dedc81590 2024-04-27T03:33:27.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.574+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 23 ==== osd_op_reply(68 gc.6 [create,call,call,call] v27'39 uv29 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.574+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:69 6.1d 6:beaa4be4:gc::gc.7:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc9fbd0 con 0x7f4428016440 2024-04-27T03:33:27.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.575+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 16 ==== osd_op_reply(69 gc.7 [create,call,call,call] v27'61 uv42 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.575+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:70 6.12 6:4f78e9a2:gc::gc.8:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0130 con 0x7f44200c0060 2024-04-27T03:33:27.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.576+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 15 ==== osd_op_reply(70 gc.8 [create,call,call,call] v27'31 uv21 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.582 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.576+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:71 6.1c 6:3e426232:gc::gc.9:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca05e0 con 0x560dedc85f80 2024-04-27T03:33:27.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.577+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 17 ==== osd_op_reply(71 gc.9 [create,call,call,call] v27'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 274+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.577+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:72 6.3 6:c31f41d7:gc::gc.10:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca05e0 con 0x7f44200c0060 2024-04-27T03:33:27.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.578+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 16 ==== osd_op_reply(72 gc.10 [create,call,call,call] v27'91 uv61 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.583 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.578+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:73 6.13 6:c8333aab:gc::gc.11:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0670 con 0x7f4428016440 2024-04-27T03:33:27.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.579+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 17 ==== osd_op_reply(73 gc.11 [create,call,call,call] v27'41 uv24 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.584 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.579+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:74 6.15 6:ab569f81:gc::gc.12:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0700 con 0x7f44200c0060 2024-04-27T03:33:27.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.580+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 17 ==== osd_op_reply(74 gc.12 [create,call,call,call] v27'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.585 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.580+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:75 6.0 6:0667bb13:gc::gc.13:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0790 con 0x7f4428016440 2024-04-27T03:33:27.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.581+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 18 ==== osd_op_reply(75 gc.13 [create,call,call,call] v27'74 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.586 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.581+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:76 6.4 6:227f63b5:gc::gc.14:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0790 con 0x560dedc81590 2024-04-27T03:33:27.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.582+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 24 ==== osd_op_reply(76 gc.14 [create,call,call,call] v27'282 uv168 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.587 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.582+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:77 6.7 6:e07a6548:gc::gc.15:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0790 con 0x560dedc85f80 2024-04-27T03:33:27.588 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.583+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 18 ==== osd_op_reply(77 gc.15 [create,call,call,call] v27'84 uv52 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.583+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:78 6.3 6:c623b173:gc::gc.16:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0790 con 0x7f44200c0060 2024-04-27T03:33:27.589 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.584+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 18 ==== osd_op_reply(78 gc.16 [create,call,call,call] v27'92 uv57 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.584+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:79 6.16 6:6849fd06:gc::gc.17:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0850 con 0x560dedc85f80 2024-04-27T03:33:27.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.585+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 19 ==== osd_op_reply(79 gc.17 [create,call,call,call] v27'43 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.590 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.585+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:80 6.d 6:b019ea83:gc::gc.18:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0850 con 0x560dedc81590 2024-04-27T03:33:27.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.587+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 25 ==== osd_op_reply(80 gc.18 [create,call,call,call] v27'39 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.592 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.587+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:81 6.4 6:24c8e1ae:gc::gc.19:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0820 con 0x560dedc81590 2024-04-27T03:33:27.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.588+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 26 ==== osd_op_reply(81 gc.19 [create,call,call,call] v27'283 uv170 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.593 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.588+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:82 6.7 6:e622f76a:gc::gc.20:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0790 con 0x560dedc85f80 2024-04-27T03:33:27.594 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.589+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 20 ==== osd_op_reply(82 gc.20 [create,call,call,call] v27'85 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.589+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:83 6.a 6:5127e642:gc::gc.21:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0790 con 0x7f44200c0060 2024-04-27T03:33:27.595 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.590+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 19 ==== osd_op_reply(83 gc.21 [create,call,call,call] v27'47 uv37 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.590+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:84 6.11 6:8eb99c53:gc::gc.22:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0790 con 0x560dedc85f80 2024-04-27T03:33:27.596 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.591+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 21 ==== osd_op_reply(84 gc.22 [create,call,call,call] v27'39 uv29 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.591+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:85 6.8 6:17d7e8ed:gc::gc.23:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0700 con 0x560dedc85f80 2024-04-27T03:33:27.597 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.592+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 22 ==== osd_op_reply(85 gc.23 [create,call,call,call] v27'50 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.592+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:86 6.1b 6:db9d130d:gc::gc.24:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0700 con 0x560dedc85f80 2024-04-27T03:33:27.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.593+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 23 ==== osd_op_reply(86 gc.24 [create,call,call,call] v27'33 uv23 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.598 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.593+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:87 6.9 6:921ecdb1:gc::gc.25:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0700 con 0x560dedc85f80 2024-04-27T03:33:27.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.594+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 24 ==== osd_op_reply(87 gc.25 [create,call,call,call] v27'70 uv47 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.599 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.594+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:88 6.1f 6:fde4dd55:gc::gc.26:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0700 con 0x7f4428016440 2024-04-27T03:33:27.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.595+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 19 ==== osd_op_reply(88 gc.26 [create,call,call,call] v27'41 uv31 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.595+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:89 6.1a 6:5909e671:gc::gc.27:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0700 con 0x560dedc85f80 2024-04-27T03:33:27.601 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.596+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 25 ==== osd_op_reply(89 gc.27 [create,call,call,call] v27'45 uv33 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.602 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.596+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:90 6.1d 6:b88e3271:gc::gc.28:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0700 con 0x7f4428016440 2024-04-27T03:33:27.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.597+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 20 ==== osd_op_reply(90 gc.28 [create,call,call,call] v27'62 uv40 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.603 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.598+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:91 6.7 6:e47c9322:gc::gc.29:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0700 con 0x560dedc85f80 2024-04-27T03:33:27.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.599+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 26 ==== osd_op_reply(91 gc.29 [create,call,call,call] v27'86 uv56 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.604 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.599+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:92 6.0 6:013d6c33:gc::gc.30:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0700 con 0x7f4428016440 2024-04-27T03:33:27.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.600+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 21 ==== osd_op_reply(92 gc.30 [create,call,call,call] v27'75 uv54 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.605 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.600+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:93 6.f 6:f24cb4a8:gc::gc.31:head [create,call version.check_conds in=74b,call rgw_gc.rgw_gc_queue_init in=45b,call version.set in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedca0700 con 0x7f44200c0060 2024-04-27T03:33:27.606 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.601+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 20 ==== osd_op_reply(93 gc.31 [create,call,call,call] v27'37 uv27 ondisk = -125 ((125) Operation canceled)) ==== 275+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.601+0000 7f42e57e2640 20 reqs_thread_entry: start 2024-04-27T03:33:27.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.601+0000 7f4434933800 20 init_complete bucket index max shards: 11 2024-04-27T03:33:27.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.601+0000 7f4434933800 20 Filter name: none 2024-04-27T03:33:27.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.601+0000 7f4256ffd640 20 reqs_thread_entry: start 2024-04-27T03:33:27.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.601+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2024-04-27T03:33:27.607 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.601+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:94 6.1d 6:beaa4be4:gc::gc.7:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.603+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 22 ==== osd_op_reply(94 gc.7 [call] v27'63 uv63 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.603+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:95 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.603+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 23 ==== osd_op_reply(95 gc.7 [call out=15b] v0'0 uv63 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.609 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.603+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.603+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:96 6.1d 6:beaa4be4:gc::gc.7:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f4428016440 2024-04-27T03:33:27.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.603+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 24 ==== osd_op_reply(96 gc.7 [call out=24b] v0'0 uv63 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.603+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:97 6.1d 6:beaa4be4:gc::gc.7:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.603+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 25 ==== osd_op_reply(97 gc.7 [call out=15b] v0'0 uv63 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.603+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.603+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:98 6.1d 6:beaa4be4:gc::gc.7:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.604+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 26 ==== osd_op_reply(98 gc.7 [call out=15b] v0'0 uv63 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.604+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.610 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.604+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:99 6.1d 6:beaa4be4:gc::gc.7:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.605+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 27 ==== osd_op_reply(99 gc.7 [call] v27'64 uv64 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.605+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2024-04-27T03:33:27.611 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.605+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:100 6.12 6:4f78e9a2:gc::gc.8:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.606+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 21 ==== osd_op_reply(100 gc.8 [call] v27'32 uv32 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.606+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:101 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.606+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 22 ==== osd_op_reply(101 gc.8 [call out=15b] v0'0 uv32 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.606+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.606+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:102 6.12 6:4f78e9a2:gc::gc.8:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f44200c0060 2024-04-27T03:33:27.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.607+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 23 ==== osd_op_reply(102 gc.8 [call out=24b] v0'0 uv32 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.607+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:103 6.12 6:4f78e9a2:gc::gc.8:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.607+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 24 ==== osd_op_reply(103 gc.8 [call out=15b] v0'0 uv32 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.607+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.613 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.607+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:104 6.12 6:4f78e9a2:gc::gc.8:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.607+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 25 ==== osd_op_reply(104 gc.8 [call out=15b] v0'0 uv32 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.607+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.614 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.607+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:105 6.12 6:4f78e9a2:gc::gc.8:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.608+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 26 ==== osd_op_reply(105 gc.8 [call] v27'33 uv33 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.608+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.608+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:106 6.1c 6:3e426232:gc::gc.9:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.609+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 27 ==== osd_op_reply(106 gc.9 [call] v27'38 uv38 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.609+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:107 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.609+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 28 ==== osd_op_reply(107 gc.9 [call out=15b] v0'0 uv38 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.609+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.609+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:108 6.1c 6:3e426232:gc::gc.9:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc85f80 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.610+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 29 ==== osd_op_reply(108 gc.9 [call out=24b] v0'0 uv38 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.610+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:109 6.1c 6:3e426232:gc::gc.9:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.610+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 30 ==== osd_op_reply(109 gc.9 [call out=15b] v0'0 uv38 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.610+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.616 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.610+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:110 6.1c 6:3e426232:gc::gc.9:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.610+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 31 ==== osd_op_reply(110 gc.9 [call out=15b] v0'0 uv38 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.610+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.617 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.610+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:111 6.1c 6:3e426232:gc::gc.9:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.618 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.611+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 32 ==== osd_op_reply(111 gc.9 [call] v27'39 uv39 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.611+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2024-04-27T03:33:27.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.611+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:112 6.3 6:c31f41d7:gc::gc.10:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.612+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 27 ==== osd_op_reply(112 gc.10 [call] v27'93 uv93 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.612+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:113 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.612+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 28 ==== osd_op_reply(113 gc.10 [call out=15b] v0'0 uv93 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.612+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.612+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:114 6.3 6:c31f41d7:gc::gc.10:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f44200c0060 2024-04-27T03:33:27.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.612+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 29 ==== osd_op_reply(114 gc.10 [call out=24b] v0'0 uv93 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.612+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:115 6.3 6:c31f41d7:gc::gc.10:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.613+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 30 ==== osd_op_reply(115 gc.10 [call out=15b] v0'0 uv93 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.613+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.619 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.613+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:116 6.3 6:c31f41d7:gc::gc.10:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.613+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 31 ==== osd_op_reply(116 gc.10 [call out=15b] v0'0 uv93 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.613+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.620 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.613+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:117 6.3 6:c31f41d7:gc::gc.10:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.621 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.614+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 32 ==== osd_op_reply(117 gc.10 [call] v27'94 uv94 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.614+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2024-04-27T03:33:27.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.614+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:118 6.13 6:c8333aab:gc::gc.11:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.615+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 28 ==== osd_op_reply(118 gc.11 [call] v27'42 uv42 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.615+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:119 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.615+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 29 ==== osd_op_reply(119 gc.11 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.615+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.615+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:120 6.13 6:c8333aab:gc::gc.11:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f4428016440 2024-04-27T03:33:27.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.615+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 30 ==== osd_op_reply(120 gc.11 [call out=24b] v0'0 uv42 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.615+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:121 6.13 6:c8333aab:gc::gc.11:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.616+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 31 ==== osd_op_reply(121 gc.11 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.622 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.616+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.616+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:122 6.13 6:c8333aab:gc::gc.11:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.616+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 32 ==== osd_op_reply(122 gc.11 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.616+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.623 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.616+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:123 6.13 6:c8333aab:gc::gc.11:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.617+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 33 ==== osd_op_reply(123 gc.11 [call] v27'43 uv43 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.617+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2024-04-27T03:33:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.617+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:124 6.15 6:ab569f81:gc::gc.12:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.618+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 33 ==== osd_op_reply(124 gc.12 [call] v27'38 uv38 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.618+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:125 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.618+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 34 ==== osd_op_reply(125 gc.12 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.618+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.618+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:126 6.15 6:ab569f81:gc::gc.12:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f44200c0060 2024-04-27T03:33:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.619+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 35 ==== osd_op_reply(126 gc.12 [call out=24b] v0'0 uv38 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.619+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:127 6.15 6:ab569f81:gc::gc.12:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.625 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.619+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 36 ==== osd_op_reply(127 gc.12 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.619+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.619+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:128 6.15 6:ab569f81:gc::gc.12:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.619+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 37 ==== osd_op_reply(128 gc.12 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.619+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.626 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.619+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:129 6.15 6:ab569f81:gc::gc.12:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.620+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 38 ==== osd_op_reply(129 gc.12 [call] v27'39 uv39 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.620+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2024-04-27T03:33:27.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.620+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:130 6.0 6:0667bb13:gc::gc.13:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.621+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 34 ==== osd_op_reply(130 gc.13 [call] v27'76 uv76 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.627 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.621+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:131 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.622+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 35 ==== osd_op_reply(131 gc.13 [call out=15b] v0'0 uv76 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.628 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.622+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.622+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:132 6.0 6:0667bb13:gc::gc.13:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f4428016440 2024-04-27T03:33:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.622+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 36 ==== osd_op_reply(132 gc.13 [call out=24b] v0'0 uv76 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.622+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:133 6.0 6:0667bb13:gc::gc.13:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.622+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 37 ==== osd_op_reply(133 gc.13 [call out=15b] v0'0 uv76 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.622+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.622+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:134 6.0 6:0667bb13:gc::gc.13:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.622+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 38 ==== osd_op_reply(134 gc.13 [call out=15b] v0'0 uv76 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.622+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.629 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.622+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:135 6.0 6:0667bb13:gc::gc.13:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.624+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 39 ==== osd_op_reply(135 gc.13 [call] v27'77 uv77 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.624+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2024-04-27T03:33:27.630 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.624+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:136 6.4 6:227f63b5:gc::gc.14:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.625+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 27 ==== osd_op_reply(136 gc.14 [call] v27'284 uv284 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.631 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.625+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:137 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.625+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 28 ==== osd_op_reply(137 gc.14 [call out=15b] v0'0 uv284 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.625+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.625+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:138 6.4 6:227f63b5:gc::gc.14:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc81590 2024-04-27T03:33:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.625+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 29 ==== osd_op_reply(138 gc.14 [call out=24b] v0'0 uv284 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.625+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:139 6.4 6:227f63b5:gc::gc.14:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.625+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 30 ==== osd_op_reply(139 gc.14 [call out=15b] v0'0 uv284 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.625+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.625+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:140 6.4 6:227f63b5:gc::gc.14:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.626+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 31 ==== osd_op_reply(140 gc.14 [call out=15b] v0'0 uv284 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.626+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.632 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.626+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:141 6.4 6:227f63b5:gc::gc.14:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.626+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 32 ==== osd_op_reply(141 gc.14 [call] v27'285 uv285 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.626+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2024-04-27T03:33:27.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.626+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:142 6.7 6:e07a6548:gc::gc.15:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.634 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.628+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 33 ==== osd_op_reply(142 gc.15 [call] v27'87 uv87 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.628+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:143 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.628+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 34 ==== osd_op_reply(143 gc.15 [call out=15b] v0'0 uv87 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.628+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.628+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:144 6.7 6:e07a6548:gc::gc.15:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc85f80 2024-04-27T03:33:27.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.628+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 35 ==== osd_op_reply(144 gc.15 [call out=24b] v0'0 uv87 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.628+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:145 6.7 6:e07a6548:gc::gc.15:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.628+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 36 ==== osd_op_reply(145 gc.15 [call out=15b] v0'0 uv87 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.628+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.628+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:146 6.7 6:e07a6548:gc::gc.15:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.629+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 37 ==== osd_op_reply(146 gc.15 [call out=15b] v0'0 uv87 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.629+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.635 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.629+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:147 6.7 6:e07a6548:gc::gc.15:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.630+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 38 ==== osd_op_reply(147 gc.15 [call] v27'88 uv88 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.630+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2024-04-27T03:33:27.637 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.630+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:148 6.3 6:c623b173:gc::gc.16:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.631+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 39 ==== osd_op_reply(148 gc.16 [call] v27'95 uv95 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.631+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:149 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.631+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 40 ==== osd_op_reply(149 gc.16 [call out=15b] v0'0 uv95 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.631+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.631+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:150 6.3 6:c623b173:gc::gc.16:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f44200c0060 2024-04-27T03:33:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.631+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 41 ==== osd_op_reply(150 gc.16 [call out=24b] v0'0 uv95 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.631+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:151 6.3 6:c623b173:gc::gc.16:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.632+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 42 ==== osd_op_reply(151 gc.16 [call out=15b] v0'0 uv95 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.632+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.632+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:152 6.3 6:c623b173:gc::gc.16:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.632+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 43 ==== osd_op_reply(152 gc.16 [call out=15b] v0'0 uv95 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.638 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.632+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.639 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.632+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:153 6.3 6:c623b173:gc::gc.16:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.633+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 44 ==== osd_op_reply(153 gc.16 [call] v27'96 uv96 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.640 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.633+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.633+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:154 6.16 6:6849fd06:gc::gc.17:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.634+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 39 ==== osd_op_reply(154 gc.17 [call] v27'44 uv44 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.634+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:155 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.634+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 40 ==== osd_op_reply(155 gc.17 [call out=15b] v0'0 uv44 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.634+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.634+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:156 6.16 6:6849fd06:gc::gc.17:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc85f80 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.634+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 41 ==== osd_op_reply(156 gc.17 [call out=24b] v0'0 uv44 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.634+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:157 6.16 6:6849fd06:gc::gc.17:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.635+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 42 ==== osd_op_reply(157 gc.17 [call out=15b] v0'0 uv44 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.635+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.635+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:158 6.16 6:6849fd06:gc::gc.17:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.635+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 43 ==== osd_op_reply(158 gc.17 [call out=15b] v0'0 uv44 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.641 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.635+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.635+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:159 6.16 6:6849fd06:gc::gc.17:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.636+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 44 ==== osd_op_reply(159 gc.17 [call] v27'45 uv45 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.636+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2024-04-27T03:33:27.642 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.636+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:160 6.d 6:b019ea83:gc::gc.18:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.637+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 33 ==== osd_op_reply(160 gc.18 [call] v27'40 uv40 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.637+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:161 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.637+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 34 ==== osd_op_reply(161 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.637+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.637+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:162 6.d 6:b019ea83:gc::gc.18:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc81590 2024-04-27T03:33:27.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.638+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 35 ==== osd_op_reply(162 gc.18 [call out=24b] v0'0 uv40 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.644 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.638+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:163 6.d 6:b019ea83:gc::gc.18:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.638+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 36 ==== osd_op_reply(163 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.638+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.638+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:164 6.d 6:b019ea83:gc::gc.18:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.638+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 37 ==== osd_op_reply(164 gc.18 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.638+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.638+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:165 6.d 6:b019ea83:gc::gc.18:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.639+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 38 ==== osd_op_reply(165 gc.18 [call] v27'41 uv41 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.639+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2024-04-27T03:33:27.645 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.639+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:166 6.4 6:24c8e1ae:gc::gc.19:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.640+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 39 ==== osd_op_reply(166 gc.19 [call] v27'286 uv286 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.640+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:167 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.641+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 40 ==== osd_op_reply(167 gc.19 [call out=15b] v0'0 uv286 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.641+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.641+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:168 6.4 6:24c8e1ae:gc::gc.19:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc81590 2024-04-27T03:33:27.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.641+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 41 ==== osd_op_reply(168 gc.19 [call out=24b] v0'0 uv286 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.641+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:169 6.4 6:24c8e1ae:gc::gc.19:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.641+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 42 ==== osd_op_reply(169 gc.19 [call out=15b] v0'0 uv286 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.641+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.647 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.641+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:170 6.4 6:24c8e1ae:gc::gc.19:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.641+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 43 ==== osd_op_reply(170 gc.19 [call out=15b] v0'0 uv286 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.641+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.653 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.641+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:171 6.4 6:24c8e1ae:gc::gc.19:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.642+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 44 ==== osd_op_reply(171 gc.19 [call] v27'287 uv287 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.642+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.642+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:172 6.7 6:e622f76a:gc::gc.20:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.643+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 45 ==== osd_op_reply(172 gc.20 [call] v27'89 uv89 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.643+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:173 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.643+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 46 ==== osd_op_reply(173 gc.20 [call out=15b] v0'0 uv89 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.643+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.643+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:174 6.7 6:e622f76a:gc::gc.20:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc85f80 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.644+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 47 ==== osd_op_reply(174 gc.20 [call out=24b] v0'0 uv89 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.644+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:175 6.7 6:e622f76a:gc::gc.20:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.644+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 48 ==== osd_op_reply(175 gc.20 [call out=15b] v0'0 uv89 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.644+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.654 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.644+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:176 6.7 6:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.644+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 49 ==== osd_op_reply(176 gc.20 [call out=15b] v0'0 uv89 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.644+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.644+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:177 6.7 6:e622f76a:gc::gc.20:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.645+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 50 ==== osd_op_reply(177 gc.20 [call] v27'90 uv90 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.645+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.645+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:178 6.a 6:5127e642:gc::gc.21:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.646+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 45 ==== osd_op_reply(178 gc.21 [call] v27'48 uv48 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.646+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:179 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.646+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 46 ==== osd_op_reply(179 gc.21 [call out=15b] v0'0 uv48 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.646+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.646+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:180 6.a 6:5127e642:gc::gc.21:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f44200c0060 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.647+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 47 ==== osd_op_reply(180 gc.21 [call out=24b] v0'0 uv48 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4424037040 con 0x7f44200c0060 2024-04-27T03:33:27.655 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.647+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:181 6.a 6:5127e642:gc::gc.21:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.647+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 48 ==== osd_op_reply(181 gc.21 [call out=15b] v0'0 uv48 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.647+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.647+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:182 6.a 6:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.647+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 49 ==== osd_op_reply(182 gc.21 [call out=15b] v0'0 uv48 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.647+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.656 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.647+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:183 6.a 6:5127e642:gc::gc.21:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.648+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 50 ==== osd_op_reply(183 gc.21 [call] v27'49 uv49 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.648+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2024-04-27T03:33:27.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.648+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:184 6.11 6:8eb99c53:gc::gc.22:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.659 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.649+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 51 ==== osd_op_reply(184 gc.22 [call] v27'40 uv40 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.649+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:185 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.649+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 52 ==== osd_op_reply(185 gc.22 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.649+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.649+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:186 6.11 6:8eb99c53:gc::gc.22:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc85f80 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.650+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 53 ==== osd_op_reply(186 gc.22 [call out=24b] v0'0 uv40 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.650+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:187 6.11 6:8eb99c53:gc::gc.22:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.650+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 54 ==== osd_op_reply(187 gc.22 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.650+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.650+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:188 6.11 6:8eb99c53:gc::gc.22:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.650+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 55 ==== osd_op_reply(188 gc.22 [call out=15b] v0'0 uv40 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.650+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.650+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:189 6.11 6:8eb99c53:gc::gc.22:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.660 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.651+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 56 ==== osd_op_reply(189 gc.22 [call] v27'41 uv41 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.651+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.651+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:190 6.8 6:17d7e8ed:gc::gc.23:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.652+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 57 ==== osd_op_reply(190 gc.23 [call] v27'51 uv51 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.652+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:191 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.652+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 58 ==== osd_op_reply(191 gc.23 [call out=15b] v0'0 uv51 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.652+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.652+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:192 6.8 6:17d7e8ed:gc::gc.23:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc85f80 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.652+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 59 ==== osd_op_reply(192 gc.23 [call out=24b] v0'0 uv51 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.652+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:193 6.8 6:17d7e8ed:gc::gc.23:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.653+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 60 ==== osd_op_reply(193 gc.23 [call out=15b] v0'0 uv51 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.653+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.653+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:194 6.8 6:17d7e8ed:gc::gc.23:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.661 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.653+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 61 ==== osd_op_reply(194 gc.23 [call out=15b] v0'0 uv51 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.653+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.653+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:195 6.8 6:17d7e8ed:gc::gc.23:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.654+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 62 ==== osd_op_reply(195 gc.23 [call] v27'52 uv52 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.654+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2024-04-27T03:33:27.662 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.654+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:196 6.1b 6:db9d130d:gc::gc.24:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.655+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 63 ==== osd_op_reply(196 gc.24 [call] v27'34 uv34 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.655+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:197 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.655+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 64 ==== osd_op_reply(197 gc.24 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.655+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.655+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:198 6.1b 6:db9d130d:gc::gc.24:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc85f80 2024-04-27T03:33:27.665 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.655+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 65 ==== osd_op_reply(198 gc.24 [call out=24b] v0'0 uv34 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.655+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:199 6.1b 6:db9d130d:gc::gc.24:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.655+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 66 ==== osd_op_reply(199 gc.24 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.655+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.655+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:200 6.1b 6:db9d130d:gc::gc.24:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.656+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 67 ==== osd_op_reply(200 gc.24 [call out=15b] v0'0 uv34 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.656+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.656+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:201 6.1b 6:db9d130d:gc::gc.24:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.656+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 68 ==== osd_op_reply(201 gc.24 [call] v27'35 uv35 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.656+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2024-04-27T03:33:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.656+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:202 6.9 6:921ecdb1:gc::gc.25:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.657+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 69 ==== osd_op_reply(202 gc.25 [call] v27'71 uv71 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.666 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.657+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:203 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.658+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 70 ==== osd_op_reply(203 gc.25 [call out=15b] v0'0 uv71 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.658+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.658+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:204 6.9 6:921ecdb1:gc::gc.25:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc85f80 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.658+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 71 ==== osd_op_reply(204 gc.25 [call out=24b] v0'0 uv71 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.658+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:205 6.9 6:921ecdb1:gc::gc.25:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.658+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 72 ==== osd_op_reply(205 gc.25 [call out=15b] v0'0 uv71 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.658+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.658+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:206 6.9 6:921ecdb1:gc::gc.25:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.659+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 73 ==== osd_op_reply(206 gc.25 [call out=15b] v0'0 uv71 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.659+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.659+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:207 6.9 6:921ecdb1:gc::gc.25:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.659+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 74 ==== osd_op_reply(207 gc.25 [call] v27'72 uv72 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.667 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.659+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2024-04-27T03:33:27.668 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.659+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:208 6.1f 6:fde4dd55:gc::gc.26:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.661+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 40 ==== osd_op_reply(208 gc.26 [call] v27'42 uv42 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.661+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:209 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.661+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 41 ==== osd_op_reply(209 gc.26 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.661+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.661+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:210 6.1f 6:fde4dd55:gc::gc.26:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f4428016440 2024-04-27T03:33:27.671 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.661+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 42 ==== osd_op_reply(210 gc.26 [call out=24b] v0'0 uv42 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.661+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:211 6.1f 6:fde4dd55:gc::gc.26:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.662+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 43 ==== osd_op_reply(211 gc.26 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.662+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.662+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:212 6.1f 6:fde4dd55:gc::gc.26:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.662+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 44 ==== osd_op_reply(212 gc.26 [call out=15b] v0'0 uv42 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.662+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.662+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:213 6.1f 6:fde4dd55:gc::gc.26:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.663+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 45 ==== osd_op_reply(213 gc.26 [call] v27'43 uv43 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.663+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.663+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:214 6.1a 6:5909e671:gc::gc.27:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.664+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 75 ==== osd_op_reply(214 gc.27 [call] v27'46 uv46 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.664+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:215 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.664+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 76 ==== osd_op_reply(215 gc.27 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.672 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.664+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.664+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:216 6.1a 6:5909e671:gc::gc.27:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc85f80 2024-04-27T03:33:27.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.664+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 77 ==== osd_op_reply(216 gc.27 [call out=24b] v0'0 uv46 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.664+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:217 6.1a 6:5909e671:gc::gc.27:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.665+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 78 ==== osd_op_reply(217 gc.27 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.665+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.665+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:218 6.1a 6:5909e671:gc::gc.27:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.665+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 79 ==== osd_op_reply(218 gc.27 [call out=15b] v0'0 uv46 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.665+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.665+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:219 6.1a 6:5909e671:gc::gc.27:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.666+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 80 ==== osd_op_reply(219 gc.27 [call] v27'47 uv47 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.666+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2024-04-27T03:33:27.673 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.666+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:220 6.1d 6:b88e3271:gc::gc.28:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.667+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 46 ==== osd_op_reply(220 gc.28 [call] v27'65 uv65 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.667+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:221 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.667+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 47 ==== osd_op_reply(221 gc.28 [call out=15b] v0'0 uv65 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.667+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.667+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:222 6.1d 6:b88e3271:gc::gc.28:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f4428016440 2024-04-27T03:33:27.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.667+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 48 ==== osd_op_reply(222 gc.28 [call out=24b] v0'0 uv65 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.667+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:223 6.1d 6:b88e3271:gc::gc.28:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.675 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.668+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 49 ==== osd_op_reply(223 gc.28 [call out=15b] v0'0 uv65 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.668+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.668+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:224 6.1d 6:b88e3271:gc::gc.28:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.668+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 50 ==== osd_op_reply(224 gc.28 [call out=15b] v0'0 uv65 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.668+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.668+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:225 6.1d 6:b88e3271:gc::gc.28:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.669+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 51 ==== osd_op_reply(225 gc.28 [call] v27'66 uv66 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.669+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2024-04-27T03:33:27.676 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.669+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:226 6.7 6:e47c9322:gc::gc.29:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.670+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 81 ==== osd_op_reply(226 gc.29 [call] v27'91 uv91 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.670+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:227 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.671+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 82 ==== osd_op_reply(227 gc.29 [call out=15b] v0'0 uv91 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.671+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.671+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:228 6.7 6:e47c9322:gc::gc.29:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc85f80 2024-04-27T03:33:27.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.671+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 83 ==== osd_op_reply(228 gc.29 [call out=24b] v0'0 uv91 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.671+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:229 6.7 6:e47c9322:gc::gc.29:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.678 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.671+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 84 ==== osd_op_reply(229 gc.29 [call out=15b] v0'0 uv91 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.671+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.671+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:230 6.7 6:e47c9322:gc::gc.29:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.671+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 85 ==== osd_op_reply(230 gc.29 [call out=15b] v0'0 uv91 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.671+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.671+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:231 6.7 6:e47c9322:gc::gc.29:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc85f80 2024-04-27T03:33:27.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.672+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 86 ==== osd_op_reply(231 gc.29 [call] v27'92 uv92 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.672+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2024-04-27T03:33:27.679 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.672+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:232 6.0 6:013d6c33:gc::gc.30:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.673+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 52 ==== osd_op_reply(232 gc.30 [call] v27'78 uv78 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.673+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:233 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.674+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 53 ==== osd_op_reply(233 gc.30 [call out=15b] v0'0 uv78 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.674+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.674+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:234 6.0 6:013d6c33:gc::gc.30:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f4428016440 2024-04-27T03:33:27.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.674+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 54 ==== osd_op_reply(234 gc.30 [call out=24b] v0'0 uv78 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.674+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:235 6.0 6:013d6c33:gc::gc.30:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.674+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 55 ==== osd_op_reply(235 gc.30 [call out=15b] v0'0 uv78 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.674+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.674+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:236 6.0 6:013d6c33:gc::gc.30:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.681 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.674+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 56 ==== osd_op_reply(236 gc.30 [call out=15b] v0'0 uv78 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.675+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.682 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.675+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:237 6.0 6:013d6c33:gc::gc.30:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.683 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.676+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 57 ==== osd_op_reply(237 gc.30 [call] v27'79 uv79 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.676+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.676+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:238 6.f 6:f24cb4a8:gc::gc.31:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.677+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 51 ==== osd_op_reply(238 gc.31 [call] v27'38 uv38 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.677+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:239 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.677+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 52 ==== osd_op_reply(239 gc.31 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.677+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.677+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:240 6.f 6:f24cb4a8:gc::gc.31:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f44200c0060 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.677+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 53 ==== osd_op_reply(240 gc.31 [call out=24b] v0'0 uv38 ondisk = 0) ==== 149+0+24 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.677+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:241 6.f 6:f24cb4a8:gc::gc.31:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.678+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 54 ==== osd_op_reply(241 gc.31 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.678+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.678+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:242 6.f 6:f24cb4a8:gc::gc.31:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.684 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.678+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 55 ==== osd_op_reply(242 gc.31 [call out=15b] v0'0 uv38 ondisk = 0) ==== 149+0+15 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.678+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.678+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:243 6.f 6:f24cb4a8:gc::gc.31:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f44200c0060 2024-04-27T03:33:27.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.679+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 56 ==== osd_op_reply(243 gc.31 [call] v27'39 uv39 ondisk = 0) ==== 149+0+0 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.679+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2024-04-27T03:33:27.685 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.679+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:244 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.680+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 45 ==== osd_op_reply(244 gc.0 [call] v27'62 uv62 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.680+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:245 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.681+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 46 ==== osd_op_reply(245 gc.0 [call out=15b] v0'0 uv62 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.681+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.681+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:246 6.19 6:9a6dd8c0:gc::gc.0:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x560dedc81590 2024-04-27T03:33:27.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.681+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 47 ==== osd_op_reply(246 gc.0 [call out=24b] v0'0 uv62 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.687 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.681+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:247 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.681+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 48 ==== osd_op_reply(247 gc.0 [call out=15b] v0'0 uv62 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.681+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.681+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:248 6.19 6:9a6dd8c0:gc::gc.0:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.681+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 49 ==== osd_op_reply(248 gc.0 [call out=15b] v0'0 uv62 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.681+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.688 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.681+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:249 6.19 6:9a6dd8c0:gc::gc.0:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x560dedc81590 2024-04-27T03:33:27.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.682+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 50 ==== osd_op_reply(249 gc.0 [call] v27'63 uv63 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.682+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2024-04-27T03:33:27.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.682+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:250 6.18 6:1e85421f:gc::gc.1:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.683+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 58 ==== osd_op_reply(250 gc.1 [call] v27'40 uv40 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.683+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:251 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.684+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 59 ==== osd_op_reply(251 gc.1 [call out=15b] v0'0 uv40 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.691 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.684+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.684+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:252 6.18 6:1e85421f:gc::gc.1:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccbc10 con 0x7f4428016440 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.684+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 60 ==== osd_op_reply(252 gc.1 [call out=24b] v0'0 uv40 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.684+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:253 6.18 6:1e85421f:gc::gc.1:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.684+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 61 ==== osd_op_reply(253 gc.1 [call out=15b] v0'0 uv40 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.684+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.684+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:254 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcccc20 con 0x7f4428016440 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.684+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 62 ==== osd_op_reply(254 gc.1 [call out=1716b] v0'0 uv40 ondisk = 0) ==== 148+0+1716 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 20 garbage collection: RGWGC::process iterating over entry tag='708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.17022807051448174797', time=2024-04-27T03:33:35.661308+0000, chain.objs.size()=7 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_1 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:255 3.36 3:6c41ab8c:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_1:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedccd240 con 0x7f44200c0060 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_2 2024-04-27T03:33:27.692 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:256 3.23 3:c668c9ef:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_2:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedccfe80 con 0x7f4428016440 2024-04-27T03:33:27.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_3 2024-04-27T03:33:27.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:257 3.4 3:21073cb7:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_3:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd27b0 con 0x7f4428016440 2024-04-27T03:33:27.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_4 2024-04-27T03:33:27.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:258 3.3c 3:3f0ee4e0:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_4:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd5080 con 0x7f44200c0060 2024-04-27T03:33:27.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_5 2024-04-27T03:33:27.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:259 3.1e 3:7b7f5fa2:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_5:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd78f0 con 0x7f4428016440 2024-04-27T03:33:27.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_6 2024-04-27T03:33:27.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:260 3.2f 3:f4bd7174:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_6:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcda160 con 0x560dedc85f80 2024-04-27T03:33:27.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_7 2024-04-27T03:33:27.693 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.685+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:261 3.1b 3:d9b1fc06:::708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_7:head [call refcount.put in=85b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd78f0 con 0x560dedc85f80 2024-04-27T03:33:27.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.687+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 87 ==== osd_op_reply(261 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_7 [call] v27'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.687+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 88 ==== osd_op_reply(260 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_6 [call] v27'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.687+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 57 ==== osd_op_reply(258 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_4 [call] v27'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.688+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 63 ==== osd_op_reply(259 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_5 [call] v27'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.688+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 64 ==== osd_op_reply(257 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_3 [call] v27'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.688+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 58 ==== osd_op_reply(255 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_1 [call] v27'2 uv2 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.688+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 65 ==== osd_op_reply(256 708f831a-84ca-40fe-a0c7-f3b916a7f9aa.4217.11__shadow_.OuQn4bcH_7Lbt7tJsbQP80kjxsBScf1_2 [call] v27'4 uv4 ondisk = 0) ==== 231+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.688+0000 7f4434933800 5 garbage collection: RGWGC::process removing entries, marker: 2024-04-27T03:33:27.695 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.688+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:262 6.18 6:1e85421f:gc::gc.1:head [call rgw_gc.rgw_gc_queue_remove_entries in=47b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcda160 con 0x7f4428016440 2024-04-27T03:33:27.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.689+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 66 ==== osd_op_reply(262 gc.1 [call] v27'41 uv41 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.696 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.689+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:263 6.18 6:1e85421f:gc::gc.1:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2b50 con 0x7f4428016440 2024-04-27T03:33:27.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.690+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 67 ==== osd_op_reply(263 gc.1 [call] v27'42 uv42 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.690+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2024-04-27T03:33:27.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.690+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:264 6.b 6:d242335b:gc::gc.2:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2b50 con 0x560dedc85f80 2024-04-27T03:33:27.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.691+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 89 ==== osd_op_reply(264 gc.2 [call] v27'41 uv41 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.691+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:265 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2b50 con 0x560dedc85f80 2024-04-27T03:33:27.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.691+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 90 ==== osd_op_reply(265 gc.2 [call out=15b] v0'0 uv41 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.691+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.691+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:266 6.b 6:d242335b:gc::gc.2:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x560dedc85f80 2024-04-27T03:33:27.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.691+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 91 ==== osd_op_reply(266 gc.2 [call out=24b] v0'0 uv41 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.691+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:267 6.b 6:d242335b:gc::gc.2:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x560dedc85f80 2024-04-27T03:33:27.698 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.692+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 92 ==== osd_op_reply(267 gc.2 [call out=15b] v0'0 uv41 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.692+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.692+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:268 6.b 6:d242335b:gc::gc.2:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x560dedc85f80 2024-04-27T03:33:27.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.692+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 93 ==== osd_op_reply(268 gc.2 [call out=15b] v0'0 uv41 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.692+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.699 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.692+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:269 6.b 6:d242335b:gc::gc.2:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x560dedc85f80 2024-04-27T03:33:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.693+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 94 ==== osd_op_reply(269 gc.2 [call] v27'42 uv42 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.693+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2024-04-27T03:33:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.693+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:270 6.4 6:2166d8e1:gc::gc.3:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x560dedc81590 2024-04-27T03:33:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.694+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 51 ==== osd_op_reply(270 gc.3 [call] v27'288 uv288 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.694+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:271 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x560dedc81590 2024-04-27T03:33:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.694+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 52 ==== osd_op_reply(271 gc.3 [call out=15b] v0'0 uv288 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.694+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.694+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:272 6.4 6:2166d8e1:gc::gc.3:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x560dedc81590 2024-04-27T03:33:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.694+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 53 ==== osd_op_reply(272 gc.3 [call out=24b] v0'0 uv288 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.701 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.694+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:273 6.4 6:2166d8e1:gc::gc.3:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x560dedc81590 2024-04-27T03:33:27.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.695+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 54 ==== osd_op_reply(273 gc.3 [call out=15b] v0'0 uv288 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.695+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.695+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:274 6.4 6:2166d8e1:gc::gc.3:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x560dedc81590 2024-04-27T03:33:27.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.695+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 55 ==== osd_op_reply(274 gc.3 [call out=15b] v0'0 uv288 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.695+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.702 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.695+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:275 6.4 6:2166d8e1:gc::gc.3:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x560dedc81590 2024-04-27T03:33:27.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.696+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 56 ==== osd_op_reply(275 gc.3 [call] v27'289 uv289 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.696+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2024-04-27T03:33:27.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.696+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:276 6.3 6:c60da066:gc::gc.4:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x7f44200c0060 2024-04-27T03:33:27.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.697+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 59 ==== osd_op_reply(276 gc.4 [call] v27'97 uv97 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.704 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.697+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:277 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x7f44200c0060 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.697+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 60 ==== osd_op_reply(277 gc.4 [call out=15b] v0'0 uv97 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.697+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.697+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:278 6.3 6:c60da066:gc::gc.4:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x7f44200c0060 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.697+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 61 ==== osd_op_reply(278 gc.4 [call out=24b] v0'0 uv97 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.697+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:279 6.3 6:c60da066:gc::gc.4:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x7f44200c0060 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.698+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 62 ==== osd_op_reply(279 gc.4 [call out=15b] v0'0 uv97 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.698+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.698+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:280 6.3 6:c60da066:gc::gc.4:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x7f44200c0060 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.698+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 63 ==== osd_op_reply(280 gc.4 [call out=15b] v0'0 uv97 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.698+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.698+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:281 6.3 6:c60da066:gc::gc.4:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x7f44200c0060 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.699+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 64 ==== osd_op_reply(281 gc.4 [call] v27'98 uv98 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.705 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.699+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2024-04-27T03:33:27.706 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.699+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:282 6.9 6:954be8d2:gc::gc.5:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd2ac0 con 0x560dedc85f80 2024-04-27T03:33:27.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.700+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 95 ==== osd_op_reply(282 gc.5 [call] v27'73 uv73 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.707 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.700+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:283 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x560dedc85f80 2024-04-27T03:33:27.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.700+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 96 ==== osd_op_reply(283 gc.5 [call out=15b] v0'0 uv73 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.700+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.700+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:284 6.9 6:954be8d2:gc::gc.5:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedcd1780 con 0x560dedc85f80 2024-04-27T03:33:27.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.700+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 97 ==== osd_op_reply(284 gc.5 [call out=24b] v0'0 uv73 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.700+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:285 6.9 6:954be8d2:gc::gc.5:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x560dedc85f80 2024-04-27T03:33:27.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.701+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 98 ==== osd_op_reply(285 gc.5 [call out=15b] v0'0 uv73 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.701+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.701+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:286 6.9 6:954be8d2:gc::gc.5:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x560dedc85f80 2024-04-27T03:33:27.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.701+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 99 ==== osd_op_reply(286 gc.5 [call out=15b] v0'0 uv73 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.701+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.701+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:287 6.9 6:954be8d2:gc::gc.5:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x560dedc85f80 2024-04-27T03:33:27.708 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.702+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 100 ==== osd_op_reply(287 gc.5 [call] v27'74 uv74 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.702+0000 7f4434933800 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2024-04-27T03:33:27.709 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.702+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:288 6.e 6:748722ff:gc::gc.6:head [call lock.lock in=50b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x560dedc81590 2024-04-27T03:33:27.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.703+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 57 ==== osd_op_reply(288 gc.6 [call] v27'40 uv40 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.703+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:289 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x560dedc81590 2024-04-27T03:33:27.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.704+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 58 ==== osd_op_reply(289 gc.6 [call out=15b] v0'0 uv40 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.710 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.704+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.704+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:290 6.e 6:748722ff:gc::gc.6:head [call version.read in=11b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x560dedc81590 2024-04-27T03:33:27.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.704+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 59 ==== osd_op_reply(290 gc.6 [call out=24b] v0'0 uv40 ondisk = 0) ==== 148+0+24 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.704+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:291 6.e 6:748722ff:gc::gc.6:head [call rgw.gc_list in=25b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x560dedc81590 2024-04-27T03:33:27.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.704+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 60 ==== osd_op_reply(291 gc.6 [call out=15b] v0'0 uv40 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.704+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_list returned NO non expired entries, so setting cache entry to TRUE 2024-04-27T03:33:27.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.704+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:292 6.e 6:748722ff:gc::gc.6:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x560dedc81590 2024-04-27T03:33:27.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.705+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 61 ==== osd_op_reply(292 gc.6 [call out=15b] v0'0 uv40 ondisk = 0) ==== 148+0+15 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.705+0000 7f4434933800 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2024-04-27T03:33:27.711 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.705+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:293 6.e 6:748722ff:gc::gc.6:head [call lock.unlock in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x560dedc81590 2024-04-27T03:33:27.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.706+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 62 ==== osd_op_reply(293 gc.6 [call] v27'41 uv41 ondisk = 0) ==== 148+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.707+0000 7f42e67e4640 2 rgw data changes log: RGWDataChangesLog::ChangesRenewThread: start 2024-04-27T03:33:27.712 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.707+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:294 6.4 6:22d26bf9:::data_loggenerations_metadata:head [watch unwatch cookie 94617823942272] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x560dedc81590 2024-04-27T03:33:27.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.708+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 63 ==== osd_op_reply(294 data_loggenerations_metadata [watch unwatch cookie 94617823942272] v27'290 uv1 ondisk = 0) ==== 172+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.713 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.708+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:295 7.1f 7:f95f44c2:::notify.0:head [watch unwatch cookie 94617823917392] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x7f44200c0060 2024-04-27T03:33:27.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.709+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 65 ==== osd_op_reply(295 notify.0 [watch unwatch cookie 94617823917392] v27'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.709+0000 7f4434933800 20 remove_watcher() i=0 2024-04-27T03:33:27.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.709+0000 7f4434933800 2 removed watcher, disabling cache 2024-04-27T03:33:27.714 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.709+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:296 7.0 7:05bf5b68:::notify.1:head [watch unwatch cookie 94617823924384] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedccad70 con 0x560dedc81590 2024-04-27T03:33:27.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.710+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 64 ==== osd_op_reply(296 notify.1 [watch unwatch cookie 94617823924384] v27'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.715 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.710+0000 7f4434933800 20 remove_watcher() i=1 2024-04-27T03:33:27.716 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.710+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- osd_op(unknown.0.0:297 7.15 7:a93a5511:::notify.2:head [watch unwatch cookie 94617823927984] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8d0a0 con 0x560dedc85f80 2024-04-27T03:33:27.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.711+0000 7f44305d1640 1 -- 172.21.15.165:0/3080026603 <== osd.3 v2:172.21.15.165:6808/2380918637 101 ==== osd_op_reply(297 notify.2 [watch unwatch cookie 94617823927984] v27'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f44200cf040 con 0x560dedc85f80 2024-04-27T03:33:27.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.711+0000 7f4434933800 20 remove_watcher() i=2 2024-04-27T03:33:27.717 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.711+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- osd_op(unknown.0.0:298 7.e 7:7759931f:::notify.3:head [watch unwatch cookie 94617823933792] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8deb0 con 0x7f44200c0060 2024-04-27T03:33:27.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.712+0000 7f442fdd0640 1 -- 172.21.15.165:0/3080026603 <== osd.2 v2:172.21.15.165:6816/975457786 66 ==== osd_op_reply(298 notify.3 [watch unwatch cookie 94617823933792] v27'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4424055040 con 0x7f44200c0060 2024-04-27T03:33:27.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.712+0000 7f4434933800 20 remove_watcher() i=3 2024-04-27T03:33:27.718 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.712+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- osd_op(unknown.0.0:299 7.d 7:b4812045:::notify.4:head [watch unwatch cookie 94617823937344] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8d0a0 con 0x560dedc81590 2024-04-27T03:33:27.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.713+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.1 v2:172.21.15.165:6800/79879041 65 ==== osd_op_reply(299 notify.4 [watch unwatch cookie 94617823937344] v27'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x560dedc81590 2024-04-27T03:33:27.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.713+0000 7f4434933800 20 remove_watcher() i=4 2024-04-27T03:33:27.719 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.713+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:300 7.3 7:c609908c:::notify.5:head [watch unwatch cookie 94617823919600] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8f560 con 0x7f4428016440 2024-04-27T03:33:27.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.714+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 68 ==== osd_op_reply(300 notify.5 [watch unwatch cookie 94617823919600] v27'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.714+0000 7f4434933800 20 remove_watcher() i=5 2024-04-27T03:33:27.720 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.714+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:301 7.14 7:2b04a3e9:::notify.6:head [watch unwatch cookie 94617823941040] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc90340 con 0x7f4428016440 2024-04-27T03:33:27.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.715+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 69 ==== osd_op_reply(301 notify.6 [watch unwatch cookie 94617823941040] v27'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.715+0000 7f4434933800 20 remove_watcher() i=6 2024-04-27T03:33:27.721 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.715+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- osd_op(unknown.0.0:302 7.9 7:93e5b521:::notify.7:head [watch unwatch cookie 94617823944896] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e27) -- 0x560dedc8d0a0 con 0x7f4428016440 2024-04-27T03:33:27.724 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.716+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 <== osd.0 v2:172.21.15.89:6800/2353668432 70 ==== osd_op_reply(302 notify.7 [watch unwatch cookie 94617823944896] v27'285 uv283 ondisk = 0) ==== 152+0+0 (crc 0 0 0) 0x7f4428044040 con 0x7f4428016440 2024-04-27T03:33:27.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.716+0000 7f4434933800 20 remove_watcher() i=7 2024-04-27T03:33:27.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4428016440 msgr2=0x7f44280168c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7f4428016440 0x7f44280168c0 crc :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560dedc81590 msgr2=0x560dedc83a50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560dedc81590 0x560dedc83a50 crc :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f44200c0060 msgr2=0x7f44200ab790 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f44200c0060 0x7f44200ab790 crc :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560dedc85f80 msgr2=0x560dedc88370 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560dedc85f80 0x560dedc88370 crc :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4300036900 msgr2=0x7f4300038dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4300036900 0x7f4300038dc0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x560dedc538e0 tx=0x7f4424010000 comp rx=0 tx=0).stop 2024-04-27T03:33:27.725 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 msgr2=0x560dedc75f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4430dd2640 1 -- 172.21.15.165:0/3080026603 reap_dead start 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 0x560dedc75f20 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f44200524f0 tx=0x7f442004fea0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 shutdown_connections 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 --2- 172.21.15.165:0/3080026603 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc6ead0 0x560dedc75f20 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 >> 172.21.15.165:0/3080026603 conn(0x560dedc6ade0 msgr2=0x560dedc68440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 shutdown_connections 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/3080026603 wait complete. 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/755672737 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4308036950 msgr2=0x7f4308038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 --2- 172.21.15.165:0/755672737 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4308036950 0x7f4308038e10 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x560dedc4af50 tx=0x7f442400f000 comp rx=0 tx=0).stop 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/755672737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc61940 msgr2=0x560dedc61d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 --2- 172.21.15.165:0/755672737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc61940 0x560dedc61d60 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f4420066560 tx=0x7f4420006650 comp rx=0 tx=0).stop 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/755672737 shutdown_connections 2024-04-27T03:33:27.726 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 --2- 172.21.15.165:0/755672737 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4308036950 0x7f4308038e10 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 --2- 172.21.15.165:0/755672737 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedc61940 0x560dedc61d60 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/755672737 >> 172.21.15.165:0/755672737 conn(0x560dedc559e0 msgr2=0x560dedc55de0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/755672737 shutdown_connections 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.717+0000 7f4434933800 1 -- 172.21.15.165:0/755672737 wait complete. 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560dedb470e0 msgr2=0x560dedc4a070 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x560dedb470e0 0x560dedc4a070 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x560dedc50930 msgr2=0x560dedc52d80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x560dedc50930 0x560dedc52d80 crc :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560dedc4c9a0 msgr2=0x560dedc4ed90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x560dedc4c9a0 0x560dedc4ed90 crc :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4318036950 msgr2=0x7f4318038e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4318036950 0x7f4318038e10 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f4424000f60 tx=0x7f4424001370 comp rx=0 tx=0).stop 2024-04-27T03:33:27.727 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 msgr2=0x560dedc42ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 --2- 172.21.15.165:0/964831903 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x560dedb46780 0x560dedc42ed0 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7f442001cfe0 tx=0x7f4420003020 comp rx=0 tx=0).stop 2024-04-27T03:33:27.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4430dd2640 1 -- 172.21.15.165:0/964831903 reap_dead start 2024-04-27T03:33:27.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 shutdown_connections 2024-04-27T03:33:27.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 >> 172.21.15.165:0/964831903 conn(0x560dedb42300 msgr2=0x560dedc354d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:27.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 shutdown_connections 2024-04-27T03:33:27.728 INFO:teuthology.orchestra.run.smithi165.stderr:2024-04-27T03:33:27.718+0000 7f4434933800 1 -- 172.21.15.165:0/964831903 wait complete. 2024-04-27T03:33:27.732 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2024-04-27T03:33:27.741 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2024-04-27T03:33:27.742 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T03:33:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.835+0000 7efd97759640 1 Processor -- start 2024-04-27T03:33:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.835+0000 7efd97759640 1 -- start start 2024-04-27T03:33:27.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.835+0000 7efd97759640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efd90071090 0x7efd90131430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.835+0000 7efd97759640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7efd90098800 con 0x7efd90096430 2024-04-27T03:33:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.835+0000 7efd97759640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7efd90071930 con 0x7efd90071090 2024-04-27T03:33:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.836+0000 7efd94cf4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7efd90071090 0x7efd90131430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.836+0000 7efd954f5640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7efd90096430 0x7efd90096810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:47994/0 (socket says 172.21.15.89:47994) 2024-04-27T03:33:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.836+0000 7efd954f5640 1 -- 172.21.15.89:0/2861356317 learned_addr learned my addr 172.21.15.89:0/2861356317 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.836+0000 7efd94cf4640 1 -- 172.21.15.89:0/2861356317 >> v1:172.21.15.89:6789/0 conn(0x7efd90096430 legacy=0x7efd90096810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.836+0000 7efd94cf4640 1 -- 172.21.15.89:0/2861356317 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd90071f50 con 0x7efd90071090 2024-04-27T03:33:27.842 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.836+0000 7efd87fff640 1 -- 172.21.15.89:0/2861356317 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 2056106107 0 0) 0x7efd90098800 con 0x7efd90096430 2024-04-27T03:33:27.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.837+0000 7efd94cf4640 1 --2- 172.21.15.89:0/2861356317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd90071090 0x7efd90131430 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7efd80009720 tx=0x7efd8002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd79818a126c7325 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.837+0000 7efd87fff640 1 -- 172.21.15.89:0/2861356317 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd800316d0 con 0x7efd90071090 2024-04-27T03:33:27.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.837+0000 7efd87fff640 1 -- 172.21.15.89:0/2861356317 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd80037070 con 0x7efd90071090 2024-04-27T03:33:27.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.837+0000 7efd87fff640 1 -- 172.21.15.89:0/2861356317 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd800319c0 con 0x7efd90071090 2024-04-27T03:33:27.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.837+0000 7efd97759640 1 -- 172.21.15.89:0/2861356317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd90071090 msgr2=0x7efd90131430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:27.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.837+0000 7efd97759640 1 --2- 172.21.15.89:0/2861356317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd90071090 0x7efd90131430 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7efd80009720 tx=0x7efd8002b840 comp rx=0 tx=0).stop 2024-04-27T03:33:27.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.837+0000 7efd97759640 1 -- 172.21.15.89:0/2861356317 shutdown_connections 2024-04-27T03:33:27.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.837+0000 7efd97759640 1 --2- 172.21.15.89:0/2861356317 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd90071090 0x7efd90131430 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:27.845 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.837+0000 7efd97759640 1 -- 172.21.15.89:0/2861356317 >> 172.21.15.89:0/2861356317 conn(0x7efd90099680 msgr2=0x7efd90099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.837+0000 7efd97759640 1 -- 172.21.15.89:0/2861356317 shutdown_connections 2024-04-27T03:33:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.837+0000 7efd97759640 1 -- 172.21.15.89:0/2861356317 wait complete. 2024-04-27T03:33:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.838+0000 7efd97759640 1 Processor -- start 2024-04-27T03:33:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.838+0000 7efd97759640 1 -- start start 2024-04-27T03:33:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.839+0000 7efd97759640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd90071090 0x7efd90121a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.839+0000 7efd97759640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7efd90131af0 con 0x7efd90071090 2024-04-27T03:33:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.839+0000 7efd954f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd90071090 0x7efd90121a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.839+0000 7efd954f5640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd90071090 0x7efd90121a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60184/0 (socket says 172.21.15.89:60184) 2024-04-27T03:33:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.839+0000 7efd954f5640 1 -- 172.21.15.89:0/618360838 learned_addr learned my addr 172.21.15.89:0/618360838 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.839+0000 7efd954f5640 1 -- 172.21.15.89:0/618360838 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd90122f30 con 0x7efd90071090 2024-04-27T03:33:27.846 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.839+0000 7efd954f5640 1 --2- 172.21.15.89:0/618360838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd90071090 0x7efd90121a90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c420 tx=0x7efd8c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.839+0000 7efd85ffb640 1 -- 172.21.15.89:0/618360838 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd8c018070 con 0x7efd90071090 2024-04-27T03:33:27.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.839+0000 7efd85ffb640 1 -- 172.21.15.89:0/618360838 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7efd8c0074d0 con 0x7efd90071090 2024-04-27T03:33:27.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.839+0000 7efd97759640 1 -- 172.21.15.89:0/618360838 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd901237c0 con 0x7efd90071090 2024-04-27T03:33:27.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.839+0000 7efd85ffb640 1 -- 172.21.15.89:0/618360838 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd8c004950 con 0x7efd90071090 2024-04-27T03:33:27.847 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.839+0000 7efd97759640 1 -- 172.21.15.89:0/618360838 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd901221b0 con 0x7efd90071090 2024-04-27T03:33:27.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.840+0000 7efd85ffb640 1 -- 172.21.15.89:0/618360838 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7efd8c01b070 con 0x7efd90071090 2024-04-27T03:33:27.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.840+0000 7efd85ffb640 1 --2- 172.21.15.89:0/618360838 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efd74038060 0x7efd7403a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:27.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.840+0000 7efd85ffb640 1 -- 172.21.15.89:0/618360838 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7efd8c04ac40 con 0x7efd90071090 2024-04-27T03:33:27.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.840+0000 7efd97759640 1 -- 172.21.15.89:0/618360838 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd90062da0 con 0x7efd90071090 2024-04-27T03:33:27.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.840+0000 7efd94cf4640 1 --2- 172.21.15.89:0/618360838 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efd74038060 0x7efd7403a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:27.848 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.841+0000 7efd94cf4640 1 --2- 172.21.15.89:0/618360838 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efd74038060 0x7efd7403a520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efd80036000 tx=0x7efd80008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:27.850 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:27.845+0000 7efd85ffb640 1 -- 172.21.15.89:0/618360838 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7efd8c011960 con 0x7efd90071090 2024-04-27T03:33:28.017 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.011+0000 7efd97759640 1 -- 172.21.15.89:0/618360838 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd9006c170 con 0x7efd74038060 2024-04-27T03:33:28.032 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.027+0000 7efd85ffb640 1 -- 172.21.15.89:0/618360838 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665463 (secure 0 0 0) 0x7efd9006c170 con 0x7efd74038060 2024-04-27T03:33:28.042 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:33:28.048 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-04-27T03:33:28.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.031+0000 7efd97759640 1 -- 172.21.15.89:0/618360838 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efd74038060 msgr2=0x7efd7403a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.031+0000 7efd97759640 1 --2- 172.21.15.89:0/618360838 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efd74038060 0x7efd7403a520 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7efd80036000 tx=0x7efd80008030 comp rx=0 tx=0).stop 2024-04-27T03:33:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.031+0000 7efd97759640 1 -- 172.21.15.89:0/618360838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd90071090 msgr2=0x7efd90121a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.031+0000 7efd97759640 1 --2- 172.21.15.89:0/618360838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd90071090 0x7efd90121a90 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7efd8c00c420 tx=0x7efd8c00c8e0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.031+0000 7efd97759640 1 -- 172.21.15.89:0/618360838 shutdown_connections 2024-04-27T03:33:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.032+0000 7efd97759640 1 --2- 172.21.15.89:0/618360838 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7efd74038060 0x7efd7403a520 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.032+0000 7efd97759640 1 --2- 172.21.15.89:0/618360838 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7efd90071090 0x7efd90121a90 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.032+0000 7efd97759640 1 -- 172.21.15.89:0/618360838 >> 172.21.15.89:0/618360838 conn(0x7efd90099680 msgr2=0x7efd90129ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.032+0000 7efd97759640 1 -- 172.21.15.89:0/618360838 shutdown_connections 2024-04-27T03:33:28.051 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.032+0000 7efd97759640 1 -- 172.21.15.89:0/618360838 wait complete. 2024-04-27T03:33:28.052 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":125,"stamp":"2024-04-27T03:33:26.270331+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26412790,"num_objects":259,"num_object_clones":0,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":0,"num_read":5162,"num_read_kb":4712,"num_write":2296,"num_write_kb":61372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4671,"ondisk_log_size":4671,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":198568,"kb_used_data":53064,"kb_used_omap":6,"kb_used_meta":145465,"kb_avail":377288792,"statfs":{"total":386547056640,"available":386343723008,"internally_reserved":0,"allocated":54337536,"data_stored":53272200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":148956950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25811688,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":497,"num_read_kb":452,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003630"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433166+0000","last_change":"2024-04-27T03:30:11.362820+0000","last_active":"2024-04-27T03:31:01.433166+0000","last_peered":"2024-04-27T03:31:01.433166+0000","last_clean":"2024-04-27T03:31:01.433166+0000","last_became_active":"2024-04-27T03:30:11.362574+0000","last_became_peered":"2024-04-27T03:30:11.362574+0000","last_unstale":"2024-04-27T03:31:01.433166+0000","last_undegraded":"2024-04-27T03:31:01.433166+0000","last_fullsized":"2024-04-27T03:31:01.433166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.746108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933798+0000","last_change":"2024-04-27T03:30:13.384819+0000","last_active":"2024-04-27T03:33:20.933798+0000","last_peered":"2024-04-27T03:33:20.933798+0000","last_clean":"2024-04-27T03:33:20.933798+0000","last_became_active":"2024-04-27T03:30:13.384580+0000","last_became_peered":"2024-04-27T03:30:13.384580+0000","last_unstale":"2024-04-27T03:33:20.933798+0000","last_undegraded":"2024-04-27T03:33:20.933798+0000","last_fullsized":"2024-04-27T03:33:20.933798+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:07:51.011104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.921795+0000","last_change":"2024-04-27T03:30:11.361366+0000","last_active":"2024-04-27T03:33:20.921795+0000","last_peered":"2024-04-27T03:33:20.921795+0000","last_clean":"2024-04-27T03:33:20.921795+0000","last_became_active":"2024-04-27T03:30:11.360857+0000","last_became_peered":"2024-04-27T03:30:11.360857+0000","last_unstale":"2024-04-27T03:33:20.921795+0000","last_undegraded":"2024-04-27T03:33:20.921795+0000","last_fullsized":"2024-04-27T03:33:20.921795+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:11.138117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"27'12","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.695008+0000","last_change":"2024-04-27T03:30:13.376545+0000","last_active":"2024-04-27T03:33:20.695008+0000","last_peered":"2024-04-27T03:33:20.695008+0000","last_clean":"2024-04-27T03:33:20.695008+0000","last_became_active":"2024-04-27T03:30:13.376221+0000","last_became_peered":"2024-04-27T03:30:13.376221+0000","last_unstale":"2024-04-27T03:33:20.695008+0000","last_undegraded":"2024-04-27T03:33:20.695008+0000","last_fullsized":"2024-04-27T03:33:20.695008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:08.985512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440740+0000","last_change":"2024-04-27T03:30:11.363460+0000","last_active":"2024-04-27T03:31:01.440740+0000","last_peered":"2024-04-27T03:31:01.440740+0000","last_clean":"2024-04-27T03:31:01.440740+0000","last_became_active":"2024-04-27T03:30:11.362739+0000","last_became_peered":"2024-04-27T03:30:11.362739+0000","last_unstale":"2024-04-27T03:31:01.440740+0000","last_undegraded":"2024-04-27T03:31:01.440740+0000","last_fullsized":"2024-04-27T03:31:01.440740+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:12.842711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"27'7","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298536+0000","last_change":"2024-04-27T03:30:13.376819+0000","last_active":"2024-04-27T03:32:44.298536+0000","last_peered":"2024-04-27T03:32:44.298536+0000","last_clean":"2024-04-27T03:32:44.298536+0000","last_became_active":"2024-04-27T03:30:13.376524+0000","last_became_peered":"2024-04-27T03:30:13.376524+0000","last_unstale":"2024-04-27T03:32:44.298536+0000","last_undegraded":"2024-04-27T03:32:44.298536+0000","last_fullsized":"2024-04-27T03:32:44.298536+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:54:30.451033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.293096+0000","last_change":"2024-04-27T03:30:11.361039+0000","last_active":"2024-04-27T03:32:14.293096+0000","last_peered":"2024-04-27T03:32:14.293096+0000","last_clean":"2024-04-27T03:32:14.293096+0000","last_became_active":"2024-04-27T03:30:11.360188+0000","last_became_peered":"2024-04-27T03:30:11.360188+0000","last_unstale":"2024-04-27T03:32:14.293096+0000","last_undegraded":"2024-04-27T03:32:14.293096+0000","last_fullsized":"2024-04-27T03:32:14.293096+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:22.719424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"27'16","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009953+0000","last_change":"2024-04-27T03:30:13.376574+0000","last_active":"2024-04-27T03:33:21.009953+0000","last_peered":"2024-04-27T03:33:21.009953+0000","last_clean":"2024-04-27T03:33:21.009953+0000","last_became_active":"2024-04-27T03:30:13.376288+0000","last_became_peered":"2024-04-27T03:30:13.376288+0000","last_unstale":"2024-04-27T03:33:21.009953+0000","last_undegraded":"2024-04-27T03:33:21.009953+0000","last_fullsized":"2024-04-27T03:33:21.009953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:15.187356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439707+0000","last_change":"2024-04-27T03:30:11.355274+0000","last_active":"2024-04-27T03:31:01.439707+0000","last_peered":"2024-04-27T03:31:01.439707+0000","last_clean":"2024-04-27T03:31:01.439707+0000","last_became_active":"2024-04-27T03:30:11.354989+0000","last_became_peered":"2024-04-27T03:30:11.354989+0000","last_unstale":"2024-04-27T03:31:01.439707+0000","last_undegraded":"2024-04-27T03:31:01.439707+0000","last_fullsized":"2024-04-27T03:31:01.439707+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:11.281867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933479+0000","last_change":"2024-04-27T03:30:13.374182+0000","last_active":"2024-04-27T03:33:20.933479+0000","last_peered":"2024-04-27T03:33:20.933479+0000","last_clean":"2024-04-27T03:33:20.933479+0000","last_became_active":"2024-04-27T03:30:13.373424+0000","last_became_peered":"2024-04-27T03:30:13.373424+0000","last_unstale":"2024-04-27T03:33:20.933479+0000","last_undegraded":"2024-04-27T03:33:20.933479+0000","last_fullsized":"2024-04-27T03:33:20.933479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:02.825441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.657187+0000","last_change":"2024-04-27T03:30:11.361019+0000","last_active":"2024-04-27T03:33:20.657187+0000","last_peered":"2024-04-27T03:33:20.657187+0000","last_clean":"2024-04-27T03:33:20.657187+0000","last_became_active":"2024-04-27T03:30:11.360219+0000","last_became_peered":"2024-04-27T03:30:11.360219+0000","last_unstale":"2024-04-27T03:33:20.657187+0000","last_undegraded":"2024-04-27T03:33:20.657187+0000","last_fullsized":"2024-04-27T03:33:20.657187+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:45.376863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"25'8","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440191+0000","last_change":"2024-04-27T03:30:13.385912+0000","last_active":"2024-04-27T03:31:01.440191+0000","last_peered":"2024-04-27T03:31:01.440191+0000","last_clean":"2024-04-27T03:31:01.440191+0000","last_became_active":"2024-04-27T03:30:13.385665+0000","last_became_peered":"2024-04-27T03:30:13.385665+0000","last_unstale":"2024-04-27T03:31:01.440191+0000","last_undegraded":"2024-04-27T03:31:01.440191+0000","last_fullsized":"2024-04-27T03:31:01.440191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:49.082066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437953+0000","last_change":"2024-04-27T03:30:11.373049+0000","last_active":"2024-04-27T03:31:01.437953+0000","last_peered":"2024-04-27T03:31:01.437953+0000","last_clean":"2024-04-27T03:31:01.437953+0000","last_became_active":"2024-04-27T03:30:11.372704+0000","last_became_peered":"2024-04-27T03:30:11.372704+0000","last_unstale":"2024-04-27T03:31:01.437953+0000","last_undegraded":"2024-04-27T03:31:01.437953+0000","last_fullsized":"2024-04-27T03:31:01.437953+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:03.639873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"27'4","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981936+0000","last_change":"2024-04-27T03:30:13.374520+0000","last_active":"2024-04-27T03:33:20.981936+0000","last_peered":"2024-04-27T03:33:20.981936+0000","last_clean":"2024-04-27T03:33:20.981936+0000","last_became_active":"2024-04-27T03:30:13.374199+0000","last_became_peered":"2024-04-27T03:30:13.374199+0000","last_unstale":"2024-04-27T03:33:20.981936+0000","last_undegraded":"2024-04-27T03:33:20.981936+0000","last_fullsized":"2024-04-27T03:33:20.981936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:33.811635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.926966+0000","last_change":"2024-04-27T03:30:11.358470+0000","last_active":"2024-04-27T03:33:20.926966+0000","last_peered":"2024-04-27T03:33:20.926966+0000","last_clean":"2024-04-27T03:33:20.926966+0000","last_became_active":"2024-04-27T03:30:11.358181+0000","last_became_peered":"2024-04-27T03:30:11.358181+0000","last_unstale":"2024-04-27T03:33:20.926966+0000","last_undegraded":"2024-04-27T03:33:20.926966+0000","last_fullsized":"2024-04-27T03:33:20.926966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.921690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.794975+0000","last_change":"2024-04-27T03:30:13.379341+0000","last_active":"2024-04-27T03:33:20.794975+0000","last_peered":"2024-04-27T03:33:20.794975+0000","last_clean":"2024-04-27T03:33:20.794975+0000","last_became_active":"2024-04-27T03:30:13.379059+0000","last_became_peered":"2024-04-27T03:30:13.379059+0000","last_unstale":"2024-04-27T03:33:20.794975+0000","last_undegraded":"2024-04-27T03:33:20.794975+0000","last_fullsized":"2024-04-27T03:33:20.794975+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:56.906775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"27'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.010139+0000","last_change":"2024-04-27T03:30:13.383971+0000","last_active":"2024-04-27T03:33:21.010139+0000","last_peered":"2024-04-27T03:33:21.010139+0000","last_clean":"2024-04-27T03:33:21.010139+0000","last_became_active":"2024-04-27T03:30:13.383694+0000","last_became_peered":"2024-04-27T03:30:13.383694+0000","last_unstale":"2024-04-27T03:33:21.010139+0000","last_undegraded":"2024-04-27T03:33:21.010139+0000","last_fullsized":"2024-04-27T03:33:21.010139+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:36:25.083284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436571+0000","last_change":"2024-04-27T03:30:11.370455+0000","last_active":"2024-04-27T03:31:01.436571+0000","last_peered":"2024-04-27T03:31:01.436571+0000","last_clean":"2024-04-27T03:31:01.436571+0000","last_became_active":"2024-04-27T03:30:11.370157+0000","last_became_peered":"2024-04-27T03:30:11.370157+0000","last_unstale":"2024-04-27T03:31:01.436571+0000","last_undegraded":"2024-04-27T03:31:01.436571+0000","last_fullsized":"2024-04-27T03:31:01.436571+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:06.887756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"27'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009774+0000","last_change":"2024-04-27T03:30:13.373269+0000","last_active":"2024-04-27T03:33:21.009774+0000","last_peered":"2024-04-27T03:33:21.009774+0000","last_clean":"2024-04-27T03:33:21.009774+0000","last_became_active":"2024-04-27T03:30:13.372973+0000","last_became_peered":"2024-04-27T03:30:13.372973+0000","last_unstale":"2024-04-27T03:33:21.009774+0000","last_undegraded":"2024-04-27T03:33:21.009774+0000","last_fullsized":"2024-04-27T03:33:21.009774+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:57.593400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297374+0000","last_change":"2024-04-27T03:30:11.363764+0000","last_active":"2024-04-27T03:32:39.297374+0000","last_peered":"2024-04-27T03:32:39.297374+0000","last_clean":"2024-04-27T03:32:39.297374+0000","last_became_active":"2024-04-27T03:30:11.363109+0000","last_became_peered":"2024-04-27T03:30:11.363109+0000","last_unstale":"2024-04-27T03:32:39.297374+0000","last_undegraded":"2024-04-27T03:32:39.297374+0000","last_fullsized":"2024-04-27T03:32:39.297374+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:52:31.047433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"27'9","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988406+0000","last_change":"2024-04-27T03:30:13.384110+0000","last_active":"2024-04-27T03:33:20.988406+0000","last_peered":"2024-04-27T03:33:20.988406+0000","last_clean":"2024-04-27T03:33:20.988406+0000","last_became_active":"2024-04-27T03:30:13.383742+0000","last_became_peered":"2024-04-27T03:30:13.383742+0000","last_unstale":"2024-04-27T03:33:20.988406+0000","last_undegraded":"2024-04-27T03:33:20.988406+0000","last_fullsized":"2024-04-27T03:33:20.988406+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:14.799339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"25'4","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436806+0000","last_change":"2024-04-27T03:30:11.370635+0000","last_active":"2024-04-27T03:31:01.436806+0000","last_peered":"2024-04-27T03:31:01.436806+0000","last_clean":"2024-04-27T03:31:01.436806+0000","last_became_active":"2024-04-27T03:30:11.370380+0000","last_became_peered":"2024-04-27T03:30:11.370380+0000","last_unstale":"2024-04-27T03:31:01.436806+0000","last_undegraded":"2024-04-27T03:31:01.436806+0000","last_fullsized":"2024-04-27T03:31:01.436806+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:45.376863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"27'6","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009766+0000","last_change":"2024-04-27T03:31:01.446266+0000","last_active":"2024-04-27T03:33:21.009766+0000","last_peered":"2024-04-27T03:33:21.009766+0000","last_clean":"2024-04-27T03:33:21.009766+0000","last_became_active":"2024-04-27T03:31:01.440762+0000","last_became_peered":"2024-04-27T03:31:01.440762+0000","last_unstale":"2024-04-27T03:33:21.009766+0000","last_undegraded":"2024-04-27T03:33:21.009766+0000","last_fullsized":"2024-04-27T03:33:21.009766+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:58.987011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.279469+0000","last_change":"2024-04-27T03:30:11.369834+0000","last_active":"2024-04-27T03:32:20.279469+0000","last_peered":"2024-04-27T03:32:20.279469+0000","last_clean":"2024-04-27T03:32:20.279469+0000","last_became_active":"2024-04-27T03:30:11.369551+0000","last_became_peered":"2024-04-27T03:30:11.369551+0000","last_unstale":"2024-04-27T03:32:20.279469+0000","last_undegraded":"2024-04-27T03:32:20.279469+0000","last_fullsized":"2024-04-27T03:32:20.279469+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.921690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"27'2","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981818+0000","last_change":"2024-04-27T03:30:13.382150+0000","last_active":"2024-04-27T03:33:20.981818+0000","last_peered":"2024-04-27T03:33:20.981818+0000","last_clean":"2024-04-27T03:33:20.981818+0000","last_became_active":"2024-04-27T03:30:13.381901+0000","last_became_peered":"2024-04-27T03:30:13.381901+0000","last_unstale":"2024-04-27T03:33:20.981818+0000","last_undegraded":"2024-04-27T03:33:20.981818+0000","last_fullsized":"2024-04-27T03:33:20.981818+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:05.646245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"25'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431969+0000","last_change":"2024-04-27T03:30:11.361129+0000","last_active":"2024-04-27T03:31:01.431969+0000","last_peered":"2024-04-27T03:31:01.431969+0000","last_clean":"2024-04-27T03:31:01.431969+0000","last_became_active":"2024-04-27T03:30:11.360515+0000","last_became_peered":"2024-04-27T03:30:11.360515+0000","last_unstale":"2024-04-27T03:31:01.431969+0000","last_undegraded":"2024-04-27T03:31:01.431969+0000","last_fullsized":"2024-04-27T03:31:01.431969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:52:31.047433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"27'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.307097+0000","last_change":"2024-04-27T03:30:13.382249+0000","last_active":"2024-04-27T03:32:20.307097+0000","last_peered":"2024-04-27T03:32:20.307097+0000","last_clean":"2024-04-27T03:32:20.307097+0000","last_became_active":"2024-04-27T03:30:13.378932+0000","last_became_peered":"2024-04-27T03:30:13.378932+0000","last_unstale":"2024-04-27T03:32:20.307097+0000","last_undegraded":"2024-04-27T03:32:20.307097+0000","last_fullsized":"2024-04-27T03:32:20.307097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:35.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437123+0000","last_change":"2024-04-27T03:30:11.372964+0000","last_active":"2024-04-27T03:31:01.437123+0000","last_peered":"2024-04-27T03:31:01.437123+0000","last_clean":"2024-04-27T03:31:01.437123+0000","last_became_active":"2024-04-27T03:30:11.372673+0000","last_became_peered":"2024-04-27T03:30:11.372673+0000","last_unstale":"2024-04-27T03:31:01.437123+0000","last_undegraded":"2024-04-27T03:31:01.437123+0000","last_fullsized":"2024-04-27T03:31:01.437123+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:08:10.593535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"27'35","reported_seq":99,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.957246+0000","last_change":"2024-04-27T03:30:21.413597+0000","last_active":"2024-04-27T03:33:20.957246+0000","last_peered":"2024-04-27T03:33:20.957246+0000","last_clean":"2024-04-27T03:33:20.957246+0000","last_became_active":"2024-04-27T03:30:21.413255+0000","last_became_peered":"2024-04-27T03:30:21.413255+0000","last_unstale":"2024-04-27T03:33:20.957246+0000","last_undegraded":"2024-04-27T03:33:20.957246+0000","last_fullsized":"2024-04-27T03:33:20.957246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:11:19.581338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"27'3","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296833+0000","last_change":"2024-04-27T03:30:13.377382+0000","last_active":"2024-04-27T03:32:34.296833+0000","last_peered":"2024-04-27T03:32:34.296833+0000","last_clean":"2024-04-27T03:32:34.296833+0000","last_became_active":"2024-04-27T03:30:13.377110+0000","last_became_peered":"2024-04-27T03:30:13.377110+0000","last_unstale":"2024-04-27T03:32:34.296833+0000","last_undegraded":"2024-04-27T03:32:34.296833+0000","last_fullsized":"2024-04-27T03:32:34.296833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:21:27.452807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439438+0000","last_change":"2024-04-27T03:30:11.360152+0000","last_active":"2024-04-27T03:31:01.439438+0000","last_peered":"2024-04-27T03:31:01.439438+0000","last_clean":"2024-04-27T03:31:01.439438+0000","last_became_active":"2024-04-27T03:30:11.359892+0000","last_became_peered":"2024-04-27T03:30:11.359892+0000","last_unstale":"2024-04-27T03:31:01.439438+0000","last_undegraded":"2024-04-27T03:31:01.439438+0000","last_fullsized":"2024-04-27T03:31:01.439438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:07:49.004733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440221+0000","last_change":"2024-04-27T03:30:15.381168+0000","last_active":"2024-04-27T03:31:01.440221+0000","last_peered":"2024-04-27T03:31:01.440221+0000","last_clean":"2024-04-27T03:31:01.440221+0000","last_became_active":"2024-04-27T03:30:15.380864+0000","last_became_peered":"2024-04-27T03:30:15.380864+0000","last_unstale":"2024-04-27T03:31:01.440221+0000","last_undegraded":"2024-04-27T03:31:01.440221+0000","last_fullsized":"2024-04-27T03:31:01.440221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:57:52.741876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"27'42","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.914579+0000","last_change":"2024-04-27T03:30:17.388325+0000","last_active":"2024-04-27T03:32:18.914579+0000","last_peered":"2024-04-27T03:32:18.914579+0000","last_clean":"2024-04-27T03:32:18.914579+0000","last_became_active":"2024-04-27T03:30:17.387391+0000","last_became_peered":"2024-04-27T03:30:17.387391+0000","last_unstale":"2024-04-27T03:32:18.914579+0000","last_undegraded":"2024-04-27T03:32:18.914579+0000","last_fullsized":"2024-04-27T03:32:18.914579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:02:27.282043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433271+0000","last_change":"2024-04-27T03:30:19.407954+0000","last_active":"2024-04-27T03:31:01.433271+0000","last_peered":"2024-04-27T03:31:01.433271+0000","last_clean":"2024-04-27T03:31:01.433271+0000","last_became_active":"2024-04-27T03:30:19.407637+0000","last_became_peered":"2024-04-27T03:30:19.407637+0000","last_unstale":"2024-04-27T03:31:01.433271+0000","last_undegraded":"2024-04-27T03:31:01.433271+0000","last_fullsized":"2024-04-27T03:31:01.433271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:12.277984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"27'9","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.436568+0000","last_change":"2024-04-27T03:30:21.414223+0000","last_active":"2024-04-27T03:31:36.436568+0000","last_peered":"2024-04-27T03:31:36.436568+0000","last_clean":"2024-04-27T03:31:36.436568+0000","last_became_active":"2024-04-27T03:30:21.413940+0000","last_became_peered":"2024-04-27T03:30:21.413940+0000","last_unstale":"2024-04-27T03:31:36.436568+0000","last_undegraded":"2024-04-27T03:31:36.436568+0000","last_fullsized":"2024-04-27T03:31:36.436568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:44.051317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"27'16","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981635+0000","last_change":"2024-04-27T03:30:13.381748+0000","last_active":"2024-04-27T03:33:20.981635+0000","last_peered":"2024-04-27T03:33:20.981635+0000","last_clean":"2024-04-27T03:33:20.981635+0000","last_became_active":"2024-04-27T03:30:13.381487+0000","last_became_peered":"2024-04-27T03:30:13.381487+0000","last_unstale":"2024-04-27T03:33:20.981635+0000","last_undegraded":"2024-04-27T03:33:20.981635+0000","last_fullsized":"2024-04-27T03:33:20.981635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:52.946275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435664+0000","last_change":"2024-04-27T03:30:11.359690+0000","last_active":"2024-04-27T03:31:01.435664+0000","last_peered":"2024-04-27T03:31:01.435664+0000","last_clean":"2024-04-27T03:31:01.435664+0000","last_became_active":"2024-04-27T03:30:11.359446+0000","last_became_peered":"2024-04-27T03:30:11.359446+0000","last_unstale":"2024-04-27T03:31:01.435664+0000","last_undegraded":"2024-04-27T03:31:01.435664+0000","last_fullsized":"2024-04-27T03:31:01.435664+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.921690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431730+0000","last_change":"2024-04-27T03:30:15.380566+0000","last_active":"2024-04-27T03:31:01.431730+0000","last_peered":"2024-04-27T03:31:01.431730+0000","last_clean":"2024-04-27T03:31:01.431730+0000","last_became_active":"2024-04-27T03:30:15.380313+0000","last_became_peered":"2024-04-27T03:30:15.380313+0000","last_unstale":"2024-04-27T03:31:01.431730+0000","last_undegraded":"2024-04-27T03:31:01.431730+0000","last_fullsized":"2024-04-27T03:31:01.431730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:45:16.754705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"25'12","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433370+0000","last_change":"2024-04-27T03:30:17.389399+0000","last_active":"2024-04-27T03:31:01.433370+0000","last_peered":"2024-04-27T03:31:01.433370+0000","last_clean":"2024-04-27T03:31:01.433370+0000","last_became_active":"2024-04-27T03:30:17.389126+0000","last_became_peered":"2024-04-27T03:30:17.389126+0000","last_unstale":"2024-04-27T03:31:01.433370+0000","last_undegraded":"2024-04-27T03:31:01.433370+0000","last_fullsized":"2024-04-27T03:31:01.433370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:42:03.150440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432915+0000","last_change":"2024-04-27T03:30:19.405456+0000","last_active":"2024-04-27T03:31:01.432915+0000","last_peered":"2024-04-27T03:31:01.432915+0000","last_clean":"2024-04-27T03:31:01.432915+0000","last_became_active":"2024-04-27T03:30:19.405170+0000","last_became_peered":"2024-04-27T03:30:19.405170+0000","last_unstale":"2024-04-27T03:31:01.432915+0000","last_undegraded":"2024-04-27T03:31:01.432915+0000","last_fullsized":"2024-04-27T03:31:01.432915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:31:41.885815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'6","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.427217+0000","last_change":"2024-04-27T03:30:21.408308+0000","last_active":"2024-04-27T03:31:36.427217+0000","last_peered":"2024-04-27T03:31:36.427217+0000","last_clean":"2024-04-27T03:31:36.427217+0000","last_became_active":"2024-04-27T03:30:21.407917+0000","last_became_peered":"2024-04-27T03:30:21.407917+0000","last_unstale":"2024-04-27T03:31:36.427217+0000","last_undegraded":"2024-04-27T03:31:36.427217+0000","last_fullsized":"2024-04-27T03:31:36.427217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:23.492592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":95,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.792066+0000","last_change":"2024-04-27T03:30:13.378024+0000","last_active":"2024-04-27T03:33:20.792066+0000","last_peered":"2024-04-27T03:33:20.792066+0000","last_clean":"2024-04-27T03:33:20.792066+0000","last_became_active":"2024-04-27T03:30:13.377423+0000","last_became_peered":"2024-04-27T03:30:13.377423+0000","last_unstale":"2024-04-27T03:33:20.792066+0000","last_undegraded":"2024-04-27T03:33:20.792066+0000","last_fullsized":"2024-04-27T03:33:20.792066+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:22.057065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437475+0000","last_change":"2024-04-27T03:30:11.359542+0000","last_active":"2024-04-27T03:31:01.437475+0000","last_peered":"2024-04-27T03:31:01.437475+0000","last_clean":"2024-04-27T03:31:01.437475+0000","last_became_active":"2024-04-27T03:30:11.359261+0000","last_became_peered":"2024-04-27T03:30:11.359261+0000","last_unstale":"2024-04-27T03:31:01.437475+0000","last_undegraded":"2024-04-27T03:31:01.437475+0000","last_fullsized":"2024-04-27T03:31:01.437475+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:19.905319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433488+0000","last_change":"2024-04-27T03:30:15.383902+0000","last_active":"2024-04-27T03:31:01.433488+0000","last_peered":"2024-04-27T03:31:01.433488+0000","last_clean":"2024-04-27T03:31:01.433488+0000","last_became_active":"2024-04-27T03:30:15.383618+0000","last_became_peered":"2024-04-27T03:30:15.383618+0000","last_unstale":"2024-04-27T03:31:01.433488+0000","last_undegraded":"2024-04-27T03:31:01.433488+0000","last_fullsized":"2024-04-27T03:31:01.433488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:54.227106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"25'23","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433761+0000","last_change":"2024-04-27T03:30:17.392488+0000","last_active":"2024-04-27T03:31:01.433761+0000","last_peered":"2024-04-27T03:31:01.433761+0000","last_clean":"2024-04-27T03:31:01.433761+0000","last_became_active":"2024-04-27T03:30:17.392223+0000","last_became_peered":"2024-04-27T03:30:17.392223+0000","last_unstale":"2024-04-27T03:31:01.433761+0000","last_undegraded":"2024-04-27T03:31:01.433761+0000","last_fullsized":"2024-04-27T03:31:01.433761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:56.237623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"27'282","reported_seq":381,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.463012+0000","last_change":"2024-04-27T03:30:19.406264+0000","last_active":"2024-04-27T03:33:20.463012+0000","last_peered":"2024-04-27T03:33:20.463012+0000","last_clean":"2024-04-27T03:33:20.463012+0000","last_became_active":"2024-04-27T03:30:19.405730+0000","last_became_peered":"2024-04-27T03:30:19.405730+0000","last_unstale":"2024-04-27T03:33:20.463012+0000","last_undegraded":"2024-04-27T03:33:20.463012+0000","last_fullsized":"2024-04-27T03:33:20.463012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:24.551790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436546+0000","last_change":"2024-04-27T03:30:21.409739+0000","last_active":"2024-04-27T03:31:01.436546+0000","last_peered":"2024-04-27T03:31:01.436546+0000","last_clean":"2024-04-27T03:31:01.436546+0000","last_became_active":"2024-04-27T03:30:21.409156+0000","last_became_peered":"2024-04-27T03:30:21.409156+0000","last_unstale":"2024-04-27T03:31:01.436546+0000","last_undegraded":"2024-04-27T03:31:01.436546+0000","last_fullsized":"2024-04-27T03:31:01.436546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:56:51.564750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.802359+0000","last_change":"2024-04-27T03:30:13.385803+0000","last_active":"2024-04-27T03:33:20.802359+0000","last_peered":"2024-04-27T03:33:20.802359+0000","last_clean":"2024-04-27T03:33:20.802359+0000","last_became_active":"2024-04-27T03:30:13.385491+0000","last_became_peered":"2024-04-27T03:30:13.385491+0000","last_unstale":"2024-04-27T03:33:20.802359+0000","last_undegraded":"2024-04-27T03:33:20.802359+0000","last_fullsized":"2024-04-27T03:33:20.802359+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:14.849083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431754+0000","last_change":"2024-04-27T03:30:11.370401+0000","last_active":"2024-04-27T03:31:01.431754+0000","last_peered":"2024-04-27T03:31:01.431754+0000","last_clean":"2024-04-27T03:31:01.431754+0000","last_became_active":"2024-04-27T03:30:11.369939+0000","last_became_peered":"2024-04-27T03:30:11.369939+0000","last_unstale":"2024-04-27T03:31:01.431754+0000","last_undegraded":"2024-04-27T03:31:01.431754+0000","last_fullsized":"2024-04-27T03:31:01.431754+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:00.825717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439146+0000","last_change":"2024-04-27T03:30:15.384389+0000","last_active":"2024-04-27T03:31:01.439146+0000","last_peered":"2024-04-27T03:31:01.439146+0000","last_clean":"2024-04-27T03:31:01.439146+0000","last_became_active":"2024-04-27T03:30:15.384114+0000","last_became_peered":"2024-04-27T03:30:15.384114+0000","last_unstale":"2024-04-27T03:31:01.439146+0000","last_undegraded":"2024-04-27T03:31:01.439146+0000","last_fullsized":"2024-04-27T03:31:01.439146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:17.509463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"27'36","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.908651+0000","last_change":"2024-04-27T03:30:17.392377+0000","last_active":"2024-04-27T03:32:18.908651+0000","last_peered":"2024-04-27T03:32:18.908651+0000","last_clean":"2024-04-27T03:32:18.908651+0000","last_became_active":"2024-04-27T03:30:17.391880+0000","last_became_peered":"2024-04-27T03:30:17.391880+0000","last_unstale":"2024-04-27T03:32:18.908651+0000","last_undegraded":"2024-04-27T03:32:18.908651+0000","last_fullsized":"2024-04-27T03:32:18.908651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:13.007456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"27'282","reported_seq":405,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.137865+0000","last_change":"2024-04-27T03:30:19.405073+0000","last_active":"2024-04-27T03:33:21.137865+0000","last_peered":"2024-04-27T03:33:21.137865+0000","last_clean":"2024-04-27T03:33:21.137865+0000","last_became_active":"2024-04-27T03:30:19.404744+0000","last_became_peered":"2024-04-27T03:30:19.404744+0000","last_unstale":"2024-04-27T03:33:21.137865+0000","last_undegraded":"2024-04-27T03:33:21.137865+0000","last_fullsized":"2024-04-27T03:33:21.137865+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:31:41.885815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'7","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.706493+0000","last_change":"2024-04-27T03:30:13.387974+0000","last_active":"2024-04-27T03:33:20.706493+0000","last_peered":"2024-04-27T03:33:20.706493+0000","last_clean":"2024-04-27T03:33:20.706493+0000","last_became_active":"2024-04-27T03:30:13.387335+0000","last_became_peered":"2024-04-27T03:30:13.387335+0000","last_unstale":"2024-04-27T03:33:20.706493+0000","last_undegraded":"2024-04-27T03:33:20.706493+0000","last_fullsized":"2024-04-27T03:33:20.706493+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:32:09.116022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433282+0000","last_change":"2024-04-27T03:30:11.372061+0000","last_active":"2024-04-27T03:31:01.433282+0000","last_peered":"2024-04-27T03:31:01.433282+0000","last_clean":"2024-04-27T03:31:01.433282+0000","last_became_active":"2024-04-27T03:30:11.371600+0000","last_became_peered":"2024-04-27T03:30:11.371600+0000","last_unstale":"2024-04-27T03:31:01.433282+0000","last_undegraded":"2024-04-27T03:31:01.433282+0000","last_fullsized":"2024-04-27T03:31:01.433282+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.059046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433141+0000","last_change":"2024-04-27T03:30:15.381767+0000","last_active":"2024-04-27T03:31:01.433141+0000","last_peered":"2024-04-27T03:31:01.433141+0000","last_clean":"2024-04-27T03:31:01.433141+0000","last_became_active":"2024-04-27T03:30:15.381359+0000","last_became_peered":"2024-04-27T03:30:15.381359+0000","last_unstale":"2024-04-27T03:31:01.433141+0000","last_undegraded":"2024-04-27T03:31:01.433141+0000","last_fullsized":"2024-04-27T03:31:01.433141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:19:12.341976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"27'46","reported_seq":99,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.918834+0000","last_change":"2024-04-27T03:30:17.390288+0000","last_active":"2024-04-27T03:32:18.918834+0000","last_peered":"2024-04-27T03:32:18.918834+0000","last_clean":"2024-04-27T03:32:18.918834+0000","last_became_active":"2024-04-27T03:30:17.390021+0000","last_became_peered":"2024-04-27T03:30:17.390021+0000","last_unstale":"2024-04-27T03:32:18.918834+0000","last_undegraded":"2024-04-27T03:32:18.918834+0000","last_fullsized":"2024-04-27T03:32:18.918834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:58.407158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":53,"num_read_kb":41,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431506+0000","last_change":"2024-04-27T03:30:19.405394+0000","last_active":"2024-04-27T03:31:01.431506+0000","last_peered":"2024-04-27T03:31:01.431506+0000","last_clean":"2024-04-27T03:31:01.431506+0000","last_became_active":"2024-04-27T03:30:19.404948+0000","last_became_peered":"2024-04-27T03:30:19.404948+0000","last_unstale":"2024-04-27T03:31:01.431506+0000","last_undegraded":"2024-04-27T03:31:01.431506+0000","last_fullsized":"2024-04-27T03:31:01.431506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:59.540815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"27'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.438491+0000","last_change":"2024-04-27T03:30:21.412598+0000","last_active":"2024-04-27T03:31:36.438491+0000","last_peered":"2024-04-27T03:31:36.438491+0000","last_clean":"2024-04-27T03:31:36.438491+0000","last_became_active":"2024-04-27T03:30:21.412331+0000","last_became_peered":"2024-04-27T03:30:21.412331+0000","last_unstale":"2024-04-27T03:31:36.438491+0000","last_undegraded":"2024-04-27T03:31:36.438491+0000","last_fullsized":"2024-04-27T03:31:36.438491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:21.686877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"25'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432638+0000","last_change":"2024-04-27T03:30:13.373532+0000","last_active":"2024-04-27T03:31:01.432638+0000","last_peered":"2024-04-27T03:31:01.432638+0000","last_clean":"2024-04-27T03:31:01.432638+0000","last_became_active":"2024-04-27T03:30:13.373226+0000","last_became_peered":"2024-04-27T03:30:13.373226+0000","last_unstale":"2024-04-27T03:31:01.432638+0000","last_undegraded":"2024-04-27T03:31:01.432638+0000","last_fullsized":"2024-04-27T03:31:01.432638+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:38.976155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437884+0000","last_change":"2024-04-27T03:30:11.373710+0000","last_active":"2024-04-27T03:31:01.437884+0000","last_peered":"2024-04-27T03:31:01.437884+0000","last_clean":"2024-04-27T03:31:01.437884+0000","last_became_active":"2024-04-27T03:30:11.373430+0000","last_became_peered":"2024-04-27T03:30:11.373430+0000","last_unstale":"2024-04-27T03:31:01.437884+0000","last_undegraded":"2024-04-27T03:31:01.437884+0000","last_fullsized":"2024-04-27T03:31:01.437884+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:52:58.316121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"18'1","reported_seq":78,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.994968+0000","last_change":"2024-04-27T03:31:01.447513+0000","last_active":"2024-04-27T03:32:19.994968+0000","last_peered":"2024-04-27T03:32:19.994968+0000","last_clean":"2024-04-27T03:32:19.994968+0000","last_became_active":"2024-04-27T03:31:01.443259+0000","last_became_peered":"2024-04-27T03:31:01.443259+0000","last_unstale":"2024-04-27T03:32:19.994968+0000","last_undegraded":"2024-04-27T03:32:19.994968+0000","last_fullsized":"2024-04-27T03:32:19.994968+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:54.147127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"27'39","reported_seq":87,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296781+0000","last_change":"2024-04-27T03:30:17.389610+0000","last_active":"2024-04-27T03:32:34.296781+0000","last_peered":"2024-04-27T03:32:34.296781+0000","last_clean":"2024-04-27T03:32:34.296781+0000","last_became_active":"2024-04-27T03:30:17.388994+0000","last_became_peered":"2024-04-27T03:30:17.388994+0000","last_unstale":"2024-04-27T03:32:34.296781+0000","last_undegraded":"2024-04-27T03:32:34.296781+0000","last_fullsized":"2024-04-27T03:32:34.296781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:01:03.772535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431125+0000","last_change":"2024-04-27T03:30:19.405695+0000","last_active":"2024-04-27T03:31:01.431125+0000","last_peered":"2024-04-27T03:31:01.431125+0000","last_clean":"2024-04-27T03:31:01.431125+0000","last_became_active":"2024-04-27T03:30:19.405334+0000","last_became_peered":"2024-04-27T03:30:19.405334+0000","last_unstale":"2024-04-27T03:31:01.431125+0000","last_undegraded":"2024-04-27T03:31:01.431125+0000","last_fullsized":"2024-04-27T03:31:01.431125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:44:45.246417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"27'3","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:30.912296+0000","last_change":"2024-04-27T03:30:21.410221+0000","last_active":"2024-04-27T03:31:30.912296+0000","last_peered":"2024-04-27T03:31:30.912296+0000","last_clean":"2024-04-27T03:31:30.912296+0000","last_became_active":"2024-04-27T03:30:21.409922+0000","last_became_peered":"2024-04-27T03:30:21.409922+0000","last_unstale":"2024-04-27T03:31:30.912296+0000","last_undegraded":"2024-04-27T03:31:30.912296+0000","last_fullsized":"2024-04-27T03:31:30.912296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.646754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"27'6","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.010033+0000","last_change":"2024-04-27T03:30:13.381521+0000","last_active":"2024-04-27T03:33:21.010033+0000","last_peered":"2024-04-27T03:33:21.010033+0000","last_clean":"2024-04-27T03:33:21.010033+0000","last_became_active":"2024-04-27T03:30:13.381270+0000","last_became_peered":"2024-04-27T03:30:13.381270+0000","last_unstale":"2024-04-27T03:33:21.010033+0000","last_undegraded":"2024-04-27T03:33:21.010033+0000","last_fullsized":"2024-04-27T03:33:21.010033+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:58:48.554677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440331+0000","last_change":"2024-04-27T03:30:11.372966+0000","last_active":"2024-04-27T03:31:01.440331+0000","last_peered":"2024-04-27T03:31:01.440331+0000","last_clean":"2024-04-27T03:31:01.440331+0000","last_became_active":"2024-04-27T03:30:11.372583+0000","last_became_peered":"2024-04-27T03:30:11.372583+0000","last_unstale":"2024-04-27T03:31:01.440331+0000","last_undegraded":"2024-04-27T03:31:01.440331+0000","last_fullsized":"2024-04-27T03:31:01.440331+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:54.900404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433260+0000","last_change":"2024-04-27T03:30:15.382571+0000","last_active":"2024-04-27T03:31:01.433260+0000","last_peered":"2024-04-27T03:31:01.433260+0000","last_clean":"2024-04-27T03:31:01.433260+0000","last_became_active":"2024-04-27T03:30:15.382207+0000","last_became_peered":"2024-04-27T03:30:15.382207+0000","last_unstale":"2024-04-27T03:31:01.433260+0000","last_undegraded":"2024-04-27T03:31:01.433260+0000","last_fullsized":"2024-04-27T03:31:01.433260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:56.396642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"27'49","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.920671+0000","last_change":"2024-04-27T03:30:17.390418+0000","last_active":"2024-04-27T03:32:18.920671+0000","last_peered":"2024-04-27T03:32:18.920671+0000","last_clean":"2024-04-27T03:32:18.920671+0000","last_became_active":"2024-04-27T03:30:17.389242+0000","last_became_peered":"2024-04-27T03:30:17.389242+0000","last_unstale":"2024-04-27T03:32:18.920671+0000","last_undegraded":"2024-04-27T03:32:18.920671+0000","last_fullsized":"2024-04-27T03:32:18.920671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:59.857055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":30,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"27'282","reported_seq":348,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.148028+0000","last_change":"2024-04-27T03:30:19.406718+0000","last_active":"2024-04-27T03:33:21.148028+0000","last_peered":"2024-04-27T03:33:21.148028+0000","last_clean":"2024-04-27T03:33:21.148028+0000","last_became_active":"2024-04-27T03:30:19.406364+0000","last_became_peered":"2024-04-27T03:30:19.406364+0000","last_unstale":"2024-04-27T03:33:21.148028+0000","last_undegraded":"2024-04-27T03:33:21.148028+0000","last_fullsized":"2024-04-27T03:33:21.148028+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:19.671271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432341+0000","last_change":"2024-04-27T03:30:21.408481+0000","last_active":"2024-04-27T03:31:01.432341+0000","last_peered":"2024-04-27T03:31:01.432341+0000","last_clean":"2024-04-27T03:31:01.432341+0000","last_became_active":"2024-04-27T03:30:21.408168+0000","last_became_peered":"2024-04-27T03:30:21.408168+0000","last_unstale":"2024-04-27T03:31:01.432341+0000","last_undegraded":"2024-04-27T03:31:01.432341+0000","last_fullsized":"2024-04-27T03:31:01.432341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:56.856006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"27'14","reported_seq":101,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.703564+0000","last_change":"2024-04-27T03:30:13.371773+0000","last_active":"2024-04-27T03:33:20.703564+0000","last_peered":"2024-04-27T03:33:20.703564+0000","last_clean":"2024-04-27T03:33:20.703564+0000","last_became_active":"2024-04-27T03:30:13.371503+0000","last_became_peered":"2024-04-27T03:30:13.371503+0000","last_unstale":"2024-04-27T03:33:20.703564+0000","last_undegraded":"2024-04-27T03:33:20.703564+0000","last_fullsized":"2024-04-27T03:33:20.703564+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:54.324828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438871+0000","last_change":"2024-04-27T03:30:11.357185+0000","last_active":"2024-04-27T03:31:01.438871+0000","last_peered":"2024-04-27T03:31:01.438871+0000","last_clean":"2024-04-27T03:31:01.438871+0000","last_became_active":"2024-04-27T03:30:11.356739+0000","last_became_peered":"2024-04-27T03:30:11.356739+0000","last_unstale":"2024-04-27T03:31:01.438871+0000","last_undegraded":"2024-04-27T03:31:01.438871+0000","last_fullsized":"2024-04-27T03:31:01.438871+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:31.805263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:04.282986+0000","last_change":"2024-04-27T03:31:01.443432+0000","last_active":"2024-04-27T03:31:04.282986+0000","last_peered":"2024-04-27T03:31:04.282986+0000","last_clean":"2024-04-27T03:31:04.282986+0000","last_became_active":"2024-04-27T03:31:01.443118+0000","last_became_peered":"2024-04-27T03:31:01.443118+0000","last_unstale":"2024-04-27T03:31:04.282986+0000","last_undegraded":"2024-04-27T03:31:04.282986+0000","last_fullsized":"2024-04-27T03:31:04.282986+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:38.016966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"27'68","reported_seq":136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.922498+0000","last_change":"2024-04-27T03:30:17.392542+0000","last_active":"2024-04-27T03:32:18.922498+0000","last_peered":"2024-04-27T03:32:18.922498+0000","last_clean":"2024-04-27T03:32:18.922498+0000","last_became_active":"2024-04-27T03:30:17.387318+0000","last_became_peered":"2024-04-27T03:30:17.387318+0000","last_unstale":"2024-04-27T03:32:18.922498+0000","last_undegraded":"2024-04-27T03:32:18.922498+0000","last_fullsized":"2024-04-27T03:32:18.922498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:36.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":67,"num_read_kb":56,"num_write":30,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430991+0000","last_change":"2024-04-27T03:30:19.404488+0000","last_active":"2024-04-27T03:31:01.430991+0000","last_peered":"2024-04-27T03:31:01.430991+0000","last_clean":"2024-04-27T03:31:01.430991+0000","last_became_active":"2024-04-27T03:30:19.404265+0000","last_became_peered":"2024-04-27T03:30:19.404265+0000","last_unstale":"2024-04-27T03:31:01.430991+0000","last_undegraded":"2024-04-27T03:31:01.430991+0000","last_fullsized":"2024-04-27T03:31:01.430991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:26.725030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"27'18","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.120745+0000","last_change":"2024-04-27T03:30:21.414109+0000","last_active":"2024-04-27T03:33:21.120745+0000","last_peered":"2024-04-27T03:33:21.120745+0000","last_clean":"2024-04-27T03:33:21.120745+0000","last_became_active":"2024-04-27T03:30:21.413757+0000","last_became_peered":"2024-04-27T03:30:21.413757+0000","last_unstale":"2024-04-27T03:33:21.120745+0000","last_undegraded":"2024-04-27T03:33:21.120745+0000","last_fullsized":"2024-04-27T03:33:21.120745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:41.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981981+0000","last_change":"2024-04-27T03:30:13.380233+0000","last_active":"2024-04-27T03:33:20.981981+0000","last_peered":"2024-04-27T03:33:20.981981+0000","last_clean":"2024-04-27T03:33:20.981981+0000","last_became_active":"2024-04-27T03:30:13.379861+0000","last_became_peered":"2024-04-27T03:30:13.379861+0000","last_unstale":"2024-04-27T03:33:20.981981+0000","last_undegraded":"2024-04-27T03:33:20.981981+0000","last_fullsized":"2024-04-27T03:33:20.981981+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:46.205600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431725+0000","last_change":"2024-04-27T03:30:11.357503+0000","last_active":"2024-04-27T03:31:01.431725+0000","last_peered":"2024-04-27T03:31:01.431725+0000","last_clean":"2024-04-27T03:31:01.431725+0000","last_became_active":"2024-04-27T03:30:11.357176+0000","last_became_peered":"2024-04-27T03:30:11.357176+0000","last_unstale":"2024-04-27T03:31:01.431725+0000","last_undegraded":"2024-04-27T03:31:01.431725+0000","last_fullsized":"2024-04-27T03:31:01.431725+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:06.887756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432802+0000","last_change":"2024-04-27T03:30:15.382614+0000","last_active":"2024-04-27T03:31:01.432802+0000","last_peered":"2024-04-27T03:31:01.432802+0000","last_clean":"2024-04-27T03:31:01.432802+0000","last_became_active":"2024-04-27T03:30:15.382290+0000","last_became_peered":"2024-04-27T03:30:15.382290+0000","last_unstale":"2024-04-27T03:31:01.432802+0000","last_undegraded":"2024-04-27T03:31:01.432802+0000","last_fullsized":"2024-04-27T03:31:01.432802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:16.810766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"27'38","reported_seq":87,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.902396+0000","last_change":"2024-04-27T03:30:17.386194+0000","last_active":"2024-04-27T03:32:18.902396+0000","last_peered":"2024-04-27T03:32:18.902396+0000","last_clean":"2024-04-27T03:32:18.902396+0000","last_became_active":"2024-04-27T03:30:17.385902+0000","last_became_peered":"2024-04-27T03:30:17.385902+0000","last_unstale":"2024-04-27T03:32:18.902396+0000","last_undegraded":"2024-04-27T03:32:18.902396+0000","last_fullsized":"2024-04-27T03:32:18.902396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:45:51.459337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438538+0000","last_change":"2024-04-27T03:30:19.406061+0000","last_active":"2024-04-27T03:31:01.438538+0000","last_peered":"2024-04-27T03:31:01.438538+0000","last_clean":"2024-04-27T03:31:01.438538+0000","last_became_active":"2024-04-27T03:30:19.405450+0000","last_became_peered":"2024-04-27T03:30:19.405450+0000","last_unstale":"2024-04-27T03:31:01.438538+0000","last_undegraded":"2024-04-27T03:31:01.438538+0000","last_fullsized":"2024-04-27T03:31:01.438538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:26:03.989010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"27'9","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.701473+0000","last_change":"2024-04-27T03:30:21.415884+0000","last_active":"2024-04-27T03:33:20.701473+0000","last_peered":"2024-04-27T03:33:20.701473+0000","last_clean":"2024-04-27T03:33:20.701473+0000","last_became_active":"2024-04-27T03:30:21.415494+0000","last_became_peered":"2024-04-27T03:30:21.415494+0000","last_unstale":"2024-04-27T03:33:20.701473+0000","last_undegraded":"2024-04-27T03:33:20.701473+0000","last_fullsized":"2024-04-27T03:33:20.701473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:34.804682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'6","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009826+0000","last_change":"2024-04-27T03:30:13.388008+0000","last_active":"2024-04-27T03:33:21.009826+0000","last_peered":"2024-04-27T03:33:21.009826+0000","last_clean":"2024-04-27T03:33:21.009826+0000","last_became_active":"2024-04-27T03:30:13.387533+0000","last_became_peered":"2024-04-27T03:30:13.387533+0000","last_unstale":"2024-04-27T03:33:21.009826+0000","last_undegraded":"2024-04-27T03:33:21.009826+0000","last_fullsized":"2024-04-27T03:33:21.009826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:53.142322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437632+0000","last_change":"2024-04-27T03:30:11.373706+0000","last_active":"2024-04-27T03:31:01.437632+0000","last_peered":"2024-04-27T03:31:01.437632+0000","last_clean":"2024-04-27T03:31:01.437632+0000","last_became_active":"2024-04-27T03:30:11.373440+0000","last_became_peered":"2024-04-27T03:30:11.373440+0000","last_unstale":"2024-04-27T03:31:01.437632+0000","last_undegraded":"2024-04-27T03:31:01.437632+0000","last_fullsized":"2024-04-27T03:31:01.437632+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:41:06.626342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432793+0000","last_change":"2024-04-27T03:30:15.383722+0000","last_active":"2024-04-27T03:31:01.432793+0000","last_peered":"2024-04-27T03:31:01.432793+0000","last_clean":"2024-04-27T03:31:01.432793+0000","last_became_active":"2024-04-27T03:30:15.382913+0000","last_became_peered":"2024-04-27T03:30:15.382913+0000","last_unstale":"2024-04-27T03:31:01.432793+0000","last_undegraded":"2024-04-27T03:31:01.432793+0000","last_fullsized":"2024-04-27T03:31:01.432793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:59:19.478810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"27'36","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.928891+0000","last_change":"2024-04-27T03:30:17.382348+0000","last_active":"2024-04-27T03:32:18.928891+0000","last_peered":"2024-04-27T03:32:18.928891+0000","last_clean":"2024-04-27T03:32:18.928891+0000","last_became_active":"2024-04-27T03:30:17.381106+0000","last_became_peered":"2024-04-27T03:30:17.381106+0000","last_unstale":"2024-04-27T03:32:18.928891+0000","last_undegraded":"2024-04-27T03:32:18.928891+0000","last_fullsized":"2024-04-27T03:32:18.928891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:36:29.105227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"27'282","reported_seq":403,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.124909+0000","last_change":"2024-04-27T03:30:19.403728+0000","last_active":"2024-04-27T03:33:21.124909+0000","last_peered":"2024-04-27T03:33:21.124909+0000","last_clean":"2024-04-27T03:33:21.124909+0000","last_became_active":"2024-04-27T03:30:19.403413+0000","last_became_peered":"2024-04-27T03:30:19.403413+0000","last_unstale":"2024-04-27T03:33:21.124909+0000","last_undegraded":"2024-04-27T03:33:21.124909+0000","last_fullsized":"2024-04-27T03:33:21.124909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.087236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"27'14","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.289774+0000","last_change":"2024-04-27T03:30:21.403250+0000","last_active":"2024-04-27T03:32:29.289774+0000","last_peered":"2024-04-27T03:32:29.289774+0000","last_clean":"2024-04-27T03:32:29.289774+0000","last_became_active":"2024-04-27T03:30:21.402102+0000","last_became_peered":"2024-04-27T03:30:21.402102+0000","last_unstale":"2024-04-27T03:32:29.289774+0000","last_undegraded":"2024-04-27T03:32:29.289774+0000","last_fullsized":"2024-04-27T03:32:29.289774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:00.181478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":23,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'12","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.795005+0000","last_change":"2024-04-27T03:30:13.383203+0000","last_active":"2024-04-27T03:33:20.795005+0000","last_peered":"2024-04-27T03:33:20.795005+0000","last_clean":"2024-04-27T03:33:20.795005+0000","last_became_active":"2024-04-27T03:30:13.382899+0000","last_became_peered":"2024-04-27T03:30:13.382899+0000","last_unstale":"2024-04-27T03:33:20.795005+0000","last_undegraded":"2024-04-27T03:33:20.795005+0000","last_fullsized":"2024-04-27T03:33:20.795005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:57.593400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'4","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.924480+0000","last_change":"2024-04-27T03:31:01.444153+0000","last_active":"2024-04-27T03:33:20.924480+0000","last_peered":"2024-04-27T03:33:20.924480+0000","last_clean":"2024-04-27T03:33:20.924480+0000","last_became_active":"2024-04-27T03:31:01.439776+0000","last_became_peered":"2024-04-27T03:31:01.439776+0000","last_unstale":"2024-04-27T03:33:20.924480+0000","last_undegraded":"2024-04-27T03:33:20.924480+0000","last_fullsized":"2024-04-27T03:33:20.924480+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:15.239233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"27'3","reported_seq":413,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.994698+0000","last_change":"2024-04-27T03:30:15.382936+0000","last_active":"2024-04-27T03:32:19.994698+0000","last_peered":"2024-04-27T03:32:19.994698+0000","last_clean":"2024-04-27T03:32:19.994698+0000","last_became_active":"2024-04-27T03:30:15.382502+0000","last_became_peered":"2024-04-27T03:30:15.382502+0000","last_unstale":"2024-04-27T03:32:19.994698+0000","last_undegraded":"2024-04-27T03:32:19.994698+0000","last_fullsized":"2024-04-27T03:32:19.994698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:17.198784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"25'10","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438996+0000","last_change":"2024-04-27T03:30:17.389548+0000","last_active":"2024-04-27T03:31:01.438996+0000","last_peered":"2024-04-27T03:31:01.438996+0000","last_clean":"2024-04-27T03:31:01.438996+0000","last_became_active":"2024-04-27T03:30:17.387739+0000","last_became_peered":"2024-04-27T03:30:17.387739+0000","last_unstale":"2024-04-27T03:31:01.438996+0000","last_undegraded":"2024-04-27T03:31:01.438996+0000","last_fullsized":"2024-04-27T03:31:01.438996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:03:39.120700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"27'282","reported_seq":510,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.076259+0000","last_change":"2024-04-27T03:30:19.402853+0000","last_active":"2024-04-27T03:33:21.076259+0000","last_peered":"2024-04-27T03:33:21.076259+0000","last_clean":"2024-04-27T03:33:21.076259+0000","last_became_active":"2024-04-27T03:30:19.402484+0000","last_became_peered":"2024-04-27T03:30:19.402484+0000","last_unstale":"2024-04-27T03:33:21.076259+0000","last_undegraded":"2024-04-27T03:33:21.076259+0000","last_fullsized":"2024-04-27T03:33:21.076259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:31.846278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"27'4","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.132546+0000","last_change":"2024-04-27T03:30:21.408801+0000","last_active":"2024-04-27T03:32:20.132546+0000","last_peered":"2024-04-27T03:32:20.132546+0000","last_clean":"2024-04-27T03:32:20.132546+0000","last_became_active":"2024-04-27T03:30:21.408512+0000","last_became_peered":"2024-04-27T03:30:21.408512+0000","last_unstale":"2024-04-27T03:32:20.132546+0000","last_undegraded":"2024-04-27T03:32:20.132546+0000","last_fullsized":"2024-04-27T03:32:20.132546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:49:39.647798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'14","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.792438+0000","last_change":"2024-04-27T03:30:13.381639+0000","last_active":"2024-04-27T03:33:20.792438+0000","last_peered":"2024-04-27T03:33:20.792438+0000","last_clean":"2024-04-27T03:33:20.792438+0000","last_became_active":"2024-04-27T03:30:13.380540+0000","last_became_peered":"2024-04-27T03:30:13.380540+0000","last_unstale":"2024-04-27T03:33:20.792438+0000","last_undegraded":"2024-04-27T03:33:20.792438+0000","last_fullsized":"2024-04-27T03:33:20.792438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:44.819638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.190061+0000","last_change":"2024-04-27T03:30:11.372470+0000","last_active":"2024-04-27T03:32:20.190061+0000","last_peered":"2024-04-27T03:32:20.190061+0000","last_clean":"2024-04-27T03:32:20.190061+0000","last_became_active":"2024-04-27T03:30:11.372193+0000","last_became_peered":"2024-04-27T03:30:11.372193+0000","last_unstale":"2024-04-27T03:32:20.190061+0000","last_undegraded":"2024-04-27T03:32:20.190061+0000","last_fullsized":"2024-04-27T03:32:20.190061+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:16.294699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438746+0000","last_change":"2024-04-27T03:30:15.385100+0000","last_active":"2024-04-27T03:31:01.438746+0000","last_peered":"2024-04-27T03:31:01.438746+0000","last_clean":"2024-04-27T03:31:01.438746+0000","last_became_active":"2024-04-27T03:30:15.383924+0000","last_became_peered":"2024-04-27T03:30:15.383924+0000","last_unstale":"2024-04-27T03:31:01.438746+0000","last_undegraded":"2024-04-27T03:31:01.438746+0000","last_fullsized":"2024-04-27T03:31:01.438746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:08.259240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"27'38","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.915730+0000","last_change":"2024-04-27T03:30:17.388297+0000","last_active":"2024-04-27T03:32:18.915730+0000","last_peered":"2024-04-27T03:32:18.915730+0000","last_clean":"2024-04-27T03:32:18.915730+0000","last_became_active":"2024-04-27T03:30:17.387578+0000","last_became_peered":"2024-04-27T03:30:17.387578+0000","last_unstale":"2024-04-27T03:32:18.915730+0000","last_undegraded":"2024-04-27T03:32:18.915730+0000","last_fullsized":"2024-04-27T03:32:18.915730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:26.963495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433032+0000","last_change":"2024-04-27T03:30:19.405992+0000","last_active":"2024-04-27T03:31:01.433032+0000","last_peered":"2024-04-27T03:31:01.433032+0000","last_clean":"2024-04-27T03:31:01.433032+0000","last_became_active":"2024-04-27T03:30:19.405512+0000","last_became_peered":"2024-04-27T03:30:19.405512+0000","last_unstale":"2024-04-27T03:31:01.433032+0000","last_undegraded":"2024-04-27T03:31:01.433032+0000","last_fullsized":"2024-04-27T03:31:01.433032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:15.297931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"27'14","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.046419+0000","last_change":"2024-04-27T03:30:21.407207+0000","last_active":"2024-04-27T03:33:21.046419+0000","last_peered":"2024-04-27T03:33:21.046419+0000","last_clean":"2024-04-27T03:33:21.046419+0000","last_became_active":"2024-04-27T03:30:21.406883+0000","last_became_peered":"2024-04-27T03:30:21.406883+0000","last_unstale":"2024-04-27T03:33:21.046419+0000","last_undegraded":"2024-04-27T03:33:21.046419+0000","last_fullsized":"2024-04-27T03:33:21.046419+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:00.181478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":59,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'20","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933899+0000","last_change":"2024-04-27T03:30:13.387655+0000","last_active":"2024-04-27T03:33:20.933899+0000","last_peered":"2024-04-27T03:33:20.933899+0000","last_clean":"2024-04-27T03:33:20.933899+0000","last_became_active":"2024-04-27T03:30:13.387180+0000","last_became_peered":"2024-04-27T03:30:13.387180+0000","last_unstale":"2024-04-27T03:33:20.933899+0000","last_undegraded":"2024-04-27T03:33:20.933899+0000","last_fullsized":"2024-04-27T03:33:20.933899+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:24.510289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431660+0000","last_change":"2024-04-27T03:30:06.104151+0000","last_active":"2024-04-27T03:31:01.431660+0000","last_peered":"2024-04-27T03:31:01.431660+0000","last_clean":"2024-04-27T03:31:01.431660+0000","last_became_active":"2024-04-27T03:30:06.102893+0000","last_became_peered":"2024-04-27T03:30:06.102893+0000","last_unstale":"2024-04-27T03:31:01.431660+0000","last_undegraded":"2024-04-27T03:31:01.431660+0000","last_fullsized":"2024-04-27T03:31:01.431660+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:27:47.408864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439201+0000","last_change":"2024-04-27T03:30:06.106875+0000","last_active":"2024-04-27T03:31:01.439201+0000","last_peered":"2024-04-27T03:31:01.439201+0000","last_clean":"2024-04-27T03:31:01.439201+0000","last_became_active":"2024-04-27T03:30:06.106511+0000","last_became_peered":"2024-04-27T03:30:06.106511+0000","last_unstale":"2024-04-27T03:31:01.439201+0000","last_undegraded":"2024-04-27T03:31:01.439201+0000","last_fullsized":"2024-04-27T03:31:01.439201+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:56:59.055441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.005690+0000","last_change":"2024-04-27T03:30:11.362699+0000","last_active":"2024-04-27T03:33:21.005690+0000","last_peered":"2024-04-27T03:33:21.005690+0000","last_clean":"2024-04-27T03:33:21.005690+0000","last_became_active":"2024-04-27T03:30:11.362445+0000","last_became_peered":"2024-04-27T03:30:11.362445+0000","last_unstale":"2024-04-27T03:33:21.005690+0000","last_undegraded":"2024-04-27T03:33:21.005690+0000","last_fullsized":"2024-04-27T03:33:21.005690+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:11:24.757438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.441975+0000","last_change":"2024-04-27T03:31:01.441027+0000","last_active":"2024-04-27T03:31:01.441975+0000","last_peered":"2024-04-27T03:31:01.441975+0000","last_clean":"2024-04-27T03:31:01.441975+0000","last_became_active":"2024-04-27T03:31:01.440678+0000","last_became_peered":"2024-04-27T03:31:01.440678+0000","last_unstale":"2024-04-27T03:31:01.441975+0000","last_undegraded":"2024-04-27T03:31:01.441975+0000","last_fullsized":"2024-04-27T03:31:01.441975+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:22.897973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"27'83","reported_seq":158,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.927035+0000","last_change":"2024-04-27T03:30:17.388976+0000","last_active":"2024-04-27T03:32:18.927035+0000","last_peered":"2024-04-27T03:32:18.927035+0000","last_clean":"2024-04-27T03:32:18.927035+0000","last_became_active":"2024-04-27T03:30:17.387707+0000","last_became_peered":"2024-04-27T03:30:17.387707+0000","last_unstale":"2024-04-27T03:32:18.927035+0000","last_undegraded":"2024-04-27T03:32:18.927035+0000","last_fullsized":"2024-04-27T03:32:18.927035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:03.008955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":63,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433316+0000","last_change":"2024-04-27T03:30:19.405693+0000","last_active":"2024-04-27T03:31:01.433316+0000","last_peered":"2024-04-27T03:31:01.433316+0000","last_clean":"2024-04-27T03:31:01.433316+0000","last_became_active":"2024-04-27T03:30:19.405359+0000","last_became_peered":"2024-04-27T03:30:19.405359+0000","last_unstale":"2024-04-27T03:31:01.433316+0000","last_undegraded":"2024-04-27T03:31:01.433316+0000","last_fullsized":"2024-04-27T03:31:01.433316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:36.634879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"27'81","reported_seq":162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.022864+0000","last_change":"2024-04-27T03:30:21.404603+0000","last_active":"2024-04-27T03:33:21.022864+0000","last_peered":"2024-04-27T03:33:21.022864+0000","last_clean":"2024-04-27T03:33:21.022864+0000","last_became_active":"2024-04-27T03:30:21.404300+0000","last_became_peered":"2024-04-27T03:30:21.404300+0000","last_unstale":"2024-04-27T03:33:21.022864+0000","last_undegraded":"2024-04-27T03:33:21.022864+0000","last_fullsized":"2024-04-27T03:33:21.022864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:05.598061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439232+0000","last_change":"2024-04-27T03:30:13.378248+0000","last_active":"2024-04-27T03:31:01.439232+0000","last_peered":"2024-04-27T03:31:01.439232+0000","last_clean":"2024-04-27T03:31:01.439232+0000","last_became_active":"2024-04-27T03:30:13.377713+0000","last_became_peered":"2024-04-27T03:30:13.377713+0000","last_unstale":"2024-04-27T03:31:01.439232+0000","last_undegraded":"2024-04-27T03:31:01.439232+0000","last_fullsized":"2024-04-27T03:31:01.439232+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:09.267760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436259+0000","last_change":"2024-04-27T03:30:06.106890+0000","last_active":"2024-04-27T03:31:01.436259+0000","last_peered":"2024-04-27T03:31:01.436259+0000","last_clean":"2024-04-27T03:31:01.436259+0000","last_became_active":"2024-04-27T03:30:06.106474+0000","last_became_peered":"2024-04-27T03:30:06.106474+0000","last_unstale":"2024-04-27T03:31:01.436259+0000","last_undegraded":"2024-04-27T03:31:01.436259+0000","last_fullsized":"2024-04-27T03:31:01.436259+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:12.652758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.643945+0000","last_change":"2024-04-27T03:30:11.360679+0000","last_active":"2024-04-27T03:31:36.643945+0000","last_peered":"2024-04-27T03:31:36.643945+0000","last_clean":"2024-04-27T03:31:36.643945+0000","last_became_active":"2024-04-27T03:30:11.360244+0000","last_became_peered":"2024-04-27T03:30:11.360244+0000","last_unstale":"2024-04-27T03:31:36.643945+0000","last_undegraded":"2024-04-27T03:31:36.643945+0000","last_fullsized":"2024-04-27T03:31:36.643945+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.539212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440319+0000","last_change":"2024-04-27T03:30:15.379725+0000","last_active":"2024-04-27T03:31:01.440319+0000","last_peered":"2024-04-27T03:31:01.440319+0000","last_clean":"2024-04-27T03:31:01.440319+0000","last_became_active":"2024-04-27T03:30:15.379448+0000","last_became_peered":"2024-04-27T03:30:15.379448+0000","last_unstale":"2024-04-27T03:31:01.440319+0000","last_undegraded":"2024-04-27T03:31:01.440319+0000","last_fullsized":"2024-04-27T03:31:01.440319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:06.243654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"27'279","reported_seq":488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462874+0000","last_change":"2024-04-27T03:30:17.391189+0000","last_active":"2024-04-27T03:33:20.462874+0000","last_peered":"2024-04-27T03:33:20.462874+0000","last_clean":"2024-04-27T03:33:20.462874+0000","last_became_active":"2024-04-27T03:30:17.390884+0000","last_became_peered":"2024-04-27T03:30:17.390884+0000","last_unstale":"2024-04-27T03:33:20.462874+0000","last_undegraded":"2024-04-27T03:33:20.462874+0000","last_fullsized":"2024-04-27T03:33:20.462874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:57.164266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":366,"num_read_kb":350,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439611+0000","last_change":"2024-04-27T03:30:19.408904+0000","last_active":"2024-04-27T03:31:01.439611+0000","last_peered":"2024-04-27T03:31:01.439611+0000","last_clean":"2024-04-27T03:31:01.439611+0000","last_became_active":"2024-04-27T03:30:19.405777+0000","last_became_peered":"2024-04-27T03:30:19.405777+0000","last_unstale":"2024-04-27T03:31:01.439611+0000","last_undegraded":"2024-04-27T03:31:01.439611+0000","last_fullsized":"2024-04-27T03:31:01.439611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:57.389557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"27'7","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.968472+0000","last_change":"2024-04-27T03:30:21.409632+0000","last_active":"2024-04-27T03:33:20.968472+0000","last_peered":"2024-04-27T03:33:20.968472+0000","last_clean":"2024-04-27T03:33:20.968472+0000","last_became_active":"2024-04-27T03:30:21.409359+0000","last_became_peered":"2024-04-27T03:30:21.409359+0000","last_unstale":"2024-04-27T03:33:20.968472+0000","last_undegraded":"2024-04-27T03:33:20.968472+0000","last_fullsized":"2024-04-27T03:33:20.968472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:41.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"25'10","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433170+0000","last_change":"2024-04-27T03:30:13.385862+0000","last_active":"2024-04-27T03:31:01.433170+0000","last_peered":"2024-04-27T03:31:01.433170+0000","last_clean":"2024-04-27T03:31:01.433170+0000","last_became_active":"2024-04-27T03:30:13.385521+0000","last_became_peered":"2024-04-27T03:30:13.385521+0000","last_unstale":"2024-04-27T03:31:01.433170+0000","last_undegraded":"2024-04-27T03:31:01.433170+0000","last_fullsized":"2024-04-27T03:31:01.433170+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:38.976155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435195+0000","last_change":"2024-04-27T03:30:06.103722+0000","last_active":"2024-04-27T03:31:01.435195+0000","last_peered":"2024-04-27T03:31:01.435195+0000","last_clean":"2024-04-27T03:31:01.435195+0000","last_became_active":"2024-04-27T03:30:06.103441+0000","last_became_peered":"2024-04-27T03:30:06.103441+0000","last_unstale":"2024-04-27T03:31:01.435195+0000","last_undegraded":"2024-04-27T03:31:01.435195+0000","last_fullsized":"2024-04-27T03:31:01.435195+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:59.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430689+0000","last_change":"2024-04-27T03:30:11.360789+0000","last_active":"2024-04-27T03:31:01.430689+0000","last_peered":"2024-04-27T03:31:01.430689+0000","last_clean":"2024-04-27T03:31:01.430689+0000","last_became_active":"2024-04-27T03:30:11.360443+0000","last_became_peered":"2024-04-27T03:30:11.360443+0000","last_unstale":"2024-04-27T03:31:01.430689+0000","last_undegraded":"2024-04-27T03:31:01.430689+0000","last_fullsized":"2024-04-27T03:31:01.430689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.863119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"18'1","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437997+0000","last_change":"2024-04-27T03:30:15.383569+0000","last_active":"2024-04-27T03:31:01.437997+0000","last_peered":"2024-04-27T03:31:01.437997+0000","last_clean":"2024-04-27T03:31:01.437997+0000","last_became_active":"2024-04-27T03:30:15.383213+0000","last_became_peered":"2024-04-27T03:30:15.383213+0000","last_unstale":"2024-04-27T03:31:01.437997+0000","last_undegraded":"2024-04-27T03:31:01.437997+0000","last_fullsized":"2024-04-27T03:31:01.437997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:48.217027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"27'16","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462975+0000","last_change":"2024-04-27T03:30:17.388886+0000","last_active":"2024-04-27T03:33:20.462975+0000","last_peered":"2024-04-27T03:33:20.462975+0000","last_clean":"2024-04-27T03:33:20.462975+0000","last_became_active":"2024-04-27T03:30:17.388588+0000","last_became_peered":"2024-04-27T03:30:17.388588+0000","last_unstale":"2024-04-27T03:33:20.462975+0000","last_undegraded":"2024-04-27T03:33:20.462975+0000","last_fullsized":"2024-04-27T03:33:20.462975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:59:21.489327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431871+0000","last_change":"2024-04-27T03:30:19.407292+0000","last_active":"2024-04-27T03:31:01.431871+0000","last_peered":"2024-04-27T03:31:01.431871+0000","last_clean":"2024-04-27T03:31:01.431871+0000","last_became_active":"2024-04-27T03:30:19.407004+0000","last_became_peered":"2024-04-27T03:30:19.407004+0000","last_unstale":"2024-04-27T03:31:01.431871+0000","last_undegraded":"2024-04-27T03:31:01.431871+0000","last_fullsized":"2024-04-27T03:31:01.431871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:10.262399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"25'2","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438400+0000","last_change":"2024-04-27T03:30:21.417022+0000","last_active":"2024-04-27T03:31:01.438400+0000","last_peered":"2024-04-27T03:31:01.438400+0000","last_clean":"2024-04-27T03:31:01.438400+0000","last_became_active":"2024-04-27T03:30:21.416389+0000","last_became_peered":"2024-04-27T03:30:21.416389+0000","last_unstale":"2024-04-27T03:31:01.438400+0000","last_undegraded":"2024-04-27T03:31:01.438400+0000","last_fullsized":"2024-04-27T03:31:01.438400+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:11:27.333113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'8","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.696826+0000","last_change":"2024-04-27T03:30:13.369982+0000","last_active":"2024-04-27T03:33:20.696826+0000","last_peered":"2024-04-27T03:33:20.696826+0000","last_clean":"2024-04-27T03:33:20.696826+0000","last_became_active":"2024-04-27T03:30:13.369638+0000","last_became_peered":"2024-04-27T03:30:13.369638+0000","last_unstale":"2024-04-27T03:33:20.696826+0000","last_undegraded":"2024-04-27T03:33:20.696826+0000","last_fullsized":"2024-04-27T03:33:20.696826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:57.593400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436751+0000","last_change":"2024-04-27T03:30:06.107772+0000","last_active":"2024-04-27T03:31:01.436751+0000","last_peered":"2024-04-27T03:31:01.436751+0000","last_clean":"2024-04-27T03:31:01.436751+0000","last_became_active":"2024-04-27T03:30:06.107356+0000","last_became_peered":"2024-04-27T03:30:06.107356+0000","last_unstale":"2024-04-27T03:31:01.436751+0000","last_undegraded":"2024-04-27T03:31:01.436751+0000","last_fullsized":"2024-04-27T03:31:01.436751+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:05:28.414626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436437+0000","last_change":"2024-04-27T03:30:11.357072+0000","last_active":"2024-04-27T03:31:01.436437+0000","last_peered":"2024-04-27T03:31:01.436437+0000","last_clean":"2024-04-27T03:31:01.436437+0000","last_became_active":"2024-04-27T03:30:11.356685+0000","last_became_peered":"2024-04-27T03:30:11.356685+0000","last_unstale":"2024-04-27T03:31:01.436437+0000","last_undegraded":"2024-04-27T03:31:01.436437+0000","last_fullsized":"2024-04-27T03:31:01.436437+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.863119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432748+0000","last_change":"2024-04-27T03:30:15.382673+0000","last_active":"2024-04-27T03:31:01.432748+0000","last_peered":"2024-04-27T03:31:01.432748+0000","last_clean":"2024-04-27T03:31:01.432748+0000","last_became_active":"2024-04-27T03:30:15.382326+0000","last_became_peered":"2024-04-27T03:30:15.382326+0000","last_unstale":"2024-04-27T03:31:01.432748+0000","last_undegraded":"2024-04-27T03:31:01.432748+0000","last_fullsized":"2024-04-27T03:31:01.432748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:37.500097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"27'13","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.319621+0000","last_change":"2024-04-27T03:30:17.390316+0000","last_active":"2024-04-27T03:33:20.319621+0000","last_peered":"2024-04-27T03:33:20.319621+0000","last_clean":"2024-04-27T03:33:20.319621+0000","last_became_active":"2024-04-27T03:30:17.389373+0000","last_became_peered":"2024-04-27T03:30:17.389373+0000","last_unstale":"2024-04-27T03:33:20.319621+0000","last_undegraded":"2024-04-27T03:33:20.319621+0000","last_fullsized":"2024-04-27T03:33:20.319621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:08.896469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438561+0000","last_change":"2024-04-27T03:30:19.406634+0000","last_active":"2024-04-27T03:31:01.438561+0000","last_peered":"2024-04-27T03:31:01.438561+0000","last_clean":"2024-04-27T03:31:01.438561+0000","last_became_active":"2024-04-27T03:30:19.405953+0000","last_became_peered":"2024-04-27T03:30:19.405953+0000","last_unstale":"2024-04-27T03:31:01.438561+0000","last_undegraded":"2024-04-27T03:31:01.438561+0000","last_fullsized":"2024-04-27T03:31:01.438561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:02.743749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"27'6","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.073585+0000","last_change":"2024-04-27T03:30:21.413040+0000","last_active":"2024-04-27T03:33:21.073585+0000","last_peered":"2024-04-27T03:33:21.073585+0000","last_clean":"2024-04-27T03:33:21.073585+0000","last_became_active":"2024-04-27T03:30:21.412790+0000","last_became_peered":"2024-04-27T03:30:21.412790+0000","last_unstale":"2024-04-27T03:33:21.073585+0000","last_undegraded":"2024-04-27T03:33:21.073585+0000","last_fullsized":"2024-04-27T03:33:21.073585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:58.583851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'18","reported_seq":86,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933802+0000","last_change":"2024-04-27T03:30:13.381418+0000","last_active":"2024-04-27T03:33:20.933802+0000","last_peered":"2024-04-27T03:33:20.933802+0000","last_clean":"2024-04-27T03:33:20.933802+0000","last_became_active":"2024-04-27T03:30:13.380888+0000","last_became_peered":"2024-04-27T03:30:13.380888+0000","last_unstale":"2024-04-27T03:33:20.933802+0000","last_undegraded":"2024-04-27T03:33:20.933802+0000","last_fullsized":"2024-04-27T03:33:20.933802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:28.256872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":73,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433189+0000","last_change":"2024-04-27T03:30:11.362550+0000","last_active":"2024-04-27T03:31:01.433189+0000","last_peered":"2024-04-27T03:31:01.433189+0000","last_clean":"2024-04-27T03:31:01.433189+0000","last_became_active":"2024-04-27T03:30:11.361786+0000","last_became_peered":"2024-04-27T03:30:11.361786+0000","last_unstale":"2024-04-27T03:31:01.433189+0000","last_undegraded":"2024-04-27T03:31:01.433189+0000","last_fullsized":"2024-04-27T03:31:01.433189+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.539212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431862+0000","last_change":"2024-04-27T03:30:06.107861+0000","last_active":"2024-04-27T03:31:01.431862+0000","last_peered":"2024-04-27T03:31:01.431862+0000","last_clean":"2024-04-27T03:31:01.431862+0000","last_became_active":"2024-04-27T03:30:06.107594+0000","last_became_peered":"2024-04-27T03:30:06.107594+0000","last_unstale":"2024-04-27T03:31:01.431862+0000","last_undegraded":"2024-04-27T03:31:01.431862+0000","last_fullsized":"2024-04-27T03:31:01.431862+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:10.319152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"27'1","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.289051+0000","last_change":"2024-04-27T03:30:15.381770+0000","last_active":"2024-04-27T03:32:24.289051+0000","last_peered":"2024-04-27T03:32:24.289051+0000","last_clean":"2024-04-27T03:32:24.289051+0000","last_became_active":"2024-04-27T03:30:15.381392+0000","last_became_peered":"2024-04-27T03:30:15.381392+0000","last_unstale":"2024-04-27T03:32:24.289051+0000","last_undegraded":"2024-04-27T03:32:24.289051+0000","last_fullsized":"2024-04-27T03:32:24.289051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:01.291555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"27'73","reported_seq":144,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288088+0000","last_change":"2024-04-27T03:30:17.390751+0000","last_active":"2024-04-27T03:32:19.288088+0000","last_peered":"2024-04-27T03:32:19.288088+0000","last_clean":"2024-04-27T03:32:19.288088+0000","last_became_active":"2024-04-27T03:30:17.390409+0000","last_became_peered":"2024-04-27T03:30:17.390409+0000","last_unstale":"2024-04-27T03:32:19.288088+0000","last_undegraded":"2024-04-27T03:32:19.288088+0000","last_fullsized":"2024-04-27T03:32:19.288088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.209300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":77,"num_read_kb":63,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435401+0000","last_change":"2024-04-27T03:30:19.403292+0000","last_active":"2024-04-27T03:31:01.435401+0000","last_peered":"2024-04-27T03:31:01.435401+0000","last_clean":"2024-04-27T03:31:01.435401+0000","last_became_active":"2024-04-27T03:30:19.402977+0000","last_became_peered":"2024-04-27T03:30:19.402977+0000","last_unstale":"2024-04-27T03:31:01.435401+0000","last_undegraded":"2024-04-27T03:31:01.435401+0000","last_fullsized":"2024-04-27T03:31:01.435401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:24.818605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"27'11","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.984330+0000","last_change":"2024-04-27T03:30:21.410763+0000","last_active":"2024-04-27T03:33:20.984330+0000","last_peered":"2024-04-27T03:33:20.984330+0000","last_clean":"2024-04-27T03:33:20.984330+0000","last_became_active":"2024-04-27T03:30:21.410511+0000","last_became_peered":"2024-04-27T03:30:21.410511+0000","last_unstale":"2024-04-27T03:33:20.984330+0000","last_undegraded":"2024-04-27T03:33:20.984330+0000","last_fullsized":"2024-04-27T03:33:20.984330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:58.583851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.019081+0000","last_change":"2024-04-27T03:30:13.380442+0000","last_active":"2024-04-27T03:33:21.019081+0000","last_peered":"2024-04-27T03:33:21.019081+0000","last_clean":"2024-04-27T03:33:21.019081+0000","last_became_active":"2024-04-27T03:30:13.379985+0000","last_became_peered":"2024-04-27T03:30:13.379985+0000","last_unstale":"2024-04-27T03:33:21.019081+0000","last_undegraded":"2024-04-27T03:33:21.019081+0000","last_fullsized":"2024-04-27T03:33:21.019081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:19.215823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288186+0000","last_change":"2024-04-27T03:30:11.361129+0000","last_active":"2024-04-27T03:32:19.288186+0000","last_peered":"2024-04-27T03:32:19.288186+0000","last_clean":"2024-04-27T03:32:19.288186+0000","last_became_active":"2024-04-27T03:30:11.360352+0000","last_became_peered":"2024-04-27T03:30:11.360352+0000","last_unstale":"2024-04-27T03:32:19.288186+0000","last_undegraded":"2024-04-27T03:32:19.288186+0000","last_fullsized":"2024-04-27T03:32:19.288186+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:22.798184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440363+0000","last_change":"2024-04-27T03:30:06.106816+0000","last_active":"2024-04-27T03:31:01.440363+0000","last_peered":"2024-04-27T03:31:01.440363+0000","last_clean":"2024-04-27T03:31:01.440363+0000","last_became_active":"2024-04-27T03:30:06.106475+0000","last_became_peered":"2024-04-27T03:30:06.106475+0000","last_unstale":"2024-04-27T03:31:01.440363+0000","last_undegraded":"2024-04-27T03:31:01.440363+0000","last_fullsized":"2024-04-27T03:31:01.440363+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:36.362210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437040+0000","last_change":"2024-04-27T03:30:15.380295+0000","last_active":"2024-04-27T03:31:01.437040+0000","last_peered":"2024-04-27T03:31:01.437040+0000","last_clean":"2024-04-27T03:31:01.437040+0000","last_became_active":"2024-04-27T03:30:15.380030+0000","last_became_peered":"2024-04-27T03:30:15.380030+0000","last_unstale":"2024-04-27T03:31:01.437040+0000","last_undegraded":"2024-04-27T03:31:01.437040+0000","last_fullsized":"2024-04-27T03:31:01.437040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:27.827534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"25'20","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433411+0000","last_change":"2024-04-27T03:30:17.393593+0000","last_active":"2024-04-27T03:31:01.433411+0000","last_peered":"2024-04-27T03:31:01.433411+0000","last_clean":"2024-04-27T03:31:01.433411+0000","last_became_active":"2024-04-27T03:30:17.393253+0000","last_became_peered":"2024-04-27T03:30:17.393253+0000","last_unstale":"2024-04-27T03:31:01.433411+0000","last_undegraded":"2024-04-27T03:31:01.433411+0000","last_fullsized":"2024-04-27T03:31:01.433411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:56.968218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"27'282","reported_seq":386,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462731+0000","last_change":"2024-04-27T03:30:19.402596+0000","last_active":"2024-04-27T03:33:20.462731+0000","last_peered":"2024-04-27T03:33:20.462731+0000","last_clean":"2024-04-27T03:33:20.462731+0000","last_became_active":"2024-04-27T03:30:19.402319+0000","last_became_peered":"2024-04-27T03:30:19.402319+0000","last_unstale":"2024-04-27T03:33:20.462731+0000","last_undegraded":"2024-04-27T03:33:20.462731+0000","last_fullsized":"2024-04-27T03:33:20.462731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:21.528207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438342+0000","last_change":"2024-04-27T03:30:21.414388+0000","last_active":"2024-04-27T03:31:01.438342+0000","last_peered":"2024-04-27T03:31:01.438342+0000","last_clean":"2024-04-27T03:31:01.438342+0000","last_became_active":"2024-04-27T03:30:21.414109+0000","last_became_peered":"2024-04-27T03:30:21.414109+0000","last_unstale":"2024-04-27T03:31:01.438342+0000","last_undegraded":"2024-04-27T03:31:01.438342+0000","last_fullsized":"2024-04-27T03:31:01.438342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:34.636870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436753+0000","last_change":"2024-04-27T03:30:13.388070+0000","last_active":"2024-04-27T03:31:01.436753+0000","last_peered":"2024-04-27T03:31:01.436753+0000","last_clean":"2024-04-27T03:31:01.436753+0000","last_became_active":"2024-04-27T03:30:13.387764+0000","last_became_peered":"2024-04-27T03:30:13.387764+0000","last_unstale":"2024-04-27T03:31:01.436753+0000","last_undegraded":"2024-04-27T03:31:01.436753+0000","last_fullsized":"2024-04-27T03:31:01.436753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:16.285485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.918891+0000","last_change":"2024-04-27T03:30:11.356555+0000","last_active":"2024-04-27T03:33:20.918891+0000","last_peered":"2024-04-27T03:33:20.918891+0000","last_clean":"2024-04-27T03:33:20.918891+0000","last_became_active":"2024-04-27T03:30:11.355704+0000","last_became_peered":"2024-04-27T03:30:11.355704+0000","last_unstale":"2024-04-27T03:33:20.918891+0000","last_undegraded":"2024-04-27T03:33:20.918891+0000","last_fullsized":"2024-04-27T03:33:20.918891+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:35.764527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432650+0000","last_change":"2024-04-27T03:30:06.104067+0000","last_active":"2024-04-27T03:31:01.432650+0000","last_peered":"2024-04-27T03:31:01.432650+0000","last_clean":"2024-04-27T03:31:01.432650+0000","last_became_active":"2024-04-27T03:30:06.103759+0000","last_became_peered":"2024-04-27T03:30:06.103759+0000","last_unstale":"2024-04-27T03:31:01.432650+0000","last_undegraded":"2024-04-27T03:31:01.432650+0000","last_fullsized":"2024-04-27T03:31:01.432650+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:27:47.408864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"27'4","reported_seq":226,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.984492+0000","last_change":"2024-04-27T03:30:15.384765+0000","last_active":"2024-04-27T03:32:19.984492+0000","last_peered":"2024-04-27T03:32:19.984492+0000","last_clean":"2024-04-27T03:32:19.984492+0000","last_became_active":"2024-04-27T03:30:15.384372+0000","last_became_peered":"2024-04-27T03:30:15.384372+0000","last_unstale":"2024-04-27T03:32:19.984492+0000","last_undegraded":"2024-04-27T03:32:19.984492+0000","last_fullsized":"2024-04-27T03:32:19.984492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:55.971988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":298,"num_read_kb":298,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'20","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295153+0000","last_change":"2024-04-27T03:30:17.389511+0000","last_active":"2024-04-27T03:32:24.295153+0000","last_peered":"2024-04-27T03:32:24.295153+0000","last_clean":"2024-04-27T03:32:24.295153+0000","last_became_active":"2024-04-27T03:30:17.389154+0000","last_became_peered":"2024-04-27T03:30:17.389154+0000","last_unstale":"2024-04-27T03:32:24.295153+0000","last_undegraded":"2024-04-27T03:32:24.295153+0000","last_fullsized":"2024-04-27T03:32:24.295153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:41:46.387083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"27'282","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.121400+0000","last_change":"2024-04-27T03:30:19.405158+0000","last_active":"2024-04-27T03:33:21.121400+0000","last_peered":"2024-04-27T03:33:21.121400+0000","last_clean":"2024-04-27T03:33:21.121400+0000","last_became_active":"2024-04-27T03:30:19.404867+0000","last_became_peered":"2024-04-27T03:30:19.404867+0000","last_unstale":"2024-04-27T03:33:21.121400+0000","last_undegraded":"2024-04-27T03:33:21.121400+0000","last_fullsized":"2024-04-27T03:33:21.121400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:12.277984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"25'3","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434777+0000","last_change":"2024-04-27T03:30:21.413577+0000","last_active":"2024-04-27T03:31:01.434777+0000","last_peered":"2024-04-27T03:31:01.434777+0000","last_clean":"2024-04-27T03:31:01.434777+0000","last_became_active":"2024-04-27T03:30:21.413264+0000","last_became_peered":"2024-04-27T03:30:21.413264+0000","last_unstale":"2024-04-27T03:31:01.434777+0000","last_undegraded":"2024-04-27T03:31:01.434777+0000","last_fullsized":"2024-04-27T03:31:01.434777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:23.492592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"27'4","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.696091+0000","last_change":"2024-04-27T03:30:13.383496+0000","last_active":"2024-04-27T03:33:20.696091+0000","last_peered":"2024-04-27T03:33:20.696091+0000","last_clean":"2024-04-27T03:33:20.696091+0000","last_became_active":"2024-04-27T03:30:13.383208+0000","last_became_peered":"2024-04-27T03:30:13.383208+0000","last_unstale":"2024-04-27T03:33:20.696091+0000","last_undegraded":"2024-04-27T03:33:20.696091+0000","last_fullsized":"2024-04-27T03:33:20.696091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:13.288239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436165+0000","last_change":"2024-04-27T03:30:11.370648+0000","last_active":"2024-04-27T03:31:01.436165+0000","last_peered":"2024-04-27T03:31:01.436165+0000","last_clean":"2024-04-27T03:31:01.436165+0000","last_became_active":"2024-04-27T03:30:11.369568+0000","last_became_peered":"2024-04-27T03:30:11.369568+0000","last_unstale":"2024-04-27T03:31:01.436165+0000","last_undegraded":"2024-04-27T03:31:01.436165+0000","last_fullsized":"2024-04-27T03:31:01.436165+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:51.203886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438979+0000","last_change":"2024-04-27T03:30:06.107954+0000","last_active":"2024-04-27T03:31:01.438979+0000","last_peered":"2024-04-27T03:31:01.438979+0000","last_clean":"2024-04-27T03:31:01.438979+0000","last_became_active":"2024-04-27T03:30:06.107600+0000","last_became_peered":"2024-04-27T03:30:06.107600+0000","last_unstale":"2024-04-27T03:31:01.438979+0000","last_undegraded":"2024-04-27T03:31:01.438979+0000","last_fullsized":"2024-04-27T03:31:01.438979+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:44.876194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437548+0000","last_change":"2024-04-27T03:30:15.380881+0000","last_active":"2024-04-27T03:31:01.437548+0000","last_peered":"2024-04-27T03:31:01.437548+0000","last_clean":"2024-04-27T03:31:01.437548+0000","last_became_active":"2024-04-27T03:30:15.380598+0000","last_became_peered":"2024-04-27T03:30:15.380598+0000","last_unstale":"2024-04-27T03:31:01.437548+0000","last_undegraded":"2024-04-27T03:31:01.437548+0000","last_fullsized":"2024-04-27T03:31:01.437548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:10.183004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"27'89","reported_seq":167,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294574+0000","last_change":"2024-04-27T03:30:17.391634+0000","last_active":"2024-04-27T03:32:24.294574+0000","last_peered":"2024-04-27T03:32:24.294574+0000","last_clean":"2024-04-27T03:32:24.294574+0000","last_became_active":"2024-04-27T03:30:17.391191+0000","last_became_peered":"2024-04-27T03:30:17.391191+0000","last_unstale":"2024-04-27T03:32:24.294574+0000","last_undegraded":"2024-04-27T03:32:24.294574+0000","last_fullsized":"2024-04-27T03:32:24.294574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:12.065352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":84,"num_read_kb":69,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431584+0000","last_change":"2024-04-27T03:30:19.404491+0000","last_active":"2024-04-27T03:31:01.431584+0000","last_peered":"2024-04-27T03:31:01.431584+0000","last_clean":"2024-04-27T03:31:01.431584+0000","last_became_active":"2024-04-27T03:30:19.404187+0000","last_became_peered":"2024-04-27T03:30:19.404187+0000","last_unstale":"2024-04-27T03:31:01.431584+0000","last_undegraded":"2024-04-27T03:31:01.431584+0000","last_fullsized":"2024-04-27T03:31:01.431584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:21.528207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432393+0000","last_change":"2024-04-27T03:30:21.408168+0000","last_active":"2024-04-27T03:31:01.432393+0000","last_peered":"2024-04-27T03:31:01.432393+0000","last_clean":"2024-04-27T03:31:01.432393+0000","last_became_active":"2024-04-27T03:30:21.407857+0000","last_became_peered":"2024-04-27T03:30:21.407857+0000","last_unstale":"2024-04-27T03:31:01.432393+0000","last_undegraded":"2024-04-27T03:31:01.432393+0000","last_fullsized":"2024-04-27T03:31:01.432393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:28.740636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"25'2","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432483+0000","last_change":"2024-04-27T03:30:21.411358+0000","last_active":"2024-04-27T03:31:01.432483+0000","last_peered":"2024-04-27T03:31:01.432483+0000","last_clean":"2024-04-27T03:31:01.432483+0000","last_became_active":"2024-04-27T03:30:21.411054+0000","last_became_peered":"2024-04-27T03:30:21.411054+0000","last_unstale":"2024-04-27T03:31:01.432483+0000","last_undegraded":"2024-04-27T03:31:01.432483+0000","last_fullsized":"2024-04-27T03:31:01.432483+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:44:47.262023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.653891+0000","last_change":"2024-04-27T03:30:13.383003+0000","last_active":"2024-04-27T03:31:36.653891+0000","last_peered":"2024-04-27T03:31:36.653891+0000","last_clean":"2024-04-27T03:31:36.653891+0000","last_became_active":"2024-04-27T03:30:13.382675+0000","last_became_peered":"2024-04-27T03:30:13.382675+0000","last_unstale":"2024-04-27T03:31:36.653891+0000","last_undegraded":"2024-04-27T03:31:36.653891+0000","last_fullsized":"2024-04-27T03:31:36.653891+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:21.911691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438857+0000","last_change":"2024-04-27T03:30:11.359531+0000","last_active":"2024-04-27T03:31:01.438857+0000","last_peered":"2024-04-27T03:31:01.438857+0000","last_clean":"2024-04-27T03:31:01.438857+0000","last_became_active":"2024-04-27T03:30:11.359195+0000","last_became_peered":"2024-04-27T03:30:11.359195+0000","last_unstale":"2024-04-27T03:31:01.438857+0000","last_undegraded":"2024-04-27T03:31:01.438857+0000","last_fullsized":"2024-04-27T03:31:01.438857+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:52.318456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434515+0000","last_change":"2024-04-27T03:30:15.379854+0000","last_active":"2024-04-27T03:31:01.434515+0000","last_peered":"2024-04-27T03:31:01.434515+0000","last_clean":"2024-04-27T03:31:01.434515+0000","last_became_active":"2024-04-27T03:30:15.379541+0000","last_became_peered":"2024-04-27T03:30:15.379541+0000","last_unstale":"2024-04-27T03:31:01.434515+0000","last_undegraded":"2024-04-27T03:31:01.434515+0000","last_fullsized":"2024-04-27T03:31:01.434515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:32:11.127449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"27'38","reported_seq":87,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.919746+0000","last_change":"2024-04-27T03:30:17.386926+0000","last_active":"2024-04-27T03:32:18.919746+0000","last_peered":"2024-04-27T03:32:18.919746+0000","last_clean":"2024-04-27T03:32:18.919746+0000","last_became_active":"2024-04-27T03:30:17.386496+0000","last_became_peered":"2024-04-27T03:30:17.386496+0000","last_unstale":"2024-04-27T03:32:18.919746+0000","last_undegraded":"2024-04-27T03:32:18.919746+0000","last_fullsized":"2024-04-27T03:32:18.919746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:56.661166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437333+0000","last_change":"2024-04-27T03:30:19.405864+0000","last_active":"2024-04-27T03:31:01.437333+0000","last_peered":"2024-04-27T03:31:01.437333+0000","last_clean":"2024-04-27T03:31:01.437333+0000","last_became_active":"2024-04-27T03:30:19.405189+0000","last_became_peered":"2024-04-27T03:30:19.405189+0000","last_unstale":"2024-04-27T03:31:01.437333+0000","last_undegraded":"2024-04-27T03:31:01.437333+0000","last_fullsized":"2024-04-27T03:31:01.437333+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:23.134716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"27'3","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.594114+0000","last_change":"2024-04-27T03:30:21.414203+0000","last_active":"2024-04-27T03:32:18.594114+0000","last_peered":"2024-04-27T03:32:18.594114+0000","last_clean":"2024-04-27T03:32:18.594114+0000","last_became_active":"2024-04-27T03:30:21.413920+0000","last_became_peered":"2024-04-27T03:30:21.413920+0000","last_unstale":"2024-04-27T03:32:18.594114+0000","last_undegraded":"2024-04-27T03:32:18.594114+0000","last_fullsized":"2024-04-27T03:32:18.594114+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:40.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"27'16","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981976+0000","last_change":"2024-04-27T03:30:13.381187+0000","last_active":"2024-04-27T03:33:20.981976+0000","last_peered":"2024-04-27T03:33:20.981976+0000","last_clean":"2024-04-27T03:33:20.981976+0000","last_became_active":"2024-04-27T03:30:13.380240+0000","last_became_peered":"2024-04-27T03:30:13.380240+0000","last_unstale":"2024-04-27T03:33:20.981976+0000","last_undegraded":"2024-04-27T03:33:20.981976+0000","last_fullsized":"2024-04-27T03:33:20.981976+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:56.024650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":31,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433005+0000","last_change":"2024-04-27T03:30:11.361106+0000","last_active":"2024-04-27T03:31:01.433005+0000","last_peered":"2024-04-27T03:31:01.433005+0000","last_clean":"2024-04-27T03:31:01.433005+0000","last_became_active":"2024-04-27T03:30:11.360217+0000","last_became_peered":"2024-04-27T03:30:11.360217+0000","last_unstale":"2024-04-27T03:31:01.433005+0000","last_undegraded":"2024-04-27T03:31:01.433005+0000","last_fullsized":"2024-04-27T03:31:01.433005+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:51.203886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439951+0000","last_change":"2024-04-27T03:30:15.381885+0000","last_active":"2024-04-27T03:31:01.439951+0000","last_peered":"2024-04-27T03:31:01.439951+0000","last_clean":"2024-04-27T03:31:01.439951+0000","last_became_active":"2024-04-27T03:30:15.381507+0000","last_became_peered":"2024-04-27T03:30:15.381507+0000","last_unstale":"2024-04-27T03:31:01.439951+0000","last_undegraded":"2024-04-27T03:31:01.439951+0000","last_fullsized":"2024-04-27T03:31:01.439951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:31:37.867070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.324334+0000","last_change":"2024-04-27T03:30:17.385578+0000","last_active":"2024-04-27T03:33:20.324334+0000","last_peered":"2024-04-27T03:33:20.324334+0000","last_clean":"2024-04-27T03:33:20.324334+0000","last_became_active":"2024-04-27T03:30:17.385318+0000","last_became_peered":"2024-04-27T03:30:17.385318+0000","last_unstale":"2024-04-27T03:33:20.324334+0000","last_undegraded":"2024-04-27T03:33:20.324334+0000","last_fullsized":"2024-04-27T03:33:20.324334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:47.385488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435327+0000","last_change":"2024-04-27T03:30:19.407084+0000","last_active":"2024-04-27T03:31:01.435327+0000","last_peered":"2024-04-27T03:31:01.435327+0000","last_clean":"2024-04-27T03:31:01.435327+0000","last_became_active":"2024-04-27T03:30:19.406736+0000","last_became_peered":"2024-04-27T03:30:19.406736+0000","last_unstale":"2024-04-27T03:31:01.435327+0000","last_undegraded":"2024-04-27T03:31:01.435327+0000","last_fullsized":"2024-04-27T03:31:01.435327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:21.217528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"27'4","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.798144+0000","last_change":"2024-04-27T03:30:21.408419+0000","last_active":"2024-04-27T03:33:20.798144+0000","last_peered":"2024-04-27T03:33:20.798144+0000","last_clean":"2024-04-27T03:33:20.798144+0000","last_became_active":"2024-04-27T03:30:21.408079+0000","last_became_peered":"2024-04-27T03:30:21.408079+0000","last_unstale":"2024-04-27T03:33:20.798144+0000","last_undegraded":"2024-04-27T03:33:20.798144+0000","last_fullsized":"2024-04-27T03:33:20.798144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.646754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009922+0000","last_change":"2024-04-27T03:30:13.375656+0000","last_active":"2024-04-27T03:33:21.009922+0000","last_peered":"2024-04-27T03:33:21.009922+0000","last_clean":"2024-04-27T03:33:21.009922+0000","last_became_active":"2024-04-27T03:30:13.375350+0000","last_became_peered":"2024-04-27T03:30:13.375350+0000","last_unstale":"2024-04-27T03:33:21.009922+0000","last_undegraded":"2024-04-27T03:33:21.009922+0000","last_fullsized":"2024-04-27T03:33:21.009922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:13.288239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434851+0000","last_change":"2024-04-27T03:30:11.361187+0000","last_active":"2024-04-27T03:31:01.434851+0000","last_peered":"2024-04-27T03:31:01.434851+0000","last_clean":"2024-04-27T03:31:01.434851+0000","last_became_active":"2024-04-27T03:30:11.360938+0000","last_became_peered":"2024-04-27T03:30:11.360938+0000","last_unstale":"2024-04-27T03:31:01.434851+0000","last_undegraded":"2024-04-27T03:31:01.434851+0000","last_fullsized":"2024-04-27T03:31:01.434851+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:11.138117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431691+0000","last_change":"2024-04-27T03:30:15.382918+0000","last_active":"2024-04-27T03:31:01.431691+0000","last_peered":"2024-04-27T03:31:01.431691+0000","last_clean":"2024-04-27T03:31:01.431691+0000","last_became_active":"2024-04-27T03:30:15.382287+0000","last_became_peered":"2024-04-27T03:30:15.382287+0000","last_unstale":"2024-04-27T03:31:01.431691+0000","last_undegraded":"2024-04-27T03:31:01.431691+0000","last_fullsized":"2024-04-27T03:31:01.431691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:21.227251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"27'40","reported_seq":99,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.385027+0000","last_change":"2024-04-27T03:30:17.388554+0000","last_active":"2024-04-27T03:33:20.385027+0000","last_peered":"2024-04-27T03:33:20.385027+0000","last_clean":"2024-04-27T03:33:20.385027+0000","last_became_active":"2024-04-27T03:30:17.388260+0000","last_became_peered":"2024-04-27T03:30:17.388260+0000","last_unstale":"2024-04-27T03:33:20.385027+0000","last_undegraded":"2024-04-27T03:33:20.385027+0000","last_fullsized":"2024-04-27T03:33:20.385027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:22.810377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432676+0000","last_change":"2024-04-27T03:30:19.402729+0000","last_active":"2024-04-27T03:31:01.432676+0000","last_peered":"2024-04-27T03:31:01.432676+0000","last_clean":"2024-04-27T03:31:01.432676+0000","last_became_active":"2024-04-27T03:30:19.402430+0000","last_became_peered":"2024-04-27T03:30:19.402430+0000","last_unstale":"2024-04-27T03:31:01.432676+0000","last_undegraded":"2024-04-27T03:31:01.432676+0000","last_fullsized":"2024-04-27T03:31:01.432676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:59.540815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438223+0000","last_change":"2024-04-27T03:30:21.416316+0000","last_active":"2024-04-27T03:31:01.438223+0000","last_peered":"2024-04-27T03:31:01.438223+0000","last_clean":"2024-04-27T03:31:01.438223+0000","last_became_active":"2024-04-27T03:30:21.415984+0000","last_became_peered":"2024-04-27T03:30:21.415984+0000","last_unstale":"2024-04-27T03:31:01.438223+0000","last_undegraded":"2024-04-27T03:31:01.438223+0000","last_fullsized":"2024-04-27T03:31:01.438223+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:30.270957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988543+0000","last_change":"2024-04-27T03:30:13.387737+0000","last_active":"2024-04-27T03:33:20.988543+0000","last_peered":"2024-04-27T03:33:20.988543+0000","last_clean":"2024-04-27T03:33:20.988543+0000","last_became_active":"2024-04-27T03:30:13.386985+0000","last_became_peered":"2024-04-27T03:30:13.386985+0000","last_unstale":"2024-04-27T03:33:20.988543+0000","last_undegraded":"2024-04-27T03:33:20.988543+0000","last_fullsized":"2024-04-27T03:33:20.988543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:43.363544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436347+0000","last_change":"2024-04-27T03:30:11.370182+0000","last_active":"2024-04-27T03:31:01.436347+0000","last_peered":"2024-04-27T03:31:01.436347+0000","last_clean":"2024-04-27T03:31:01.436347+0000","last_became_active":"2024-04-27T03:30:11.369824+0000","last_became_peered":"2024-04-27T03:30:11.369824+0000","last_unstale":"2024-04-27T03:31:01.436347+0000","last_undegraded":"2024-04-27T03:31:01.436347+0000","last_fullsized":"2024-04-27T03:31:01.436347+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:11.138117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438418+0000","last_change":"2024-04-27T03:30:15.384051+0000","last_active":"2024-04-27T03:31:01.438418+0000","last_peered":"2024-04-27T03:31:01.438418+0000","last_clean":"2024-04-27T03:31:01.438418+0000","last_became_active":"2024-04-27T03:30:15.383771+0000","last_became_peered":"2024-04-27T03:30:15.383771+0000","last_unstale":"2024-04-27T03:31:01.438418+0000","last_undegraded":"2024-04-27T03:31:01.438418+0000","last_fullsized":"2024-04-27T03:31:01.438418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:32.616135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"27'30","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.904671+0000","last_change":"2024-04-27T03:30:17.385866+0000","last_active":"2024-04-27T03:32:18.904671+0000","last_peered":"2024-04-27T03:32:18.904671+0000","last_clean":"2024-04-27T03:32:18.904671+0000","last_became_active":"2024-04-27T03:30:17.385488+0000","last_became_peered":"2024-04-27T03:30:17.385488+0000","last_unstale":"2024-04-27T03:32:18.904671+0000","last_undegraded":"2024-04-27T03:32:18.904671+0000","last_fullsized":"2024-04-27T03:32:18.904671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:39.510614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437192+0000","last_change":"2024-04-27T03:30:19.398792+0000","last_active":"2024-04-27T03:31:01.437192+0000","last_peered":"2024-04-27T03:31:01.437192+0000","last_clean":"2024-04-27T03:31:01.437192+0000","last_became_active":"2024-04-27T03:30:19.398484+0000","last_became_peered":"2024-04-27T03:30:19.398484+0000","last_unstale":"2024-04-27T03:31:01.437192+0000","last_undegraded":"2024-04-27T03:31:01.437192+0000","last_fullsized":"2024-04-27T03:31:01.437192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:16.023756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"25'2","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440063+0000","last_change":"2024-04-27T03:30:13.377494+0000","last_active":"2024-04-27T03:31:01.440063+0000","last_peered":"2024-04-27T03:31:01.440063+0000","last_clean":"2024-04-27T03:31:01.440063+0000","last_became_active":"2024-04-27T03:30:13.377217+0000","last_became_peered":"2024-04-27T03:30:13.377217+0000","last_unstale":"2024-04-27T03:31:01.440063+0000","last_undegraded":"2024-04-27T03:31:01.440063+0000","last_fullsized":"2024-04-27T03:31:01.440063+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:18.788433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"25'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436454+0000","last_change":"2024-04-27T03:30:11.370820+0000","last_active":"2024-04-27T03:31:01.436454+0000","last_peered":"2024-04-27T03:31:01.436454+0000","last_clean":"2024-04-27T03:31:01.436454+0000","last_became_active":"2024-04-27T03:30:11.370547+0000","last_became_peered":"2024-04-27T03:30:11.370547+0000","last_unstale":"2024-04-27T03:31:01.436454+0000","last_undegraded":"2024-04-27T03:31:01.436454+0000","last_fullsized":"2024-04-27T03:31:01.436454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:34.092880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433313+0000","last_change":"2024-04-27T03:30:15.382292+0000","last_active":"2024-04-27T03:31:01.433313+0000","last_peered":"2024-04-27T03:31:01.433313+0000","last_clean":"2024-04-27T03:31:01.433313+0000","last_became_active":"2024-04-27T03:30:15.381956+0000","last_became_peered":"2024-04-27T03:30:15.381956+0000","last_unstale":"2024-04-27T03:31:01.433313+0000","last_undegraded":"2024-04-27T03:31:01.433313+0000","last_fullsized":"2024-04-27T03:31:01.433313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:10.054835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"27'60","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.925854+0000","last_change":"2024-04-27T03:30:17.383982+0000","last_active":"2024-04-27T03:32:18.925854+0000","last_peered":"2024-04-27T03:32:18.925854+0000","last_clean":"2024-04-27T03:32:18.925854+0000","last_became_active":"2024-04-27T03:30:17.383178+0000","last_became_peered":"2024-04-27T03:30:17.383178+0000","last_unstale":"2024-04-27T03:32:18.925854+0000","last_undegraded":"2024-04-27T03:32:18.925854+0000","last_fullsized":"2024-04-27T03:32:18.925854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:14.015528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":48,"num_write":24,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433851+0000","last_change":"2024-04-27T03:30:19.403087+0000","last_active":"2024-04-27T03:31:01.433851+0000","last_peered":"2024-04-27T03:31:01.433851+0000","last_clean":"2024-04-27T03:31:01.433851+0000","last_became_active":"2024-04-27T03:30:19.402781+0000","last_became_peered":"2024-04-27T03:30:19.402781+0000","last_unstale":"2024-04-27T03:31:01.433851+0000","last_undegraded":"2024-04-27T03:31:01.433851+0000","last_fullsized":"2024-04-27T03:31:01.433851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:30.540461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"27'16","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.240928+0000","last_change":"2024-04-27T03:30:21.408415+0000","last_active":"2024-04-27T03:32:20.240928+0000","last_peered":"2024-04-27T03:32:20.240928+0000","last_clean":"2024-04-27T03:32:20.240928+0000","last_became_active":"2024-04-27T03:30:21.408049+0000","last_became_peered":"2024-04-27T03:30:21.408049+0000","last_unstale":"2024-04-27T03:32:20.240928+0000","last_undegraded":"2024-04-27T03:32:20.240928+0000","last_fullsized":"2024-04-27T03:32:20.240928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:09.753400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":21,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432877+0000","last_change":"2024-04-27T03:30:13.373552+0000","last_active":"2024-04-27T03:31:01.432877+0000","last_peered":"2024-04-27T03:31:01.432877+0000","last_clean":"2024-04-27T03:31:01.432877+0000","last_became_active":"2024-04-27T03:30:13.373257+0000","last_became_peered":"2024-04-27T03:30:13.373257+0000","last_unstale":"2024-04-27T03:31:01.432877+0000","last_undegraded":"2024-04-27T03:31:01.432877+0000","last_fullsized":"2024-04-27T03:31:01.432877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:07:51.011104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432526+0000","last_change":"2024-04-27T03:30:11.371431+0000","last_active":"2024-04-27T03:31:01.432526+0000","last_peered":"2024-04-27T03:31:01.432526+0000","last_clean":"2024-04-27T03:31:01.432526+0000","last_became_active":"2024-04-27T03:30:11.370091+0000","last_became_peered":"2024-04-27T03:30:11.370091+0000","last_unstale":"2024-04-27T03:31:01.432526+0000","last_undegraded":"2024-04-27T03:31:01.432526+0000","last_fullsized":"2024-04-27T03:31:01.432526+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.746108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432766+0000","last_change":"2024-04-27T03:30:15.380494+0000","last_active":"2024-04-27T03:31:01.432766+0000","last_peered":"2024-04-27T03:31:01.432766+0000","last_clean":"2024-04-27T03:31:01.432766+0000","last_became_active":"2024-04-27T03:30:15.379800+0000","last_became_peered":"2024-04-27T03:30:15.379800+0000","last_unstale":"2024-04-27T03:31:01.432766+0000","last_undegraded":"2024-04-27T03:31:01.432766+0000","last_fullsized":"2024-04-27T03:31:01.432766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:58.036077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"27'36","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.905630+0000","last_change":"2024-04-27T03:30:17.389821+0000","last_active":"2024-04-27T03:32:18.905630+0000","last_peered":"2024-04-27T03:32:18.905630+0000","last_clean":"2024-04-27T03:32:18.905630+0000","last_became_active":"2024-04-27T03:30:17.388246+0000","last_became_peered":"2024-04-27T03:30:17.388246+0000","last_unstale":"2024-04-27T03:32:18.905630+0000","last_undegraded":"2024-04-27T03:32:18.905630+0000","last_fullsized":"2024-04-27T03:32:18.905630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:10:57.956151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437582+0000","last_change":"2024-04-27T03:30:19.407011+0000","last_active":"2024-04-27T03:31:01.437582+0000","last_peered":"2024-04-27T03:31:01.437582+0000","last_clean":"2024-04-27T03:31:01.437582+0000","last_became_active":"2024-04-27T03:30:19.406707+0000","last_became_peered":"2024-04-27T03:30:19.406707+0000","last_unstale":"2024-04-27T03:31:01.437582+0000","last_undegraded":"2024-04-27T03:31:01.437582+0000","last_fullsized":"2024-04-27T03:31:01.437582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:59.990733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436390+0000","last_change":"2024-04-27T03:30:21.415821+0000","last_active":"2024-04-27T03:31:01.436390+0000","last_peered":"2024-04-27T03:31:01.436390+0000","last_clean":"2024-04-27T03:31:01.436390+0000","last_became_active":"2024-04-27T03:30:21.415441+0000","last_became_peered":"2024-04-27T03:30:21.415441+0000","last_unstale":"2024-04-27T03:31:01.436390+0000","last_undegraded":"2024-04-27T03:31:01.436390+0000","last_fullsized":"2024-04-27T03:31:01.436390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:03.042667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933527+0000","last_change":"2024-04-27T03:30:13.388229+0000","last_active":"2024-04-27T03:33:20.933527+0000","last_peered":"2024-04-27T03:33:20.933527+0000","last_clean":"2024-04-27T03:33:20.933527+0000","last_became_active":"2024-04-27T03:30:13.387397+0000","last_became_peered":"2024-04-27T03:30:13.387397+0000","last_unstale":"2024-04-27T03:33:20.933527+0000","last_undegraded":"2024-04-27T03:33:20.933527+0000","last_fullsized":"2024-04-27T03:33:20.933527+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:45:47.437394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432931+0000","last_change":"2024-04-27T03:30:11.370706+0000","last_active":"2024-04-27T03:31:01.432931+0000","last_peered":"2024-04-27T03:31:01.432931+0000","last_clean":"2024-04-27T03:31:01.432931+0000","last_became_active":"2024-04-27T03:30:11.369799+0000","last_became_peered":"2024-04-27T03:30:11.369799+0000","last_unstale":"2024-04-27T03:31:01.432931+0000","last_undegraded":"2024-04-27T03:31:01.432931+0000","last_fullsized":"2024-04-27T03:31:01.432931+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.539212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433610+0000","last_change":"2024-04-27T03:30:15.380214+0000","last_active":"2024-04-27T03:31:01.433610+0000","last_peered":"2024-04-27T03:31:01.433610+0000","last_clean":"2024-04-27T03:31:01.433610+0000","last_became_active":"2024-04-27T03:30:15.379873+0000","last_became_peered":"2024-04-27T03:30:15.379873+0000","last_unstale":"2024-04-27T03:31:01.433610+0000","last_undegraded":"2024-04-27T03:31:01.433610+0000","last_fullsized":"2024-04-27T03:31:01.433610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:53.446024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"27'40","reported_seq":89,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.923662+0000","last_change":"2024-04-27T03:30:17.387440+0000","last_active":"2024-04-27T03:32:18.923662+0000","last_peered":"2024-04-27T03:32:18.923662+0000","last_clean":"2024-04-27T03:32:18.923662+0000","last_became_active":"2024-04-27T03:30:17.387205+0000","last_became_peered":"2024-04-27T03:30:17.387205+0000","last_unstale":"2024-04-27T03:32:18.923662+0000","last_undegraded":"2024-04-27T03:32:18.923662+0000","last_fullsized":"2024-04-27T03:32:18.923662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:26.079008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":54,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432503+0000","last_change":"2024-04-27T03:30:19.405759+0000","last_active":"2024-04-27T03:31:01.432503+0000","last_peered":"2024-04-27T03:31:01.432503+0000","last_clean":"2024-04-27T03:31:01.432503+0000","last_became_active":"2024-04-27T03:30:19.405462+0000","last_became_peered":"2024-04-27T03:30:19.405462+0000","last_unstale":"2024-04-27T03:31:01.432503+0000","last_undegraded":"2024-04-27T03:31:01.432503+0000","last_fullsized":"2024-04-27T03:31:01.432503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:59.990733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"27'7","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.302334+0000","last_change":"2024-04-27T03:30:21.416894+0000","last_active":"2024-04-27T03:32:20.302334+0000","last_peered":"2024-04-27T03:32:20.302334+0000","last_clean":"2024-04-27T03:32:20.302334+0000","last_became_active":"2024-04-27T03:30:21.416592+0000","last_became_peered":"2024-04-27T03:30:21.416592+0000","last_unstale":"2024-04-27T03:32:20.302334+0000","last_undegraded":"2024-04-27T03:32:20.302334+0000","last_fullsized":"2024-04-27T03:32:20.302334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:24:21.730546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"27'6","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.304764+0000","last_change":"2024-04-27T03:30:13.370253+0000","last_active":"2024-04-27T03:32:20.304764+0000","last_peered":"2024-04-27T03:32:20.304764+0000","last_clean":"2024-04-27T03:32:20.304764+0000","last_became_active":"2024-04-27T03:30:13.369643+0000","last_became_peered":"2024-04-27T03:30:13.369643+0000","last_unstale":"2024-04-27T03:32:20.304764+0000","last_undegraded":"2024-04-27T03:32:20.304764+0000","last_fullsized":"2024-04-27T03:32:20.304764+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:28.256872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298559+0000","last_change":"2024-04-27T03:30:11.372867+0000","last_active":"2024-04-27T03:32:44.298559+0000","last_peered":"2024-04-27T03:32:44.298559+0000","last_clean":"2024-04-27T03:32:44.298559+0000","last_became_active":"2024-04-27T03:30:11.372220+0000","last_became_peered":"2024-04-27T03:30:11.372220+0000","last_unstale":"2024-04-27T03:32:44.298559+0000","last_undegraded":"2024-04-27T03:32:44.298559+0000","last_fullsized":"2024-04-27T03:32:44.298559+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:43.207328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639872,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437382+0000","last_change":"2024-04-27T03:30:15.380858+0000","last_active":"2024-04-27T03:31:01.437382+0000","last_peered":"2024-04-27T03:31:01.437382+0000","last_clean":"2024-04-27T03:31:01.437382+0000","last_became_active":"2024-04-27T03:30:15.380573+0000","last_became_peered":"2024-04-27T03:30:15.380573+0000","last_unstale":"2024-04-27T03:31:01.437382+0000","last_undegraded":"2024-04-27T03:31:01.437382+0000","last_fullsized":"2024-04-27T03:31:01.437382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:45.374972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"25'16","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438994+0000","last_change":"2024-04-27T03:30:17.389436+0000","last_active":"2024-04-27T03:31:01.438994+0000","last_peered":"2024-04-27T03:31:01.438994+0000","last_clean":"2024-04-27T03:31:01.438994+0000","last_became_active":"2024-04-27T03:30:17.388289+0000","last_became_peered":"2024-04-27T03:30:17.388289+0000","last_unstale":"2024-04-27T03:31:01.438994+0000","last_undegraded":"2024-04-27T03:31:01.438994+0000","last_fullsized":"2024-04-27T03:31:01.438994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:02:33.022377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"27'282","reported_seq":372,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462759+0000","last_change":"2024-04-27T03:30:19.407449+0000","last_active":"2024-04-27T03:33:20.462759+0000","last_peered":"2024-04-27T03:33:20.462759+0000","last_clean":"2024-04-27T03:33:20.462759+0000","last_became_active":"2024-04-27T03:30:19.406854+0000","last_became_peered":"2024-04-27T03:30:19.406854+0000","last_unstale":"2024-04-27T03:33:20.462759+0000","last_undegraded":"2024-04-27T03:33:20.462759+0000","last_fullsized":"2024-04-27T03:33:20.462759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:55.112238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435927+0000","last_change":"2024-04-27T03:30:21.415992+0000","last_active":"2024-04-27T03:31:01.435927+0000","last_peered":"2024-04-27T03:31:01.435927+0000","last_clean":"2024-04-27T03:31:01.435927+0000","last_became_active":"2024-04-27T03:30:21.415534+0000","last_became_peered":"2024-04-27T03:30:21.415534+0000","last_unstale":"2024-04-27T03:31:01.435927+0000","last_undegraded":"2024-04-27T03:31:01.435927+0000","last_fullsized":"2024-04-27T03:31:01.435927+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:16.401751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"27'24","reported_seq":102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981535+0000","last_change":"2024-04-27T03:30:13.371535+0000","last_active":"2024-04-27T03:33:20.981535+0000","last_peered":"2024-04-27T03:33:20.981535+0000","last_clean":"2024-04-27T03:33:20.981535+0000","last_became_active":"2024-04-27T03:30:13.371158+0000","last_became_peered":"2024-04-27T03:30:13.371158+0000","last_unstale":"2024-04-27T03:33:20.981535+0000","last_undegraded":"2024-04-27T03:33:20.981535+0000","last_fullsized":"2024-04-27T03:33:20.981535+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:11:26.763810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":88,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435110+0000","last_change":"2024-04-27T03:30:11.359205+0000","last_active":"2024-04-27T03:31:01.435110+0000","last_peered":"2024-04-27T03:31:01.435110+0000","last_clean":"2024-04-27T03:31:01.435110+0000","last_became_active":"2024-04-27T03:30:11.358893+0000","last_became_peered":"2024-04-27T03:30:11.358893+0000","last_unstale":"2024-04-27T03:31:01.435110+0000","last_undegraded":"2024-04-27T03:31:01.435110+0000","last_fullsized":"2024-04-27T03:31:01.435110+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:52:31.047433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436411+0000","last_change":"2024-04-27T03:30:15.381535+0000","last_active":"2024-04-27T03:31:01.436411+0000","last_peered":"2024-04-27T03:31:01.436411+0000","last_clean":"2024-04-27T03:31:01.436411+0000","last_became_active":"2024-04-27T03:30:15.381175+0000","last_became_peered":"2024-04-27T03:30:15.381175+0000","last_unstale":"2024-04-27T03:31:01.436411+0000","last_undegraded":"2024-04-27T03:31:01.436411+0000","last_fullsized":"2024-04-27T03:31:01.436411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:21:29.464234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"27'60","reported_seq":129,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.896020+0000","last_change":"2024-04-27T03:30:17.386967+0000","last_active":"2024-04-27T03:32:18.896020+0000","last_peered":"2024-04-27T03:32:18.896020+0000","last_clean":"2024-04-27T03:32:18.896020+0000","last_became_active":"2024-04-27T03:30:17.386673+0000","last_became_peered":"2024-04-27T03:30:17.386673+0000","last_unstale":"2024-04-27T03:32:18.896020+0000","last_undegraded":"2024-04-27T03:32:18.896020+0000","last_fullsized":"2024-04-27T03:32:18.896020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:54:14.640233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430575+0000","last_change":"2024-04-27T03:30:19.407289+0000","last_active":"2024-04-27T03:31:01.430575+0000","last_peered":"2024-04-27T03:31:01.430575+0000","last_clean":"2024-04-27T03:31:01.430575+0000","last_became_active":"2024-04-27T03:30:19.406990+0000","last_became_peered":"2024-04-27T03:30:19.406990+0000","last_unstale":"2024-04-27T03:31:01.430575+0000","last_undegraded":"2024-04-27T03:31:01.430575+0000","last_fullsized":"2024-04-27T03:31:01.430575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:56:49.549144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433183+0000","last_change":"2024-04-27T03:30:21.410357+0000","last_active":"2024-04-27T03:31:01.433183+0000","last_peered":"2024-04-27T03:31:01.433183+0000","last_clean":"2024-04-27T03:31:01.433183+0000","last_became_active":"2024-04-27T03:30:21.410060+0000","last_became_peered":"2024-04-27T03:30:21.410060+0000","last_unstale":"2024-04-27T03:31:01.433183+0000","last_undegraded":"2024-04-27T03:31:01.433183+0000","last_fullsized":"2024-04-27T03:31:01.433183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:40.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"27'10","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009762+0000","last_change":"2024-04-27T03:30:13.372186+0000","last_active":"2024-04-27T03:33:21.009762+0000","last_peered":"2024-04-27T03:33:21.009762+0000","last_clean":"2024-04-27T03:33:21.009762+0000","last_became_active":"2024-04-27T03:30:13.371866+0000","last_became_peered":"2024-04-27T03:30:13.371866+0000","last_unstale":"2024-04-27T03:33:21.009762+0000","last_undegraded":"2024-04-27T03:33:21.009762+0000","last_fullsized":"2024-04-27T03:33:21.009762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:45.213699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.929513+0000","last_change":"2024-04-27T03:30:11.362991+0000","last_active":"2024-04-27T03:33:20.929513+0000","last_peered":"2024-04-27T03:33:20.929513+0000","last_clean":"2024-04-27T03:33:20.929513+0000","last_became_active":"2024-04-27T03:30:11.362420+0000","last_became_peered":"2024-04-27T03:30:11.362420+0000","last_unstale":"2024-04-27T03:33:20.929513+0000","last_undegraded":"2024-04-27T03:33:20.929513+0000","last_fullsized":"2024-04-27T03:33:20.929513+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:16.411097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437111+0000","last_change":"2024-04-27T03:30:15.382620+0000","last_active":"2024-04-27T03:31:01.437111+0000","last_peered":"2024-04-27T03:31:01.437111+0000","last_clean":"2024-04-27T03:31:01.437111+0000","last_became_active":"2024-04-27T03:30:15.382305+0000","last_became_peered":"2024-04-27T03:30:15.382305+0000","last_unstale":"2024-04-27T03:31:01.437111+0000","last_undegraded":"2024-04-27T03:31:01.437111+0000","last_fullsized":"2024-04-27T03:31:01.437111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:04.836869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"27'38","reported_seq":86,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.292067+0000","last_change":"2024-04-27T03:30:17.390602+0000","last_active":"2024-04-27T03:32:44.292067+0000","last_peered":"2024-04-27T03:32:44.292067+0000","last_clean":"2024-04-27T03:32:44.292067+0000","last_became_active":"2024-04-27T03:30:17.390281+0000","last_became_peered":"2024-04-27T03:30:17.390281+0000","last_unstale":"2024-04-27T03:32:44.292067+0000","last_undegraded":"2024-04-27T03:32:44.292067+0000","last_fullsized":"2024-04-27T03:32:44.292067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:20.307428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5807,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":33,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437067+0000","last_change":"2024-04-27T03:30:19.406218+0000","last_active":"2024-04-27T03:31:01.437067+0000","last_peered":"2024-04-27T03:31:01.437067+0000","last_clean":"2024-04-27T03:31:01.437067+0000","last_became_active":"2024-04-27T03:30:19.405353+0000","last_became_peered":"2024-04-27T03:30:19.405353+0000","last_unstale":"2024-04-27T03:31:01.437067+0000","last_undegraded":"2024-04-27T03:31:01.437067+0000","last_fullsized":"2024-04-27T03:31:01.437067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:21.451111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438395+0000","last_change":"2024-04-27T03:30:21.416876+0000","last_active":"2024-04-27T03:31:01.438395+0000","last_peered":"2024-04-27T03:31:01.438395+0000","last_clean":"2024-04-27T03:31:01.438395+0000","last_became_active":"2024-04-27T03:30:21.416276+0000","last_became_peered":"2024-04-27T03:30:21.416276+0000","last_unstale":"2024-04-27T03:31:01.438395+0000","last_undegraded":"2024-04-27T03:31:01.438395+0000","last_fullsized":"2024-04-27T03:31:01.438395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:58.245817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'24","reported_seq":101,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.697833+0000","last_change":"2024-04-27T03:30:13.377820+0000","last_active":"2024-04-27T03:33:20.697833+0000","last_peered":"2024-04-27T03:33:20.697833+0000","last_clean":"2024-04-27T03:33:20.697833+0000","last_became_active":"2024-04-27T03:30:13.376415+0000","last_became_peered":"2024-04-27T03:30:13.376415+0000","last_unstale":"2024-04-27T03:33:20.697833+0000","last_undegraded":"2024-04-27T03:33:20.697833+0000","last_fullsized":"2024-04-27T03:33:20.697833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:02.028024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.287615+0000","last_change":"2024-04-27T03:30:11.361140+0000","last_active":"2024-04-27T03:32:14.287615+0000","last_peered":"2024-04-27T03:32:14.287615+0000","last_clean":"2024-04-27T03:32:14.287615+0000","last_became_active":"2024-04-27T03:30:11.360590+0000","last_became_peered":"2024-04-27T03:30:11.360590+0000","last_unstale":"2024-04-27T03:32:14.287615+0000","last_undegraded":"2024-04-27T03:32:14.287615+0000","last_fullsized":"2024-04-27T03:32:14.287615+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:52:58.316121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434401+0000","last_change":"2024-04-27T03:30:15.385912+0000","last_active":"2024-04-27T03:31:01.434401+0000","last_peered":"2024-04-27T03:31:01.434401+0000","last_clean":"2024-04-27T03:31:01.434401+0000","last_became_active":"2024-04-27T03:30:15.385614+0000","last_became_peered":"2024-04-27T03:30:15.385614+0000","last_unstale":"2024-04-27T03:31:01.434401+0000","last_undegraded":"2024-04-27T03:31:01.434401+0000","last_fullsized":"2024-04-27T03:31:01.434401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:54:32.462460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"27'32","reported_seq":78,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.921601+0000","last_change":"2024-04-27T03:30:17.387854+0000","last_active":"2024-04-27T03:32:18.921601+0000","last_peered":"2024-04-27T03:32:18.921601+0000","last_clean":"2024-04-27T03:32:18.921601+0000","last_became_active":"2024-04-27T03:30:17.386983+0000","last_became_peered":"2024-04-27T03:30:17.386983+0000","last_unstale":"2024-04-27T03:32:18.921601+0000","last_undegraded":"2024-04-27T03:32:18.921601+0000","last_fullsized":"2024-04-27T03:32:18.921601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:38.038360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":27,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436657+0000","last_change":"2024-04-27T03:30:19.402931+0000","last_active":"2024-04-27T03:31:01.436657+0000","last_peered":"2024-04-27T03:31:01.436657+0000","last_clean":"2024-04-27T03:31:01.436657+0000","last_became_active":"2024-04-27T03:30:19.402652+0000","last_became_peered":"2024-04-27T03:30:19.402652+0000","last_unstale":"2024-04-27T03:31:01.436657+0000","last_undegraded":"2024-04-27T03:31:01.436657+0000","last_fullsized":"2024-04-27T03:31:01.436657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:24.199405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"27'6","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.720254+0000","last_change":"2024-04-27T03:30:21.420988+0000","last_active":"2024-04-27T03:33:20.720254+0000","last_peered":"2024-04-27T03:33:20.720254+0000","last_clean":"2024-04-27T03:33:20.720254+0000","last_became_active":"2024-04-27T03:30:21.420725+0000","last_became_peered":"2024-04-27T03:30:21.420725+0000","last_unstale":"2024-04-27T03:33:20.720254+0000","last_undegraded":"2024-04-27T03:33:20.720254+0000","last_fullsized":"2024-04-27T03:33:20.720254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:02.117649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432186+0000","last_change":"2024-04-27T03:30:13.375384+0000","last_active":"2024-04-27T03:31:01.432186+0000","last_peered":"2024-04-27T03:31:01.432186+0000","last_clean":"2024-04-27T03:31:01.432186+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:31:01.432186+0000","last_undegraded":"2024-04-27T03:31:01.432186+0000","last_fullsized":"2024-04-27T03:31:01.432186+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:40:10.176600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436421+0000","last_change":"2024-04-27T03:30:11.370518+0000","last_active":"2024-04-27T03:31:01.436421+0000","last_peered":"2024-04-27T03:31:01.436421+0000","last_clean":"2024-04-27T03:31:01.436421+0000","last_became_active":"2024-04-27T03:30:11.369877+0000","last_became_peered":"2024-04-27T03:30:11.369877+0000","last_unstale":"2024-04-27T03:31:01.436421+0000","last_undegraded":"2024-04-27T03:31:01.436421+0000","last_fullsized":"2024-04-27T03:31:01.436421+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.059046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"27'2","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.550532+0000","last_change":"2024-04-27T03:30:15.382111+0000","last_active":"2024-04-27T03:32:17.550532+0000","last_peered":"2024-04-27T03:32:17.550532+0000","last_clean":"2024-04-27T03:32:17.550532+0000","last_became_active":"2024-04-27T03:30:15.381574+0000","last_became_peered":"2024-04-27T03:30:15.381574+0000","last_unstale":"2024-04-27T03:32:17.550532+0000","last_undegraded":"2024-04-27T03:32:17.550532+0000","last_fullsized":"2024-04-27T03:32:17.550532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:04.039451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"27'44","reported_seq":100,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.325275+0000","last_change":"2024-04-27T03:30:17.391410+0000","last_active":"2024-04-27T03:33:20.325275+0000","last_peered":"2024-04-27T03:33:20.325275+0000","last_clean":"2024-04-27T03:33:20.325275+0000","last_became_active":"2024-04-27T03:30:17.388277+0000","last_became_peered":"2024-04-27T03:30:17.388277+0000","last_unstale":"2024-04-27T03:33:20.325275+0000","last_undegraded":"2024-04-27T03:33:20.325275+0000","last_fullsized":"2024-04-27T03:33:20.325275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:05.729566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":52,"num_read_kb":46,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431300+0000","last_change":"2024-04-27T03:30:19.406905+0000","last_active":"2024-04-27T03:31:01.431300+0000","last_peered":"2024-04-27T03:31:01.431300+0000","last_clean":"2024-04-27T03:31:01.431300+0000","last_became_active":"2024-04-27T03:30:19.406565+0000","last_became_peered":"2024-04-27T03:30:19.406565+0000","last_unstale":"2024-04-27T03:31:01.431300+0000","last_undegraded":"2024-04-27T03:31:01.431300+0000","last_fullsized":"2024-04-27T03:31:01.431300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:31.846278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"27'4","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.934155+0000","last_change":"2024-04-27T03:30:21.414808+0000","last_active":"2024-04-27T03:32:18.934155+0000","last_peered":"2024-04-27T03:32:18.934155+0000","last_clean":"2024-04-27T03:32:18.934155+0000","last_became_active":"2024-04-27T03:30:21.414510+0000","last_became_peered":"2024-04-27T03:30:21.414510+0000","last_unstale":"2024-04-27T03:32:18.934155+0000","last_undegraded":"2024-04-27T03:32:18.934155+0000","last_fullsized":"2024-04-27T03:32:18.934155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:07.942157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"25'8","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436998+0000","last_change":"2024-04-27T03:30:13.387657+0000","last_active":"2024-04-27T03:31:01.436998+0000","last_peered":"2024-04-27T03:31:01.436998+0000","last_clean":"2024-04-27T03:31:01.436998+0000","last_became_active":"2024-04-27T03:30:13.387149+0000","last_became_peered":"2024-04-27T03:30:13.387149+0000","last_unstale":"2024-04-27T03:31:01.436998+0000","last_undegraded":"2024-04-27T03:31:01.436998+0000","last_fullsized":"2024-04-27T03:31:01.436998+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:00.325582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"25'6","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433385+0000","last_change":"2024-04-27T03:30:11.358174+0000","last_active":"2024-04-27T03:31:01.433385+0000","last_peered":"2024-04-27T03:31:01.433385+0000","last_clean":"2024-04-27T03:31:01.433385+0000","last_became_active":"2024-04-27T03:30:11.357878+0000","last_became_peered":"2024-04-27T03:30:11.357878+0000","last_unstale":"2024-04-27T03:31:01.433385+0000","last_undegraded":"2024-04-27T03:31:01.433385+0000","last_fullsized":"2024-04-27T03:31:01.433385+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:17.981757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.304915+0000","last_change":"2024-04-27T03:30:13.369979+0000","last_active":"2024-04-27T03:32:20.304915+0000","last_peered":"2024-04-27T03:32:20.304915+0000","last_clean":"2024-04-27T03:32:20.304915+0000","last_became_active":"2024-04-27T03:30:13.369681+0000","last_became_peered":"2024-04-27T03:30:13.369681+0000","last_unstale":"2024-04-27T03:32:20.304915+0000","last_undegraded":"2024-04-27T03:32:20.304915+0000","last_fullsized":"2024-04-27T03:32:20.304915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:37.770898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"25'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430839+0000","last_change":"2024-04-27T03:30:11.373422+0000","last_active":"2024-04-27T03:31:01.430839+0000","last_peered":"2024-04-27T03:31:01.430839+0000","last_clean":"2024-04-27T03:31:01.430839+0000","last_became_active":"2024-04-27T03:30:11.373186+0000","last_became_peered":"2024-04-27T03:30:11.373186+0000","last_unstale":"2024-04-27T03:31:01.430839+0000","last_undegraded":"2024-04-27T03:31:01.430839+0000","last_fullsized":"2024-04-27T03:31:01.430839+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:22.719424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933550+0000","last_change":"2024-04-27T03:30:13.382330+0000","last_active":"2024-04-27T03:33:20.933550+0000","last_peered":"2024-04-27T03:33:20.933550+0000","last_clean":"2024-04-27T03:33:20.933550+0000","last_became_active":"2024-04-27T03:30:13.382069+0000","last_became_peered":"2024-04-27T03:30:13.382069+0000","last_unstale":"2024-04-27T03:33:20.933550+0000","last_undegraded":"2024-04-27T03:33:20.933550+0000","last_fullsized":"2024-04-27T03:33:20.933550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:22.941552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"27'3","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.420503+0000","last_change":"2024-04-27T03:30:11.350547+0000","last_active":"2024-04-27T03:31:36.420503+0000","last_peered":"2024-04-27T03:31:36.420503+0000","last_clean":"2024-04-27T03:31:36.420503+0000","last_became_active":"2024-04-27T03:30:11.350295+0000","last_became_peered":"2024-04-27T03:30:11.350295+0000","last_unstale":"2024-04-27T03:31:36.420503+0000","last_undegraded":"2024-04-27T03:31:36.420503+0000","last_fullsized":"2024-04-27T03:31:36.420503+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:45:58.123163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933497+0000","last_change":"2024-04-27T03:30:13.370393+0000","last_active":"2024-04-27T03:33:20.933497+0000","last_peered":"2024-04-27T03:33:20.933497+0000","last_clean":"2024-04-27T03:33:20.933497+0000","last_became_active":"2024-04-27T03:30:13.370066+0000","last_became_peered":"2024-04-27T03:30:13.370066+0000","last_unstale":"2024-04-27T03:33:20.933497+0000","last_undegraded":"2024-04-27T03:33:20.933497+0000","last_fullsized":"2024-04-27T03:33:20.933497+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:33.811635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"27'3","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288156+0000","last_change":"2024-04-27T03:30:11.360775+0000","last_active":"2024-04-27T03:32:19.288156+0000","last_peered":"2024-04-27T03:32:19.288156+0000","last_clean":"2024-04-27T03:32:19.288156+0000","last_became_active":"2024-04-27T03:30:11.360376+0000","last_became_peered":"2024-04-27T03:30:11.360376+0000","last_unstale":"2024-04-27T03:32:19.288156+0000","last_undegraded":"2024-04-27T03:32:19.288156+0000","last_fullsized":"2024-04-27T03:32:19.288156+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:00.825717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.792398+0000","last_change":"2024-04-27T03:30:13.378662+0000","last_active":"2024-04-27T03:33:20.792398+0000","last_peered":"2024-04-27T03:33:20.792398+0000","last_clean":"2024-04-27T03:33:20.792398+0000","last_became_active":"2024-04-27T03:30:13.378368+0000","last_became_peered":"2024-04-27T03:30:13.378368+0000","last_unstale":"2024-04-27T03:33:20.792398+0000","last_undegraded":"2024-04-27T03:33:20.792398+0000","last_fullsized":"2024-04-27T03:33:20.792398+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:56.906775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432139+0000","last_change":"2024-04-27T03:30:11.371839+0000","last_active":"2024-04-27T03:31:01.432139+0000","last_peered":"2024-04-27T03:31:01.432139+0000","last_clean":"2024-04-27T03:31:01.432139+0000","last_became_active":"2024-04-27T03:30:11.371553+0000","last_became_peered":"2024-04-27T03:30:11.371553+0000","last_unstale":"2024-04-27T03:31:01.432139+0000","last_undegraded":"2024-04-27T03:31:01.432139+0000","last_fullsized":"2024-04-27T03:31:01.432139+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.059046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"27'10","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009740+0000","last_change":"2024-04-27T03:31:01.440910+0000","last_active":"2024-04-27T03:33:21.009740+0000","last_peered":"2024-04-27T03:33:21.009740+0000","last_clean":"2024-04-27T03:33:21.009740+0000","last_became_active":"2024-04-27T03:31:01.440576+0000","last_became_peered":"2024-04-27T03:31:01.440576+0000","last_unstale":"2024-04-27T03:33:21.009740+0000","last_undegraded":"2024-04-27T03:33:21.009740+0000","last_fullsized":"2024-04-27T03:33:21.009740+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:04.874525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.213761+0000","last_change":"2024-04-27T03:30:11.371034+0000","last_active":"2024-04-27T03:32:20.213761+0000","last_peered":"2024-04-27T03:32:20.213761+0000","last_clean":"2024-04-27T03:32:20.213761+0000","last_became_active":"2024-04-27T03:30:11.369517+0000","last_became_peered":"2024-04-27T03:30:11.369517+0000","last_unstale":"2024-04-27T03:32:20.213761+0000","last_undegraded":"2024-04-27T03:32:20.213761+0000","last_fullsized":"2024-04-27T03:32:20.213761+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:22.719424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.107522+0000","last_change":"2024-04-27T03:30:13.374070+0000","last_active":"2024-04-27T03:32:17.107522+0000","last_peered":"2024-04-27T03:32:17.107522+0000","last_clean":"2024-04-27T03:32:17.107522+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:32:17.107522+0000","last_undegraded":"2024-04-27T03:32:17.107522+0000","last_fullsized":"2024-04-27T03:32:17.107522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:54.324828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.442163+0000","last_change":"2024-04-27T03:31:01.441506+0000","last_active":"2024-04-27T03:31:01.442163+0000","last_peered":"2024-04-27T03:31:01.442163+0000","last_clean":"2024-04-27T03:31:01.442163+0000","last_became_active":"2024-04-27T03:31:01.441159+0000","last_became_peered":"2024-04-27T03:31:01.441159+0000","last_unstale":"2024-04-27T03:31:01.442163+0000","last_undegraded":"2024-04-27T03:31:01.442163+0000","last_fullsized":"2024-04-27T03:31:01.442163+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:11:09.529188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"27'3","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.802333+0000","last_change":"2024-04-27T03:30:13.378083+0000","last_active":"2024-04-27T03:33:20.802333+0000","last_peered":"2024-04-27T03:33:20.802333+0000","last_clean":"2024-04-27T03:33:20.802333+0000","last_became_active":"2024-04-27T03:30:13.377821+0000","last_became_peered":"2024-04-27T03:30:13.377821+0000","last_unstale":"2024-04-27T03:33:20.802333+0000","last_undegraded":"2024-04-27T03:33:20.802333+0000","last_fullsized":"2024-04-27T03:33:20.802333+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:18.417469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432873+0000","last_change":"2024-04-27T03:30:11.371906+0000","last_active":"2024-04-27T03:31:01.432873+0000","last_peered":"2024-04-27T03:31:01.432873+0000","last_clean":"2024-04-27T03:31:01.432873+0000","last_became_active":"2024-04-27T03:30:11.371604+0000","last_became_peered":"2024-04-27T03:30:11.371604+0000","last_unstale":"2024-04-27T03:31:01.432873+0000","last_undegraded":"2024-04-27T03:31:01.432873+0000","last_fullsized":"2024-04-27T03:31:01.432873+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:33:05.923239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"27'8","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981663+0000","last_change":"2024-04-27T03:30:13.388711+0000","last_active":"2024-04-27T03:33:20.981663+0000","last_peered":"2024-04-27T03:33:20.981663+0000","last_clean":"2024-04-27T03:33:20.981663+0000","last_became_active":"2024-04-27T03:30:13.388421+0000","last_became_peered":"2024-04-27T03:30:13.388421+0000","last_unstale":"2024-04-27T03:33:20.981663+0000","last_undegraded":"2024-04-27T03:33:20.981663+0000","last_fullsized":"2024-04-27T03:33:20.981663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:35.526991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.678119+0000","last_change":"2024-04-27T03:30:11.355453+0000","last_active":"2024-04-27T03:33:20.678119+0000","last_peered":"2024-04-27T03:33:20.678119+0000","last_clean":"2024-04-27T03:33:20.678119+0000","last_became_active":"2024-04-27T03:30:11.355183+0000","last_became_peered":"2024-04-27T03:30:11.355183+0000","last_unstale":"2024-04-27T03:33:20.678119+0000","last_undegraded":"2024-04-27T03:33:20.678119+0000","last_fullsized":"2024-04-27T03:33:20.678119+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:12.667203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"27'4","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.107011+0000","last_change":"2024-04-27T03:30:13.377155+0000","last_active":"2024-04-27T03:32:17.107011+0000","last_peered":"2024-04-27T03:32:17.107011+0000","last_clean":"2024-04-27T03:32:17.107011+0000","last_became_active":"2024-04-27T03:30:13.376875+0000","last_became_peered":"2024-04-27T03:30:13.376875+0000","last_unstale":"2024-04-27T03:32:17.107011+0000","last_undegraded":"2024-04-27T03:32:17.107011+0000","last_fullsized":"2024-04-27T03:32:17.107011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:09.993585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431158+0000","last_change":"2024-04-27T03:30:11.361373+0000","last_active":"2024-04-27T03:31:01.431158+0000","last_peered":"2024-04-27T03:31:01.431158+0000","last_clean":"2024-04-27T03:31:01.431158+0000","last_became_active":"2024-04-27T03:30:11.360299+0000","last_became_peered":"2024-04-27T03:30:11.360299+0000","last_unstale":"2024-04-27T03:31:01.431158+0000","last_undegraded":"2024-04-27T03:31:01.431158+0000","last_fullsized":"2024-04-27T03:31:01.431158+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.746108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"27'10","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981744+0000","last_change":"2024-04-27T03:30:13.376269+0000","last_active":"2024-04-27T03:33:20.981744+0000","last_peered":"2024-04-27T03:33:20.981744+0000","last_clean":"2024-04-27T03:33:20.981744+0000","last_became_active":"2024-04-27T03:30:13.376087+0000","last_became_peered":"2024-04-27T03:30:13.376087+0000","last_unstale":"2024-04-27T03:33:20.981744+0000","last_undegraded":"2024-04-27T03:33:20.981744+0000","last_fullsized":"2024-04-27T03:33:20.981744+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:18.301070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432834+0000","last_change":"2024-04-27T03:30:11.371237+0000","last_active":"2024-04-27T03:31:01.432834+0000","last_peered":"2024-04-27T03:31:01.432834+0000","last_clean":"2024-04-27T03:31:01.432834+0000","last_became_active":"2024-04-27T03:30:11.370837+0000","last_became_peered":"2024-04-27T03:30:11.370837+0000","last_unstale":"2024-04-27T03:31:01.432834+0000","last_undegraded":"2024-04-27T03:31:01.432834+0000","last_fullsized":"2024-04-27T03:31:01.432834+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.863119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"27'6","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.697535+0000","last_change":"2024-04-27T03:30:13.376130+0000","last_active":"2024-04-27T03:33:20.697535+0000","last_peered":"2024-04-27T03:33:20.697535+0000","last_clean":"2024-04-27T03:33:20.697535+0000","last_became_active":"2024-04-27T03:30:13.373511+0000","last_became_peered":"2024-04-27T03:30:13.373511+0000","last_unstale":"2024-04-27T03:33:20.697535+0000","last_undegraded":"2024-04-27T03:33:20.697535+0000","last_fullsized":"2024-04-27T03:33:20.697535+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:51.434596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.257092+0000","last_change":"2024-04-27T03:30:11.370821+0000","last_active":"2024-04-27T03:32:20.257092+0000","last_peered":"2024-04-27T03:32:20.257092+0000","last_clean":"2024-04-27T03:32:20.257092+0000","last_became_active":"2024-04-27T03:30:11.370283+0000","last_became_peered":"2024-04-27T03:30:11.370283+0000","last_unstale":"2024-04-27T03:32:20.257092+0000","last_undegraded":"2024-04-27T03:32:20.257092+0000","last_fullsized":"2024-04-27T03:32:20.257092+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:17.981757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"27'11","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.016376+0000","last_change":"2024-04-27T03:30:13.380862+0000","last_active":"2024-04-27T03:33:21.016376+0000","last_peered":"2024-04-27T03:33:21.016376+0000","last_clean":"2024-04-27T03:33:21.016376+0000","last_became_active":"2024-04-27T03:30:13.380404+0000","last_became_peered":"2024-04-27T03:30:13.380404+0000","last_unstale":"2024-04-27T03:33:21.016376+0000","last_undegraded":"2024-04-27T03:33:21.016376+0000","last_fullsized":"2024-04-27T03:33:21.016376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:14.849083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439493+0000","last_change":"2024-04-27T03:30:11.373282+0000","last_active":"2024-04-27T03:31:01.439493+0000","last_peered":"2024-04-27T03:31:01.439493+0000","last_clean":"2024-04-27T03:31:01.439493+0000","last_became_active":"2024-04-27T03:30:11.373026+0000","last_became_peered":"2024-04-27T03:30:11.373026+0000","last_unstale":"2024-04-27T03:31:01.439493+0000","last_undegraded":"2024-04-27T03:31:01.439493+0000","last_fullsized":"2024-04-27T03:31:01.439493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:40:08.170228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"27'1","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295321+0000","last_change":"2024-04-27T03:30:13.386073+0000","last_active":"2024-04-27T03:32:29.295321+0000","last_peered":"2024-04-27T03:32:29.295321+0000","last_clean":"2024-04-27T03:32:29.295321+0000","last_became_active":"2024-04-27T03:30:13.385791+0000","last_became_peered":"2024-04-27T03:30:13.385791+0000","last_unstale":"2024-04-27T03:32:29.295321+0000","last_undegraded":"2024-04-27T03:32:29.295321+0000","last_fullsized":"2024-04-27T03:32:29.295321+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:18.301070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.441318+0000","last_change":"2024-04-27T03:30:11.358239+0000","last_active":"2024-04-27T03:31:01.441318+0000","last_peered":"2024-04-27T03:31:01.441318+0000","last_clean":"2024-04-27T03:31:01.441318+0000","last_became_active":"2024-04-27T03:30:11.357898+0000","last_became_peered":"2024-04-27T03:30:11.357898+0000","last_unstale":"2024-04-27T03:31:01.441318+0000","last_undegraded":"2024-04-27T03:31:01.441318+0000","last_fullsized":"2024-04-27T03:31:01.441318+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:26.250500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"27'5","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:19.303737+0000","last_change":"2024-04-27T03:31:01.444873+0000","last_active":"2024-04-27T03:33:19.303737+0000","last_peered":"2024-04-27T03:33:19.303737+0000","last_clean":"2024-04-27T03:33:19.303737+0000","last_became_active":"2024-04-27T03:31:01.440820+0000","last_became_peered":"2024-04-27T03:31:01.440820+0000","last_unstale":"2024-04-27T03:33:19.303737+0000","last_undegraded":"2024-04-27T03:33:19.303737+0000","last_fullsized":"2024-04-27T03:33:19.303737+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:15.446814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435323+0000","last_change":"2024-04-27T03:30:11.363676+0000","last_active":"2024-04-27T03:31:01.435323+0000","last_peered":"2024-04-27T03:31:01.435323+0000","last_clean":"2024-04-27T03:31:01.435323+0000","last_became_active":"2024-04-27T03:30:11.362788+0000","last_became_peered":"2024-04-27T03:30:11.362788+0000","last_unstale":"2024-04-27T03:31:01.435323+0000","last_undegraded":"2024-04-27T03:31:01.435323+0000","last_fullsized":"2024-04-27T03:31:01.435323+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:00.825717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"25'2","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436660+0000","last_change":"2024-04-27T03:30:13.378124+0000","last_active":"2024-04-27T03:31:01.436660+0000","last_peered":"2024-04-27T03:31:01.436660+0000","last_clean":"2024-04-27T03:31:01.436660+0000","last_became_active":"2024-04-27T03:30:13.377845+0000","last_became_peered":"2024-04-27T03:30:13.377845+0000","last_unstale":"2024-04-27T03:31:01.436660+0000","last_undegraded":"2024-04-27T03:31:01.436660+0000","last_fullsized":"2024-04-27T03:31:01.436660+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:37.770898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296913+0000","last_change":"2024-04-27T03:30:11.362917+0000","last_active":"2024-04-27T03:32:34.296913+0000","last_peered":"2024-04-27T03:32:34.296913+0000","last_clean":"2024-04-27T03:32:34.296913+0000","last_became_active":"2024-04-27T03:30:11.362559+0000","last_became_peered":"2024-04-27T03:30:11.362559+0000","last_unstale":"2024-04-27T03:32:34.296913+0000","last_undegraded":"2024-04-27T03:32:34.296913+0000","last_fullsized":"2024-04-27T03:32:34.296913+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:55.587028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436858+0000","last_change":"2024-04-27T03:30:11.363675+0000","last_active":"2024-04-27T03:31:01.436858+0000","last_peered":"2024-04-27T03:31:01.436858+0000","last_clean":"2024-04-27T03:31:01.436858+0000","last_became_active":"2024-04-27T03:30:11.363340+0000","last_became_peered":"2024-04-27T03:30:11.363340+0000","last_unstale":"2024-04-27T03:31:01.436858+0000","last_undegraded":"2024-04-27T03:31:01.436858+0000","last_fullsized":"2024-04-27T03:31:01.436858+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:26.857275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.107465+0000","last_change":"2024-04-27T03:30:13.386029+0000","last_active":"2024-04-27T03:32:17.107465+0000","last_peered":"2024-04-27T03:32:17.107465+0000","last_clean":"2024-04-27T03:32:17.107465+0000","last_became_active":"2024-04-27T03:30:13.385705+0000","last_became_peered":"2024-04-27T03:30:13.385705+0000","last_unstale":"2024-04-27T03:32:17.107465+0000","last_undegraded":"2024-04-27T03:32:17.107465+0000","last_fullsized":"2024-04-27T03:32:17.107465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:18.417469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430983+0000","last_change":"2024-04-27T03:30:11.360980+0000","last_active":"2024-04-27T03:31:01.430983+0000","last_peered":"2024-04-27T03:31:01.430983+0000","last_clean":"2024-04-27T03:31:01.430983+0000","last_became_active":"2024-04-27T03:30:11.360667+0000","last_became_peered":"2024-04-27T03:30:11.360667+0000","last_unstale":"2024-04-27T03:31:01.430983+0000","last_undegraded":"2024-04-27T03:31:01.430983+0000","last_fullsized":"2024-04-27T03:31:01.430983+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:51.203886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"27'14","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.305196+0000","last_change":"2024-04-27T03:30:13.380806+0000","last_active":"2024-04-27T03:32:20.305196+0000","last_peered":"2024-04-27T03:32:20.305196+0000","last_clean":"2024-04-27T03:32:20.305196+0000","last_became_active":"2024-04-27T03:30:13.380347+0000","last_became_peered":"2024-04-27T03:30:13.380347+0000","last_unstale":"2024-04-27T03:32:20.305196+0000","last_undegraded":"2024-04-27T03:32:20.305196+0000","last_fullsized":"2024-04-27T03:32:20.305196+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:48.533060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13385,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":1690,"num_read_kb":1444,"num_write":723,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1470,"ondisk_log_size":1470,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25805696,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":128,"num_read_kb":82,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":51617792,"data_stored":51611392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705706,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44568,"kb_used_data":9232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94327272,"statfs":{"total":96636764160,"available":96591126528,"internally_reserved":0,"allocated":9453568,"data_stored":9151139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705707,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60832,"kb_used_data":25496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94311008,"statfs":{"total":96636764160,"available":96574472192,"internally_reserved":0,"allocated":26107904,"data_stored":25873899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705706,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1613824,"data_stored":1348704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":134,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56256,"kb_used_data":16760,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94315584,"statfs":{"total":96636764160,"available":96579158016,"internally_reserved":0,"allocated":17162240,"data_stored":16898458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16777216,"data_stored":16777216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":643072,"data_stored":639872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25165824,"data_stored":25165824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9031680,"data_stored":9028480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":4269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T03:33:28.063 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T03:33:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.180+0000 7fa3960ac640 1 Processor -- start 2024-04-27T03:33:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.180+0000 7fa3960ac640 1 -- start start 2024-04-27T03:33:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.180+0000 7fa3960ac640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa390127ce0 0x7fa3901280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:28.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.180+0000 7fa3960ac640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fa390127b00 con 0x7fa390070100 2024-04-27T03:33:28.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.180+0000 7fa3960ac640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fa3900708e0 con 0x7fa390127ce0 2024-04-27T03:33:28.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.180+0000 7fa38f7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fa390127ce0 0x7fa3901280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:28.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.180+0000 7fa38effd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fa390070100 0x7fa39012b000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48008/0 (socket says 172.21.15.89:48008) 2024-04-27T03:33:28.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.180+0000 7fa38effd640 1 -- 172.21.15.89:0/1874238590 learned_addr learned my addr 172.21.15.89:0/1874238590 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:28.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.181+0000 7fa38f7fe640 1 -- 172.21.15.89:0/1874238590 >> v1:172.21.15.89:6789/0 conn(0x7fa390070100 legacy=0x7fa39012b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:28.187 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.181+0000 7fa38f7fe640 1 -- 172.21.15.89:0/1874238590 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa390098800 con 0x7fa390127ce0 2024-04-27T03:33:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.181+0000 7fa38f7fe640 1 --2- 172.21.15.89:0/1874238590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa390127ce0 0x7fa3901280c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa38000b9c0 tx=0x7fa38002e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10d719b270541806 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.181+0000 7fa38e7fc640 1 -- 172.21.15.89:0/1874238590 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa38003b070 con 0x7fa390127ce0 2024-04-27T03:33:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.181+0000 7fa38e7fc640 1 -- 172.21.15.89:0/1874238590 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa38002ecf0 con 0x7fa390127ce0 2024-04-27T03:33:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.181+0000 7fa38e7fc640 1 -- 172.21.15.89:0/1874238590 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa380036440 con 0x7fa390127ce0 2024-04-27T03:33:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.182+0000 7fa3960ac640 1 -- 172.21.15.89:0/1874238590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa390127ce0 msgr2=0x7fa3901280c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.182+0000 7fa3960ac640 1 --2- 172.21.15.89:0/1874238590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa390127ce0 0x7fa3901280c0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fa38000b9c0 tx=0x7fa38002e010 comp rx=0 tx=0).stop 2024-04-27T03:33:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.182+0000 7fa3960ac640 1 -- 172.21.15.89:0/1874238590 shutdown_connections 2024-04-27T03:33:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.182+0000 7fa3960ac640 1 --2- 172.21.15.89:0/1874238590 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa390127ce0 0x7fa3901280c0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.182+0000 7fa3960ac640 1 -- 172.21.15.89:0/1874238590 >> 172.21.15.89:0/1874238590 conn(0x7fa390099680 msgr2=0x7fa390099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:28.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.182+0000 7fa3960ac640 1 -- 172.21.15.89:0/1874238590 shutdown_connections 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.182+0000 7fa3960ac640 1 -- 172.21.15.89:0/1874238590 wait complete. 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.183+0000 7fa3960ac640 1 Processor -- start 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.183+0000 7fa3960ac640 1 -- start start 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.183+0000 7fa3960ac640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa390070100 0x7fa39008e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.183+0000 7fa3960ac640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fa390128780 con 0x7fa390070100 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.183+0000 7fa38f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa390070100 0x7fa39008e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.183+0000 7fa38f7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa390070100 0x7fa39008e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60208/0 (socket says 172.21.15.89:60208) 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.183+0000 7fa38f7fe640 1 -- 172.21.15.89:0/2863100254 learned_addr learned my addr 172.21.15.89:0/2863100254 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.184+0000 7fa38f7fe640 1 -- 172.21.15.89:0/2863100254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa390125260 con 0x7fa390070100 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.184+0000 7fa38f7fe640 1 --2- 172.21.15.89:0/2863100254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa390070100 0x7fa39008e3a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa380039040 tx=0x7fa3800046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.184+0000 7fa377fff640 1 -- 172.21.15.89:0/2863100254 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa380046070 con 0x7fa390070100 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.184+0000 7fa377fff640 1 -- 172.21.15.89:0/2863100254 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa380041560 con 0x7fa390070100 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.184+0000 7fa3960ac640 1 -- 172.21.15.89:0/2863100254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa390090040 con 0x7fa390070100 2024-04-27T03:33:28.191 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.184+0000 7fa377fff640 1 -- 172.21.15.89:0/2863100254 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa38003b040 con 0x7fa390070100 2024-04-27T03:33:28.192 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.184+0000 7fa3960ac640 1 -- 172.21.15.89:0/2863100254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa39008e8e0 con 0x7fa390070100 2024-04-27T03:33:28.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.185+0000 7fa377fff640 1 -- 172.21.15.89:0/2863100254 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fa380049070 con 0x7fa390070100 2024-04-27T03:33:28.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.185+0000 7fa377fff640 1 --2- 172.21.15.89:0/2863100254 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa37c038060 0x7fa37c03a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:28.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.185+0000 7fa3960ac640 1 -- 172.21.15.89:0/2863100254 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa39008fab0 con 0x7fa390070100 2024-04-27T03:33:28.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.185+0000 7fa377fff640 1 -- 172.21.15.89:0/2863100254 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fa38006d710 con 0x7fa390070100 2024-04-27T03:33:28.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.185+0000 7fa38effd640 1 --2- 172.21.15.89:0/2863100254 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa37c038060 0x7fa37c03a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:28.193 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.186+0000 7fa38effd640 1 --2- 172.21.15.89:0/2863100254 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa37c038060 0x7fa37c03a520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa384007fd0 tx=0x7fa384007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:28.195 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.190+0000 7fa377fff640 1 -- 172.21.15.89:0/2863100254 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fa380049360 con 0x7fa390070100 2024-04-27T03:33:28.364 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.358+0000 7fa3960ac640 1 -- 172.21.15.89:0/2863100254 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fa39008f870 con 0x7fa37c038060 2024-04-27T03:33:28.378 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.373+0000 7fa377fff640 1 -- 172.21.15.89:0/2863100254 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665463 (secure 0 0 0) 0x7fa39008f870 con 0x7fa37c038060 2024-04-27T03:33:28.386 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:33:28.391 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-04-27T03:33:28.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.377+0000 7fa3960ac640 1 -- 172.21.15.89:0/2863100254 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa37c038060 msgr2=0x7fa37c03a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:28.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.377+0000 7fa3960ac640 1 --2- 172.21.15.89:0/2863100254 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa37c038060 0x7fa37c03a520 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa384007fd0 tx=0x7fa384007ac0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.377+0000 7fa3960ac640 1 -- 172.21.15.89:0/2863100254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa390070100 msgr2=0x7fa39008e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:28.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.377+0000 7fa3960ac640 1 --2- 172.21.15.89:0/2863100254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa390070100 0x7fa39008e3a0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fa380039040 tx=0x7fa3800046c0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.377+0000 7fa3960ac640 1 -- 172.21.15.89:0/2863100254 shutdown_connections 2024-04-27T03:33:28.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.377+0000 7fa3960ac640 1 --2- 172.21.15.89:0/2863100254 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fa37c038060 0x7fa37c03a520 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.394 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.377+0000 7fa3960ac640 1 --2- 172.21.15.89:0/2863100254 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fa390070100 0x7fa39008e3a0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.377+0000 7fa3960ac640 1 -- 172.21.15.89:0/2863100254 >> 172.21.15.89:0/2863100254 conn(0x7fa390099680 msgr2=0x7fa390126300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:28.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.377+0000 7fa3960ac640 1 -- 172.21.15.89:0/2863100254 shutdown_connections 2024-04-27T03:33:28.395 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.377+0000 7fa3960ac640 1 -- 172.21.15.89:0/2863100254 wait complete. 2024-04-27T03:33:28.395 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-04-27T03:33:28.271321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26412790,"num_objects":259,"num_object_clones":0,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":0,"num_read":5162,"num_read_kb":4712,"num_write":2296,"num_write_kb":61372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4671,"ondisk_log_size":4671,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":198568,"kb_used_data":53064,"kb_used_omap":6,"kb_used_meta":145465,"kb_avail":377288792,"statfs":{"total":386547056640,"available":386343723008,"internally_reserved":0,"allocated":54337536,"data_stored":53272200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":148956950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25811688,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":497,"num_read_kb":452,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004184"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433166+0000","last_change":"2024-04-27T03:30:11.362820+0000","last_active":"2024-04-27T03:31:01.433166+0000","last_peered":"2024-04-27T03:31:01.433166+0000","last_clean":"2024-04-27T03:31:01.433166+0000","last_became_active":"2024-04-27T03:30:11.362574+0000","last_became_peered":"2024-04-27T03:30:11.362574+0000","last_unstale":"2024-04-27T03:31:01.433166+0000","last_undegraded":"2024-04-27T03:31:01.433166+0000","last_fullsized":"2024-04-27T03:31:01.433166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.746108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933798+0000","last_change":"2024-04-27T03:30:13.384819+0000","last_active":"2024-04-27T03:33:20.933798+0000","last_peered":"2024-04-27T03:33:20.933798+0000","last_clean":"2024-04-27T03:33:20.933798+0000","last_became_active":"2024-04-27T03:30:13.384580+0000","last_became_peered":"2024-04-27T03:30:13.384580+0000","last_unstale":"2024-04-27T03:33:20.933798+0000","last_undegraded":"2024-04-27T03:33:20.933798+0000","last_fullsized":"2024-04-27T03:33:20.933798+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:07:51.011104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.921795+0000","last_change":"2024-04-27T03:30:11.361366+0000","last_active":"2024-04-27T03:33:20.921795+0000","last_peered":"2024-04-27T03:33:20.921795+0000","last_clean":"2024-04-27T03:33:20.921795+0000","last_became_active":"2024-04-27T03:30:11.360857+0000","last_became_peered":"2024-04-27T03:30:11.360857+0000","last_unstale":"2024-04-27T03:33:20.921795+0000","last_undegraded":"2024-04-27T03:33:20.921795+0000","last_fullsized":"2024-04-27T03:33:20.921795+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:11.138117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"27'12","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.695008+0000","last_change":"2024-04-27T03:30:13.376545+0000","last_active":"2024-04-27T03:33:20.695008+0000","last_peered":"2024-04-27T03:33:20.695008+0000","last_clean":"2024-04-27T03:33:20.695008+0000","last_became_active":"2024-04-27T03:30:13.376221+0000","last_became_peered":"2024-04-27T03:30:13.376221+0000","last_unstale":"2024-04-27T03:33:20.695008+0000","last_undegraded":"2024-04-27T03:33:20.695008+0000","last_fullsized":"2024-04-27T03:33:20.695008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:08.985512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440740+0000","last_change":"2024-04-27T03:30:11.363460+0000","last_active":"2024-04-27T03:31:01.440740+0000","last_peered":"2024-04-27T03:31:01.440740+0000","last_clean":"2024-04-27T03:31:01.440740+0000","last_became_active":"2024-04-27T03:30:11.362739+0000","last_became_peered":"2024-04-27T03:30:11.362739+0000","last_unstale":"2024-04-27T03:31:01.440740+0000","last_undegraded":"2024-04-27T03:31:01.440740+0000","last_fullsized":"2024-04-27T03:31:01.440740+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:12.842711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"27'7","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298536+0000","last_change":"2024-04-27T03:30:13.376819+0000","last_active":"2024-04-27T03:32:44.298536+0000","last_peered":"2024-04-27T03:32:44.298536+0000","last_clean":"2024-04-27T03:32:44.298536+0000","last_became_active":"2024-04-27T03:30:13.376524+0000","last_became_peered":"2024-04-27T03:30:13.376524+0000","last_unstale":"2024-04-27T03:32:44.298536+0000","last_undegraded":"2024-04-27T03:32:44.298536+0000","last_fullsized":"2024-04-27T03:32:44.298536+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:54:30.451033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.293096+0000","last_change":"2024-04-27T03:30:11.361039+0000","last_active":"2024-04-27T03:32:14.293096+0000","last_peered":"2024-04-27T03:32:14.293096+0000","last_clean":"2024-04-27T03:32:14.293096+0000","last_became_active":"2024-04-27T03:30:11.360188+0000","last_became_peered":"2024-04-27T03:30:11.360188+0000","last_unstale":"2024-04-27T03:32:14.293096+0000","last_undegraded":"2024-04-27T03:32:14.293096+0000","last_fullsized":"2024-04-27T03:32:14.293096+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:22.719424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"27'16","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009953+0000","last_change":"2024-04-27T03:30:13.376574+0000","last_active":"2024-04-27T03:33:21.009953+0000","last_peered":"2024-04-27T03:33:21.009953+0000","last_clean":"2024-04-27T03:33:21.009953+0000","last_became_active":"2024-04-27T03:30:13.376288+0000","last_became_peered":"2024-04-27T03:30:13.376288+0000","last_unstale":"2024-04-27T03:33:21.009953+0000","last_undegraded":"2024-04-27T03:33:21.009953+0000","last_fullsized":"2024-04-27T03:33:21.009953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:15.187356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439707+0000","last_change":"2024-04-27T03:30:11.355274+0000","last_active":"2024-04-27T03:31:01.439707+0000","last_peered":"2024-04-27T03:31:01.439707+0000","last_clean":"2024-04-27T03:31:01.439707+0000","last_became_active":"2024-04-27T03:30:11.354989+0000","last_became_peered":"2024-04-27T03:30:11.354989+0000","last_unstale":"2024-04-27T03:31:01.439707+0000","last_undegraded":"2024-04-27T03:31:01.439707+0000","last_fullsized":"2024-04-27T03:31:01.439707+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:11.281867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933479+0000","last_change":"2024-04-27T03:30:13.374182+0000","last_active":"2024-04-27T03:33:20.933479+0000","last_peered":"2024-04-27T03:33:20.933479+0000","last_clean":"2024-04-27T03:33:20.933479+0000","last_became_active":"2024-04-27T03:30:13.373424+0000","last_became_peered":"2024-04-27T03:30:13.373424+0000","last_unstale":"2024-04-27T03:33:20.933479+0000","last_undegraded":"2024-04-27T03:33:20.933479+0000","last_fullsized":"2024-04-27T03:33:20.933479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:02.825441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.657187+0000","last_change":"2024-04-27T03:30:11.361019+0000","last_active":"2024-04-27T03:33:20.657187+0000","last_peered":"2024-04-27T03:33:20.657187+0000","last_clean":"2024-04-27T03:33:20.657187+0000","last_became_active":"2024-04-27T03:30:11.360219+0000","last_became_peered":"2024-04-27T03:30:11.360219+0000","last_unstale":"2024-04-27T03:33:20.657187+0000","last_undegraded":"2024-04-27T03:33:20.657187+0000","last_fullsized":"2024-04-27T03:33:20.657187+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:45.376863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"25'8","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440191+0000","last_change":"2024-04-27T03:30:13.385912+0000","last_active":"2024-04-27T03:31:01.440191+0000","last_peered":"2024-04-27T03:31:01.440191+0000","last_clean":"2024-04-27T03:31:01.440191+0000","last_became_active":"2024-04-27T03:30:13.385665+0000","last_became_peered":"2024-04-27T03:30:13.385665+0000","last_unstale":"2024-04-27T03:31:01.440191+0000","last_undegraded":"2024-04-27T03:31:01.440191+0000","last_fullsized":"2024-04-27T03:31:01.440191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:49.082066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437953+0000","last_change":"2024-04-27T03:30:11.373049+0000","last_active":"2024-04-27T03:31:01.437953+0000","last_peered":"2024-04-27T03:31:01.437953+0000","last_clean":"2024-04-27T03:31:01.437953+0000","last_became_active":"2024-04-27T03:30:11.372704+0000","last_became_peered":"2024-04-27T03:30:11.372704+0000","last_unstale":"2024-04-27T03:31:01.437953+0000","last_undegraded":"2024-04-27T03:31:01.437953+0000","last_fullsized":"2024-04-27T03:31:01.437953+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:03.639873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"27'4","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981936+0000","last_change":"2024-04-27T03:30:13.374520+0000","last_active":"2024-04-27T03:33:20.981936+0000","last_peered":"2024-04-27T03:33:20.981936+0000","last_clean":"2024-04-27T03:33:20.981936+0000","last_became_active":"2024-04-27T03:30:13.374199+0000","last_became_peered":"2024-04-27T03:30:13.374199+0000","last_unstale":"2024-04-27T03:33:20.981936+0000","last_undegraded":"2024-04-27T03:33:20.981936+0000","last_fullsized":"2024-04-27T03:33:20.981936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:33.811635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.926966+0000","last_change":"2024-04-27T03:30:11.358470+0000","last_active":"2024-04-27T03:33:20.926966+0000","last_peered":"2024-04-27T03:33:20.926966+0000","last_clean":"2024-04-27T03:33:20.926966+0000","last_became_active":"2024-04-27T03:30:11.358181+0000","last_became_peered":"2024-04-27T03:30:11.358181+0000","last_unstale":"2024-04-27T03:33:20.926966+0000","last_undegraded":"2024-04-27T03:33:20.926966+0000","last_fullsized":"2024-04-27T03:33:20.926966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.921690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.794975+0000","last_change":"2024-04-27T03:30:13.379341+0000","last_active":"2024-04-27T03:33:20.794975+0000","last_peered":"2024-04-27T03:33:20.794975+0000","last_clean":"2024-04-27T03:33:20.794975+0000","last_became_active":"2024-04-27T03:30:13.379059+0000","last_became_peered":"2024-04-27T03:30:13.379059+0000","last_unstale":"2024-04-27T03:33:20.794975+0000","last_undegraded":"2024-04-27T03:33:20.794975+0000","last_fullsized":"2024-04-27T03:33:20.794975+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:56.906775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"27'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.010139+0000","last_change":"2024-04-27T03:30:13.383971+0000","last_active":"2024-04-27T03:33:21.010139+0000","last_peered":"2024-04-27T03:33:21.010139+0000","last_clean":"2024-04-27T03:33:21.010139+0000","last_became_active":"2024-04-27T03:30:13.383694+0000","last_became_peered":"2024-04-27T03:30:13.383694+0000","last_unstale":"2024-04-27T03:33:21.010139+0000","last_undegraded":"2024-04-27T03:33:21.010139+0000","last_fullsized":"2024-04-27T03:33:21.010139+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:36:25.083284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436571+0000","last_change":"2024-04-27T03:30:11.370455+0000","last_active":"2024-04-27T03:31:01.436571+0000","last_peered":"2024-04-27T03:31:01.436571+0000","last_clean":"2024-04-27T03:31:01.436571+0000","last_became_active":"2024-04-27T03:30:11.370157+0000","last_became_peered":"2024-04-27T03:30:11.370157+0000","last_unstale":"2024-04-27T03:31:01.436571+0000","last_undegraded":"2024-04-27T03:31:01.436571+0000","last_fullsized":"2024-04-27T03:31:01.436571+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:06.887756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"27'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009774+0000","last_change":"2024-04-27T03:30:13.373269+0000","last_active":"2024-04-27T03:33:21.009774+0000","last_peered":"2024-04-27T03:33:21.009774+0000","last_clean":"2024-04-27T03:33:21.009774+0000","last_became_active":"2024-04-27T03:30:13.372973+0000","last_became_peered":"2024-04-27T03:30:13.372973+0000","last_unstale":"2024-04-27T03:33:21.009774+0000","last_undegraded":"2024-04-27T03:33:21.009774+0000","last_fullsized":"2024-04-27T03:33:21.009774+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:57.593400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297374+0000","last_change":"2024-04-27T03:30:11.363764+0000","last_active":"2024-04-27T03:32:39.297374+0000","last_peered":"2024-04-27T03:32:39.297374+0000","last_clean":"2024-04-27T03:32:39.297374+0000","last_became_active":"2024-04-27T03:30:11.363109+0000","last_became_peered":"2024-04-27T03:30:11.363109+0000","last_unstale":"2024-04-27T03:32:39.297374+0000","last_undegraded":"2024-04-27T03:32:39.297374+0000","last_fullsized":"2024-04-27T03:32:39.297374+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:52:31.047433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"27'9","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988406+0000","last_change":"2024-04-27T03:30:13.384110+0000","last_active":"2024-04-27T03:33:20.988406+0000","last_peered":"2024-04-27T03:33:20.988406+0000","last_clean":"2024-04-27T03:33:20.988406+0000","last_became_active":"2024-04-27T03:30:13.383742+0000","last_became_peered":"2024-04-27T03:30:13.383742+0000","last_unstale":"2024-04-27T03:33:20.988406+0000","last_undegraded":"2024-04-27T03:33:20.988406+0000","last_fullsized":"2024-04-27T03:33:20.988406+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:14.799339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"25'4","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436806+0000","last_change":"2024-04-27T03:30:11.370635+0000","last_active":"2024-04-27T03:31:01.436806+0000","last_peered":"2024-04-27T03:31:01.436806+0000","last_clean":"2024-04-27T03:31:01.436806+0000","last_became_active":"2024-04-27T03:30:11.370380+0000","last_became_peered":"2024-04-27T03:30:11.370380+0000","last_unstale":"2024-04-27T03:31:01.436806+0000","last_undegraded":"2024-04-27T03:31:01.436806+0000","last_fullsized":"2024-04-27T03:31:01.436806+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:45.376863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"27'6","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009766+0000","last_change":"2024-04-27T03:31:01.446266+0000","last_active":"2024-04-27T03:33:21.009766+0000","last_peered":"2024-04-27T03:33:21.009766+0000","last_clean":"2024-04-27T03:33:21.009766+0000","last_became_active":"2024-04-27T03:31:01.440762+0000","last_became_peered":"2024-04-27T03:31:01.440762+0000","last_unstale":"2024-04-27T03:33:21.009766+0000","last_undegraded":"2024-04-27T03:33:21.009766+0000","last_fullsized":"2024-04-27T03:33:21.009766+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:58.987011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.279469+0000","last_change":"2024-04-27T03:30:11.369834+0000","last_active":"2024-04-27T03:32:20.279469+0000","last_peered":"2024-04-27T03:32:20.279469+0000","last_clean":"2024-04-27T03:32:20.279469+0000","last_became_active":"2024-04-27T03:30:11.369551+0000","last_became_peered":"2024-04-27T03:30:11.369551+0000","last_unstale":"2024-04-27T03:32:20.279469+0000","last_undegraded":"2024-04-27T03:32:20.279469+0000","last_fullsized":"2024-04-27T03:32:20.279469+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.921690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"27'2","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981818+0000","last_change":"2024-04-27T03:30:13.382150+0000","last_active":"2024-04-27T03:33:20.981818+0000","last_peered":"2024-04-27T03:33:20.981818+0000","last_clean":"2024-04-27T03:33:20.981818+0000","last_became_active":"2024-04-27T03:30:13.381901+0000","last_became_peered":"2024-04-27T03:30:13.381901+0000","last_unstale":"2024-04-27T03:33:20.981818+0000","last_undegraded":"2024-04-27T03:33:20.981818+0000","last_fullsized":"2024-04-27T03:33:20.981818+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:05.646245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"25'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431969+0000","last_change":"2024-04-27T03:30:11.361129+0000","last_active":"2024-04-27T03:31:01.431969+0000","last_peered":"2024-04-27T03:31:01.431969+0000","last_clean":"2024-04-27T03:31:01.431969+0000","last_became_active":"2024-04-27T03:30:11.360515+0000","last_became_peered":"2024-04-27T03:30:11.360515+0000","last_unstale":"2024-04-27T03:31:01.431969+0000","last_undegraded":"2024-04-27T03:31:01.431969+0000","last_fullsized":"2024-04-27T03:31:01.431969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:52:31.047433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"27'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.307097+0000","last_change":"2024-04-27T03:30:13.382249+0000","last_active":"2024-04-27T03:32:20.307097+0000","last_peered":"2024-04-27T03:32:20.307097+0000","last_clean":"2024-04-27T03:32:20.307097+0000","last_became_active":"2024-04-27T03:30:13.378932+0000","last_became_peered":"2024-04-27T03:30:13.378932+0000","last_unstale":"2024-04-27T03:32:20.307097+0000","last_undegraded":"2024-04-27T03:32:20.307097+0000","last_fullsized":"2024-04-27T03:32:20.307097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:35.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437123+0000","last_change":"2024-04-27T03:30:11.372964+0000","last_active":"2024-04-27T03:31:01.437123+0000","last_peered":"2024-04-27T03:31:01.437123+0000","last_clean":"2024-04-27T03:31:01.437123+0000","last_became_active":"2024-04-27T03:30:11.372673+0000","last_became_peered":"2024-04-27T03:30:11.372673+0000","last_unstale":"2024-04-27T03:31:01.437123+0000","last_undegraded":"2024-04-27T03:31:01.437123+0000","last_fullsized":"2024-04-27T03:31:01.437123+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:08:10.593535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"27'35","reported_seq":99,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.957246+0000","last_change":"2024-04-27T03:30:21.413597+0000","last_active":"2024-04-27T03:33:20.957246+0000","last_peered":"2024-04-27T03:33:20.957246+0000","last_clean":"2024-04-27T03:33:20.957246+0000","last_became_active":"2024-04-27T03:30:21.413255+0000","last_became_peered":"2024-04-27T03:30:21.413255+0000","last_unstale":"2024-04-27T03:33:20.957246+0000","last_undegraded":"2024-04-27T03:33:20.957246+0000","last_fullsized":"2024-04-27T03:33:20.957246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:11:19.581338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"27'3","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296833+0000","last_change":"2024-04-27T03:30:13.377382+0000","last_active":"2024-04-27T03:32:34.296833+0000","last_peered":"2024-04-27T03:32:34.296833+0000","last_clean":"2024-04-27T03:32:34.296833+0000","last_became_active":"2024-04-27T03:30:13.377110+0000","last_became_peered":"2024-04-27T03:30:13.377110+0000","last_unstale":"2024-04-27T03:32:34.296833+0000","last_undegraded":"2024-04-27T03:32:34.296833+0000","last_fullsized":"2024-04-27T03:32:34.296833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:21:27.452807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439438+0000","last_change":"2024-04-27T03:30:11.360152+0000","last_active":"2024-04-27T03:31:01.439438+0000","last_peered":"2024-04-27T03:31:01.439438+0000","last_clean":"2024-04-27T03:31:01.439438+0000","last_became_active":"2024-04-27T03:30:11.359892+0000","last_became_peered":"2024-04-27T03:30:11.359892+0000","last_unstale":"2024-04-27T03:31:01.439438+0000","last_undegraded":"2024-04-27T03:31:01.439438+0000","last_fullsized":"2024-04-27T03:31:01.439438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:07:49.004733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440221+0000","last_change":"2024-04-27T03:30:15.381168+0000","last_active":"2024-04-27T03:31:01.440221+0000","last_peered":"2024-04-27T03:31:01.440221+0000","last_clean":"2024-04-27T03:31:01.440221+0000","last_became_active":"2024-04-27T03:30:15.380864+0000","last_became_peered":"2024-04-27T03:30:15.380864+0000","last_unstale":"2024-04-27T03:31:01.440221+0000","last_undegraded":"2024-04-27T03:31:01.440221+0000","last_fullsized":"2024-04-27T03:31:01.440221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:57:52.741876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"27'42","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.914579+0000","last_change":"2024-04-27T03:30:17.388325+0000","last_active":"2024-04-27T03:32:18.914579+0000","last_peered":"2024-04-27T03:32:18.914579+0000","last_clean":"2024-04-27T03:32:18.914579+0000","last_became_active":"2024-04-27T03:30:17.387391+0000","last_became_peered":"2024-04-27T03:30:17.387391+0000","last_unstale":"2024-04-27T03:32:18.914579+0000","last_undegraded":"2024-04-27T03:32:18.914579+0000","last_fullsized":"2024-04-27T03:32:18.914579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:02:27.282043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433271+0000","last_change":"2024-04-27T03:30:19.407954+0000","last_active":"2024-04-27T03:31:01.433271+0000","last_peered":"2024-04-27T03:31:01.433271+0000","last_clean":"2024-04-27T03:31:01.433271+0000","last_became_active":"2024-04-27T03:30:19.407637+0000","last_became_peered":"2024-04-27T03:30:19.407637+0000","last_unstale":"2024-04-27T03:31:01.433271+0000","last_undegraded":"2024-04-27T03:31:01.433271+0000","last_fullsized":"2024-04-27T03:31:01.433271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:12.277984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"27'9","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.436568+0000","last_change":"2024-04-27T03:30:21.414223+0000","last_active":"2024-04-27T03:31:36.436568+0000","last_peered":"2024-04-27T03:31:36.436568+0000","last_clean":"2024-04-27T03:31:36.436568+0000","last_became_active":"2024-04-27T03:30:21.413940+0000","last_became_peered":"2024-04-27T03:30:21.413940+0000","last_unstale":"2024-04-27T03:31:36.436568+0000","last_undegraded":"2024-04-27T03:31:36.436568+0000","last_fullsized":"2024-04-27T03:31:36.436568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:44.051317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"27'16","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981635+0000","last_change":"2024-04-27T03:30:13.381748+0000","last_active":"2024-04-27T03:33:20.981635+0000","last_peered":"2024-04-27T03:33:20.981635+0000","last_clean":"2024-04-27T03:33:20.981635+0000","last_became_active":"2024-04-27T03:30:13.381487+0000","last_became_peered":"2024-04-27T03:30:13.381487+0000","last_unstale":"2024-04-27T03:33:20.981635+0000","last_undegraded":"2024-04-27T03:33:20.981635+0000","last_fullsized":"2024-04-27T03:33:20.981635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:52.946275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435664+0000","last_change":"2024-04-27T03:30:11.359690+0000","last_active":"2024-04-27T03:31:01.435664+0000","last_peered":"2024-04-27T03:31:01.435664+0000","last_clean":"2024-04-27T03:31:01.435664+0000","last_became_active":"2024-04-27T03:30:11.359446+0000","last_became_peered":"2024-04-27T03:30:11.359446+0000","last_unstale":"2024-04-27T03:31:01.435664+0000","last_undegraded":"2024-04-27T03:31:01.435664+0000","last_fullsized":"2024-04-27T03:31:01.435664+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.921690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431730+0000","last_change":"2024-04-27T03:30:15.380566+0000","last_active":"2024-04-27T03:31:01.431730+0000","last_peered":"2024-04-27T03:31:01.431730+0000","last_clean":"2024-04-27T03:31:01.431730+0000","last_became_active":"2024-04-27T03:30:15.380313+0000","last_became_peered":"2024-04-27T03:30:15.380313+0000","last_unstale":"2024-04-27T03:31:01.431730+0000","last_undegraded":"2024-04-27T03:31:01.431730+0000","last_fullsized":"2024-04-27T03:31:01.431730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:45:16.754705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"25'12","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433370+0000","last_change":"2024-04-27T03:30:17.389399+0000","last_active":"2024-04-27T03:31:01.433370+0000","last_peered":"2024-04-27T03:31:01.433370+0000","last_clean":"2024-04-27T03:31:01.433370+0000","last_became_active":"2024-04-27T03:30:17.389126+0000","last_became_peered":"2024-04-27T03:30:17.389126+0000","last_unstale":"2024-04-27T03:31:01.433370+0000","last_undegraded":"2024-04-27T03:31:01.433370+0000","last_fullsized":"2024-04-27T03:31:01.433370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:42:03.150440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432915+0000","last_change":"2024-04-27T03:30:19.405456+0000","last_active":"2024-04-27T03:31:01.432915+0000","last_peered":"2024-04-27T03:31:01.432915+0000","last_clean":"2024-04-27T03:31:01.432915+0000","last_became_active":"2024-04-27T03:30:19.405170+0000","last_became_peered":"2024-04-27T03:30:19.405170+0000","last_unstale":"2024-04-27T03:31:01.432915+0000","last_undegraded":"2024-04-27T03:31:01.432915+0000","last_fullsized":"2024-04-27T03:31:01.432915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:31:41.885815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'6","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.427217+0000","last_change":"2024-04-27T03:30:21.408308+0000","last_active":"2024-04-27T03:31:36.427217+0000","last_peered":"2024-04-27T03:31:36.427217+0000","last_clean":"2024-04-27T03:31:36.427217+0000","last_became_active":"2024-04-27T03:30:21.407917+0000","last_became_peered":"2024-04-27T03:30:21.407917+0000","last_unstale":"2024-04-27T03:31:36.427217+0000","last_undegraded":"2024-04-27T03:31:36.427217+0000","last_fullsized":"2024-04-27T03:31:36.427217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:23.492592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":95,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.792066+0000","last_change":"2024-04-27T03:30:13.378024+0000","last_active":"2024-04-27T03:33:20.792066+0000","last_peered":"2024-04-27T03:33:20.792066+0000","last_clean":"2024-04-27T03:33:20.792066+0000","last_became_active":"2024-04-27T03:30:13.377423+0000","last_became_peered":"2024-04-27T03:30:13.377423+0000","last_unstale":"2024-04-27T03:33:20.792066+0000","last_undegraded":"2024-04-27T03:33:20.792066+0000","last_fullsized":"2024-04-27T03:33:20.792066+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:22.057065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437475+0000","last_change":"2024-04-27T03:30:11.359542+0000","last_active":"2024-04-27T03:31:01.437475+0000","last_peered":"2024-04-27T03:31:01.437475+0000","last_clean":"2024-04-27T03:31:01.437475+0000","last_became_active":"2024-04-27T03:30:11.359261+0000","last_became_peered":"2024-04-27T03:30:11.359261+0000","last_unstale":"2024-04-27T03:31:01.437475+0000","last_undegraded":"2024-04-27T03:31:01.437475+0000","last_fullsized":"2024-04-27T03:31:01.437475+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:19.905319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433488+0000","last_change":"2024-04-27T03:30:15.383902+0000","last_active":"2024-04-27T03:31:01.433488+0000","last_peered":"2024-04-27T03:31:01.433488+0000","last_clean":"2024-04-27T03:31:01.433488+0000","last_became_active":"2024-04-27T03:30:15.383618+0000","last_became_peered":"2024-04-27T03:30:15.383618+0000","last_unstale":"2024-04-27T03:31:01.433488+0000","last_undegraded":"2024-04-27T03:31:01.433488+0000","last_fullsized":"2024-04-27T03:31:01.433488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:54.227106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"25'23","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433761+0000","last_change":"2024-04-27T03:30:17.392488+0000","last_active":"2024-04-27T03:31:01.433761+0000","last_peered":"2024-04-27T03:31:01.433761+0000","last_clean":"2024-04-27T03:31:01.433761+0000","last_became_active":"2024-04-27T03:30:17.392223+0000","last_became_peered":"2024-04-27T03:30:17.392223+0000","last_unstale":"2024-04-27T03:31:01.433761+0000","last_undegraded":"2024-04-27T03:31:01.433761+0000","last_fullsized":"2024-04-27T03:31:01.433761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:56.237623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"27'282","reported_seq":381,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.463012+0000","last_change":"2024-04-27T03:30:19.406264+0000","last_active":"2024-04-27T03:33:20.463012+0000","last_peered":"2024-04-27T03:33:20.463012+0000","last_clean":"2024-04-27T03:33:20.463012+0000","last_became_active":"2024-04-27T03:30:19.405730+0000","last_became_peered":"2024-04-27T03:30:19.405730+0000","last_unstale":"2024-04-27T03:33:20.463012+0000","last_undegraded":"2024-04-27T03:33:20.463012+0000","last_fullsized":"2024-04-27T03:33:20.463012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:24.551790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436546+0000","last_change":"2024-04-27T03:30:21.409739+0000","last_active":"2024-04-27T03:31:01.436546+0000","last_peered":"2024-04-27T03:31:01.436546+0000","last_clean":"2024-04-27T03:31:01.436546+0000","last_became_active":"2024-04-27T03:30:21.409156+0000","last_became_peered":"2024-04-27T03:30:21.409156+0000","last_unstale":"2024-04-27T03:31:01.436546+0000","last_undegraded":"2024-04-27T03:31:01.436546+0000","last_fullsized":"2024-04-27T03:31:01.436546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:56:51.564750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.802359+0000","last_change":"2024-04-27T03:30:13.385803+0000","last_active":"2024-04-27T03:33:20.802359+0000","last_peered":"2024-04-27T03:33:20.802359+0000","last_clean":"2024-04-27T03:33:20.802359+0000","last_became_active":"2024-04-27T03:30:13.385491+0000","last_became_peered":"2024-04-27T03:30:13.385491+0000","last_unstale":"2024-04-27T03:33:20.802359+0000","last_undegraded":"2024-04-27T03:33:20.802359+0000","last_fullsized":"2024-04-27T03:33:20.802359+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:14.849083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431754+0000","last_change":"2024-04-27T03:30:11.370401+0000","last_active":"2024-04-27T03:31:01.431754+0000","last_peered":"2024-04-27T03:31:01.431754+0000","last_clean":"2024-04-27T03:31:01.431754+0000","last_became_active":"2024-04-27T03:30:11.369939+0000","last_became_peered":"2024-04-27T03:30:11.369939+0000","last_unstale":"2024-04-27T03:31:01.431754+0000","last_undegraded":"2024-04-27T03:31:01.431754+0000","last_fullsized":"2024-04-27T03:31:01.431754+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:00.825717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439146+0000","last_change":"2024-04-27T03:30:15.384389+0000","last_active":"2024-04-27T03:31:01.439146+0000","last_peered":"2024-04-27T03:31:01.439146+0000","last_clean":"2024-04-27T03:31:01.439146+0000","last_became_active":"2024-04-27T03:30:15.384114+0000","last_became_peered":"2024-04-27T03:30:15.384114+0000","last_unstale":"2024-04-27T03:31:01.439146+0000","last_undegraded":"2024-04-27T03:31:01.439146+0000","last_fullsized":"2024-04-27T03:31:01.439146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:17.509463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"27'36","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.908651+0000","last_change":"2024-04-27T03:30:17.392377+0000","last_active":"2024-04-27T03:32:18.908651+0000","last_peered":"2024-04-27T03:32:18.908651+0000","last_clean":"2024-04-27T03:32:18.908651+0000","last_became_active":"2024-04-27T03:30:17.391880+0000","last_became_peered":"2024-04-27T03:30:17.391880+0000","last_unstale":"2024-04-27T03:32:18.908651+0000","last_undegraded":"2024-04-27T03:32:18.908651+0000","last_fullsized":"2024-04-27T03:32:18.908651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:13.007456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"27'282","reported_seq":405,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.137865+0000","last_change":"2024-04-27T03:30:19.405073+0000","last_active":"2024-04-27T03:33:21.137865+0000","last_peered":"2024-04-27T03:33:21.137865+0000","last_clean":"2024-04-27T03:33:21.137865+0000","last_became_active":"2024-04-27T03:30:19.404744+0000","last_became_peered":"2024-04-27T03:30:19.404744+0000","last_unstale":"2024-04-27T03:33:21.137865+0000","last_undegraded":"2024-04-27T03:33:21.137865+0000","last_fullsized":"2024-04-27T03:33:21.137865+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:31:41.885815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'7","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.706493+0000","last_change":"2024-04-27T03:30:13.387974+0000","last_active":"2024-04-27T03:33:20.706493+0000","last_peered":"2024-04-27T03:33:20.706493+0000","last_clean":"2024-04-27T03:33:20.706493+0000","last_became_active":"2024-04-27T03:30:13.387335+0000","last_became_peered":"2024-04-27T03:30:13.387335+0000","last_unstale":"2024-04-27T03:33:20.706493+0000","last_undegraded":"2024-04-27T03:33:20.706493+0000","last_fullsized":"2024-04-27T03:33:20.706493+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:32:09.116022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433282+0000","last_change":"2024-04-27T03:30:11.372061+0000","last_active":"2024-04-27T03:31:01.433282+0000","last_peered":"2024-04-27T03:31:01.433282+0000","last_clean":"2024-04-27T03:31:01.433282+0000","last_became_active":"2024-04-27T03:30:11.371600+0000","last_became_peered":"2024-04-27T03:30:11.371600+0000","last_unstale":"2024-04-27T03:31:01.433282+0000","last_undegraded":"2024-04-27T03:31:01.433282+0000","last_fullsized":"2024-04-27T03:31:01.433282+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.059046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433141+0000","last_change":"2024-04-27T03:30:15.381767+0000","last_active":"2024-04-27T03:31:01.433141+0000","last_peered":"2024-04-27T03:31:01.433141+0000","last_clean":"2024-04-27T03:31:01.433141+0000","last_became_active":"2024-04-27T03:30:15.381359+0000","last_became_peered":"2024-04-27T03:30:15.381359+0000","last_unstale":"2024-04-27T03:31:01.433141+0000","last_undegraded":"2024-04-27T03:31:01.433141+0000","last_fullsized":"2024-04-27T03:31:01.433141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:19:12.341976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"27'46","reported_seq":99,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.918834+0000","last_change":"2024-04-27T03:30:17.390288+0000","last_active":"2024-04-27T03:32:18.918834+0000","last_peered":"2024-04-27T03:32:18.918834+0000","last_clean":"2024-04-27T03:32:18.918834+0000","last_became_active":"2024-04-27T03:30:17.390021+0000","last_became_peered":"2024-04-27T03:30:17.390021+0000","last_unstale":"2024-04-27T03:32:18.918834+0000","last_undegraded":"2024-04-27T03:32:18.918834+0000","last_fullsized":"2024-04-27T03:32:18.918834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:58.407158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":53,"num_read_kb":41,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431506+0000","last_change":"2024-04-27T03:30:19.405394+0000","last_active":"2024-04-27T03:31:01.431506+0000","last_peered":"2024-04-27T03:31:01.431506+0000","last_clean":"2024-04-27T03:31:01.431506+0000","last_became_active":"2024-04-27T03:30:19.404948+0000","last_became_peered":"2024-04-27T03:30:19.404948+0000","last_unstale":"2024-04-27T03:31:01.431506+0000","last_undegraded":"2024-04-27T03:31:01.431506+0000","last_fullsized":"2024-04-27T03:31:01.431506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:59.540815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"27'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.438491+0000","last_change":"2024-04-27T03:30:21.412598+0000","last_active":"2024-04-27T03:31:36.438491+0000","last_peered":"2024-04-27T03:31:36.438491+0000","last_clean":"2024-04-27T03:31:36.438491+0000","last_became_active":"2024-04-27T03:30:21.412331+0000","last_became_peered":"2024-04-27T03:30:21.412331+0000","last_unstale":"2024-04-27T03:31:36.438491+0000","last_undegraded":"2024-04-27T03:31:36.438491+0000","last_fullsized":"2024-04-27T03:31:36.438491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:21.686877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"25'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432638+0000","last_change":"2024-04-27T03:30:13.373532+0000","last_active":"2024-04-27T03:31:01.432638+0000","last_peered":"2024-04-27T03:31:01.432638+0000","last_clean":"2024-04-27T03:31:01.432638+0000","last_became_active":"2024-04-27T03:30:13.373226+0000","last_became_peered":"2024-04-27T03:30:13.373226+0000","last_unstale":"2024-04-27T03:31:01.432638+0000","last_undegraded":"2024-04-27T03:31:01.432638+0000","last_fullsized":"2024-04-27T03:31:01.432638+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:38.976155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437884+0000","last_change":"2024-04-27T03:30:11.373710+0000","last_active":"2024-04-27T03:31:01.437884+0000","last_peered":"2024-04-27T03:31:01.437884+0000","last_clean":"2024-04-27T03:31:01.437884+0000","last_became_active":"2024-04-27T03:30:11.373430+0000","last_became_peered":"2024-04-27T03:30:11.373430+0000","last_unstale":"2024-04-27T03:31:01.437884+0000","last_undegraded":"2024-04-27T03:31:01.437884+0000","last_fullsized":"2024-04-27T03:31:01.437884+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:52:58.316121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"18'1","reported_seq":78,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.994968+0000","last_change":"2024-04-27T03:31:01.447513+0000","last_active":"2024-04-27T03:32:19.994968+0000","last_peered":"2024-04-27T03:32:19.994968+0000","last_clean":"2024-04-27T03:32:19.994968+0000","last_became_active":"2024-04-27T03:31:01.443259+0000","last_became_peered":"2024-04-27T03:31:01.443259+0000","last_unstale":"2024-04-27T03:32:19.994968+0000","last_undegraded":"2024-04-27T03:32:19.994968+0000","last_fullsized":"2024-04-27T03:32:19.994968+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:54.147127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"27'39","reported_seq":87,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296781+0000","last_change":"2024-04-27T03:30:17.389610+0000","last_active":"2024-04-27T03:32:34.296781+0000","last_peered":"2024-04-27T03:32:34.296781+0000","last_clean":"2024-04-27T03:32:34.296781+0000","last_became_active":"2024-04-27T03:30:17.388994+0000","last_became_peered":"2024-04-27T03:30:17.388994+0000","last_unstale":"2024-04-27T03:32:34.296781+0000","last_undegraded":"2024-04-27T03:32:34.296781+0000","last_fullsized":"2024-04-27T03:32:34.296781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:01:03.772535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431125+0000","last_change":"2024-04-27T03:30:19.405695+0000","last_active":"2024-04-27T03:31:01.431125+0000","last_peered":"2024-04-27T03:31:01.431125+0000","last_clean":"2024-04-27T03:31:01.431125+0000","last_became_active":"2024-04-27T03:30:19.405334+0000","last_became_peered":"2024-04-27T03:30:19.405334+0000","last_unstale":"2024-04-27T03:31:01.431125+0000","last_undegraded":"2024-04-27T03:31:01.431125+0000","last_fullsized":"2024-04-27T03:31:01.431125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:44:45.246417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"27'3","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:30.912296+0000","last_change":"2024-04-27T03:30:21.410221+0000","last_active":"2024-04-27T03:31:30.912296+0000","last_peered":"2024-04-27T03:31:30.912296+0000","last_clean":"2024-04-27T03:31:30.912296+0000","last_became_active":"2024-04-27T03:30:21.409922+0000","last_became_peered":"2024-04-27T03:30:21.409922+0000","last_unstale":"2024-04-27T03:31:30.912296+0000","last_undegraded":"2024-04-27T03:31:30.912296+0000","last_fullsized":"2024-04-27T03:31:30.912296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.646754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"27'6","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.010033+0000","last_change":"2024-04-27T03:30:13.381521+0000","last_active":"2024-04-27T03:33:21.010033+0000","last_peered":"2024-04-27T03:33:21.010033+0000","last_clean":"2024-04-27T03:33:21.010033+0000","last_became_active":"2024-04-27T03:30:13.381270+0000","last_became_peered":"2024-04-27T03:30:13.381270+0000","last_unstale":"2024-04-27T03:33:21.010033+0000","last_undegraded":"2024-04-27T03:33:21.010033+0000","last_fullsized":"2024-04-27T03:33:21.010033+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:58:48.554677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440331+0000","last_change":"2024-04-27T03:30:11.372966+0000","last_active":"2024-04-27T03:31:01.440331+0000","last_peered":"2024-04-27T03:31:01.440331+0000","last_clean":"2024-04-27T03:31:01.440331+0000","last_became_active":"2024-04-27T03:30:11.372583+0000","last_became_peered":"2024-04-27T03:30:11.372583+0000","last_unstale":"2024-04-27T03:31:01.440331+0000","last_undegraded":"2024-04-27T03:31:01.440331+0000","last_fullsized":"2024-04-27T03:31:01.440331+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:54.900404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433260+0000","last_change":"2024-04-27T03:30:15.382571+0000","last_active":"2024-04-27T03:31:01.433260+0000","last_peered":"2024-04-27T03:31:01.433260+0000","last_clean":"2024-04-27T03:31:01.433260+0000","last_became_active":"2024-04-27T03:30:15.382207+0000","last_became_peered":"2024-04-27T03:30:15.382207+0000","last_unstale":"2024-04-27T03:31:01.433260+0000","last_undegraded":"2024-04-27T03:31:01.433260+0000","last_fullsized":"2024-04-27T03:31:01.433260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:56.396642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"27'49","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.920671+0000","last_change":"2024-04-27T03:30:17.390418+0000","last_active":"2024-04-27T03:32:18.920671+0000","last_peered":"2024-04-27T03:32:18.920671+0000","last_clean":"2024-04-27T03:32:18.920671+0000","last_became_active":"2024-04-27T03:30:17.389242+0000","last_became_peered":"2024-04-27T03:30:17.389242+0000","last_unstale":"2024-04-27T03:32:18.920671+0000","last_undegraded":"2024-04-27T03:32:18.920671+0000","last_fullsized":"2024-04-27T03:32:18.920671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:59.857055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":30,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"27'282","reported_seq":348,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.148028+0000","last_change":"2024-04-27T03:30:19.406718+0000","last_active":"2024-04-27T03:33:21.148028+0000","last_peered":"2024-04-27T03:33:21.148028+0000","last_clean":"2024-04-27T03:33:21.148028+0000","last_became_active":"2024-04-27T03:30:19.406364+0000","last_became_peered":"2024-04-27T03:30:19.406364+0000","last_unstale":"2024-04-27T03:33:21.148028+0000","last_undegraded":"2024-04-27T03:33:21.148028+0000","last_fullsized":"2024-04-27T03:33:21.148028+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:19.671271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432341+0000","last_change":"2024-04-27T03:30:21.408481+0000","last_active":"2024-04-27T03:31:01.432341+0000","last_peered":"2024-04-27T03:31:01.432341+0000","last_clean":"2024-04-27T03:31:01.432341+0000","last_became_active":"2024-04-27T03:30:21.408168+0000","last_became_peered":"2024-04-27T03:30:21.408168+0000","last_unstale":"2024-04-27T03:31:01.432341+0000","last_undegraded":"2024-04-27T03:31:01.432341+0000","last_fullsized":"2024-04-27T03:31:01.432341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:56.856006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"27'14","reported_seq":101,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.703564+0000","last_change":"2024-04-27T03:30:13.371773+0000","last_active":"2024-04-27T03:33:20.703564+0000","last_peered":"2024-04-27T03:33:20.703564+0000","last_clean":"2024-04-27T03:33:20.703564+0000","last_became_active":"2024-04-27T03:30:13.371503+0000","last_became_peered":"2024-04-27T03:30:13.371503+0000","last_unstale":"2024-04-27T03:33:20.703564+0000","last_undegraded":"2024-04-27T03:33:20.703564+0000","last_fullsized":"2024-04-27T03:33:20.703564+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:54.324828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438871+0000","last_change":"2024-04-27T03:30:11.357185+0000","last_active":"2024-04-27T03:31:01.438871+0000","last_peered":"2024-04-27T03:31:01.438871+0000","last_clean":"2024-04-27T03:31:01.438871+0000","last_became_active":"2024-04-27T03:30:11.356739+0000","last_became_peered":"2024-04-27T03:30:11.356739+0000","last_unstale":"2024-04-27T03:31:01.438871+0000","last_undegraded":"2024-04-27T03:31:01.438871+0000","last_fullsized":"2024-04-27T03:31:01.438871+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:31.805263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:04.282986+0000","last_change":"2024-04-27T03:31:01.443432+0000","last_active":"2024-04-27T03:31:04.282986+0000","last_peered":"2024-04-27T03:31:04.282986+0000","last_clean":"2024-04-27T03:31:04.282986+0000","last_became_active":"2024-04-27T03:31:01.443118+0000","last_became_peered":"2024-04-27T03:31:01.443118+0000","last_unstale":"2024-04-27T03:31:04.282986+0000","last_undegraded":"2024-04-27T03:31:04.282986+0000","last_fullsized":"2024-04-27T03:31:04.282986+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:38.016966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"27'68","reported_seq":136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.922498+0000","last_change":"2024-04-27T03:30:17.392542+0000","last_active":"2024-04-27T03:32:18.922498+0000","last_peered":"2024-04-27T03:32:18.922498+0000","last_clean":"2024-04-27T03:32:18.922498+0000","last_became_active":"2024-04-27T03:30:17.387318+0000","last_became_peered":"2024-04-27T03:30:17.387318+0000","last_unstale":"2024-04-27T03:32:18.922498+0000","last_undegraded":"2024-04-27T03:32:18.922498+0000","last_fullsized":"2024-04-27T03:32:18.922498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:36.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":67,"num_read_kb":56,"num_write":30,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430991+0000","last_change":"2024-04-27T03:30:19.404488+0000","last_active":"2024-04-27T03:31:01.430991+0000","last_peered":"2024-04-27T03:31:01.430991+0000","last_clean":"2024-04-27T03:31:01.430991+0000","last_became_active":"2024-04-27T03:30:19.404265+0000","last_became_peered":"2024-04-27T03:30:19.404265+0000","last_unstale":"2024-04-27T03:31:01.430991+0000","last_undegraded":"2024-04-27T03:31:01.430991+0000","last_fullsized":"2024-04-27T03:31:01.430991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:26.725030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"27'18","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.120745+0000","last_change":"2024-04-27T03:30:21.414109+0000","last_active":"2024-04-27T03:33:21.120745+0000","last_peered":"2024-04-27T03:33:21.120745+0000","last_clean":"2024-04-27T03:33:21.120745+0000","last_became_active":"2024-04-27T03:30:21.413757+0000","last_became_peered":"2024-04-27T03:30:21.413757+0000","last_unstale":"2024-04-27T03:33:21.120745+0000","last_undegraded":"2024-04-27T03:33:21.120745+0000","last_fullsized":"2024-04-27T03:33:21.120745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:41.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981981+0000","last_change":"2024-04-27T03:30:13.380233+0000","last_active":"2024-04-27T03:33:20.981981+0000","last_peered":"2024-04-27T03:33:20.981981+0000","last_clean":"2024-04-27T03:33:20.981981+0000","last_became_active":"2024-04-27T03:30:13.379861+0000","last_became_peered":"2024-04-27T03:30:13.379861+0000","last_unstale":"2024-04-27T03:33:20.981981+0000","last_undegraded":"2024-04-27T03:33:20.981981+0000","last_fullsized":"2024-04-27T03:33:20.981981+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:46.205600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431725+0000","last_change":"2024-04-27T03:30:11.357503+0000","last_active":"2024-04-27T03:31:01.431725+0000","last_peered":"2024-04-27T03:31:01.431725+0000","last_clean":"2024-04-27T03:31:01.431725+0000","last_became_active":"2024-04-27T03:30:11.357176+0000","last_became_peered":"2024-04-27T03:30:11.357176+0000","last_unstale":"2024-04-27T03:31:01.431725+0000","last_undegraded":"2024-04-27T03:31:01.431725+0000","last_fullsized":"2024-04-27T03:31:01.431725+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:06.887756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432802+0000","last_change":"2024-04-27T03:30:15.382614+0000","last_active":"2024-04-27T03:31:01.432802+0000","last_peered":"2024-04-27T03:31:01.432802+0000","last_clean":"2024-04-27T03:31:01.432802+0000","last_became_active":"2024-04-27T03:30:15.382290+0000","last_became_peered":"2024-04-27T03:30:15.382290+0000","last_unstale":"2024-04-27T03:31:01.432802+0000","last_undegraded":"2024-04-27T03:31:01.432802+0000","last_fullsized":"2024-04-27T03:31:01.432802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:16.810766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"27'38","reported_seq":87,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.902396+0000","last_change":"2024-04-27T03:30:17.386194+0000","last_active":"2024-04-27T03:32:18.902396+0000","last_peered":"2024-04-27T03:32:18.902396+0000","last_clean":"2024-04-27T03:32:18.902396+0000","last_became_active":"2024-04-27T03:30:17.385902+0000","last_became_peered":"2024-04-27T03:30:17.385902+0000","last_unstale":"2024-04-27T03:32:18.902396+0000","last_undegraded":"2024-04-27T03:32:18.902396+0000","last_fullsized":"2024-04-27T03:32:18.902396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:45:51.459337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438538+0000","last_change":"2024-04-27T03:30:19.406061+0000","last_active":"2024-04-27T03:31:01.438538+0000","last_peered":"2024-04-27T03:31:01.438538+0000","last_clean":"2024-04-27T03:31:01.438538+0000","last_became_active":"2024-04-27T03:30:19.405450+0000","last_became_peered":"2024-04-27T03:30:19.405450+0000","last_unstale":"2024-04-27T03:31:01.438538+0000","last_undegraded":"2024-04-27T03:31:01.438538+0000","last_fullsized":"2024-04-27T03:31:01.438538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:26:03.989010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"27'9","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.701473+0000","last_change":"2024-04-27T03:30:21.415884+0000","last_active":"2024-04-27T03:33:20.701473+0000","last_peered":"2024-04-27T03:33:20.701473+0000","last_clean":"2024-04-27T03:33:20.701473+0000","last_became_active":"2024-04-27T03:30:21.415494+0000","last_became_peered":"2024-04-27T03:30:21.415494+0000","last_unstale":"2024-04-27T03:33:20.701473+0000","last_undegraded":"2024-04-27T03:33:20.701473+0000","last_fullsized":"2024-04-27T03:33:20.701473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:34.804682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'6","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009826+0000","last_change":"2024-04-27T03:30:13.388008+0000","last_active":"2024-04-27T03:33:21.009826+0000","last_peered":"2024-04-27T03:33:21.009826+0000","last_clean":"2024-04-27T03:33:21.009826+0000","last_became_active":"2024-04-27T03:30:13.387533+0000","last_became_peered":"2024-04-27T03:30:13.387533+0000","last_unstale":"2024-04-27T03:33:21.009826+0000","last_undegraded":"2024-04-27T03:33:21.009826+0000","last_fullsized":"2024-04-27T03:33:21.009826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:53.142322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437632+0000","last_change":"2024-04-27T03:30:11.373706+0000","last_active":"2024-04-27T03:31:01.437632+0000","last_peered":"2024-04-27T03:31:01.437632+0000","last_clean":"2024-04-27T03:31:01.437632+0000","last_became_active":"2024-04-27T03:30:11.373440+0000","last_became_peered":"2024-04-27T03:30:11.373440+0000","last_unstale":"2024-04-27T03:31:01.437632+0000","last_undegraded":"2024-04-27T03:31:01.437632+0000","last_fullsized":"2024-04-27T03:31:01.437632+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:41:06.626342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432793+0000","last_change":"2024-04-27T03:30:15.383722+0000","last_active":"2024-04-27T03:31:01.432793+0000","last_peered":"2024-04-27T03:31:01.432793+0000","last_clean":"2024-04-27T03:31:01.432793+0000","last_became_active":"2024-04-27T03:30:15.382913+0000","last_became_peered":"2024-04-27T03:30:15.382913+0000","last_unstale":"2024-04-27T03:31:01.432793+0000","last_undegraded":"2024-04-27T03:31:01.432793+0000","last_fullsized":"2024-04-27T03:31:01.432793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:59:19.478810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"27'36","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.928891+0000","last_change":"2024-04-27T03:30:17.382348+0000","last_active":"2024-04-27T03:32:18.928891+0000","last_peered":"2024-04-27T03:32:18.928891+0000","last_clean":"2024-04-27T03:32:18.928891+0000","last_became_active":"2024-04-27T03:30:17.381106+0000","last_became_peered":"2024-04-27T03:30:17.381106+0000","last_unstale":"2024-04-27T03:32:18.928891+0000","last_undegraded":"2024-04-27T03:32:18.928891+0000","last_fullsized":"2024-04-27T03:32:18.928891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:36:29.105227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"27'282","reported_seq":403,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.124909+0000","last_change":"2024-04-27T03:30:19.403728+0000","last_active":"2024-04-27T03:33:21.124909+0000","last_peered":"2024-04-27T03:33:21.124909+0000","last_clean":"2024-04-27T03:33:21.124909+0000","last_became_active":"2024-04-27T03:30:19.403413+0000","last_became_peered":"2024-04-27T03:30:19.403413+0000","last_unstale":"2024-04-27T03:33:21.124909+0000","last_undegraded":"2024-04-27T03:33:21.124909+0000","last_fullsized":"2024-04-27T03:33:21.124909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.087236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"27'14","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.289774+0000","last_change":"2024-04-27T03:30:21.403250+0000","last_active":"2024-04-27T03:32:29.289774+0000","last_peered":"2024-04-27T03:32:29.289774+0000","last_clean":"2024-04-27T03:32:29.289774+0000","last_became_active":"2024-04-27T03:30:21.402102+0000","last_became_peered":"2024-04-27T03:30:21.402102+0000","last_unstale":"2024-04-27T03:32:29.289774+0000","last_undegraded":"2024-04-27T03:32:29.289774+0000","last_fullsized":"2024-04-27T03:32:29.289774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:00.181478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":23,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'12","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.795005+0000","last_change":"2024-04-27T03:30:13.383203+0000","last_active":"2024-04-27T03:33:20.795005+0000","last_peered":"2024-04-27T03:33:20.795005+0000","last_clean":"2024-04-27T03:33:20.795005+0000","last_became_active":"2024-04-27T03:30:13.382899+0000","last_became_peered":"2024-04-27T03:30:13.382899+0000","last_unstale":"2024-04-27T03:33:20.795005+0000","last_undegraded":"2024-04-27T03:33:20.795005+0000","last_fullsized":"2024-04-27T03:33:20.795005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:57.593400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'4","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.924480+0000","last_change":"2024-04-27T03:31:01.444153+0000","last_active":"2024-04-27T03:33:20.924480+0000","last_peered":"2024-04-27T03:33:20.924480+0000","last_clean":"2024-04-27T03:33:20.924480+0000","last_became_active":"2024-04-27T03:31:01.439776+0000","last_became_peered":"2024-04-27T03:31:01.439776+0000","last_unstale":"2024-04-27T03:33:20.924480+0000","last_undegraded":"2024-04-27T03:33:20.924480+0000","last_fullsized":"2024-04-27T03:33:20.924480+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:15.239233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"27'3","reported_seq":413,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.994698+0000","last_change":"2024-04-27T03:30:15.382936+0000","last_active":"2024-04-27T03:32:19.994698+0000","last_peered":"2024-04-27T03:32:19.994698+0000","last_clean":"2024-04-27T03:32:19.994698+0000","last_became_active":"2024-04-27T03:30:15.382502+0000","last_became_peered":"2024-04-27T03:30:15.382502+0000","last_unstale":"2024-04-27T03:32:19.994698+0000","last_undegraded":"2024-04-27T03:32:19.994698+0000","last_fullsized":"2024-04-27T03:32:19.994698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:17.198784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"25'10","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438996+0000","last_change":"2024-04-27T03:30:17.389548+0000","last_active":"2024-04-27T03:31:01.438996+0000","last_peered":"2024-04-27T03:31:01.438996+0000","last_clean":"2024-04-27T03:31:01.438996+0000","last_became_active":"2024-04-27T03:30:17.387739+0000","last_became_peered":"2024-04-27T03:30:17.387739+0000","last_unstale":"2024-04-27T03:31:01.438996+0000","last_undegraded":"2024-04-27T03:31:01.438996+0000","last_fullsized":"2024-04-27T03:31:01.438996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:03:39.120700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"27'282","reported_seq":510,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.076259+0000","last_change":"2024-04-27T03:30:19.402853+0000","last_active":"2024-04-27T03:33:21.076259+0000","last_peered":"2024-04-27T03:33:21.076259+0000","last_clean":"2024-04-27T03:33:21.076259+0000","last_became_active":"2024-04-27T03:30:19.402484+0000","last_became_peered":"2024-04-27T03:30:19.402484+0000","last_unstale":"2024-04-27T03:33:21.076259+0000","last_undegraded":"2024-04-27T03:33:21.076259+0000","last_fullsized":"2024-04-27T03:33:21.076259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:31.846278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"27'4","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.132546+0000","last_change":"2024-04-27T03:30:21.408801+0000","last_active":"2024-04-27T03:32:20.132546+0000","last_peered":"2024-04-27T03:32:20.132546+0000","last_clean":"2024-04-27T03:32:20.132546+0000","last_became_active":"2024-04-27T03:30:21.408512+0000","last_became_peered":"2024-04-27T03:30:21.408512+0000","last_unstale":"2024-04-27T03:32:20.132546+0000","last_undegraded":"2024-04-27T03:32:20.132546+0000","last_fullsized":"2024-04-27T03:32:20.132546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:49:39.647798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'14","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.792438+0000","last_change":"2024-04-27T03:30:13.381639+0000","last_active":"2024-04-27T03:33:20.792438+0000","last_peered":"2024-04-27T03:33:20.792438+0000","last_clean":"2024-04-27T03:33:20.792438+0000","last_became_active":"2024-04-27T03:30:13.380540+0000","last_became_peered":"2024-04-27T03:30:13.380540+0000","last_unstale":"2024-04-27T03:33:20.792438+0000","last_undegraded":"2024-04-27T03:33:20.792438+0000","last_fullsized":"2024-04-27T03:33:20.792438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:44.819638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.190061+0000","last_change":"2024-04-27T03:30:11.372470+0000","last_active":"2024-04-27T03:32:20.190061+0000","last_peered":"2024-04-27T03:32:20.190061+0000","last_clean":"2024-04-27T03:32:20.190061+0000","last_became_active":"2024-04-27T03:30:11.372193+0000","last_became_peered":"2024-04-27T03:30:11.372193+0000","last_unstale":"2024-04-27T03:32:20.190061+0000","last_undegraded":"2024-04-27T03:32:20.190061+0000","last_fullsized":"2024-04-27T03:32:20.190061+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:16.294699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438746+0000","last_change":"2024-04-27T03:30:15.385100+0000","last_active":"2024-04-27T03:31:01.438746+0000","last_peered":"2024-04-27T03:31:01.438746+0000","last_clean":"2024-04-27T03:31:01.438746+0000","last_became_active":"2024-04-27T03:30:15.383924+0000","last_became_peered":"2024-04-27T03:30:15.383924+0000","last_unstale":"2024-04-27T03:31:01.438746+0000","last_undegraded":"2024-04-27T03:31:01.438746+0000","last_fullsized":"2024-04-27T03:31:01.438746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:08.259240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"27'38","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.915730+0000","last_change":"2024-04-27T03:30:17.388297+0000","last_active":"2024-04-27T03:32:18.915730+0000","last_peered":"2024-04-27T03:32:18.915730+0000","last_clean":"2024-04-27T03:32:18.915730+0000","last_became_active":"2024-04-27T03:30:17.387578+0000","last_became_peered":"2024-04-27T03:30:17.387578+0000","last_unstale":"2024-04-27T03:32:18.915730+0000","last_undegraded":"2024-04-27T03:32:18.915730+0000","last_fullsized":"2024-04-27T03:32:18.915730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:26.963495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433032+0000","last_change":"2024-04-27T03:30:19.405992+0000","last_active":"2024-04-27T03:31:01.433032+0000","last_peered":"2024-04-27T03:31:01.433032+0000","last_clean":"2024-04-27T03:31:01.433032+0000","last_became_active":"2024-04-27T03:30:19.405512+0000","last_became_peered":"2024-04-27T03:30:19.405512+0000","last_unstale":"2024-04-27T03:31:01.433032+0000","last_undegraded":"2024-04-27T03:31:01.433032+0000","last_fullsized":"2024-04-27T03:31:01.433032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:15.297931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"27'14","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.046419+0000","last_change":"2024-04-27T03:30:21.407207+0000","last_active":"2024-04-27T03:33:21.046419+0000","last_peered":"2024-04-27T03:33:21.046419+0000","last_clean":"2024-04-27T03:33:21.046419+0000","last_became_active":"2024-04-27T03:30:21.406883+0000","last_became_peered":"2024-04-27T03:30:21.406883+0000","last_unstale":"2024-04-27T03:33:21.046419+0000","last_undegraded":"2024-04-27T03:33:21.046419+0000","last_fullsized":"2024-04-27T03:33:21.046419+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:00.181478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":59,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'20","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933899+0000","last_change":"2024-04-27T03:30:13.387655+0000","last_active":"2024-04-27T03:33:20.933899+0000","last_peered":"2024-04-27T03:33:20.933899+0000","last_clean":"2024-04-27T03:33:20.933899+0000","last_became_active":"2024-04-27T03:30:13.387180+0000","last_became_peered":"2024-04-27T03:30:13.387180+0000","last_unstale":"2024-04-27T03:33:20.933899+0000","last_undegraded":"2024-04-27T03:33:20.933899+0000","last_fullsized":"2024-04-27T03:33:20.933899+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:24.510289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431660+0000","last_change":"2024-04-27T03:30:06.104151+0000","last_active":"2024-04-27T03:31:01.431660+0000","last_peered":"2024-04-27T03:31:01.431660+0000","last_clean":"2024-04-27T03:31:01.431660+0000","last_became_active":"2024-04-27T03:30:06.102893+0000","last_became_peered":"2024-04-27T03:30:06.102893+0000","last_unstale":"2024-04-27T03:31:01.431660+0000","last_undegraded":"2024-04-27T03:31:01.431660+0000","last_fullsized":"2024-04-27T03:31:01.431660+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:27:47.408864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439201+0000","last_change":"2024-04-27T03:30:06.106875+0000","last_active":"2024-04-27T03:31:01.439201+0000","last_peered":"2024-04-27T03:31:01.439201+0000","last_clean":"2024-04-27T03:31:01.439201+0000","last_became_active":"2024-04-27T03:30:06.106511+0000","last_became_peered":"2024-04-27T03:30:06.106511+0000","last_unstale":"2024-04-27T03:31:01.439201+0000","last_undegraded":"2024-04-27T03:31:01.439201+0000","last_fullsized":"2024-04-27T03:31:01.439201+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:56:59.055441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.005690+0000","last_change":"2024-04-27T03:30:11.362699+0000","last_active":"2024-04-27T03:33:21.005690+0000","last_peered":"2024-04-27T03:33:21.005690+0000","last_clean":"2024-04-27T03:33:21.005690+0000","last_became_active":"2024-04-27T03:30:11.362445+0000","last_became_peered":"2024-04-27T03:30:11.362445+0000","last_unstale":"2024-04-27T03:33:21.005690+0000","last_undegraded":"2024-04-27T03:33:21.005690+0000","last_fullsized":"2024-04-27T03:33:21.005690+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:11:24.757438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.441975+0000","last_change":"2024-04-27T03:31:01.441027+0000","last_active":"2024-04-27T03:31:01.441975+0000","last_peered":"2024-04-27T03:31:01.441975+0000","last_clean":"2024-04-27T03:31:01.441975+0000","last_became_active":"2024-04-27T03:31:01.440678+0000","last_became_peered":"2024-04-27T03:31:01.440678+0000","last_unstale":"2024-04-27T03:31:01.441975+0000","last_undegraded":"2024-04-27T03:31:01.441975+0000","last_fullsized":"2024-04-27T03:31:01.441975+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:22.897973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"27'83","reported_seq":158,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.927035+0000","last_change":"2024-04-27T03:30:17.388976+0000","last_active":"2024-04-27T03:32:18.927035+0000","last_peered":"2024-04-27T03:32:18.927035+0000","last_clean":"2024-04-27T03:32:18.927035+0000","last_became_active":"2024-04-27T03:30:17.387707+0000","last_became_peered":"2024-04-27T03:30:17.387707+0000","last_unstale":"2024-04-27T03:32:18.927035+0000","last_undegraded":"2024-04-27T03:32:18.927035+0000","last_fullsized":"2024-04-27T03:32:18.927035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:03.008955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":63,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433316+0000","last_change":"2024-04-27T03:30:19.405693+0000","last_active":"2024-04-27T03:31:01.433316+0000","last_peered":"2024-04-27T03:31:01.433316+0000","last_clean":"2024-04-27T03:31:01.433316+0000","last_became_active":"2024-04-27T03:30:19.405359+0000","last_became_peered":"2024-04-27T03:30:19.405359+0000","last_unstale":"2024-04-27T03:31:01.433316+0000","last_undegraded":"2024-04-27T03:31:01.433316+0000","last_fullsized":"2024-04-27T03:31:01.433316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:36.634879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"27'81","reported_seq":162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.022864+0000","last_change":"2024-04-27T03:30:21.404603+0000","last_active":"2024-04-27T03:33:21.022864+0000","last_peered":"2024-04-27T03:33:21.022864+0000","last_clean":"2024-04-27T03:33:21.022864+0000","last_became_active":"2024-04-27T03:30:21.404300+0000","last_became_peered":"2024-04-27T03:30:21.404300+0000","last_unstale":"2024-04-27T03:33:21.022864+0000","last_undegraded":"2024-04-27T03:33:21.022864+0000","last_fullsized":"2024-04-27T03:33:21.022864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:05.598061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439232+0000","last_change":"2024-04-27T03:30:13.378248+0000","last_active":"2024-04-27T03:31:01.439232+0000","last_peered":"2024-04-27T03:31:01.439232+0000","last_clean":"2024-04-27T03:31:01.439232+0000","last_became_active":"2024-04-27T03:30:13.377713+0000","last_became_peered":"2024-04-27T03:30:13.377713+0000","last_unstale":"2024-04-27T03:31:01.439232+0000","last_undegraded":"2024-04-27T03:31:01.439232+0000","last_fullsized":"2024-04-27T03:31:01.439232+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:09.267760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436259+0000","last_change":"2024-04-27T03:30:06.106890+0000","last_active":"2024-04-27T03:31:01.436259+0000","last_peered":"2024-04-27T03:31:01.436259+0000","last_clean":"2024-04-27T03:31:01.436259+0000","last_became_active":"2024-04-27T03:30:06.106474+0000","last_became_peered":"2024-04-27T03:30:06.106474+0000","last_unstale":"2024-04-27T03:31:01.436259+0000","last_undegraded":"2024-04-27T03:31:01.436259+0000","last_fullsized":"2024-04-27T03:31:01.436259+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:12.652758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.643945+0000","last_change":"2024-04-27T03:30:11.360679+0000","last_active":"2024-04-27T03:31:36.643945+0000","last_peered":"2024-04-27T03:31:36.643945+0000","last_clean":"2024-04-27T03:31:36.643945+0000","last_became_active":"2024-04-27T03:30:11.360244+0000","last_became_peered":"2024-04-27T03:30:11.360244+0000","last_unstale":"2024-04-27T03:31:36.643945+0000","last_undegraded":"2024-04-27T03:31:36.643945+0000","last_fullsized":"2024-04-27T03:31:36.643945+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.539212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440319+0000","last_change":"2024-04-27T03:30:15.379725+0000","last_active":"2024-04-27T03:31:01.440319+0000","last_peered":"2024-04-27T03:31:01.440319+0000","last_clean":"2024-04-27T03:31:01.440319+0000","last_became_active":"2024-04-27T03:30:15.379448+0000","last_became_peered":"2024-04-27T03:30:15.379448+0000","last_unstale":"2024-04-27T03:31:01.440319+0000","last_undegraded":"2024-04-27T03:31:01.440319+0000","last_fullsized":"2024-04-27T03:31:01.440319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:06.243654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"27'279","reported_seq":488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462874+0000","last_change":"2024-04-27T03:30:17.391189+0000","last_active":"2024-04-27T03:33:20.462874+0000","last_peered":"2024-04-27T03:33:20.462874+0000","last_clean":"2024-04-27T03:33:20.462874+0000","last_became_active":"2024-04-27T03:30:17.390884+0000","last_became_peered":"2024-04-27T03:30:17.390884+0000","last_unstale":"2024-04-27T03:33:20.462874+0000","last_undegraded":"2024-04-27T03:33:20.462874+0000","last_fullsized":"2024-04-27T03:33:20.462874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:57.164266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":366,"num_read_kb":350,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439611+0000","last_change":"2024-04-27T03:30:19.408904+0000","last_active":"2024-04-27T03:31:01.439611+0000","last_peered":"2024-04-27T03:31:01.439611+0000","last_clean":"2024-04-27T03:31:01.439611+0000","last_became_active":"2024-04-27T03:30:19.405777+0000","last_became_peered":"2024-04-27T03:30:19.405777+0000","last_unstale":"2024-04-27T03:31:01.439611+0000","last_undegraded":"2024-04-27T03:31:01.439611+0000","last_fullsized":"2024-04-27T03:31:01.439611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:57.389557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"27'7","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.968472+0000","last_change":"2024-04-27T03:30:21.409632+0000","last_active":"2024-04-27T03:33:20.968472+0000","last_peered":"2024-04-27T03:33:20.968472+0000","last_clean":"2024-04-27T03:33:20.968472+0000","last_became_active":"2024-04-27T03:30:21.409359+0000","last_became_peered":"2024-04-27T03:30:21.409359+0000","last_unstale":"2024-04-27T03:33:20.968472+0000","last_undegraded":"2024-04-27T03:33:20.968472+0000","last_fullsized":"2024-04-27T03:33:20.968472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:41.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"25'10","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433170+0000","last_change":"2024-04-27T03:30:13.385862+0000","last_active":"2024-04-27T03:31:01.433170+0000","last_peered":"2024-04-27T03:31:01.433170+0000","last_clean":"2024-04-27T03:31:01.433170+0000","last_became_active":"2024-04-27T03:30:13.385521+0000","last_became_peered":"2024-04-27T03:30:13.385521+0000","last_unstale":"2024-04-27T03:31:01.433170+0000","last_undegraded":"2024-04-27T03:31:01.433170+0000","last_fullsized":"2024-04-27T03:31:01.433170+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:38.976155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435195+0000","last_change":"2024-04-27T03:30:06.103722+0000","last_active":"2024-04-27T03:31:01.435195+0000","last_peered":"2024-04-27T03:31:01.435195+0000","last_clean":"2024-04-27T03:31:01.435195+0000","last_became_active":"2024-04-27T03:30:06.103441+0000","last_became_peered":"2024-04-27T03:30:06.103441+0000","last_unstale":"2024-04-27T03:31:01.435195+0000","last_undegraded":"2024-04-27T03:31:01.435195+0000","last_fullsized":"2024-04-27T03:31:01.435195+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:59.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430689+0000","last_change":"2024-04-27T03:30:11.360789+0000","last_active":"2024-04-27T03:31:01.430689+0000","last_peered":"2024-04-27T03:31:01.430689+0000","last_clean":"2024-04-27T03:31:01.430689+0000","last_became_active":"2024-04-27T03:30:11.360443+0000","last_became_peered":"2024-04-27T03:30:11.360443+0000","last_unstale":"2024-04-27T03:31:01.430689+0000","last_undegraded":"2024-04-27T03:31:01.430689+0000","last_fullsized":"2024-04-27T03:31:01.430689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.863119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"18'1","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437997+0000","last_change":"2024-04-27T03:30:15.383569+0000","last_active":"2024-04-27T03:31:01.437997+0000","last_peered":"2024-04-27T03:31:01.437997+0000","last_clean":"2024-04-27T03:31:01.437997+0000","last_became_active":"2024-04-27T03:30:15.383213+0000","last_became_peered":"2024-04-27T03:30:15.383213+0000","last_unstale":"2024-04-27T03:31:01.437997+0000","last_undegraded":"2024-04-27T03:31:01.437997+0000","last_fullsized":"2024-04-27T03:31:01.437997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:48.217027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"27'16","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462975+0000","last_change":"2024-04-27T03:30:17.388886+0000","last_active":"2024-04-27T03:33:20.462975+0000","last_peered":"2024-04-27T03:33:20.462975+0000","last_clean":"2024-04-27T03:33:20.462975+0000","last_became_active":"2024-04-27T03:30:17.388588+0000","last_became_peered":"2024-04-27T03:30:17.388588+0000","last_unstale":"2024-04-27T03:33:20.462975+0000","last_undegraded":"2024-04-27T03:33:20.462975+0000","last_fullsized":"2024-04-27T03:33:20.462975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:59:21.489327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431871+0000","last_change":"2024-04-27T03:30:19.407292+0000","last_active":"2024-04-27T03:31:01.431871+0000","last_peered":"2024-04-27T03:31:01.431871+0000","last_clean":"2024-04-27T03:31:01.431871+0000","last_became_active":"2024-04-27T03:30:19.407004+0000","last_became_peered":"2024-04-27T03:30:19.407004+0000","last_unstale":"2024-04-27T03:31:01.431871+0000","last_undegraded":"2024-04-27T03:31:01.431871+0000","last_fullsized":"2024-04-27T03:31:01.431871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:10.262399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"25'2","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438400+0000","last_change":"2024-04-27T03:30:21.417022+0000","last_active":"2024-04-27T03:31:01.438400+0000","last_peered":"2024-04-27T03:31:01.438400+0000","last_clean":"2024-04-27T03:31:01.438400+0000","last_became_active":"2024-04-27T03:30:21.416389+0000","last_became_peered":"2024-04-27T03:30:21.416389+0000","last_unstale":"2024-04-27T03:31:01.438400+0000","last_undegraded":"2024-04-27T03:31:01.438400+0000","last_fullsized":"2024-04-27T03:31:01.438400+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:11:27.333113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'8","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.696826+0000","last_change":"2024-04-27T03:30:13.369982+0000","last_active":"2024-04-27T03:33:20.696826+0000","last_peered":"2024-04-27T03:33:20.696826+0000","last_clean":"2024-04-27T03:33:20.696826+0000","last_became_active":"2024-04-27T03:30:13.369638+0000","last_became_peered":"2024-04-27T03:30:13.369638+0000","last_unstale":"2024-04-27T03:33:20.696826+0000","last_undegraded":"2024-04-27T03:33:20.696826+0000","last_fullsized":"2024-04-27T03:33:20.696826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:57.593400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436751+0000","last_change":"2024-04-27T03:30:06.107772+0000","last_active":"2024-04-27T03:31:01.436751+0000","last_peered":"2024-04-27T03:31:01.436751+0000","last_clean":"2024-04-27T03:31:01.436751+0000","last_became_active":"2024-04-27T03:30:06.107356+0000","last_became_peered":"2024-04-27T03:30:06.107356+0000","last_unstale":"2024-04-27T03:31:01.436751+0000","last_undegraded":"2024-04-27T03:31:01.436751+0000","last_fullsized":"2024-04-27T03:31:01.436751+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:05:28.414626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436437+0000","last_change":"2024-04-27T03:30:11.357072+0000","last_active":"2024-04-27T03:31:01.436437+0000","last_peered":"2024-04-27T03:31:01.436437+0000","last_clean":"2024-04-27T03:31:01.436437+0000","last_became_active":"2024-04-27T03:30:11.356685+0000","last_became_peered":"2024-04-27T03:30:11.356685+0000","last_unstale":"2024-04-27T03:31:01.436437+0000","last_undegraded":"2024-04-27T03:31:01.436437+0000","last_fullsized":"2024-04-27T03:31:01.436437+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.863119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432748+0000","last_change":"2024-04-27T03:30:15.382673+0000","last_active":"2024-04-27T03:31:01.432748+0000","last_peered":"2024-04-27T03:31:01.432748+0000","last_clean":"2024-04-27T03:31:01.432748+0000","last_became_active":"2024-04-27T03:30:15.382326+0000","last_became_peered":"2024-04-27T03:30:15.382326+0000","last_unstale":"2024-04-27T03:31:01.432748+0000","last_undegraded":"2024-04-27T03:31:01.432748+0000","last_fullsized":"2024-04-27T03:31:01.432748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:37.500097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"27'13","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.319621+0000","last_change":"2024-04-27T03:30:17.390316+0000","last_active":"2024-04-27T03:33:20.319621+0000","last_peered":"2024-04-27T03:33:20.319621+0000","last_clean":"2024-04-27T03:33:20.319621+0000","last_became_active":"2024-04-27T03:30:17.389373+0000","last_became_peered":"2024-04-27T03:30:17.389373+0000","last_unstale":"2024-04-27T03:33:20.319621+0000","last_undegraded":"2024-04-27T03:33:20.319621+0000","last_fullsized":"2024-04-27T03:33:20.319621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:08.896469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438561+0000","last_change":"2024-04-27T03:30:19.406634+0000","last_active":"2024-04-27T03:31:01.438561+0000","last_peered":"2024-04-27T03:31:01.438561+0000","last_clean":"2024-04-27T03:31:01.438561+0000","last_became_active":"2024-04-27T03:30:19.405953+0000","last_became_peered":"2024-04-27T03:30:19.405953+0000","last_unstale":"2024-04-27T03:31:01.438561+0000","last_undegraded":"2024-04-27T03:31:01.438561+0000","last_fullsized":"2024-04-27T03:31:01.438561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:02.743749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"27'6","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.073585+0000","last_change":"2024-04-27T03:30:21.413040+0000","last_active":"2024-04-27T03:33:21.073585+0000","last_peered":"2024-04-27T03:33:21.073585+0000","last_clean":"2024-04-27T03:33:21.073585+0000","last_became_active":"2024-04-27T03:30:21.412790+0000","last_became_peered":"2024-04-27T03:30:21.412790+0000","last_unstale":"2024-04-27T03:33:21.073585+0000","last_undegraded":"2024-04-27T03:33:21.073585+0000","last_fullsized":"2024-04-27T03:33:21.073585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:58.583851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'18","reported_seq":86,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933802+0000","last_change":"2024-04-27T03:30:13.381418+0000","last_active":"2024-04-27T03:33:20.933802+0000","last_peered":"2024-04-27T03:33:20.933802+0000","last_clean":"2024-04-27T03:33:20.933802+0000","last_became_active":"2024-04-27T03:30:13.380888+0000","last_became_peered":"2024-04-27T03:30:13.380888+0000","last_unstale":"2024-04-27T03:33:20.933802+0000","last_undegraded":"2024-04-27T03:33:20.933802+0000","last_fullsized":"2024-04-27T03:33:20.933802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:28.256872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":73,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433189+0000","last_change":"2024-04-27T03:30:11.362550+0000","last_active":"2024-04-27T03:31:01.433189+0000","last_peered":"2024-04-27T03:31:01.433189+0000","last_clean":"2024-04-27T03:31:01.433189+0000","last_became_active":"2024-04-27T03:30:11.361786+0000","last_became_peered":"2024-04-27T03:30:11.361786+0000","last_unstale":"2024-04-27T03:31:01.433189+0000","last_undegraded":"2024-04-27T03:31:01.433189+0000","last_fullsized":"2024-04-27T03:31:01.433189+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.539212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431862+0000","last_change":"2024-04-27T03:30:06.107861+0000","last_active":"2024-04-27T03:31:01.431862+0000","last_peered":"2024-04-27T03:31:01.431862+0000","last_clean":"2024-04-27T03:31:01.431862+0000","last_became_active":"2024-04-27T03:30:06.107594+0000","last_became_peered":"2024-04-27T03:30:06.107594+0000","last_unstale":"2024-04-27T03:31:01.431862+0000","last_undegraded":"2024-04-27T03:31:01.431862+0000","last_fullsized":"2024-04-27T03:31:01.431862+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:10.319152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"27'1","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.289051+0000","last_change":"2024-04-27T03:30:15.381770+0000","last_active":"2024-04-27T03:32:24.289051+0000","last_peered":"2024-04-27T03:32:24.289051+0000","last_clean":"2024-04-27T03:32:24.289051+0000","last_became_active":"2024-04-27T03:30:15.381392+0000","last_became_peered":"2024-04-27T03:30:15.381392+0000","last_unstale":"2024-04-27T03:32:24.289051+0000","last_undegraded":"2024-04-27T03:32:24.289051+0000","last_fullsized":"2024-04-27T03:32:24.289051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:01.291555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"27'73","reported_seq":144,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288088+0000","last_change":"2024-04-27T03:30:17.390751+0000","last_active":"2024-04-27T03:32:19.288088+0000","last_peered":"2024-04-27T03:32:19.288088+0000","last_clean":"2024-04-27T03:32:19.288088+0000","last_became_active":"2024-04-27T03:30:17.390409+0000","last_became_peered":"2024-04-27T03:30:17.390409+0000","last_unstale":"2024-04-27T03:32:19.288088+0000","last_undegraded":"2024-04-27T03:32:19.288088+0000","last_fullsized":"2024-04-27T03:32:19.288088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.209300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":77,"num_read_kb":63,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435401+0000","last_change":"2024-04-27T03:30:19.403292+0000","last_active":"2024-04-27T03:31:01.435401+0000","last_peered":"2024-04-27T03:31:01.435401+0000","last_clean":"2024-04-27T03:31:01.435401+0000","last_became_active":"2024-04-27T03:30:19.402977+0000","last_became_peered":"2024-04-27T03:30:19.402977+0000","last_unstale":"2024-04-27T03:31:01.435401+0000","last_undegraded":"2024-04-27T03:31:01.435401+0000","last_fullsized":"2024-04-27T03:31:01.435401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:24.818605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"27'11","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.984330+0000","last_change":"2024-04-27T03:30:21.410763+0000","last_active":"2024-04-27T03:33:20.984330+0000","last_peered":"2024-04-27T03:33:20.984330+0000","last_clean":"2024-04-27T03:33:20.984330+0000","last_became_active":"2024-04-27T03:30:21.410511+0000","last_became_peered":"2024-04-27T03:30:21.410511+0000","last_unstale":"2024-04-27T03:33:20.984330+0000","last_undegraded":"2024-04-27T03:33:20.984330+0000","last_fullsized":"2024-04-27T03:33:20.984330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:58.583851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.019081+0000","last_change":"2024-04-27T03:30:13.380442+0000","last_active":"2024-04-27T03:33:21.019081+0000","last_peered":"2024-04-27T03:33:21.019081+0000","last_clean":"2024-04-27T03:33:21.019081+0000","last_became_active":"2024-04-27T03:30:13.379985+0000","last_became_peered":"2024-04-27T03:30:13.379985+0000","last_unstale":"2024-04-27T03:33:21.019081+0000","last_undegraded":"2024-04-27T03:33:21.019081+0000","last_fullsized":"2024-04-27T03:33:21.019081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:19.215823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288186+0000","last_change":"2024-04-27T03:30:11.361129+0000","last_active":"2024-04-27T03:32:19.288186+0000","last_peered":"2024-04-27T03:32:19.288186+0000","last_clean":"2024-04-27T03:32:19.288186+0000","last_became_active":"2024-04-27T03:30:11.360352+0000","last_became_peered":"2024-04-27T03:30:11.360352+0000","last_unstale":"2024-04-27T03:32:19.288186+0000","last_undegraded":"2024-04-27T03:32:19.288186+0000","last_fullsized":"2024-04-27T03:32:19.288186+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:22.798184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440363+0000","last_change":"2024-04-27T03:30:06.106816+0000","last_active":"2024-04-27T03:31:01.440363+0000","last_peered":"2024-04-27T03:31:01.440363+0000","last_clean":"2024-04-27T03:31:01.440363+0000","last_became_active":"2024-04-27T03:30:06.106475+0000","last_became_peered":"2024-04-27T03:30:06.106475+0000","last_unstale":"2024-04-27T03:31:01.440363+0000","last_undegraded":"2024-04-27T03:31:01.440363+0000","last_fullsized":"2024-04-27T03:31:01.440363+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:36.362210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437040+0000","last_change":"2024-04-27T03:30:15.380295+0000","last_active":"2024-04-27T03:31:01.437040+0000","last_peered":"2024-04-27T03:31:01.437040+0000","last_clean":"2024-04-27T03:31:01.437040+0000","last_became_active":"2024-04-27T03:30:15.380030+0000","last_became_peered":"2024-04-27T03:30:15.380030+0000","last_unstale":"2024-04-27T03:31:01.437040+0000","last_undegraded":"2024-04-27T03:31:01.437040+0000","last_fullsized":"2024-04-27T03:31:01.437040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:27.827534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"25'20","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433411+0000","last_change":"2024-04-27T03:30:17.393593+0000","last_active":"2024-04-27T03:31:01.433411+0000","last_peered":"2024-04-27T03:31:01.433411+0000","last_clean":"2024-04-27T03:31:01.433411+0000","last_became_active":"2024-04-27T03:30:17.393253+0000","last_became_peered":"2024-04-27T03:30:17.393253+0000","last_unstale":"2024-04-27T03:31:01.433411+0000","last_undegraded":"2024-04-27T03:31:01.433411+0000","last_fullsized":"2024-04-27T03:31:01.433411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:56.968218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"27'282","reported_seq":386,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462731+0000","last_change":"2024-04-27T03:30:19.402596+0000","last_active":"2024-04-27T03:33:20.462731+0000","last_peered":"2024-04-27T03:33:20.462731+0000","last_clean":"2024-04-27T03:33:20.462731+0000","last_became_active":"2024-04-27T03:30:19.402319+0000","last_became_peered":"2024-04-27T03:30:19.402319+0000","last_unstale":"2024-04-27T03:33:20.462731+0000","last_undegraded":"2024-04-27T03:33:20.462731+0000","last_fullsized":"2024-04-27T03:33:20.462731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:21.528207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438342+0000","last_change":"2024-04-27T03:30:21.414388+0000","last_active":"2024-04-27T03:31:01.438342+0000","last_peered":"2024-04-27T03:31:01.438342+0000","last_clean":"2024-04-27T03:31:01.438342+0000","last_became_active":"2024-04-27T03:30:21.414109+0000","last_became_peered":"2024-04-27T03:30:21.414109+0000","last_unstale":"2024-04-27T03:31:01.438342+0000","last_undegraded":"2024-04-27T03:31:01.438342+0000","last_fullsized":"2024-04-27T03:31:01.438342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:34.636870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436753+0000","last_change":"2024-04-27T03:30:13.388070+0000","last_active":"2024-04-27T03:31:01.436753+0000","last_peered":"2024-04-27T03:31:01.436753+0000","last_clean":"2024-04-27T03:31:01.436753+0000","last_became_active":"2024-04-27T03:30:13.387764+0000","last_became_peered":"2024-04-27T03:30:13.387764+0000","last_unstale":"2024-04-27T03:31:01.436753+0000","last_undegraded":"2024-04-27T03:31:01.436753+0000","last_fullsized":"2024-04-27T03:31:01.436753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:16.285485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.918891+0000","last_change":"2024-04-27T03:30:11.356555+0000","last_active":"2024-04-27T03:33:20.918891+0000","last_peered":"2024-04-27T03:33:20.918891+0000","last_clean":"2024-04-27T03:33:20.918891+0000","last_became_active":"2024-04-27T03:30:11.355704+0000","last_became_peered":"2024-04-27T03:30:11.355704+0000","last_unstale":"2024-04-27T03:33:20.918891+0000","last_undegraded":"2024-04-27T03:33:20.918891+0000","last_fullsized":"2024-04-27T03:33:20.918891+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:35.764527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432650+0000","last_change":"2024-04-27T03:30:06.104067+0000","last_active":"2024-04-27T03:31:01.432650+0000","last_peered":"2024-04-27T03:31:01.432650+0000","last_clean":"2024-04-27T03:31:01.432650+0000","last_became_active":"2024-04-27T03:30:06.103759+0000","last_became_peered":"2024-04-27T03:30:06.103759+0000","last_unstale":"2024-04-27T03:31:01.432650+0000","last_undegraded":"2024-04-27T03:31:01.432650+0000","last_fullsized":"2024-04-27T03:31:01.432650+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:27:47.408864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"27'4","reported_seq":226,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.984492+0000","last_change":"2024-04-27T03:30:15.384765+0000","last_active":"2024-04-27T03:32:19.984492+0000","last_peered":"2024-04-27T03:32:19.984492+0000","last_clean":"2024-04-27T03:32:19.984492+0000","last_became_active":"2024-04-27T03:30:15.384372+0000","last_became_peered":"2024-04-27T03:30:15.384372+0000","last_unstale":"2024-04-27T03:32:19.984492+0000","last_undegraded":"2024-04-27T03:32:19.984492+0000","last_fullsized":"2024-04-27T03:32:19.984492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:55.971988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":298,"num_read_kb":298,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'20","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295153+0000","last_change":"2024-04-27T03:30:17.389511+0000","last_active":"2024-04-27T03:32:24.295153+0000","last_peered":"2024-04-27T03:32:24.295153+0000","last_clean":"2024-04-27T03:32:24.295153+0000","last_became_active":"2024-04-27T03:30:17.389154+0000","last_became_peered":"2024-04-27T03:30:17.389154+0000","last_unstale":"2024-04-27T03:32:24.295153+0000","last_undegraded":"2024-04-27T03:32:24.295153+0000","last_fullsized":"2024-04-27T03:32:24.295153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:41:46.387083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"27'282","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.121400+0000","last_change":"2024-04-27T03:30:19.405158+0000","last_active":"2024-04-27T03:33:21.121400+0000","last_peered":"2024-04-27T03:33:21.121400+0000","last_clean":"2024-04-27T03:33:21.121400+0000","last_became_active":"2024-04-27T03:30:19.404867+0000","last_became_peered":"2024-04-27T03:30:19.404867+0000","last_unstale":"2024-04-27T03:33:21.121400+0000","last_undegraded":"2024-04-27T03:33:21.121400+0000","last_fullsized":"2024-04-27T03:33:21.121400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:12.277984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"25'3","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434777+0000","last_change":"2024-04-27T03:30:21.413577+0000","last_active":"2024-04-27T03:31:01.434777+0000","last_peered":"2024-04-27T03:31:01.434777+0000","last_clean":"2024-04-27T03:31:01.434777+0000","last_became_active":"2024-04-27T03:30:21.413264+0000","last_became_peered":"2024-04-27T03:30:21.413264+0000","last_unstale":"2024-04-27T03:31:01.434777+0000","last_undegraded":"2024-04-27T03:31:01.434777+0000","last_fullsized":"2024-04-27T03:31:01.434777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:23.492592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"27'4","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.696091+0000","last_change":"2024-04-27T03:30:13.383496+0000","last_active":"2024-04-27T03:33:20.696091+0000","last_peered":"2024-04-27T03:33:20.696091+0000","last_clean":"2024-04-27T03:33:20.696091+0000","last_became_active":"2024-04-27T03:30:13.383208+0000","last_became_peered":"2024-04-27T03:30:13.383208+0000","last_unstale":"2024-04-27T03:33:20.696091+0000","last_undegraded":"2024-04-27T03:33:20.696091+0000","last_fullsized":"2024-04-27T03:33:20.696091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:13.288239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436165+0000","last_change":"2024-04-27T03:30:11.370648+0000","last_active":"2024-04-27T03:31:01.436165+0000","last_peered":"2024-04-27T03:31:01.436165+0000","last_clean":"2024-04-27T03:31:01.436165+0000","last_became_active":"2024-04-27T03:30:11.369568+0000","last_became_peered":"2024-04-27T03:30:11.369568+0000","last_unstale":"2024-04-27T03:31:01.436165+0000","last_undegraded":"2024-04-27T03:31:01.436165+0000","last_fullsized":"2024-04-27T03:31:01.436165+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:51.203886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438979+0000","last_change":"2024-04-27T03:30:06.107954+0000","last_active":"2024-04-27T03:31:01.438979+0000","last_peered":"2024-04-27T03:31:01.438979+0000","last_clean":"2024-04-27T03:31:01.438979+0000","last_became_active":"2024-04-27T03:30:06.107600+0000","last_became_peered":"2024-04-27T03:30:06.107600+0000","last_unstale":"2024-04-27T03:31:01.438979+0000","last_undegraded":"2024-04-27T03:31:01.438979+0000","last_fullsized":"2024-04-27T03:31:01.438979+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:44.876194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437548+0000","last_change":"2024-04-27T03:30:15.380881+0000","last_active":"2024-04-27T03:31:01.437548+0000","last_peered":"2024-04-27T03:31:01.437548+0000","last_clean":"2024-04-27T03:31:01.437548+0000","last_became_active":"2024-04-27T03:30:15.380598+0000","last_became_peered":"2024-04-27T03:30:15.380598+0000","last_unstale":"2024-04-27T03:31:01.437548+0000","last_undegraded":"2024-04-27T03:31:01.437548+0000","last_fullsized":"2024-04-27T03:31:01.437548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:10.183004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"27'89","reported_seq":167,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294574+0000","last_change":"2024-04-27T03:30:17.391634+0000","last_active":"2024-04-27T03:32:24.294574+0000","last_peered":"2024-04-27T03:32:24.294574+0000","last_clean":"2024-04-27T03:32:24.294574+0000","last_became_active":"2024-04-27T03:30:17.391191+0000","last_became_peered":"2024-04-27T03:30:17.391191+0000","last_unstale":"2024-04-27T03:32:24.294574+0000","last_undegraded":"2024-04-27T03:32:24.294574+0000","last_fullsized":"2024-04-27T03:32:24.294574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:12.065352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":84,"num_read_kb":69,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431584+0000","last_change":"2024-04-27T03:30:19.404491+0000","last_active":"2024-04-27T03:31:01.431584+0000","last_peered":"2024-04-27T03:31:01.431584+0000","last_clean":"2024-04-27T03:31:01.431584+0000","last_became_active":"2024-04-27T03:30:19.404187+0000","last_became_peered":"2024-04-27T03:30:19.404187+0000","last_unstale":"2024-04-27T03:31:01.431584+0000","last_undegraded":"2024-04-27T03:31:01.431584+0000","last_fullsized":"2024-04-27T03:31:01.431584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:21.528207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432393+0000","last_change":"2024-04-27T03:30:21.408168+0000","last_active":"2024-04-27T03:31:01.432393+0000","last_peered":"2024-04-27T03:31:01.432393+0000","last_clean":"2024-04-27T03:31:01.432393+0000","last_became_active":"2024-04-27T03:30:21.407857+0000","last_became_peered":"2024-04-27T03:30:21.407857+0000","last_unstale":"2024-04-27T03:31:01.432393+0000","last_undegraded":"2024-04-27T03:31:01.432393+0000","last_fullsized":"2024-04-27T03:31:01.432393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:28.740636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"25'2","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432483+0000","last_change":"2024-04-27T03:30:21.411358+0000","last_active":"2024-04-27T03:31:01.432483+0000","last_peered":"2024-04-27T03:31:01.432483+0000","last_clean":"2024-04-27T03:31:01.432483+0000","last_became_active":"2024-04-27T03:30:21.411054+0000","last_became_peered":"2024-04-27T03:30:21.411054+0000","last_unstale":"2024-04-27T03:31:01.432483+0000","last_undegraded":"2024-04-27T03:31:01.432483+0000","last_fullsized":"2024-04-27T03:31:01.432483+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:44:47.262023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.653891+0000","last_change":"2024-04-27T03:30:13.383003+0000","last_active":"2024-04-27T03:31:36.653891+0000","last_peered":"2024-04-27T03:31:36.653891+0000","last_clean":"2024-04-27T03:31:36.653891+0000","last_became_active":"2024-04-27T03:30:13.382675+0000","last_became_peered":"2024-04-27T03:30:13.382675+0000","last_unstale":"2024-04-27T03:31:36.653891+0000","last_undegraded":"2024-04-27T03:31:36.653891+0000","last_fullsized":"2024-04-27T03:31:36.653891+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:21.911691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438857+0000","last_change":"2024-04-27T03:30:11.359531+0000","last_active":"2024-04-27T03:31:01.438857+0000","last_peered":"2024-04-27T03:31:01.438857+0000","last_clean":"2024-04-27T03:31:01.438857+0000","last_became_active":"2024-04-27T03:30:11.359195+0000","last_became_peered":"2024-04-27T03:30:11.359195+0000","last_unstale":"2024-04-27T03:31:01.438857+0000","last_undegraded":"2024-04-27T03:31:01.438857+0000","last_fullsized":"2024-04-27T03:31:01.438857+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:52.318456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434515+0000","last_change":"2024-04-27T03:30:15.379854+0000","last_active":"2024-04-27T03:31:01.434515+0000","last_peered":"2024-04-27T03:31:01.434515+0000","last_clean":"2024-04-27T03:31:01.434515+0000","last_became_active":"2024-04-27T03:30:15.379541+0000","last_became_peered":"2024-04-27T03:30:15.379541+0000","last_unstale":"2024-04-27T03:31:01.434515+0000","last_undegraded":"2024-04-27T03:31:01.434515+0000","last_fullsized":"2024-04-27T03:31:01.434515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:32:11.127449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"27'38","reported_seq":87,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.919746+0000","last_change":"2024-04-27T03:30:17.386926+0000","last_active":"2024-04-27T03:32:18.919746+0000","last_peered":"2024-04-27T03:32:18.919746+0000","last_clean":"2024-04-27T03:32:18.919746+0000","last_became_active":"2024-04-27T03:30:17.386496+0000","last_became_peered":"2024-04-27T03:30:17.386496+0000","last_unstale":"2024-04-27T03:32:18.919746+0000","last_undegraded":"2024-04-27T03:32:18.919746+0000","last_fullsized":"2024-04-27T03:32:18.919746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:56.661166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437333+0000","last_change":"2024-04-27T03:30:19.405864+0000","last_active":"2024-04-27T03:31:01.437333+0000","last_peered":"2024-04-27T03:31:01.437333+0000","last_clean":"2024-04-27T03:31:01.437333+0000","last_became_active":"2024-04-27T03:30:19.405189+0000","last_became_peered":"2024-04-27T03:30:19.405189+0000","last_unstale":"2024-04-27T03:31:01.437333+0000","last_undegraded":"2024-04-27T03:31:01.437333+0000","last_fullsized":"2024-04-27T03:31:01.437333+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:23.134716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"27'3","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.594114+0000","last_change":"2024-04-27T03:30:21.414203+0000","last_active":"2024-04-27T03:32:18.594114+0000","last_peered":"2024-04-27T03:32:18.594114+0000","last_clean":"2024-04-27T03:32:18.594114+0000","last_became_active":"2024-04-27T03:30:21.413920+0000","last_became_peered":"2024-04-27T03:30:21.413920+0000","last_unstale":"2024-04-27T03:32:18.594114+0000","last_undegraded":"2024-04-27T03:32:18.594114+0000","last_fullsized":"2024-04-27T03:32:18.594114+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:40.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"27'16","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981976+0000","last_change":"2024-04-27T03:30:13.381187+0000","last_active":"2024-04-27T03:33:20.981976+0000","last_peered":"2024-04-27T03:33:20.981976+0000","last_clean":"2024-04-27T03:33:20.981976+0000","last_became_active":"2024-04-27T03:30:13.380240+0000","last_became_peered":"2024-04-27T03:30:13.380240+0000","last_unstale":"2024-04-27T03:33:20.981976+0000","last_undegraded":"2024-04-27T03:33:20.981976+0000","last_fullsized":"2024-04-27T03:33:20.981976+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:56.024650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":31,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433005+0000","last_change":"2024-04-27T03:30:11.361106+0000","last_active":"2024-04-27T03:31:01.433005+0000","last_peered":"2024-04-27T03:31:01.433005+0000","last_clean":"2024-04-27T03:31:01.433005+0000","last_became_active":"2024-04-27T03:30:11.360217+0000","last_became_peered":"2024-04-27T03:30:11.360217+0000","last_unstale":"2024-04-27T03:31:01.433005+0000","last_undegraded":"2024-04-27T03:31:01.433005+0000","last_fullsized":"2024-04-27T03:31:01.433005+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:51.203886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439951+0000","last_change":"2024-04-27T03:30:15.381885+0000","last_active":"2024-04-27T03:31:01.439951+0000","last_peered":"2024-04-27T03:31:01.439951+0000","last_clean":"2024-04-27T03:31:01.439951+0000","last_became_active":"2024-04-27T03:30:15.381507+0000","last_became_peered":"2024-04-27T03:30:15.381507+0000","last_unstale":"2024-04-27T03:31:01.439951+0000","last_undegraded":"2024-04-27T03:31:01.439951+0000","last_fullsized":"2024-04-27T03:31:01.439951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:31:37.867070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.324334+0000","last_change":"2024-04-27T03:30:17.385578+0000","last_active":"2024-04-27T03:33:20.324334+0000","last_peered":"2024-04-27T03:33:20.324334+0000","last_clean":"2024-04-27T03:33:20.324334+0000","last_became_active":"2024-04-27T03:30:17.385318+0000","last_became_peered":"2024-04-27T03:30:17.385318+0000","last_unstale":"2024-04-27T03:33:20.324334+0000","last_undegraded":"2024-04-27T03:33:20.324334+0000","last_fullsized":"2024-04-27T03:33:20.324334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:47.385488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435327+0000","last_change":"2024-04-27T03:30:19.407084+0000","last_active":"2024-04-27T03:31:01.435327+0000","last_peered":"2024-04-27T03:31:01.435327+0000","last_clean":"2024-04-27T03:31:01.435327+0000","last_became_active":"2024-04-27T03:30:19.406736+0000","last_became_peered":"2024-04-27T03:30:19.406736+0000","last_unstale":"2024-04-27T03:31:01.435327+0000","last_undegraded":"2024-04-27T03:31:01.435327+0000","last_fullsized":"2024-04-27T03:31:01.435327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:21.217528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"27'4","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.798144+0000","last_change":"2024-04-27T03:30:21.408419+0000","last_active":"2024-04-27T03:33:20.798144+0000","last_peered":"2024-04-27T03:33:20.798144+0000","last_clean":"2024-04-27T03:33:20.798144+0000","last_became_active":"2024-04-27T03:30:21.408079+0000","last_became_peered":"2024-04-27T03:30:21.408079+0000","last_unstale":"2024-04-27T03:33:20.798144+0000","last_undegraded":"2024-04-27T03:33:20.798144+0000","last_fullsized":"2024-04-27T03:33:20.798144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.646754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009922+0000","last_change":"2024-04-27T03:30:13.375656+0000","last_active":"2024-04-27T03:33:21.009922+0000","last_peered":"2024-04-27T03:33:21.009922+0000","last_clean":"2024-04-27T03:33:21.009922+0000","last_became_active":"2024-04-27T03:30:13.375350+0000","last_became_peered":"2024-04-27T03:30:13.375350+0000","last_unstale":"2024-04-27T03:33:21.009922+0000","last_undegraded":"2024-04-27T03:33:21.009922+0000","last_fullsized":"2024-04-27T03:33:21.009922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:13.288239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434851+0000","last_change":"2024-04-27T03:30:11.361187+0000","last_active":"2024-04-27T03:31:01.434851+0000","last_peered":"2024-04-27T03:31:01.434851+0000","last_clean":"2024-04-27T03:31:01.434851+0000","last_became_active":"2024-04-27T03:30:11.360938+0000","last_became_peered":"2024-04-27T03:30:11.360938+0000","last_unstale":"2024-04-27T03:31:01.434851+0000","last_undegraded":"2024-04-27T03:31:01.434851+0000","last_fullsized":"2024-04-27T03:31:01.434851+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:11.138117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431691+0000","last_change":"2024-04-27T03:30:15.382918+0000","last_active":"2024-04-27T03:31:01.431691+0000","last_peered":"2024-04-27T03:31:01.431691+0000","last_clean":"2024-04-27T03:31:01.431691+0000","last_became_active":"2024-04-27T03:30:15.382287+0000","last_became_peered":"2024-04-27T03:30:15.382287+0000","last_unstale":"2024-04-27T03:31:01.431691+0000","last_undegraded":"2024-04-27T03:31:01.431691+0000","last_fullsized":"2024-04-27T03:31:01.431691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:21.227251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"27'40","reported_seq":99,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.385027+0000","last_change":"2024-04-27T03:30:17.388554+0000","last_active":"2024-04-27T03:33:20.385027+0000","last_peered":"2024-04-27T03:33:20.385027+0000","last_clean":"2024-04-27T03:33:20.385027+0000","last_became_active":"2024-04-27T03:30:17.388260+0000","last_became_peered":"2024-04-27T03:30:17.388260+0000","last_unstale":"2024-04-27T03:33:20.385027+0000","last_undegraded":"2024-04-27T03:33:20.385027+0000","last_fullsized":"2024-04-27T03:33:20.385027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:22.810377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432676+0000","last_change":"2024-04-27T03:30:19.402729+0000","last_active":"2024-04-27T03:31:01.432676+0000","last_peered":"2024-04-27T03:31:01.432676+0000","last_clean":"2024-04-27T03:31:01.432676+0000","last_became_active":"2024-04-27T03:30:19.402430+0000","last_became_peered":"2024-04-27T03:30:19.402430+0000","last_unstale":"2024-04-27T03:31:01.432676+0000","last_undegraded":"2024-04-27T03:31:01.432676+0000","last_fullsized":"2024-04-27T03:31:01.432676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:59.540815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438223+0000","last_change":"2024-04-27T03:30:21.416316+0000","last_active":"2024-04-27T03:31:01.438223+0000","last_peered":"2024-04-27T03:31:01.438223+0000","last_clean":"2024-04-27T03:31:01.438223+0000","last_became_active":"2024-04-27T03:30:21.415984+0000","last_became_peered":"2024-04-27T03:30:21.415984+0000","last_unstale":"2024-04-27T03:31:01.438223+0000","last_undegraded":"2024-04-27T03:31:01.438223+0000","last_fullsized":"2024-04-27T03:31:01.438223+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:30.270957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988543+0000","last_change":"2024-04-27T03:30:13.387737+0000","last_active":"2024-04-27T03:33:20.988543+0000","last_peered":"2024-04-27T03:33:20.988543+0000","last_clean":"2024-04-27T03:33:20.988543+0000","last_became_active":"2024-04-27T03:30:13.386985+0000","last_became_peered":"2024-04-27T03:30:13.386985+0000","last_unstale":"2024-04-27T03:33:20.988543+0000","last_undegraded":"2024-04-27T03:33:20.988543+0000","last_fullsized":"2024-04-27T03:33:20.988543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:43.363544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436347+0000","last_change":"2024-04-27T03:30:11.370182+0000","last_active":"2024-04-27T03:31:01.436347+0000","last_peered":"2024-04-27T03:31:01.436347+0000","last_clean":"2024-04-27T03:31:01.436347+0000","last_became_active":"2024-04-27T03:30:11.369824+0000","last_became_peered":"2024-04-27T03:30:11.369824+0000","last_unstale":"2024-04-27T03:31:01.436347+0000","last_undegraded":"2024-04-27T03:31:01.436347+0000","last_fullsized":"2024-04-27T03:31:01.436347+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:11.138117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438418+0000","last_change":"2024-04-27T03:30:15.384051+0000","last_active":"2024-04-27T03:31:01.438418+0000","last_peered":"2024-04-27T03:31:01.438418+0000","last_clean":"2024-04-27T03:31:01.438418+0000","last_became_active":"2024-04-27T03:30:15.383771+0000","last_became_peered":"2024-04-27T03:30:15.383771+0000","last_unstale":"2024-04-27T03:31:01.438418+0000","last_undegraded":"2024-04-27T03:31:01.438418+0000","last_fullsized":"2024-04-27T03:31:01.438418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:32.616135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"27'30","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.904671+0000","last_change":"2024-04-27T03:30:17.385866+0000","last_active":"2024-04-27T03:32:18.904671+0000","last_peered":"2024-04-27T03:32:18.904671+0000","last_clean":"2024-04-27T03:32:18.904671+0000","last_became_active":"2024-04-27T03:30:17.385488+0000","last_became_peered":"2024-04-27T03:30:17.385488+0000","last_unstale":"2024-04-27T03:32:18.904671+0000","last_undegraded":"2024-04-27T03:32:18.904671+0000","last_fullsized":"2024-04-27T03:32:18.904671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:39.510614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437192+0000","last_change":"2024-04-27T03:30:19.398792+0000","last_active":"2024-04-27T03:31:01.437192+0000","last_peered":"2024-04-27T03:31:01.437192+0000","last_clean":"2024-04-27T03:31:01.437192+0000","last_became_active":"2024-04-27T03:30:19.398484+0000","last_became_peered":"2024-04-27T03:30:19.398484+0000","last_unstale":"2024-04-27T03:31:01.437192+0000","last_undegraded":"2024-04-27T03:31:01.437192+0000","last_fullsized":"2024-04-27T03:31:01.437192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:16.023756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"25'2","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440063+0000","last_change":"2024-04-27T03:30:13.377494+0000","last_active":"2024-04-27T03:31:01.440063+0000","last_peered":"2024-04-27T03:31:01.440063+0000","last_clean":"2024-04-27T03:31:01.440063+0000","last_became_active":"2024-04-27T03:30:13.377217+0000","last_became_peered":"2024-04-27T03:30:13.377217+0000","last_unstale":"2024-04-27T03:31:01.440063+0000","last_undegraded":"2024-04-27T03:31:01.440063+0000","last_fullsized":"2024-04-27T03:31:01.440063+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:18.788433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"25'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436454+0000","last_change":"2024-04-27T03:30:11.370820+0000","last_active":"2024-04-27T03:31:01.436454+0000","last_peered":"2024-04-27T03:31:01.436454+0000","last_clean":"2024-04-27T03:31:01.436454+0000","last_became_active":"2024-04-27T03:30:11.370547+0000","last_became_peered":"2024-04-27T03:30:11.370547+0000","last_unstale":"2024-04-27T03:31:01.436454+0000","last_undegraded":"2024-04-27T03:31:01.436454+0000","last_fullsized":"2024-04-27T03:31:01.436454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:34.092880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433313+0000","last_change":"2024-04-27T03:30:15.382292+0000","last_active":"2024-04-27T03:31:01.433313+0000","last_peered":"2024-04-27T03:31:01.433313+0000","last_clean":"2024-04-27T03:31:01.433313+0000","last_became_active":"2024-04-27T03:30:15.381956+0000","last_became_peered":"2024-04-27T03:30:15.381956+0000","last_unstale":"2024-04-27T03:31:01.433313+0000","last_undegraded":"2024-04-27T03:31:01.433313+0000","last_fullsized":"2024-04-27T03:31:01.433313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:10.054835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"27'60","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.925854+0000","last_change":"2024-04-27T03:30:17.383982+0000","last_active":"2024-04-27T03:32:18.925854+0000","last_peered":"2024-04-27T03:32:18.925854+0000","last_clean":"2024-04-27T03:32:18.925854+0000","last_became_active":"2024-04-27T03:30:17.383178+0000","last_became_peered":"2024-04-27T03:30:17.383178+0000","last_unstale":"2024-04-27T03:32:18.925854+0000","last_undegraded":"2024-04-27T03:32:18.925854+0000","last_fullsized":"2024-04-27T03:32:18.925854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:14.015528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":48,"num_write":24,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433851+0000","last_change":"2024-04-27T03:30:19.403087+0000","last_active":"2024-04-27T03:31:01.433851+0000","last_peered":"2024-04-27T03:31:01.433851+0000","last_clean":"2024-04-27T03:31:01.433851+0000","last_became_active":"2024-04-27T03:30:19.402781+0000","last_became_peered":"2024-04-27T03:30:19.402781+0000","last_unstale":"2024-04-27T03:31:01.433851+0000","last_undegraded":"2024-04-27T03:31:01.433851+0000","last_fullsized":"2024-04-27T03:31:01.433851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:30.540461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"27'16","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.240928+0000","last_change":"2024-04-27T03:30:21.408415+0000","last_active":"2024-04-27T03:32:20.240928+0000","last_peered":"2024-04-27T03:32:20.240928+0000","last_clean":"2024-04-27T03:32:20.240928+0000","last_became_active":"2024-04-27T03:30:21.408049+0000","last_became_peered":"2024-04-27T03:30:21.408049+0000","last_unstale":"2024-04-27T03:32:20.240928+0000","last_undegraded":"2024-04-27T03:32:20.240928+0000","last_fullsized":"2024-04-27T03:32:20.240928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:09.753400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":21,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432877+0000","last_change":"2024-04-27T03:30:13.373552+0000","last_active":"2024-04-27T03:31:01.432877+0000","last_peered":"2024-04-27T03:31:01.432877+0000","last_clean":"2024-04-27T03:31:01.432877+0000","last_became_active":"2024-04-27T03:30:13.373257+0000","last_became_peered":"2024-04-27T03:30:13.373257+0000","last_unstale":"2024-04-27T03:31:01.432877+0000","last_undegraded":"2024-04-27T03:31:01.432877+0000","last_fullsized":"2024-04-27T03:31:01.432877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:07:51.011104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432526+0000","last_change":"2024-04-27T03:30:11.371431+0000","last_active":"2024-04-27T03:31:01.432526+0000","last_peered":"2024-04-27T03:31:01.432526+0000","last_clean":"2024-04-27T03:31:01.432526+0000","last_became_active":"2024-04-27T03:30:11.370091+0000","last_became_peered":"2024-04-27T03:30:11.370091+0000","last_unstale":"2024-04-27T03:31:01.432526+0000","last_undegraded":"2024-04-27T03:31:01.432526+0000","last_fullsized":"2024-04-27T03:31:01.432526+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.746108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432766+0000","last_change":"2024-04-27T03:30:15.380494+0000","last_active":"2024-04-27T03:31:01.432766+0000","last_peered":"2024-04-27T03:31:01.432766+0000","last_clean":"2024-04-27T03:31:01.432766+0000","last_became_active":"2024-04-27T03:30:15.379800+0000","last_became_peered":"2024-04-27T03:30:15.379800+0000","last_unstale":"2024-04-27T03:31:01.432766+0000","last_undegraded":"2024-04-27T03:31:01.432766+0000","last_fullsized":"2024-04-27T03:31:01.432766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:58.036077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"27'36","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.905630+0000","last_change":"2024-04-27T03:30:17.389821+0000","last_active":"2024-04-27T03:32:18.905630+0000","last_peered":"2024-04-27T03:32:18.905630+0000","last_clean":"2024-04-27T03:32:18.905630+0000","last_became_active":"2024-04-27T03:30:17.388246+0000","last_became_peered":"2024-04-27T03:30:17.388246+0000","last_unstale":"2024-04-27T03:32:18.905630+0000","last_undegraded":"2024-04-27T03:32:18.905630+0000","last_fullsized":"2024-04-27T03:32:18.905630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:10:57.956151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437582+0000","last_change":"2024-04-27T03:30:19.407011+0000","last_active":"2024-04-27T03:31:01.437582+0000","last_peered":"2024-04-27T03:31:01.437582+0000","last_clean":"2024-04-27T03:31:01.437582+0000","last_became_active":"2024-04-27T03:30:19.406707+0000","last_became_peered":"2024-04-27T03:30:19.406707+0000","last_unstale":"2024-04-27T03:31:01.437582+0000","last_undegraded":"2024-04-27T03:31:01.437582+0000","last_fullsized":"2024-04-27T03:31:01.437582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:59.990733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436390+0000","last_change":"2024-04-27T03:30:21.415821+0000","last_active":"2024-04-27T03:31:01.436390+0000","last_peered":"2024-04-27T03:31:01.436390+0000","last_clean":"2024-04-27T03:31:01.436390+0000","last_became_active":"2024-04-27T03:30:21.415441+0000","last_became_peered":"2024-04-27T03:30:21.415441+0000","last_unstale":"2024-04-27T03:31:01.436390+0000","last_undegraded":"2024-04-27T03:31:01.436390+0000","last_fullsized":"2024-04-27T03:31:01.436390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:03.042667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933527+0000","last_change":"2024-04-27T03:30:13.388229+0000","last_active":"2024-04-27T03:33:20.933527+0000","last_peered":"2024-04-27T03:33:20.933527+0000","last_clean":"2024-04-27T03:33:20.933527+0000","last_became_active":"2024-04-27T03:30:13.387397+0000","last_became_peered":"2024-04-27T03:30:13.387397+0000","last_unstale":"2024-04-27T03:33:20.933527+0000","last_undegraded":"2024-04-27T03:33:20.933527+0000","last_fullsized":"2024-04-27T03:33:20.933527+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:45:47.437394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432931+0000","last_change":"2024-04-27T03:30:11.370706+0000","last_active":"2024-04-27T03:31:01.432931+0000","last_peered":"2024-04-27T03:31:01.432931+0000","last_clean":"2024-04-27T03:31:01.432931+0000","last_became_active":"2024-04-27T03:30:11.369799+0000","last_became_peered":"2024-04-27T03:30:11.369799+0000","last_unstale":"2024-04-27T03:31:01.432931+0000","last_undegraded":"2024-04-27T03:31:01.432931+0000","last_fullsized":"2024-04-27T03:31:01.432931+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.539212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433610+0000","last_change":"2024-04-27T03:30:15.380214+0000","last_active":"2024-04-27T03:31:01.433610+0000","last_peered":"2024-04-27T03:31:01.433610+0000","last_clean":"2024-04-27T03:31:01.433610+0000","last_became_active":"2024-04-27T03:30:15.379873+0000","last_became_peered":"2024-04-27T03:30:15.379873+0000","last_unstale":"2024-04-27T03:31:01.433610+0000","last_undegraded":"2024-04-27T03:31:01.433610+0000","last_fullsized":"2024-04-27T03:31:01.433610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:53.446024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"27'40","reported_seq":89,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.923662+0000","last_change":"2024-04-27T03:30:17.387440+0000","last_active":"2024-04-27T03:32:18.923662+0000","last_peered":"2024-04-27T03:32:18.923662+0000","last_clean":"2024-04-27T03:32:18.923662+0000","last_became_active":"2024-04-27T03:30:17.387205+0000","last_became_peered":"2024-04-27T03:30:17.387205+0000","last_unstale":"2024-04-27T03:32:18.923662+0000","last_undegraded":"2024-04-27T03:32:18.923662+0000","last_fullsized":"2024-04-27T03:32:18.923662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:26.079008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":54,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432503+0000","last_change":"2024-04-27T03:30:19.405759+0000","last_active":"2024-04-27T03:31:01.432503+0000","last_peered":"2024-04-27T03:31:01.432503+0000","last_clean":"2024-04-27T03:31:01.432503+0000","last_became_active":"2024-04-27T03:30:19.405462+0000","last_became_peered":"2024-04-27T03:30:19.405462+0000","last_unstale":"2024-04-27T03:31:01.432503+0000","last_undegraded":"2024-04-27T03:31:01.432503+0000","last_fullsized":"2024-04-27T03:31:01.432503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:59.990733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"27'7","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.302334+0000","last_change":"2024-04-27T03:30:21.416894+0000","last_active":"2024-04-27T03:32:20.302334+0000","last_peered":"2024-04-27T03:32:20.302334+0000","last_clean":"2024-04-27T03:32:20.302334+0000","last_became_active":"2024-04-27T03:30:21.416592+0000","last_became_peered":"2024-04-27T03:30:21.416592+0000","last_unstale":"2024-04-27T03:32:20.302334+0000","last_undegraded":"2024-04-27T03:32:20.302334+0000","last_fullsized":"2024-04-27T03:32:20.302334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:24:21.730546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"27'6","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.304764+0000","last_change":"2024-04-27T03:30:13.370253+0000","last_active":"2024-04-27T03:32:20.304764+0000","last_peered":"2024-04-27T03:32:20.304764+0000","last_clean":"2024-04-27T03:32:20.304764+0000","last_became_active":"2024-04-27T03:30:13.369643+0000","last_became_peered":"2024-04-27T03:30:13.369643+0000","last_unstale":"2024-04-27T03:32:20.304764+0000","last_undegraded":"2024-04-27T03:32:20.304764+0000","last_fullsized":"2024-04-27T03:32:20.304764+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:28.256872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298559+0000","last_change":"2024-04-27T03:30:11.372867+0000","last_active":"2024-04-27T03:32:44.298559+0000","last_peered":"2024-04-27T03:32:44.298559+0000","last_clean":"2024-04-27T03:32:44.298559+0000","last_became_active":"2024-04-27T03:30:11.372220+0000","last_became_peered":"2024-04-27T03:30:11.372220+0000","last_unstale":"2024-04-27T03:32:44.298559+0000","last_undegraded":"2024-04-27T03:32:44.298559+0000","last_fullsized":"2024-04-27T03:32:44.298559+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:43.207328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639872,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437382+0000","last_change":"2024-04-27T03:30:15.380858+0000","last_active":"2024-04-27T03:31:01.437382+0000","last_peered":"2024-04-27T03:31:01.437382+0000","last_clean":"2024-04-27T03:31:01.437382+0000","last_became_active":"2024-04-27T03:30:15.380573+0000","last_became_peered":"2024-04-27T03:30:15.380573+0000","last_unstale":"2024-04-27T03:31:01.437382+0000","last_undegraded":"2024-04-27T03:31:01.437382+0000","last_fullsized":"2024-04-27T03:31:01.437382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:45.374972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"25'16","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438994+0000","last_change":"2024-04-27T03:30:17.389436+0000","last_active":"2024-04-27T03:31:01.438994+0000","last_peered":"2024-04-27T03:31:01.438994+0000","last_clean":"2024-04-27T03:31:01.438994+0000","last_became_active":"2024-04-27T03:30:17.388289+0000","last_became_peered":"2024-04-27T03:30:17.388289+0000","last_unstale":"2024-04-27T03:31:01.438994+0000","last_undegraded":"2024-04-27T03:31:01.438994+0000","last_fullsized":"2024-04-27T03:31:01.438994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:02:33.022377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"27'282","reported_seq":372,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462759+0000","last_change":"2024-04-27T03:30:19.407449+0000","last_active":"2024-04-27T03:33:20.462759+0000","last_peered":"2024-04-27T03:33:20.462759+0000","last_clean":"2024-04-27T03:33:20.462759+0000","last_became_active":"2024-04-27T03:30:19.406854+0000","last_became_peered":"2024-04-27T03:30:19.406854+0000","last_unstale":"2024-04-27T03:33:20.462759+0000","last_undegraded":"2024-04-27T03:33:20.462759+0000","last_fullsized":"2024-04-27T03:33:20.462759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:55.112238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435927+0000","last_change":"2024-04-27T03:30:21.415992+0000","last_active":"2024-04-27T03:31:01.435927+0000","last_peered":"2024-04-27T03:31:01.435927+0000","last_clean":"2024-04-27T03:31:01.435927+0000","last_became_active":"2024-04-27T03:30:21.415534+0000","last_became_peered":"2024-04-27T03:30:21.415534+0000","last_unstale":"2024-04-27T03:31:01.435927+0000","last_undegraded":"2024-04-27T03:31:01.435927+0000","last_fullsized":"2024-04-27T03:31:01.435927+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:16.401751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"27'24","reported_seq":102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981535+0000","last_change":"2024-04-27T03:30:13.371535+0000","last_active":"2024-04-27T03:33:20.981535+0000","last_peered":"2024-04-27T03:33:20.981535+0000","last_clean":"2024-04-27T03:33:20.981535+0000","last_became_active":"2024-04-27T03:30:13.371158+0000","last_became_peered":"2024-04-27T03:30:13.371158+0000","last_unstale":"2024-04-27T03:33:20.981535+0000","last_undegraded":"2024-04-27T03:33:20.981535+0000","last_fullsized":"2024-04-27T03:33:20.981535+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:11:26.763810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":88,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435110+0000","last_change":"2024-04-27T03:30:11.359205+0000","last_active":"2024-04-27T03:31:01.435110+0000","last_peered":"2024-04-27T03:31:01.435110+0000","last_clean":"2024-04-27T03:31:01.435110+0000","last_became_active":"2024-04-27T03:30:11.358893+0000","last_became_peered":"2024-04-27T03:30:11.358893+0000","last_unstale":"2024-04-27T03:31:01.435110+0000","last_undegraded":"2024-04-27T03:31:01.435110+0000","last_fullsized":"2024-04-27T03:31:01.435110+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:52:31.047433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436411+0000","last_change":"2024-04-27T03:30:15.381535+0000","last_active":"2024-04-27T03:31:01.436411+0000","last_peered":"2024-04-27T03:31:01.436411+0000","last_clean":"2024-04-27T03:31:01.436411+0000","last_became_active":"2024-04-27T03:30:15.381175+0000","last_became_peered":"2024-04-27T03:30:15.381175+0000","last_unstale":"2024-04-27T03:31:01.436411+0000","last_undegraded":"2024-04-27T03:31:01.436411+0000","last_fullsized":"2024-04-27T03:31:01.436411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:21:29.464234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"27'60","reported_seq":129,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.896020+0000","last_change":"2024-04-27T03:30:17.386967+0000","last_active":"2024-04-27T03:32:18.896020+0000","last_peered":"2024-04-27T03:32:18.896020+0000","last_clean":"2024-04-27T03:32:18.896020+0000","last_became_active":"2024-04-27T03:30:17.386673+0000","last_became_peered":"2024-04-27T03:30:17.386673+0000","last_unstale":"2024-04-27T03:32:18.896020+0000","last_undegraded":"2024-04-27T03:32:18.896020+0000","last_fullsized":"2024-04-27T03:32:18.896020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:54:14.640233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430575+0000","last_change":"2024-04-27T03:30:19.407289+0000","last_active":"2024-04-27T03:31:01.430575+0000","last_peered":"2024-04-27T03:31:01.430575+0000","last_clean":"2024-04-27T03:31:01.430575+0000","last_became_active":"2024-04-27T03:30:19.406990+0000","last_became_peered":"2024-04-27T03:30:19.406990+0000","last_unstale":"2024-04-27T03:31:01.430575+0000","last_undegraded":"2024-04-27T03:31:01.430575+0000","last_fullsized":"2024-04-27T03:31:01.430575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:56:49.549144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433183+0000","last_change":"2024-04-27T03:30:21.410357+0000","last_active":"2024-04-27T03:31:01.433183+0000","last_peered":"2024-04-27T03:31:01.433183+0000","last_clean":"2024-04-27T03:31:01.433183+0000","last_became_active":"2024-04-27T03:30:21.410060+0000","last_became_peered":"2024-04-27T03:30:21.410060+0000","last_unstale":"2024-04-27T03:31:01.433183+0000","last_undegraded":"2024-04-27T03:31:01.433183+0000","last_fullsized":"2024-04-27T03:31:01.433183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:40.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"27'10","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009762+0000","last_change":"2024-04-27T03:30:13.372186+0000","last_active":"2024-04-27T03:33:21.009762+0000","last_peered":"2024-04-27T03:33:21.009762+0000","last_clean":"2024-04-27T03:33:21.009762+0000","last_became_active":"2024-04-27T03:30:13.371866+0000","last_became_peered":"2024-04-27T03:30:13.371866+0000","last_unstale":"2024-04-27T03:33:21.009762+0000","last_undegraded":"2024-04-27T03:33:21.009762+0000","last_fullsized":"2024-04-27T03:33:21.009762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:45.213699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.929513+0000","last_change":"2024-04-27T03:30:11.362991+0000","last_active":"2024-04-27T03:33:20.929513+0000","last_peered":"2024-04-27T03:33:20.929513+0000","last_clean":"2024-04-27T03:33:20.929513+0000","last_became_active":"2024-04-27T03:30:11.362420+0000","last_became_peered":"2024-04-27T03:30:11.362420+0000","last_unstale":"2024-04-27T03:33:20.929513+0000","last_undegraded":"2024-04-27T03:33:20.929513+0000","last_fullsized":"2024-04-27T03:33:20.929513+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:16.411097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437111+0000","last_change":"2024-04-27T03:30:15.382620+0000","last_active":"2024-04-27T03:31:01.437111+0000","last_peered":"2024-04-27T03:31:01.437111+0000","last_clean":"2024-04-27T03:31:01.437111+0000","last_became_active":"2024-04-27T03:30:15.382305+0000","last_became_peered":"2024-04-27T03:30:15.382305+0000","last_unstale":"2024-04-27T03:31:01.437111+0000","last_undegraded":"2024-04-27T03:31:01.437111+0000","last_fullsized":"2024-04-27T03:31:01.437111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:04.836869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"27'38","reported_seq":86,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.292067+0000","last_change":"2024-04-27T03:30:17.390602+0000","last_active":"2024-04-27T03:32:44.292067+0000","last_peered":"2024-04-27T03:32:44.292067+0000","last_clean":"2024-04-27T03:32:44.292067+0000","last_became_active":"2024-04-27T03:30:17.390281+0000","last_became_peered":"2024-04-27T03:30:17.390281+0000","last_unstale":"2024-04-27T03:32:44.292067+0000","last_undegraded":"2024-04-27T03:32:44.292067+0000","last_fullsized":"2024-04-27T03:32:44.292067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:20.307428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5807,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":33,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437067+0000","last_change":"2024-04-27T03:30:19.406218+0000","last_active":"2024-04-27T03:31:01.437067+0000","last_peered":"2024-04-27T03:31:01.437067+0000","last_clean":"2024-04-27T03:31:01.437067+0000","last_became_active":"2024-04-27T03:30:19.405353+0000","last_became_peered":"2024-04-27T03:30:19.405353+0000","last_unstale":"2024-04-27T03:31:01.437067+0000","last_undegraded":"2024-04-27T03:31:01.437067+0000","last_fullsized":"2024-04-27T03:31:01.437067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:21.451111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438395+0000","last_change":"2024-04-27T03:30:21.416876+0000","last_active":"2024-04-27T03:31:01.438395+0000","last_peered":"2024-04-27T03:31:01.438395+0000","last_clean":"2024-04-27T03:31:01.438395+0000","last_became_active":"2024-04-27T03:30:21.416276+0000","last_became_peered":"2024-04-27T03:30:21.416276+0000","last_unstale":"2024-04-27T03:31:01.438395+0000","last_undegraded":"2024-04-27T03:31:01.438395+0000","last_fullsized":"2024-04-27T03:31:01.438395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:58.245817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'24","reported_seq":101,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.697833+0000","last_change":"2024-04-27T03:30:13.377820+0000","last_active":"2024-04-27T03:33:20.697833+0000","last_peered":"2024-04-27T03:33:20.697833+0000","last_clean":"2024-04-27T03:33:20.697833+0000","last_became_active":"2024-04-27T03:30:13.376415+0000","last_became_peered":"2024-04-27T03:30:13.376415+0000","last_unstale":"2024-04-27T03:33:20.697833+0000","last_undegraded":"2024-04-27T03:33:20.697833+0000","last_fullsized":"2024-04-27T03:33:20.697833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:02.028024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.287615+0000","last_change":"2024-04-27T03:30:11.361140+0000","last_active":"2024-04-27T03:32:14.287615+0000","last_peered":"2024-04-27T03:32:14.287615+0000","last_clean":"2024-04-27T03:32:14.287615+0000","last_became_active":"2024-04-27T03:30:11.360590+0000","last_became_peered":"2024-04-27T03:30:11.360590+0000","last_unstale":"2024-04-27T03:32:14.287615+0000","last_undegraded":"2024-04-27T03:32:14.287615+0000","last_fullsized":"2024-04-27T03:32:14.287615+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:52:58.316121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434401+0000","last_change":"2024-04-27T03:30:15.385912+0000","last_active":"2024-04-27T03:31:01.434401+0000","last_peered":"2024-04-27T03:31:01.434401+0000","last_clean":"2024-04-27T03:31:01.434401+0000","last_became_active":"2024-04-27T03:30:15.385614+0000","last_became_peered":"2024-04-27T03:30:15.385614+0000","last_unstale":"2024-04-27T03:31:01.434401+0000","last_undegraded":"2024-04-27T03:31:01.434401+0000","last_fullsized":"2024-04-27T03:31:01.434401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:54:32.462460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"27'32","reported_seq":78,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.921601+0000","last_change":"2024-04-27T03:30:17.387854+0000","last_active":"2024-04-27T03:32:18.921601+0000","last_peered":"2024-04-27T03:32:18.921601+0000","last_clean":"2024-04-27T03:32:18.921601+0000","last_became_active":"2024-04-27T03:30:17.386983+0000","last_became_peered":"2024-04-27T03:30:17.386983+0000","last_unstale":"2024-04-27T03:32:18.921601+0000","last_undegraded":"2024-04-27T03:32:18.921601+0000","last_fullsized":"2024-04-27T03:32:18.921601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:38.038360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":27,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436657+0000","last_change":"2024-04-27T03:30:19.402931+0000","last_active":"2024-04-27T03:31:01.436657+0000","last_peered":"2024-04-27T03:31:01.436657+0000","last_clean":"2024-04-27T03:31:01.436657+0000","last_became_active":"2024-04-27T03:30:19.402652+0000","last_became_peered":"2024-04-27T03:30:19.402652+0000","last_unstale":"2024-04-27T03:31:01.436657+0000","last_undegraded":"2024-04-27T03:31:01.436657+0000","last_fullsized":"2024-04-27T03:31:01.436657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:24.199405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"27'6","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.720254+0000","last_change":"2024-04-27T03:30:21.420988+0000","last_active":"2024-04-27T03:33:20.720254+0000","last_peered":"2024-04-27T03:33:20.720254+0000","last_clean":"2024-04-27T03:33:20.720254+0000","last_became_active":"2024-04-27T03:30:21.420725+0000","last_became_peered":"2024-04-27T03:30:21.420725+0000","last_unstale":"2024-04-27T03:33:20.720254+0000","last_undegraded":"2024-04-27T03:33:20.720254+0000","last_fullsized":"2024-04-27T03:33:20.720254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:02.117649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432186+0000","last_change":"2024-04-27T03:30:13.375384+0000","last_active":"2024-04-27T03:31:01.432186+0000","last_peered":"2024-04-27T03:31:01.432186+0000","last_clean":"2024-04-27T03:31:01.432186+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:31:01.432186+0000","last_undegraded":"2024-04-27T03:31:01.432186+0000","last_fullsized":"2024-04-27T03:31:01.432186+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:40:10.176600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436421+0000","last_change":"2024-04-27T03:30:11.370518+0000","last_active":"2024-04-27T03:31:01.436421+0000","last_peered":"2024-04-27T03:31:01.436421+0000","last_clean":"2024-04-27T03:31:01.436421+0000","last_became_active":"2024-04-27T03:30:11.369877+0000","last_became_peered":"2024-04-27T03:30:11.369877+0000","last_unstale":"2024-04-27T03:31:01.436421+0000","last_undegraded":"2024-04-27T03:31:01.436421+0000","last_fullsized":"2024-04-27T03:31:01.436421+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.059046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"27'2","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.550532+0000","last_change":"2024-04-27T03:30:15.382111+0000","last_active":"2024-04-27T03:32:17.550532+0000","last_peered":"2024-04-27T03:32:17.550532+0000","last_clean":"2024-04-27T03:32:17.550532+0000","last_became_active":"2024-04-27T03:30:15.381574+0000","last_became_peered":"2024-04-27T03:30:15.381574+0000","last_unstale":"2024-04-27T03:32:17.550532+0000","last_undegraded":"2024-04-27T03:32:17.550532+0000","last_fullsized":"2024-04-27T03:32:17.550532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:04.039451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"27'44","reported_seq":100,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.325275+0000","last_change":"2024-04-27T03:30:17.391410+0000","last_active":"2024-04-27T03:33:20.325275+0000","last_peered":"2024-04-27T03:33:20.325275+0000","last_clean":"2024-04-27T03:33:20.325275+0000","last_became_active":"2024-04-27T03:30:17.388277+0000","last_became_peered":"2024-04-27T03:30:17.388277+0000","last_unstale":"2024-04-27T03:33:20.325275+0000","last_undegraded":"2024-04-27T03:33:20.325275+0000","last_fullsized":"2024-04-27T03:33:20.325275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:05.729566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":52,"num_read_kb":46,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431300+0000","last_change":"2024-04-27T03:30:19.406905+0000","last_active":"2024-04-27T03:31:01.431300+0000","last_peered":"2024-04-27T03:31:01.431300+0000","last_clean":"2024-04-27T03:31:01.431300+0000","last_became_active":"2024-04-27T03:30:19.406565+0000","last_became_peered":"2024-04-27T03:30:19.406565+0000","last_unstale":"2024-04-27T03:31:01.431300+0000","last_undegraded":"2024-04-27T03:31:01.431300+0000","last_fullsized":"2024-04-27T03:31:01.431300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:31.846278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"27'4","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.934155+0000","last_change":"2024-04-27T03:30:21.414808+0000","last_active":"2024-04-27T03:32:18.934155+0000","last_peered":"2024-04-27T03:32:18.934155+0000","last_clean":"2024-04-27T03:32:18.934155+0000","last_became_active":"2024-04-27T03:30:21.414510+0000","last_became_peered":"2024-04-27T03:30:21.414510+0000","last_unstale":"2024-04-27T03:32:18.934155+0000","last_undegraded":"2024-04-27T03:32:18.934155+0000","last_fullsized":"2024-04-27T03:32:18.934155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:07.942157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"25'8","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436998+0000","last_change":"2024-04-27T03:30:13.387657+0000","last_active":"2024-04-27T03:31:01.436998+0000","last_peered":"2024-04-27T03:31:01.436998+0000","last_clean":"2024-04-27T03:31:01.436998+0000","last_became_active":"2024-04-27T03:30:13.387149+0000","last_became_peered":"2024-04-27T03:30:13.387149+0000","last_unstale":"2024-04-27T03:31:01.436998+0000","last_undegraded":"2024-04-27T03:31:01.436998+0000","last_fullsized":"2024-04-27T03:31:01.436998+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:00.325582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"25'6","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433385+0000","last_change":"2024-04-27T03:30:11.358174+0000","last_active":"2024-04-27T03:31:01.433385+0000","last_peered":"2024-04-27T03:31:01.433385+0000","last_clean":"2024-04-27T03:31:01.433385+0000","last_became_active":"2024-04-27T03:30:11.357878+0000","last_became_peered":"2024-04-27T03:30:11.357878+0000","last_unstale":"2024-04-27T03:31:01.433385+0000","last_undegraded":"2024-04-27T03:31:01.433385+0000","last_fullsized":"2024-04-27T03:31:01.433385+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:17.981757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.304915+0000","last_change":"2024-04-27T03:30:13.369979+0000","last_active":"2024-04-27T03:32:20.304915+0000","last_peered":"2024-04-27T03:32:20.304915+0000","last_clean":"2024-04-27T03:32:20.304915+0000","last_became_active":"2024-04-27T03:30:13.369681+0000","last_became_peered":"2024-04-27T03:30:13.369681+0000","last_unstale":"2024-04-27T03:32:20.304915+0000","last_undegraded":"2024-04-27T03:32:20.304915+0000","last_fullsized":"2024-04-27T03:32:20.304915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:37.770898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"25'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430839+0000","last_change":"2024-04-27T03:30:11.373422+0000","last_active":"2024-04-27T03:31:01.430839+0000","last_peered":"2024-04-27T03:31:01.430839+0000","last_clean":"2024-04-27T03:31:01.430839+0000","last_became_active":"2024-04-27T03:30:11.373186+0000","last_became_peered":"2024-04-27T03:30:11.373186+0000","last_unstale":"2024-04-27T03:31:01.430839+0000","last_undegraded":"2024-04-27T03:31:01.430839+0000","last_fullsized":"2024-04-27T03:31:01.430839+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:22.719424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933550+0000","last_change":"2024-04-27T03:30:13.382330+0000","last_active":"2024-04-27T03:33:20.933550+0000","last_peered":"2024-04-27T03:33:20.933550+0000","last_clean":"2024-04-27T03:33:20.933550+0000","last_became_active":"2024-04-27T03:30:13.382069+0000","last_became_peered":"2024-04-27T03:30:13.382069+0000","last_unstale":"2024-04-27T03:33:20.933550+0000","last_undegraded":"2024-04-27T03:33:20.933550+0000","last_fullsized":"2024-04-27T03:33:20.933550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:22.941552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"27'3","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.420503+0000","last_change":"2024-04-27T03:30:11.350547+0000","last_active":"2024-04-27T03:31:36.420503+0000","last_peered":"2024-04-27T03:31:36.420503+0000","last_clean":"2024-04-27T03:31:36.420503+0000","last_became_active":"2024-04-27T03:30:11.350295+0000","last_became_peered":"2024-04-27T03:30:11.350295+0000","last_unstale":"2024-04-27T03:31:36.420503+0000","last_undegraded":"2024-04-27T03:31:36.420503+0000","last_fullsized":"2024-04-27T03:31:36.420503+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:45:58.123163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933497+0000","last_change":"2024-04-27T03:30:13.370393+0000","last_active":"2024-04-27T03:33:20.933497+0000","last_peered":"2024-04-27T03:33:20.933497+0000","last_clean":"2024-04-27T03:33:20.933497+0000","last_became_active":"2024-04-27T03:30:13.370066+0000","last_became_peered":"2024-04-27T03:30:13.370066+0000","last_unstale":"2024-04-27T03:33:20.933497+0000","last_undegraded":"2024-04-27T03:33:20.933497+0000","last_fullsized":"2024-04-27T03:33:20.933497+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:33.811635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"27'3","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288156+0000","last_change":"2024-04-27T03:30:11.360775+0000","last_active":"2024-04-27T03:32:19.288156+0000","last_peered":"2024-04-27T03:32:19.288156+0000","last_clean":"2024-04-27T03:32:19.288156+0000","last_became_active":"2024-04-27T03:30:11.360376+0000","last_became_peered":"2024-04-27T03:30:11.360376+0000","last_unstale":"2024-04-27T03:32:19.288156+0000","last_undegraded":"2024-04-27T03:32:19.288156+0000","last_fullsized":"2024-04-27T03:32:19.288156+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:00.825717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.792398+0000","last_change":"2024-04-27T03:30:13.378662+0000","last_active":"2024-04-27T03:33:20.792398+0000","last_peered":"2024-04-27T03:33:20.792398+0000","last_clean":"2024-04-27T03:33:20.792398+0000","last_became_active":"2024-04-27T03:30:13.378368+0000","last_became_peered":"2024-04-27T03:30:13.378368+0000","last_unstale":"2024-04-27T03:33:20.792398+0000","last_undegraded":"2024-04-27T03:33:20.792398+0000","last_fullsized":"2024-04-27T03:33:20.792398+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:56.906775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432139+0000","last_change":"2024-04-27T03:30:11.371839+0000","last_active":"2024-04-27T03:31:01.432139+0000","last_peered":"2024-04-27T03:31:01.432139+0000","last_clean":"2024-04-27T03:31:01.432139+0000","last_became_active":"2024-04-27T03:30:11.371553+0000","last_became_peered":"2024-04-27T03:30:11.371553+0000","last_unstale":"2024-04-27T03:31:01.432139+0000","last_undegraded":"2024-04-27T03:31:01.432139+0000","last_fullsized":"2024-04-27T03:31:01.432139+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.059046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"27'10","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009740+0000","last_change":"2024-04-27T03:31:01.440910+0000","last_active":"2024-04-27T03:33:21.009740+0000","last_peered":"2024-04-27T03:33:21.009740+0000","last_clean":"2024-04-27T03:33:21.009740+0000","last_became_active":"2024-04-27T03:31:01.440576+0000","last_became_peered":"2024-04-27T03:31:01.440576+0000","last_unstale":"2024-04-27T03:33:21.009740+0000","last_undegraded":"2024-04-27T03:33:21.009740+0000","last_fullsized":"2024-04-27T03:33:21.009740+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:04.874525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.213761+0000","last_change":"2024-04-27T03:30:11.371034+0000","last_active":"2024-04-27T03:32:20.213761+0000","last_peered":"2024-04-27T03:32:20.213761+0000","last_clean":"2024-04-27T03:32:20.213761+0000","last_became_active":"2024-04-27T03:30:11.369517+0000","last_became_peered":"2024-04-27T03:30:11.369517+0000","last_unstale":"2024-04-27T03:32:20.213761+0000","last_undegraded":"2024-04-27T03:32:20.213761+0000","last_fullsized":"2024-04-27T03:32:20.213761+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:22.719424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.107522+0000","last_change":"2024-04-27T03:30:13.374070+0000","last_active":"2024-04-27T03:32:17.107522+0000","last_peered":"2024-04-27T03:32:17.107522+0000","last_clean":"2024-04-27T03:32:17.107522+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:32:17.107522+0000","last_undegraded":"2024-04-27T03:32:17.107522+0000","last_fullsized":"2024-04-27T03:32:17.107522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:54.324828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.442163+0000","last_change":"2024-04-27T03:31:01.441506+0000","last_active":"2024-04-27T03:31:01.442163+0000","last_peered":"2024-04-27T03:31:01.442163+0000","last_clean":"2024-04-27T03:31:01.442163+0000","last_became_active":"2024-04-27T03:31:01.441159+0000","last_became_peered":"2024-04-27T03:31:01.441159+0000","last_unstale":"2024-04-27T03:31:01.442163+0000","last_undegraded":"2024-04-27T03:31:01.442163+0000","last_fullsized":"2024-04-27T03:31:01.442163+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:11:09.529188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"27'3","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.802333+0000","last_change":"2024-04-27T03:30:13.378083+0000","last_active":"2024-04-27T03:33:20.802333+0000","last_peered":"2024-04-27T03:33:20.802333+0000","last_clean":"2024-04-27T03:33:20.802333+0000","last_became_active":"2024-04-27T03:30:13.377821+0000","last_became_peered":"2024-04-27T03:30:13.377821+0000","last_unstale":"2024-04-27T03:33:20.802333+0000","last_undegraded":"2024-04-27T03:33:20.802333+0000","last_fullsized":"2024-04-27T03:33:20.802333+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:18.417469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432873+0000","last_change":"2024-04-27T03:30:11.371906+0000","last_active":"2024-04-27T03:31:01.432873+0000","last_peered":"2024-04-27T03:31:01.432873+0000","last_clean":"2024-04-27T03:31:01.432873+0000","last_became_active":"2024-04-27T03:30:11.371604+0000","last_became_peered":"2024-04-27T03:30:11.371604+0000","last_unstale":"2024-04-27T03:31:01.432873+0000","last_undegraded":"2024-04-27T03:31:01.432873+0000","last_fullsized":"2024-04-27T03:31:01.432873+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:33:05.923239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"27'8","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981663+0000","last_change":"2024-04-27T03:30:13.388711+0000","last_active":"2024-04-27T03:33:20.981663+0000","last_peered":"2024-04-27T03:33:20.981663+0000","last_clean":"2024-04-27T03:33:20.981663+0000","last_became_active":"2024-04-27T03:30:13.388421+0000","last_became_peered":"2024-04-27T03:30:13.388421+0000","last_unstale":"2024-04-27T03:33:20.981663+0000","last_undegraded":"2024-04-27T03:33:20.981663+0000","last_fullsized":"2024-04-27T03:33:20.981663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:35.526991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.678119+0000","last_change":"2024-04-27T03:30:11.355453+0000","last_active":"2024-04-27T03:33:20.678119+0000","last_peered":"2024-04-27T03:33:20.678119+0000","last_clean":"2024-04-27T03:33:20.678119+0000","last_became_active":"2024-04-27T03:30:11.355183+0000","last_became_peered":"2024-04-27T03:30:11.355183+0000","last_unstale":"2024-04-27T03:33:20.678119+0000","last_undegraded":"2024-04-27T03:33:20.678119+0000","last_fullsized":"2024-04-27T03:33:20.678119+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:12.667203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"27'4","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.107011+0000","last_change":"2024-04-27T03:30:13.377155+0000","last_active":"2024-04-27T03:32:17.107011+0000","last_peered":"2024-04-27T03:32:17.107011+0000","last_clean":"2024-04-27T03:32:17.107011+0000","last_became_active":"2024-04-27T03:30:13.376875+0000","last_became_peered":"2024-04-27T03:30:13.376875+0000","last_unstale":"2024-04-27T03:32:17.107011+0000","last_undegraded":"2024-04-27T03:32:17.107011+0000","last_fullsized":"2024-04-27T03:32:17.107011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:09.993585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431158+0000","last_change":"2024-04-27T03:30:11.361373+0000","last_active":"2024-04-27T03:31:01.431158+0000","last_peered":"2024-04-27T03:31:01.431158+0000","last_clean":"2024-04-27T03:31:01.431158+0000","last_became_active":"2024-04-27T03:30:11.360299+0000","last_became_peered":"2024-04-27T03:30:11.360299+0000","last_unstale":"2024-04-27T03:31:01.431158+0000","last_undegraded":"2024-04-27T03:31:01.431158+0000","last_fullsized":"2024-04-27T03:31:01.431158+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.746108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"27'10","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981744+0000","last_change":"2024-04-27T03:30:13.376269+0000","last_active":"2024-04-27T03:33:20.981744+0000","last_peered":"2024-04-27T03:33:20.981744+0000","last_clean":"2024-04-27T03:33:20.981744+0000","last_became_active":"2024-04-27T03:30:13.376087+0000","last_became_peered":"2024-04-27T03:30:13.376087+0000","last_unstale":"2024-04-27T03:33:20.981744+0000","last_undegraded":"2024-04-27T03:33:20.981744+0000","last_fullsized":"2024-04-27T03:33:20.981744+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:18.301070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432834+0000","last_change":"2024-04-27T03:30:11.371237+0000","last_active":"2024-04-27T03:31:01.432834+0000","last_peered":"2024-04-27T03:31:01.432834+0000","last_clean":"2024-04-27T03:31:01.432834+0000","last_became_active":"2024-04-27T03:30:11.370837+0000","last_became_peered":"2024-04-27T03:30:11.370837+0000","last_unstale":"2024-04-27T03:31:01.432834+0000","last_undegraded":"2024-04-27T03:31:01.432834+0000","last_fullsized":"2024-04-27T03:31:01.432834+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.863119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"27'6","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.697535+0000","last_change":"2024-04-27T03:30:13.376130+0000","last_active":"2024-04-27T03:33:20.697535+0000","last_peered":"2024-04-27T03:33:20.697535+0000","last_clean":"2024-04-27T03:33:20.697535+0000","last_became_active":"2024-04-27T03:30:13.373511+0000","last_became_peered":"2024-04-27T03:30:13.373511+0000","last_unstale":"2024-04-27T03:33:20.697535+0000","last_undegraded":"2024-04-27T03:33:20.697535+0000","last_fullsized":"2024-04-27T03:33:20.697535+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:51.434596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.257092+0000","last_change":"2024-04-27T03:30:11.370821+0000","last_active":"2024-04-27T03:32:20.257092+0000","last_peered":"2024-04-27T03:32:20.257092+0000","last_clean":"2024-04-27T03:32:20.257092+0000","last_became_active":"2024-04-27T03:30:11.370283+0000","last_became_peered":"2024-04-27T03:30:11.370283+0000","last_unstale":"2024-04-27T03:32:20.257092+0000","last_undegraded":"2024-04-27T03:32:20.257092+0000","last_fullsized":"2024-04-27T03:32:20.257092+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:17.981757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"27'11","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.016376+0000","last_change":"2024-04-27T03:30:13.380862+0000","last_active":"2024-04-27T03:33:21.016376+0000","last_peered":"2024-04-27T03:33:21.016376+0000","last_clean":"2024-04-27T03:33:21.016376+0000","last_became_active":"2024-04-27T03:30:13.380404+0000","last_became_peered":"2024-04-27T03:30:13.380404+0000","last_unstale":"2024-04-27T03:33:21.016376+0000","last_undegraded":"2024-04-27T03:33:21.016376+0000","last_fullsized":"2024-04-27T03:33:21.016376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:14.849083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439493+0000","last_change":"2024-04-27T03:30:11.373282+0000","last_active":"2024-04-27T03:31:01.439493+0000","last_peered":"2024-04-27T03:31:01.439493+0000","last_clean":"2024-04-27T03:31:01.439493+0000","last_became_active":"2024-04-27T03:30:11.373026+0000","last_became_peered":"2024-04-27T03:30:11.373026+0000","last_unstale":"2024-04-27T03:31:01.439493+0000","last_undegraded":"2024-04-27T03:31:01.439493+0000","last_fullsized":"2024-04-27T03:31:01.439493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:40:08.170228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"27'1","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295321+0000","last_change":"2024-04-27T03:30:13.386073+0000","last_active":"2024-04-27T03:32:29.295321+0000","last_peered":"2024-04-27T03:32:29.295321+0000","last_clean":"2024-04-27T03:32:29.295321+0000","last_became_active":"2024-04-27T03:30:13.385791+0000","last_became_peered":"2024-04-27T03:30:13.385791+0000","last_unstale":"2024-04-27T03:32:29.295321+0000","last_undegraded":"2024-04-27T03:32:29.295321+0000","last_fullsized":"2024-04-27T03:32:29.295321+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:18.301070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.441318+0000","last_change":"2024-04-27T03:30:11.358239+0000","last_active":"2024-04-27T03:31:01.441318+0000","last_peered":"2024-04-27T03:31:01.441318+0000","last_clean":"2024-04-27T03:31:01.441318+0000","last_became_active":"2024-04-27T03:30:11.357898+0000","last_became_peered":"2024-04-27T03:30:11.357898+0000","last_unstale":"2024-04-27T03:31:01.441318+0000","last_undegraded":"2024-04-27T03:31:01.441318+0000","last_fullsized":"2024-04-27T03:31:01.441318+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:26.250500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"27'5","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:19.303737+0000","last_change":"2024-04-27T03:31:01.444873+0000","last_active":"2024-04-27T03:33:19.303737+0000","last_peered":"2024-04-27T03:33:19.303737+0000","last_clean":"2024-04-27T03:33:19.303737+0000","last_became_active":"2024-04-27T03:31:01.440820+0000","last_became_peered":"2024-04-27T03:31:01.440820+0000","last_unstale":"2024-04-27T03:33:19.303737+0000","last_undegraded":"2024-04-27T03:33:19.303737+0000","last_fullsized":"2024-04-27T03:33:19.303737+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:15.446814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435323+0000","last_change":"2024-04-27T03:30:11.363676+0000","last_active":"2024-04-27T03:31:01.435323+0000","last_peered":"2024-04-27T03:31:01.435323+0000","last_clean":"2024-04-27T03:31:01.435323+0000","last_became_active":"2024-04-27T03:30:11.362788+0000","last_became_peered":"2024-04-27T03:30:11.362788+0000","last_unstale":"2024-04-27T03:31:01.435323+0000","last_undegraded":"2024-04-27T03:31:01.435323+0000","last_fullsized":"2024-04-27T03:31:01.435323+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:00.825717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"25'2","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436660+0000","last_change":"2024-04-27T03:30:13.378124+0000","last_active":"2024-04-27T03:31:01.436660+0000","last_peered":"2024-04-27T03:31:01.436660+0000","last_clean":"2024-04-27T03:31:01.436660+0000","last_became_active":"2024-04-27T03:30:13.377845+0000","last_became_peered":"2024-04-27T03:30:13.377845+0000","last_unstale":"2024-04-27T03:31:01.436660+0000","last_undegraded":"2024-04-27T03:31:01.436660+0000","last_fullsized":"2024-04-27T03:31:01.436660+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:37.770898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296913+0000","last_change":"2024-04-27T03:30:11.362917+0000","last_active":"2024-04-27T03:32:34.296913+0000","last_peered":"2024-04-27T03:32:34.296913+0000","last_clean":"2024-04-27T03:32:34.296913+0000","last_became_active":"2024-04-27T03:30:11.362559+0000","last_became_peered":"2024-04-27T03:30:11.362559+0000","last_unstale":"2024-04-27T03:32:34.296913+0000","last_undegraded":"2024-04-27T03:32:34.296913+0000","last_fullsized":"2024-04-27T03:32:34.296913+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:55.587028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436858+0000","last_change":"2024-04-27T03:30:11.363675+0000","last_active":"2024-04-27T03:31:01.436858+0000","last_peered":"2024-04-27T03:31:01.436858+0000","last_clean":"2024-04-27T03:31:01.436858+0000","last_became_active":"2024-04-27T03:30:11.363340+0000","last_became_peered":"2024-04-27T03:30:11.363340+0000","last_unstale":"2024-04-27T03:31:01.436858+0000","last_undegraded":"2024-04-27T03:31:01.436858+0000","last_fullsized":"2024-04-27T03:31:01.436858+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:26.857275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.107465+0000","last_change":"2024-04-27T03:30:13.386029+0000","last_active":"2024-04-27T03:32:17.107465+0000","last_peered":"2024-04-27T03:32:17.107465+0000","last_clean":"2024-04-27T03:32:17.107465+0000","last_became_active":"2024-04-27T03:30:13.385705+0000","last_became_peered":"2024-04-27T03:30:13.385705+0000","last_unstale":"2024-04-27T03:32:17.107465+0000","last_undegraded":"2024-04-27T03:32:17.107465+0000","last_fullsized":"2024-04-27T03:32:17.107465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:18.417469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430983+0000","last_change":"2024-04-27T03:30:11.360980+0000","last_active":"2024-04-27T03:31:01.430983+0000","last_peered":"2024-04-27T03:31:01.430983+0000","last_clean":"2024-04-27T03:31:01.430983+0000","last_became_active":"2024-04-27T03:30:11.360667+0000","last_became_peered":"2024-04-27T03:30:11.360667+0000","last_unstale":"2024-04-27T03:31:01.430983+0000","last_undegraded":"2024-04-27T03:31:01.430983+0000","last_fullsized":"2024-04-27T03:31:01.430983+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:51.203886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"27'14","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.305196+0000","last_change":"2024-04-27T03:30:13.380806+0000","last_active":"2024-04-27T03:32:20.305196+0000","last_peered":"2024-04-27T03:32:20.305196+0000","last_clean":"2024-04-27T03:32:20.305196+0000","last_became_active":"2024-04-27T03:30:13.380347+0000","last_became_peered":"2024-04-27T03:30:13.380347+0000","last_unstale":"2024-04-27T03:32:20.305196+0000","last_undegraded":"2024-04-27T03:32:20.305196+0000","last_fullsized":"2024-04-27T03:32:20.305196+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:48.533060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13385,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":1690,"num_read_kb":1444,"num_write":723,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1470,"ondisk_log_size":1470,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25805696,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":128,"num_read_kb":82,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":51617792,"data_stored":51611392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705706,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44568,"kb_used_data":9232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94327272,"statfs":{"total":96636764160,"available":96591126528,"internally_reserved":0,"allocated":9453568,"data_stored":9151139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705707,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60832,"kb_used_data":25496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94311008,"statfs":{"total":96636764160,"available":96574472192,"internally_reserved":0,"allocated":26107904,"data_stored":25873899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705706,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1613824,"data_stored":1348704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":134,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56256,"kb_used_data":16760,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94315584,"statfs":{"total":96636764160,"available":96579158016,"internally_reserved":0,"allocated":17162240,"data_stored":16898458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16777216,"data_stored":16777216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":643072,"data_stored":639872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25165824,"data_stored":25165824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9031680,"data_stored":9028480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":4269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T03:33:28.404 INFO:tasks.ceph.ceph_manager.ceph:clean! 2024-04-27T03:33:28.405 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T03:33:28.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.524+0000 7f56f3c6a640 1 Processor -- start 2024-04-27T03:33:28.530 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.524+0000 7f56f3c6a640 1 -- start start 2024-04-27T03:33:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.525+0000 7f56f3c6a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f56ec06fd50 0x7f56ec133860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.525+0000 7f56f3c6a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f56ec133da0 con 0x7f56ec06f260 2024-04-27T03:33:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.525+0000 7f56f3c6a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f56ec070bc0 con 0x7f56ec06fd50 2024-04-27T03:33:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.525+0000 7f56f1a06640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f56ec06f260 0x7f56ec06f640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48014/0 (socket says 172.21.15.89:48014) 2024-04-27T03:33:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.525+0000 7f56f1a06640 1 -- 172.21.15.89:0/1464037748 learned_addr learned my addr 172.21.15.89:0/1464037748 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:28.532 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.525+0000 7f56f1205640 1 --2- 172.21.15.89:0/1464037748 >> v2:172.21.15.89:3300/0 conn(0x7f56ec06fd50 0x7f56ec133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.525+0000 7f56f0a04640 1 -- 172.21.15.89:0/1464037748 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3854494834 0 0) 0x7f56ec133da0 con 0x7f56ec06f260 2024-04-27T03:33:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.525+0000 7f56f0a04640 1 -- 172.21.15.89:0/1464037748 --> v1:172.21.15.89:6789/0 -- auth(proto 2 36 bytes epoch 0) -- 0x7f56d8003520 con 0x7f56ec06f260 2024-04-27T03:33:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.525+0000 7f56f1205640 1 -- 172.21.15.89:0/1464037748 >> v1:172.21.15.89:6789/0 conn(0x7f56ec06f260 legacy=0x7f56ec06f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.525+0000 7f56f1205640 1 -- 172.21.15.89:0/1464037748 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec134f80 con 0x7f56ec06fd50 2024-04-27T03:33:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.526+0000 7f56f1205640 1 --2- 172.21.15.89:0/1464037748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ec06fd50 0x7f56ec133860 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009720 tx=0x7f56dc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e3e8d240060ef0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.526+0000 7f56f0a04640 1 -- 172.21.15.89:0/1464037748 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56dc0316d0 con 0x7f56ec06fd50 2024-04-27T03:33:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.526+0000 7f56f0a04640 1 -- 172.21.15.89:0/1464037748 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56dc037070 con 0x7f56ec06fd50 2024-04-27T03:33:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.526+0000 7f56f0a04640 1 -- 172.21.15.89:0/1464037748 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56dc0319c0 con 0x7f56ec06fd50 2024-04-27T03:33:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.526+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1464037748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ec06fd50 msgr2=0x7f56ec133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.526+0000 7f56f3c6a640 1 --2- 172.21.15.89:0/1464037748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ec06fd50 0x7f56ec133860 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009720 tx=0x7f56dc02b840 comp rx=0 tx=0).stop 2024-04-27T03:33:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.527+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1464037748 shutdown_connections 2024-04-27T03:33:28.533 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.527+0000 7f56f3c6a640 1 --2- 172.21.15.89:0/1464037748 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ec06fd50 0x7f56ec133860 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.527+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1464037748 >> 172.21.15.89:0/1464037748 conn(0x7f56ec09e170 msgr2=0x7f56ec06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:28.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.527+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1464037748 shutdown_connections 2024-04-27T03:33:28.534 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.527+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1464037748 wait complete. 2024-04-27T03:33:28.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.527+0000 7f56f3c6a640 1 Processor -- start 2024-04-27T03:33:28.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.527+0000 7f56f3c6a640 1 -- start start 2024-04-27T03:33:28.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.528+0000 7f56f3c6a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ec06f260 0x7f56ec1247f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:28.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.528+0000 7f56f3c6a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f56ec134300 con 0x7f56ec06f260 2024-04-27T03:33:28.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.528+0000 7f56f1a06640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ec06f260 0x7f56ec1247f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:28.536 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.528+0000 7f56f1a06640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ec06f260 0x7f56ec1247f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60224/0 (socket says 172.21.15.89:60224) 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.528+0000 7f56f1a06640 1 -- 172.21.15.89:0/1657900795 learned_addr learned my addr 172.21.15.89:0/1657900795 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.528+0000 7f56f1a06640 1 -- 172.21.15.89:0/1657900795 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56ec131cf0 con 0x7f56ec06f260 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.528+0000 7f56f1a06640 1 --2- 172.21.15.89:0/1657900795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ec06f260 0x7f56ec1247f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f56e400cb40 tx=0x7f56e40021e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.528+0000 7f56e27fc640 1 -- 172.21.15.89:0/1657900795 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56e4017070 con 0x7f56ec06f260 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.528+0000 7f56e27fc640 1 -- 172.21.15.89:0/1657900795 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f56e400ab80 con 0x7f56ec06f260 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.528+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1657900795 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56ec131610 con 0x7f56ec06f260 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.528+0000 7f56e27fc640 1 -- 172.21.15.89:0/1657900795 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f56e4012400 con 0x7f56ec06f260 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.528+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1657900795 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56ec132eb0 con 0x7f56ec06f260 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.529+0000 7f56e27fc640 1 -- 172.21.15.89:0/1657900795 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f56e401a070 con 0x7f56ec06f260 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.529+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1657900795 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56ec131ab0 con 0x7f56ec06f260 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.529+0000 7f56e27fc640 1 --2- 172.21.15.89:0/1657900795 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f56d0038060 0x7f56d003a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.529+0000 7f56e27fc640 1 -- 172.21.15.89:0/1657900795 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f56e404b750 con 0x7f56ec06f260 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.530+0000 7f56f1205640 1 --2- 172.21.15.89:0/1657900795 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f56d0038060 0x7f56d003a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:28.537 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.530+0000 7f56f1205640 1 --2- 172.21.15.89:0/1657900795 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f56d0038060 0x7f56d003a520 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009720 tx=0x7f56dc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:28.539 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.534+0000 7f56e27fc640 1 -- 172.21.15.89:0/1657900795 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f56e401a760 con 0x7f56ec06f260 2024-04-27T03:33:28.708 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.702+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1657900795 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f56ec0a18a0 con 0x7f56d0038060 2024-04-27T03:33:28.722 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.717+0000 7f56e27fc640 1 -- 172.21.15.89:0/1657900795 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665463 (secure 0 0 0) 0x7f56ec0a18a0 con 0x7f56d0038060 2024-04-27T03:33:28.731 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:33:28.736 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-04-27T03:33:28.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.721+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1657900795 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f56d0038060 msgr2=0x7f56d003a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:28.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.721+0000 7f56f3c6a640 1 --2- 172.21.15.89:0/1657900795 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f56d0038060 0x7f56d003a520 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f56dc009720 tx=0x7f56dc008030 comp rx=0 tx=0).stop 2024-04-27T03:33:28.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.721+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1657900795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ec06f260 msgr2=0x7f56ec1247f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:28.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.721+0000 7f56f3c6a640 1 --2- 172.21.15.89:0/1657900795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ec06f260 0x7f56ec1247f0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7f56e400cb40 tx=0x7f56e40021e0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.721+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1657900795 shutdown_connections 2024-04-27T03:33:28.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.721+0000 7f56f3c6a640 1 --2- 172.21.15.89:0/1657900795 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f56d0038060 0x7f56d003a520 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.721+0000 7f56f3c6a640 1 --2- 172.21.15.89:0/1657900795 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f56ec06f260 0x7f56ec1247f0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.721+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1657900795 >> 172.21.15.89:0/1657900795 conn(0x7f56ec09e170 msgr2=0x7f56ec09e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:28.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.722+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1657900795 shutdown_connections 2024-04-27T03:33:28.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.722+0000 7f56f3c6a640 1 -- 172.21.15.89:0/1657900795 wait complete. 2024-04-27T03:33:28.740 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":126,"stamp":"2024-04-27T03:33:28.271321+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":26412790,"num_objects":259,"num_object_clones":0,"num_object_copies":518,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":259,"num_whiteouts":0,"num_read":5162,"num_read_kb":4712,"num_write":2296,"num_write_kb":61372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4671,"ondisk_log_size":4671,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":198568,"kb_used_data":53064,"kb_used_omap":6,"kb_used_meta":145465,"kb_avail":377288792,"statfs":{"total":386547056640,"available":386343723008,"internally_reserved":0,"allocated":54337536,"data_stored":53272200,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":148956950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":25811688,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":497,"num_read_kb":452,"num_write":384,"num_write_kb":29371,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004184"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433166+0000","last_change":"2024-04-27T03:30:11.362820+0000","last_active":"2024-04-27T03:31:01.433166+0000","last_peered":"2024-04-27T03:31:01.433166+0000","last_clean":"2024-04-27T03:31:01.433166+0000","last_became_active":"2024-04-27T03:30:11.362574+0000","last_became_peered":"2024-04-27T03:30:11.362574+0000","last_unstale":"2024-04-27T03:31:01.433166+0000","last_undegraded":"2024-04-27T03:31:01.433166+0000","last_fullsized":"2024-04-27T03:31:01.433166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.746108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933798+0000","last_change":"2024-04-27T03:30:13.384819+0000","last_active":"2024-04-27T03:33:20.933798+0000","last_peered":"2024-04-27T03:33:20.933798+0000","last_clean":"2024-04-27T03:33:20.933798+0000","last_became_active":"2024-04-27T03:30:13.384580+0000","last_became_peered":"2024-04-27T03:30:13.384580+0000","last_unstale":"2024-04-27T03:33:20.933798+0000","last_undegraded":"2024-04-27T03:33:20.933798+0000","last_fullsized":"2024-04-27T03:33:20.933798+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:07:51.011104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.921795+0000","last_change":"2024-04-27T03:30:11.361366+0000","last_active":"2024-04-27T03:33:20.921795+0000","last_peered":"2024-04-27T03:33:20.921795+0000","last_clean":"2024-04-27T03:33:20.921795+0000","last_became_active":"2024-04-27T03:30:11.360857+0000","last_became_peered":"2024-04-27T03:30:11.360857+0000","last_unstale":"2024-04-27T03:33:20.921795+0000","last_undegraded":"2024-04-27T03:33:20.921795+0000","last_fullsized":"2024-04-27T03:33:20.921795+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:11.138117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"27'12","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.695008+0000","last_change":"2024-04-27T03:30:13.376545+0000","last_active":"2024-04-27T03:33:20.695008+0000","last_peered":"2024-04-27T03:33:20.695008+0000","last_clean":"2024-04-27T03:33:20.695008+0000","last_became_active":"2024-04-27T03:30:13.376221+0000","last_became_peered":"2024-04-27T03:30:13.376221+0000","last_unstale":"2024-04-27T03:33:20.695008+0000","last_undegraded":"2024-04-27T03:33:20.695008+0000","last_fullsized":"2024-04-27T03:33:20.695008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:08.985512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440740+0000","last_change":"2024-04-27T03:30:11.363460+0000","last_active":"2024-04-27T03:31:01.440740+0000","last_peered":"2024-04-27T03:31:01.440740+0000","last_clean":"2024-04-27T03:31:01.440740+0000","last_became_active":"2024-04-27T03:30:11.362739+0000","last_became_peered":"2024-04-27T03:30:11.362739+0000","last_unstale":"2024-04-27T03:31:01.440740+0000","last_undegraded":"2024-04-27T03:31:01.440740+0000","last_fullsized":"2024-04-27T03:31:01.440740+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:12.842711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"27'7","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298536+0000","last_change":"2024-04-27T03:30:13.376819+0000","last_active":"2024-04-27T03:32:44.298536+0000","last_peered":"2024-04-27T03:32:44.298536+0000","last_clean":"2024-04-27T03:32:44.298536+0000","last_became_active":"2024-04-27T03:30:13.376524+0000","last_became_peered":"2024-04-27T03:30:13.376524+0000","last_unstale":"2024-04-27T03:32:44.298536+0000","last_undegraded":"2024-04-27T03:32:44.298536+0000","last_fullsized":"2024-04-27T03:32:44.298536+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:54:30.451033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.293096+0000","last_change":"2024-04-27T03:30:11.361039+0000","last_active":"2024-04-27T03:32:14.293096+0000","last_peered":"2024-04-27T03:32:14.293096+0000","last_clean":"2024-04-27T03:32:14.293096+0000","last_became_active":"2024-04-27T03:30:11.360188+0000","last_became_peered":"2024-04-27T03:30:11.360188+0000","last_unstale":"2024-04-27T03:32:14.293096+0000","last_undegraded":"2024-04-27T03:32:14.293096+0000","last_fullsized":"2024-04-27T03:32:14.293096+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:22.719424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"27'16","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009953+0000","last_change":"2024-04-27T03:30:13.376574+0000","last_active":"2024-04-27T03:33:21.009953+0000","last_peered":"2024-04-27T03:33:21.009953+0000","last_clean":"2024-04-27T03:33:21.009953+0000","last_became_active":"2024-04-27T03:30:13.376288+0000","last_became_peered":"2024-04-27T03:30:13.376288+0000","last_unstale":"2024-04-27T03:33:21.009953+0000","last_undegraded":"2024-04-27T03:33:21.009953+0000","last_fullsized":"2024-04-27T03:33:21.009953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:15.187356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439707+0000","last_change":"2024-04-27T03:30:11.355274+0000","last_active":"2024-04-27T03:31:01.439707+0000","last_peered":"2024-04-27T03:31:01.439707+0000","last_clean":"2024-04-27T03:31:01.439707+0000","last_became_active":"2024-04-27T03:30:11.354989+0000","last_became_peered":"2024-04-27T03:30:11.354989+0000","last_unstale":"2024-04-27T03:31:01.439707+0000","last_undegraded":"2024-04-27T03:31:01.439707+0000","last_fullsized":"2024-04-27T03:31:01.439707+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:11.281867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933479+0000","last_change":"2024-04-27T03:30:13.374182+0000","last_active":"2024-04-27T03:33:20.933479+0000","last_peered":"2024-04-27T03:33:20.933479+0000","last_clean":"2024-04-27T03:33:20.933479+0000","last_became_active":"2024-04-27T03:30:13.373424+0000","last_became_peered":"2024-04-27T03:30:13.373424+0000","last_unstale":"2024-04-27T03:33:20.933479+0000","last_undegraded":"2024-04-27T03:33:20.933479+0000","last_fullsized":"2024-04-27T03:33:20.933479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:02.825441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.657187+0000","last_change":"2024-04-27T03:30:11.361019+0000","last_active":"2024-04-27T03:33:20.657187+0000","last_peered":"2024-04-27T03:33:20.657187+0000","last_clean":"2024-04-27T03:33:20.657187+0000","last_became_active":"2024-04-27T03:30:11.360219+0000","last_became_peered":"2024-04-27T03:30:11.360219+0000","last_unstale":"2024-04-27T03:33:20.657187+0000","last_undegraded":"2024-04-27T03:33:20.657187+0000","last_fullsized":"2024-04-27T03:33:20.657187+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:45.376863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"25'8","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440191+0000","last_change":"2024-04-27T03:30:13.385912+0000","last_active":"2024-04-27T03:31:01.440191+0000","last_peered":"2024-04-27T03:31:01.440191+0000","last_clean":"2024-04-27T03:31:01.440191+0000","last_became_active":"2024-04-27T03:30:13.385665+0000","last_became_peered":"2024-04-27T03:30:13.385665+0000","last_unstale":"2024-04-27T03:31:01.440191+0000","last_undegraded":"2024-04-27T03:31:01.440191+0000","last_fullsized":"2024-04-27T03:31:01.440191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:49.082066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437953+0000","last_change":"2024-04-27T03:30:11.373049+0000","last_active":"2024-04-27T03:31:01.437953+0000","last_peered":"2024-04-27T03:31:01.437953+0000","last_clean":"2024-04-27T03:31:01.437953+0000","last_became_active":"2024-04-27T03:30:11.372704+0000","last_became_peered":"2024-04-27T03:30:11.372704+0000","last_unstale":"2024-04-27T03:31:01.437953+0000","last_undegraded":"2024-04-27T03:31:01.437953+0000","last_fullsized":"2024-04-27T03:31:01.437953+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:03.639873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"27'4","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981936+0000","last_change":"2024-04-27T03:30:13.374520+0000","last_active":"2024-04-27T03:33:20.981936+0000","last_peered":"2024-04-27T03:33:20.981936+0000","last_clean":"2024-04-27T03:33:20.981936+0000","last_became_active":"2024-04-27T03:30:13.374199+0000","last_became_peered":"2024-04-27T03:30:13.374199+0000","last_unstale":"2024-04-27T03:33:20.981936+0000","last_undegraded":"2024-04-27T03:33:20.981936+0000","last_fullsized":"2024-04-27T03:33:20.981936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:33.811635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.926966+0000","last_change":"2024-04-27T03:30:11.358470+0000","last_active":"2024-04-27T03:33:20.926966+0000","last_peered":"2024-04-27T03:33:20.926966+0000","last_clean":"2024-04-27T03:33:20.926966+0000","last_became_active":"2024-04-27T03:30:11.358181+0000","last_became_peered":"2024-04-27T03:30:11.358181+0000","last_unstale":"2024-04-27T03:33:20.926966+0000","last_undegraded":"2024-04-27T03:33:20.926966+0000","last_fullsized":"2024-04-27T03:33:20.926966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.921690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.794975+0000","last_change":"2024-04-27T03:30:13.379341+0000","last_active":"2024-04-27T03:33:20.794975+0000","last_peered":"2024-04-27T03:33:20.794975+0000","last_clean":"2024-04-27T03:33:20.794975+0000","last_became_active":"2024-04-27T03:30:13.379059+0000","last_became_peered":"2024-04-27T03:30:13.379059+0000","last_unstale":"2024-04-27T03:33:20.794975+0000","last_undegraded":"2024-04-27T03:33:20.794975+0000","last_fullsized":"2024-04-27T03:33:20.794975+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:56.906775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"27'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.010139+0000","last_change":"2024-04-27T03:30:13.383971+0000","last_active":"2024-04-27T03:33:21.010139+0000","last_peered":"2024-04-27T03:33:21.010139+0000","last_clean":"2024-04-27T03:33:21.010139+0000","last_became_active":"2024-04-27T03:30:13.383694+0000","last_became_peered":"2024-04-27T03:30:13.383694+0000","last_unstale":"2024-04-27T03:33:21.010139+0000","last_undegraded":"2024-04-27T03:33:21.010139+0000","last_fullsized":"2024-04-27T03:33:21.010139+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:36:25.083284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436571+0000","last_change":"2024-04-27T03:30:11.370455+0000","last_active":"2024-04-27T03:31:01.436571+0000","last_peered":"2024-04-27T03:31:01.436571+0000","last_clean":"2024-04-27T03:31:01.436571+0000","last_became_active":"2024-04-27T03:30:11.370157+0000","last_became_peered":"2024-04-27T03:30:11.370157+0000","last_unstale":"2024-04-27T03:31:01.436571+0000","last_undegraded":"2024-04-27T03:31:01.436571+0000","last_fullsized":"2024-04-27T03:31:01.436571+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:06.887756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"27'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009774+0000","last_change":"2024-04-27T03:30:13.373269+0000","last_active":"2024-04-27T03:33:21.009774+0000","last_peered":"2024-04-27T03:33:21.009774+0000","last_clean":"2024-04-27T03:33:21.009774+0000","last_became_active":"2024-04-27T03:30:13.372973+0000","last_became_peered":"2024-04-27T03:30:13.372973+0000","last_unstale":"2024-04-27T03:33:21.009774+0000","last_undegraded":"2024-04-27T03:33:21.009774+0000","last_fullsized":"2024-04-27T03:33:21.009774+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:57.593400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297374+0000","last_change":"2024-04-27T03:30:11.363764+0000","last_active":"2024-04-27T03:32:39.297374+0000","last_peered":"2024-04-27T03:32:39.297374+0000","last_clean":"2024-04-27T03:32:39.297374+0000","last_became_active":"2024-04-27T03:30:11.363109+0000","last_became_peered":"2024-04-27T03:30:11.363109+0000","last_unstale":"2024-04-27T03:32:39.297374+0000","last_undegraded":"2024-04-27T03:32:39.297374+0000","last_fullsized":"2024-04-27T03:32:39.297374+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:52:31.047433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"27'9","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988406+0000","last_change":"2024-04-27T03:30:13.384110+0000","last_active":"2024-04-27T03:33:20.988406+0000","last_peered":"2024-04-27T03:33:20.988406+0000","last_clean":"2024-04-27T03:33:20.988406+0000","last_became_active":"2024-04-27T03:30:13.383742+0000","last_became_peered":"2024-04-27T03:30:13.383742+0000","last_unstale":"2024-04-27T03:33:20.988406+0000","last_undegraded":"2024-04-27T03:33:20.988406+0000","last_fullsized":"2024-04-27T03:33:20.988406+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:14.799339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"25'4","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436806+0000","last_change":"2024-04-27T03:30:11.370635+0000","last_active":"2024-04-27T03:31:01.436806+0000","last_peered":"2024-04-27T03:31:01.436806+0000","last_clean":"2024-04-27T03:31:01.436806+0000","last_became_active":"2024-04-27T03:30:11.370380+0000","last_became_peered":"2024-04-27T03:30:11.370380+0000","last_unstale":"2024-04-27T03:31:01.436806+0000","last_undegraded":"2024-04-27T03:31:01.436806+0000","last_fullsized":"2024-04-27T03:31:01.436806+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:45.376863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"27'6","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009766+0000","last_change":"2024-04-27T03:31:01.446266+0000","last_active":"2024-04-27T03:33:21.009766+0000","last_peered":"2024-04-27T03:33:21.009766+0000","last_clean":"2024-04-27T03:33:21.009766+0000","last_became_active":"2024-04-27T03:31:01.440762+0000","last_became_peered":"2024-04-27T03:31:01.440762+0000","last_unstale":"2024-04-27T03:33:21.009766+0000","last_undegraded":"2024-04-27T03:33:21.009766+0000","last_fullsized":"2024-04-27T03:33:21.009766+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:58.987011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.279469+0000","last_change":"2024-04-27T03:30:11.369834+0000","last_active":"2024-04-27T03:32:20.279469+0000","last_peered":"2024-04-27T03:32:20.279469+0000","last_clean":"2024-04-27T03:32:20.279469+0000","last_became_active":"2024-04-27T03:30:11.369551+0000","last_became_peered":"2024-04-27T03:30:11.369551+0000","last_unstale":"2024-04-27T03:32:20.279469+0000","last_undegraded":"2024-04-27T03:32:20.279469+0000","last_fullsized":"2024-04-27T03:32:20.279469+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.921690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"27'2","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981818+0000","last_change":"2024-04-27T03:30:13.382150+0000","last_active":"2024-04-27T03:33:20.981818+0000","last_peered":"2024-04-27T03:33:20.981818+0000","last_clean":"2024-04-27T03:33:20.981818+0000","last_became_active":"2024-04-27T03:30:13.381901+0000","last_became_peered":"2024-04-27T03:30:13.381901+0000","last_unstale":"2024-04-27T03:33:20.981818+0000","last_undegraded":"2024-04-27T03:33:20.981818+0000","last_fullsized":"2024-04-27T03:33:20.981818+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:05.646245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"25'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431969+0000","last_change":"2024-04-27T03:30:11.361129+0000","last_active":"2024-04-27T03:31:01.431969+0000","last_peered":"2024-04-27T03:31:01.431969+0000","last_clean":"2024-04-27T03:31:01.431969+0000","last_became_active":"2024-04-27T03:30:11.360515+0000","last_became_peered":"2024-04-27T03:30:11.360515+0000","last_unstale":"2024-04-27T03:31:01.431969+0000","last_undegraded":"2024-04-27T03:31:01.431969+0000","last_fullsized":"2024-04-27T03:31:01.431969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:52:31.047433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"27'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.307097+0000","last_change":"2024-04-27T03:30:13.382249+0000","last_active":"2024-04-27T03:32:20.307097+0000","last_peered":"2024-04-27T03:32:20.307097+0000","last_clean":"2024-04-27T03:32:20.307097+0000","last_became_active":"2024-04-27T03:30:13.378932+0000","last_became_peered":"2024-04-27T03:30:13.378932+0000","last_unstale":"2024-04-27T03:32:20.307097+0000","last_undegraded":"2024-04-27T03:32:20.307097+0000","last_fullsized":"2024-04-27T03:32:20.307097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:35.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437123+0000","last_change":"2024-04-27T03:30:11.372964+0000","last_active":"2024-04-27T03:31:01.437123+0000","last_peered":"2024-04-27T03:31:01.437123+0000","last_clean":"2024-04-27T03:31:01.437123+0000","last_became_active":"2024-04-27T03:30:11.372673+0000","last_became_peered":"2024-04-27T03:30:11.372673+0000","last_unstale":"2024-04-27T03:31:01.437123+0000","last_undegraded":"2024-04-27T03:31:01.437123+0000","last_fullsized":"2024-04-27T03:31:01.437123+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:08:10.593535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"27'35","reported_seq":99,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.957246+0000","last_change":"2024-04-27T03:30:21.413597+0000","last_active":"2024-04-27T03:33:20.957246+0000","last_peered":"2024-04-27T03:33:20.957246+0000","last_clean":"2024-04-27T03:33:20.957246+0000","last_became_active":"2024-04-27T03:30:21.413255+0000","last_became_peered":"2024-04-27T03:30:21.413255+0000","last_unstale":"2024-04-27T03:33:20.957246+0000","last_undegraded":"2024-04-27T03:33:20.957246+0000","last_fullsized":"2024-04-27T03:33:20.957246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:11:19.581338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"27'3","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296833+0000","last_change":"2024-04-27T03:30:13.377382+0000","last_active":"2024-04-27T03:32:34.296833+0000","last_peered":"2024-04-27T03:32:34.296833+0000","last_clean":"2024-04-27T03:32:34.296833+0000","last_became_active":"2024-04-27T03:30:13.377110+0000","last_became_peered":"2024-04-27T03:30:13.377110+0000","last_unstale":"2024-04-27T03:32:34.296833+0000","last_undegraded":"2024-04-27T03:32:34.296833+0000","last_fullsized":"2024-04-27T03:32:34.296833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:21:27.452807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439438+0000","last_change":"2024-04-27T03:30:11.360152+0000","last_active":"2024-04-27T03:31:01.439438+0000","last_peered":"2024-04-27T03:31:01.439438+0000","last_clean":"2024-04-27T03:31:01.439438+0000","last_became_active":"2024-04-27T03:30:11.359892+0000","last_became_peered":"2024-04-27T03:30:11.359892+0000","last_unstale":"2024-04-27T03:31:01.439438+0000","last_undegraded":"2024-04-27T03:31:01.439438+0000","last_fullsized":"2024-04-27T03:31:01.439438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:07:49.004733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440221+0000","last_change":"2024-04-27T03:30:15.381168+0000","last_active":"2024-04-27T03:31:01.440221+0000","last_peered":"2024-04-27T03:31:01.440221+0000","last_clean":"2024-04-27T03:31:01.440221+0000","last_became_active":"2024-04-27T03:30:15.380864+0000","last_became_peered":"2024-04-27T03:30:15.380864+0000","last_unstale":"2024-04-27T03:31:01.440221+0000","last_undegraded":"2024-04-27T03:31:01.440221+0000","last_fullsized":"2024-04-27T03:31:01.440221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:57:52.741876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"27'42","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.914579+0000","last_change":"2024-04-27T03:30:17.388325+0000","last_active":"2024-04-27T03:32:18.914579+0000","last_peered":"2024-04-27T03:32:18.914579+0000","last_clean":"2024-04-27T03:32:18.914579+0000","last_became_active":"2024-04-27T03:30:17.387391+0000","last_became_peered":"2024-04-27T03:30:17.387391+0000","last_unstale":"2024-04-27T03:32:18.914579+0000","last_undegraded":"2024-04-27T03:32:18.914579+0000","last_fullsized":"2024-04-27T03:32:18.914579+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:02:27.282043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":24,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433271+0000","last_change":"2024-04-27T03:30:19.407954+0000","last_active":"2024-04-27T03:31:01.433271+0000","last_peered":"2024-04-27T03:31:01.433271+0000","last_clean":"2024-04-27T03:31:01.433271+0000","last_became_active":"2024-04-27T03:30:19.407637+0000","last_became_peered":"2024-04-27T03:30:19.407637+0000","last_unstale":"2024-04-27T03:31:01.433271+0000","last_undegraded":"2024-04-27T03:31:01.433271+0000","last_fullsized":"2024-04-27T03:31:01.433271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:12.277984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"27'9","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.436568+0000","last_change":"2024-04-27T03:30:21.414223+0000","last_active":"2024-04-27T03:31:36.436568+0000","last_peered":"2024-04-27T03:31:36.436568+0000","last_clean":"2024-04-27T03:31:36.436568+0000","last_became_active":"2024-04-27T03:30:21.413940+0000","last_became_peered":"2024-04-27T03:30:21.413940+0000","last_unstale":"2024-04-27T03:31:36.436568+0000","last_undegraded":"2024-04-27T03:31:36.436568+0000","last_fullsized":"2024-04-27T03:31:36.436568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:44.051317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"27'16","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981635+0000","last_change":"2024-04-27T03:30:13.381748+0000","last_active":"2024-04-27T03:33:20.981635+0000","last_peered":"2024-04-27T03:33:20.981635+0000","last_clean":"2024-04-27T03:33:20.981635+0000","last_became_active":"2024-04-27T03:30:13.381487+0000","last_became_peered":"2024-04-27T03:30:13.381487+0000","last_unstale":"2024-04-27T03:33:20.981635+0000","last_undegraded":"2024-04-27T03:33:20.981635+0000","last_fullsized":"2024-04-27T03:33:20.981635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:52.946275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435664+0000","last_change":"2024-04-27T03:30:11.359690+0000","last_active":"2024-04-27T03:31:01.435664+0000","last_peered":"2024-04-27T03:31:01.435664+0000","last_clean":"2024-04-27T03:31:01.435664+0000","last_became_active":"2024-04-27T03:30:11.359446+0000","last_became_peered":"2024-04-27T03:30:11.359446+0000","last_unstale":"2024-04-27T03:31:01.435664+0000","last_undegraded":"2024-04-27T03:31:01.435664+0000","last_fullsized":"2024-04-27T03:31:01.435664+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.921690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431730+0000","last_change":"2024-04-27T03:30:15.380566+0000","last_active":"2024-04-27T03:31:01.431730+0000","last_peered":"2024-04-27T03:31:01.431730+0000","last_clean":"2024-04-27T03:31:01.431730+0000","last_became_active":"2024-04-27T03:30:15.380313+0000","last_became_peered":"2024-04-27T03:30:15.380313+0000","last_unstale":"2024-04-27T03:31:01.431730+0000","last_undegraded":"2024-04-27T03:31:01.431730+0000","last_fullsized":"2024-04-27T03:31:01.431730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:45:16.754705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"25'12","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433370+0000","last_change":"2024-04-27T03:30:17.389399+0000","last_active":"2024-04-27T03:31:01.433370+0000","last_peered":"2024-04-27T03:31:01.433370+0000","last_clean":"2024-04-27T03:31:01.433370+0000","last_became_active":"2024-04-27T03:30:17.389126+0000","last_became_peered":"2024-04-27T03:30:17.389126+0000","last_unstale":"2024-04-27T03:31:01.433370+0000","last_undegraded":"2024-04-27T03:31:01.433370+0000","last_fullsized":"2024-04-27T03:31:01.433370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:42:03.150440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432915+0000","last_change":"2024-04-27T03:30:19.405456+0000","last_active":"2024-04-27T03:31:01.432915+0000","last_peered":"2024-04-27T03:31:01.432915+0000","last_clean":"2024-04-27T03:31:01.432915+0000","last_became_active":"2024-04-27T03:30:19.405170+0000","last_became_peered":"2024-04-27T03:30:19.405170+0000","last_unstale":"2024-04-27T03:31:01.432915+0000","last_undegraded":"2024-04-27T03:31:01.432915+0000","last_fullsized":"2024-04-27T03:31:01.432915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:31:41.885815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'6","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.427217+0000","last_change":"2024-04-27T03:30:21.408308+0000","last_active":"2024-04-27T03:31:36.427217+0000","last_peered":"2024-04-27T03:31:36.427217+0000","last_clean":"2024-04-27T03:31:36.427217+0000","last_became_active":"2024-04-27T03:30:21.407917+0000","last_became_peered":"2024-04-27T03:30:21.407917+0000","last_unstale":"2024-04-27T03:31:36.427217+0000","last_undegraded":"2024-04-27T03:31:36.427217+0000","last_fullsized":"2024-04-27T03:31:36.427217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:23.492592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":95,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.792066+0000","last_change":"2024-04-27T03:30:13.378024+0000","last_active":"2024-04-27T03:33:20.792066+0000","last_peered":"2024-04-27T03:33:20.792066+0000","last_clean":"2024-04-27T03:33:20.792066+0000","last_became_active":"2024-04-27T03:30:13.377423+0000","last_became_peered":"2024-04-27T03:30:13.377423+0000","last_unstale":"2024-04-27T03:33:20.792066+0000","last_undegraded":"2024-04-27T03:33:20.792066+0000","last_fullsized":"2024-04-27T03:33:20.792066+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:22.057065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437475+0000","last_change":"2024-04-27T03:30:11.359542+0000","last_active":"2024-04-27T03:31:01.437475+0000","last_peered":"2024-04-27T03:31:01.437475+0000","last_clean":"2024-04-27T03:31:01.437475+0000","last_became_active":"2024-04-27T03:30:11.359261+0000","last_became_peered":"2024-04-27T03:30:11.359261+0000","last_unstale":"2024-04-27T03:31:01.437475+0000","last_undegraded":"2024-04-27T03:31:01.437475+0000","last_fullsized":"2024-04-27T03:31:01.437475+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:19.905319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433488+0000","last_change":"2024-04-27T03:30:15.383902+0000","last_active":"2024-04-27T03:31:01.433488+0000","last_peered":"2024-04-27T03:31:01.433488+0000","last_clean":"2024-04-27T03:31:01.433488+0000","last_became_active":"2024-04-27T03:30:15.383618+0000","last_became_peered":"2024-04-27T03:30:15.383618+0000","last_unstale":"2024-04-27T03:31:01.433488+0000","last_undegraded":"2024-04-27T03:31:01.433488+0000","last_fullsized":"2024-04-27T03:31:01.433488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:54.227106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"25'23","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433761+0000","last_change":"2024-04-27T03:30:17.392488+0000","last_active":"2024-04-27T03:31:01.433761+0000","last_peered":"2024-04-27T03:31:01.433761+0000","last_clean":"2024-04-27T03:31:01.433761+0000","last_became_active":"2024-04-27T03:30:17.392223+0000","last_became_peered":"2024-04-27T03:30:17.392223+0000","last_unstale":"2024-04-27T03:31:01.433761+0000","last_undegraded":"2024-04-27T03:31:01.433761+0000","last_fullsized":"2024-04-27T03:31:01.433761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:56.237623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"27'282","reported_seq":381,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.463012+0000","last_change":"2024-04-27T03:30:19.406264+0000","last_active":"2024-04-27T03:33:20.463012+0000","last_peered":"2024-04-27T03:33:20.463012+0000","last_clean":"2024-04-27T03:33:20.463012+0000","last_became_active":"2024-04-27T03:30:19.405730+0000","last_became_peered":"2024-04-27T03:30:19.405730+0000","last_unstale":"2024-04-27T03:33:20.463012+0000","last_undegraded":"2024-04-27T03:33:20.463012+0000","last_fullsized":"2024-04-27T03:33:20.463012+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:24.551790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436546+0000","last_change":"2024-04-27T03:30:21.409739+0000","last_active":"2024-04-27T03:31:01.436546+0000","last_peered":"2024-04-27T03:31:01.436546+0000","last_clean":"2024-04-27T03:31:01.436546+0000","last_became_active":"2024-04-27T03:30:21.409156+0000","last_became_peered":"2024-04-27T03:30:21.409156+0000","last_unstale":"2024-04-27T03:31:01.436546+0000","last_undegraded":"2024-04-27T03:31:01.436546+0000","last_fullsized":"2024-04-27T03:31:01.436546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:56:51.564750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.802359+0000","last_change":"2024-04-27T03:30:13.385803+0000","last_active":"2024-04-27T03:33:20.802359+0000","last_peered":"2024-04-27T03:33:20.802359+0000","last_clean":"2024-04-27T03:33:20.802359+0000","last_became_active":"2024-04-27T03:30:13.385491+0000","last_became_peered":"2024-04-27T03:30:13.385491+0000","last_unstale":"2024-04-27T03:33:20.802359+0000","last_undegraded":"2024-04-27T03:33:20.802359+0000","last_fullsized":"2024-04-27T03:33:20.802359+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:14.849083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431754+0000","last_change":"2024-04-27T03:30:11.370401+0000","last_active":"2024-04-27T03:31:01.431754+0000","last_peered":"2024-04-27T03:31:01.431754+0000","last_clean":"2024-04-27T03:31:01.431754+0000","last_became_active":"2024-04-27T03:30:11.369939+0000","last_became_peered":"2024-04-27T03:30:11.369939+0000","last_unstale":"2024-04-27T03:31:01.431754+0000","last_undegraded":"2024-04-27T03:31:01.431754+0000","last_fullsized":"2024-04-27T03:31:01.431754+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:00.825717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439146+0000","last_change":"2024-04-27T03:30:15.384389+0000","last_active":"2024-04-27T03:31:01.439146+0000","last_peered":"2024-04-27T03:31:01.439146+0000","last_clean":"2024-04-27T03:31:01.439146+0000","last_became_active":"2024-04-27T03:30:15.384114+0000","last_became_peered":"2024-04-27T03:30:15.384114+0000","last_unstale":"2024-04-27T03:31:01.439146+0000","last_undegraded":"2024-04-27T03:31:01.439146+0000","last_fullsized":"2024-04-27T03:31:01.439146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:17.509463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"27'36","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.908651+0000","last_change":"2024-04-27T03:30:17.392377+0000","last_active":"2024-04-27T03:32:18.908651+0000","last_peered":"2024-04-27T03:32:18.908651+0000","last_clean":"2024-04-27T03:32:18.908651+0000","last_became_active":"2024-04-27T03:30:17.391880+0000","last_became_peered":"2024-04-27T03:30:17.391880+0000","last_unstale":"2024-04-27T03:32:18.908651+0000","last_undegraded":"2024-04-27T03:32:18.908651+0000","last_fullsized":"2024-04-27T03:32:18.908651+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:13.007456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"27'282","reported_seq":405,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.137865+0000","last_change":"2024-04-27T03:30:19.405073+0000","last_active":"2024-04-27T03:33:21.137865+0000","last_peered":"2024-04-27T03:33:21.137865+0000","last_clean":"2024-04-27T03:33:21.137865+0000","last_became_active":"2024-04-27T03:30:19.404744+0000","last_became_peered":"2024-04-27T03:30:19.404744+0000","last_unstale":"2024-04-27T03:33:21.137865+0000","last_undegraded":"2024-04-27T03:33:21.137865+0000","last_fullsized":"2024-04-27T03:33:21.137865+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:31:41.885815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'7","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.706493+0000","last_change":"2024-04-27T03:30:13.387974+0000","last_active":"2024-04-27T03:33:20.706493+0000","last_peered":"2024-04-27T03:33:20.706493+0000","last_clean":"2024-04-27T03:33:20.706493+0000","last_became_active":"2024-04-27T03:30:13.387335+0000","last_became_peered":"2024-04-27T03:30:13.387335+0000","last_unstale":"2024-04-27T03:33:20.706493+0000","last_undegraded":"2024-04-27T03:33:20.706493+0000","last_fullsized":"2024-04-27T03:33:20.706493+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:32:09.116022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433282+0000","last_change":"2024-04-27T03:30:11.372061+0000","last_active":"2024-04-27T03:31:01.433282+0000","last_peered":"2024-04-27T03:31:01.433282+0000","last_clean":"2024-04-27T03:31:01.433282+0000","last_became_active":"2024-04-27T03:30:11.371600+0000","last_became_peered":"2024-04-27T03:30:11.371600+0000","last_unstale":"2024-04-27T03:31:01.433282+0000","last_undegraded":"2024-04-27T03:31:01.433282+0000","last_fullsized":"2024-04-27T03:31:01.433282+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.059046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433141+0000","last_change":"2024-04-27T03:30:15.381767+0000","last_active":"2024-04-27T03:31:01.433141+0000","last_peered":"2024-04-27T03:31:01.433141+0000","last_clean":"2024-04-27T03:31:01.433141+0000","last_became_active":"2024-04-27T03:30:15.381359+0000","last_became_peered":"2024-04-27T03:30:15.381359+0000","last_unstale":"2024-04-27T03:31:01.433141+0000","last_undegraded":"2024-04-27T03:31:01.433141+0000","last_fullsized":"2024-04-27T03:31:01.433141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:19:12.341976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"27'46","reported_seq":99,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.918834+0000","last_change":"2024-04-27T03:30:17.390288+0000","last_active":"2024-04-27T03:32:18.918834+0000","last_peered":"2024-04-27T03:32:18.918834+0000","last_clean":"2024-04-27T03:32:18.918834+0000","last_became_active":"2024-04-27T03:30:17.390021+0000","last_became_peered":"2024-04-27T03:30:17.390021+0000","last_unstale":"2024-04-27T03:32:18.918834+0000","last_undegraded":"2024-04-27T03:32:18.918834+0000","last_fullsized":"2024-04-27T03:32:18.918834+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:58.407158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":53,"num_read_kb":41,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431506+0000","last_change":"2024-04-27T03:30:19.405394+0000","last_active":"2024-04-27T03:31:01.431506+0000","last_peered":"2024-04-27T03:31:01.431506+0000","last_clean":"2024-04-27T03:31:01.431506+0000","last_became_active":"2024-04-27T03:30:19.404948+0000","last_became_peered":"2024-04-27T03:30:19.404948+0000","last_unstale":"2024-04-27T03:31:01.431506+0000","last_undegraded":"2024-04-27T03:31:01.431506+0000","last_fullsized":"2024-04-27T03:31:01.431506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:59.540815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"27'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.438491+0000","last_change":"2024-04-27T03:30:21.412598+0000","last_active":"2024-04-27T03:31:36.438491+0000","last_peered":"2024-04-27T03:31:36.438491+0000","last_clean":"2024-04-27T03:31:36.438491+0000","last_became_active":"2024-04-27T03:30:21.412331+0000","last_became_peered":"2024-04-27T03:30:21.412331+0000","last_unstale":"2024-04-27T03:31:36.438491+0000","last_undegraded":"2024-04-27T03:31:36.438491+0000","last_fullsized":"2024-04-27T03:31:36.438491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:21.686877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"25'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432638+0000","last_change":"2024-04-27T03:30:13.373532+0000","last_active":"2024-04-27T03:31:01.432638+0000","last_peered":"2024-04-27T03:31:01.432638+0000","last_clean":"2024-04-27T03:31:01.432638+0000","last_became_active":"2024-04-27T03:30:13.373226+0000","last_became_peered":"2024-04-27T03:30:13.373226+0000","last_unstale":"2024-04-27T03:31:01.432638+0000","last_undegraded":"2024-04-27T03:31:01.432638+0000","last_fullsized":"2024-04-27T03:31:01.432638+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:38.976155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437884+0000","last_change":"2024-04-27T03:30:11.373710+0000","last_active":"2024-04-27T03:31:01.437884+0000","last_peered":"2024-04-27T03:31:01.437884+0000","last_clean":"2024-04-27T03:31:01.437884+0000","last_became_active":"2024-04-27T03:30:11.373430+0000","last_became_peered":"2024-04-27T03:30:11.373430+0000","last_unstale":"2024-04-27T03:31:01.437884+0000","last_undegraded":"2024-04-27T03:31:01.437884+0000","last_fullsized":"2024-04-27T03:31:01.437884+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:52:58.316121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"18'1","reported_seq":78,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.994968+0000","last_change":"2024-04-27T03:31:01.447513+0000","last_active":"2024-04-27T03:32:19.994968+0000","last_peered":"2024-04-27T03:32:19.994968+0000","last_clean":"2024-04-27T03:32:19.994968+0000","last_became_active":"2024-04-27T03:31:01.443259+0000","last_became_peered":"2024-04-27T03:31:01.443259+0000","last_unstale":"2024-04-27T03:32:19.994968+0000","last_undegraded":"2024-04-27T03:32:19.994968+0000","last_fullsized":"2024-04-27T03:32:19.994968+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:54.147127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":282,"num_read_kb":282,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"27'39","reported_seq":87,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296781+0000","last_change":"2024-04-27T03:30:17.389610+0000","last_active":"2024-04-27T03:32:34.296781+0000","last_peered":"2024-04-27T03:32:34.296781+0000","last_clean":"2024-04-27T03:32:34.296781+0000","last_became_active":"2024-04-27T03:30:17.388994+0000","last_became_peered":"2024-04-27T03:30:17.388994+0000","last_unstale":"2024-04-27T03:32:34.296781+0000","last_undegraded":"2024-04-27T03:32:34.296781+0000","last_fullsized":"2024-04-27T03:32:34.296781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:01:03.772535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":38,"num_read_kb":31,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431125+0000","last_change":"2024-04-27T03:30:19.405695+0000","last_active":"2024-04-27T03:31:01.431125+0000","last_peered":"2024-04-27T03:31:01.431125+0000","last_clean":"2024-04-27T03:31:01.431125+0000","last_became_active":"2024-04-27T03:30:19.405334+0000","last_became_peered":"2024-04-27T03:30:19.405334+0000","last_unstale":"2024-04-27T03:31:01.431125+0000","last_undegraded":"2024-04-27T03:31:01.431125+0000","last_fullsized":"2024-04-27T03:31:01.431125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:44:45.246417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"27'3","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:30.912296+0000","last_change":"2024-04-27T03:30:21.410221+0000","last_active":"2024-04-27T03:31:30.912296+0000","last_peered":"2024-04-27T03:31:30.912296+0000","last_clean":"2024-04-27T03:31:30.912296+0000","last_became_active":"2024-04-27T03:30:21.409922+0000","last_became_peered":"2024-04-27T03:30:21.409922+0000","last_unstale":"2024-04-27T03:31:30.912296+0000","last_undegraded":"2024-04-27T03:31:30.912296+0000","last_fullsized":"2024-04-27T03:31:30.912296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.646754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"27'6","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.010033+0000","last_change":"2024-04-27T03:30:13.381521+0000","last_active":"2024-04-27T03:33:21.010033+0000","last_peered":"2024-04-27T03:33:21.010033+0000","last_clean":"2024-04-27T03:33:21.010033+0000","last_became_active":"2024-04-27T03:30:13.381270+0000","last_became_peered":"2024-04-27T03:30:13.381270+0000","last_unstale":"2024-04-27T03:33:21.010033+0000","last_undegraded":"2024-04-27T03:33:21.010033+0000","last_fullsized":"2024-04-27T03:33:21.010033+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:58:48.554677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440331+0000","last_change":"2024-04-27T03:30:11.372966+0000","last_active":"2024-04-27T03:31:01.440331+0000","last_peered":"2024-04-27T03:31:01.440331+0000","last_clean":"2024-04-27T03:31:01.440331+0000","last_became_active":"2024-04-27T03:30:11.372583+0000","last_became_peered":"2024-04-27T03:30:11.372583+0000","last_unstale":"2024-04-27T03:31:01.440331+0000","last_undegraded":"2024-04-27T03:31:01.440331+0000","last_fullsized":"2024-04-27T03:31:01.440331+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:54.900404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433260+0000","last_change":"2024-04-27T03:30:15.382571+0000","last_active":"2024-04-27T03:31:01.433260+0000","last_peered":"2024-04-27T03:31:01.433260+0000","last_clean":"2024-04-27T03:31:01.433260+0000","last_became_active":"2024-04-27T03:30:15.382207+0000","last_became_peered":"2024-04-27T03:30:15.382207+0000","last_unstale":"2024-04-27T03:31:01.433260+0000","last_undegraded":"2024-04-27T03:31:01.433260+0000","last_fullsized":"2024-04-27T03:31:01.433260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:56.396642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"27'49","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.920671+0000","last_change":"2024-04-27T03:30:17.390418+0000","last_active":"2024-04-27T03:32:18.920671+0000","last_peered":"2024-04-27T03:32:18.920671+0000","last_clean":"2024-04-27T03:32:18.920671+0000","last_became_active":"2024-04-27T03:30:17.389242+0000","last_became_peered":"2024-04-27T03:30:17.389242+0000","last_unstale":"2024-04-27T03:32:18.920671+0000","last_undegraded":"2024-04-27T03:32:18.920671+0000","last_fullsized":"2024-04-27T03:32:18.920671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:59.857055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":56,"num_read_kb":45,"num_write":30,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"27'282","reported_seq":348,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.148028+0000","last_change":"2024-04-27T03:30:19.406718+0000","last_active":"2024-04-27T03:33:21.148028+0000","last_peered":"2024-04-27T03:33:21.148028+0000","last_clean":"2024-04-27T03:33:21.148028+0000","last_became_active":"2024-04-27T03:30:19.406364+0000","last_became_peered":"2024-04-27T03:30:19.406364+0000","last_unstale":"2024-04-27T03:33:21.148028+0000","last_undegraded":"2024-04-27T03:33:21.148028+0000","last_fullsized":"2024-04-27T03:33:21.148028+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:19.671271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432341+0000","last_change":"2024-04-27T03:30:21.408481+0000","last_active":"2024-04-27T03:31:01.432341+0000","last_peered":"2024-04-27T03:31:01.432341+0000","last_clean":"2024-04-27T03:31:01.432341+0000","last_became_active":"2024-04-27T03:30:21.408168+0000","last_became_peered":"2024-04-27T03:30:21.408168+0000","last_unstale":"2024-04-27T03:31:01.432341+0000","last_undegraded":"2024-04-27T03:31:01.432341+0000","last_fullsized":"2024-04-27T03:31:01.432341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:56.856006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"27'14","reported_seq":101,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.703564+0000","last_change":"2024-04-27T03:30:13.371773+0000","last_active":"2024-04-27T03:33:20.703564+0000","last_peered":"2024-04-27T03:33:20.703564+0000","last_clean":"2024-04-27T03:33:20.703564+0000","last_became_active":"2024-04-27T03:30:13.371503+0000","last_became_peered":"2024-04-27T03:30:13.371503+0000","last_unstale":"2024-04-27T03:33:20.703564+0000","last_undegraded":"2024-04-27T03:33:20.703564+0000","last_fullsized":"2024-04-27T03:33:20.703564+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:54.324828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438871+0000","last_change":"2024-04-27T03:30:11.357185+0000","last_active":"2024-04-27T03:31:01.438871+0000","last_peered":"2024-04-27T03:31:01.438871+0000","last_clean":"2024-04-27T03:31:01.438871+0000","last_became_active":"2024-04-27T03:30:11.356739+0000","last_became_peered":"2024-04-27T03:30:11.356739+0000","last_unstale":"2024-04-27T03:31:01.438871+0000","last_undegraded":"2024-04-27T03:31:01.438871+0000","last_fullsized":"2024-04-27T03:31:01.438871+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:31.805263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:04.282986+0000","last_change":"2024-04-27T03:31:01.443432+0000","last_active":"2024-04-27T03:31:04.282986+0000","last_peered":"2024-04-27T03:31:04.282986+0000","last_clean":"2024-04-27T03:31:04.282986+0000","last_became_active":"2024-04-27T03:31:01.443118+0000","last_became_peered":"2024-04-27T03:31:01.443118+0000","last_unstale":"2024-04-27T03:31:04.282986+0000","last_undegraded":"2024-04-27T03:31:04.282986+0000","last_fullsized":"2024-04-27T03:31:04.282986+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:38.016966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"27'68","reported_seq":136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.922498+0000","last_change":"2024-04-27T03:30:17.392542+0000","last_active":"2024-04-27T03:32:18.922498+0000","last_peered":"2024-04-27T03:32:18.922498+0000","last_clean":"2024-04-27T03:32:18.922498+0000","last_became_active":"2024-04-27T03:30:17.387318+0000","last_became_peered":"2024-04-27T03:30:17.387318+0000","last_unstale":"2024-04-27T03:32:18.922498+0000","last_undegraded":"2024-04-27T03:32:18.922498+0000","last_fullsized":"2024-04-27T03:32:18.922498+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":68,"log_dups_size":0,"ondisk_log_size":68,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:36.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":67,"num_read_kb":56,"num_write":30,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430991+0000","last_change":"2024-04-27T03:30:19.404488+0000","last_active":"2024-04-27T03:31:01.430991+0000","last_peered":"2024-04-27T03:31:01.430991+0000","last_clean":"2024-04-27T03:31:01.430991+0000","last_became_active":"2024-04-27T03:30:19.404265+0000","last_became_peered":"2024-04-27T03:30:19.404265+0000","last_unstale":"2024-04-27T03:31:01.430991+0000","last_undegraded":"2024-04-27T03:31:01.430991+0000","last_fullsized":"2024-04-27T03:31:01.430991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:26.725030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"27'18","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.120745+0000","last_change":"2024-04-27T03:30:21.414109+0000","last_active":"2024-04-27T03:33:21.120745+0000","last_peered":"2024-04-27T03:33:21.120745+0000","last_clean":"2024-04-27T03:33:21.120745+0000","last_became_active":"2024-04-27T03:30:21.413757+0000","last_became_peered":"2024-04-27T03:30:21.413757+0000","last_unstale":"2024-04-27T03:33:21.120745+0000","last_undegraded":"2024-04-27T03:33:21.120745+0000","last_fullsized":"2024-04-27T03:33:21.120745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:41.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981981+0000","last_change":"2024-04-27T03:30:13.380233+0000","last_active":"2024-04-27T03:33:20.981981+0000","last_peered":"2024-04-27T03:33:20.981981+0000","last_clean":"2024-04-27T03:33:20.981981+0000","last_became_active":"2024-04-27T03:30:13.379861+0000","last_became_peered":"2024-04-27T03:30:13.379861+0000","last_unstale":"2024-04-27T03:33:20.981981+0000","last_undegraded":"2024-04-27T03:33:20.981981+0000","last_fullsized":"2024-04-27T03:33:20.981981+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:46.205600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431725+0000","last_change":"2024-04-27T03:30:11.357503+0000","last_active":"2024-04-27T03:31:01.431725+0000","last_peered":"2024-04-27T03:31:01.431725+0000","last_clean":"2024-04-27T03:31:01.431725+0000","last_became_active":"2024-04-27T03:30:11.357176+0000","last_became_peered":"2024-04-27T03:30:11.357176+0000","last_unstale":"2024-04-27T03:31:01.431725+0000","last_undegraded":"2024-04-27T03:31:01.431725+0000","last_fullsized":"2024-04-27T03:31:01.431725+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:06.887756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432802+0000","last_change":"2024-04-27T03:30:15.382614+0000","last_active":"2024-04-27T03:31:01.432802+0000","last_peered":"2024-04-27T03:31:01.432802+0000","last_clean":"2024-04-27T03:31:01.432802+0000","last_became_active":"2024-04-27T03:30:15.382290+0000","last_became_peered":"2024-04-27T03:30:15.382290+0000","last_unstale":"2024-04-27T03:31:01.432802+0000","last_undegraded":"2024-04-27T03:31:01.432802+0000","last_fullsized":"2024-04-27T03:31:01.432802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:16.810766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"27'38","reported_seq":87,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.902396+0000","last_change":"2024-04-27T03:30:17.386194+0000","last_active":"2024-04-27T03:32:18.902396+0000","last_peered":"2024-04-27T03:32:18.902396+0000","last_clean":"2024-04-27T03:32:18.902396+0000","last_became_active":"2024-04-27T03:30:17.385902+0000","last_became_peered":"2024-04-27T03:30:17.385902+0000","last_unstale":"2024-04-27T03:32:18.902396+0000","last_undegraded":"2024-04-27T03:32:18.902396+0000","last_fullsized":"2024-04-27T03:32:18.902396+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:45:51.459337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438538+0000","last_change":"2024-04-27T03:30:19.406061+0000","last_active":"2024-04-27T03:31:01.438538+0000","last_peered":"2024-04-27T03:31:01.438538+0000","last_clean":"2024-04-27T03:31:01.438538+0000","last_became_active":"2024-04-27T03:30:19.405450+0000","last_became_peered":"2024-04-27T03:30:19.405450+0000","last_unstale":"2024-04-27T03:31:01.438538+0000","last_undegraded":"2024-04-27T03:31:01.438538+0000","last_fullsized":"2024-04-27T03:31:01.438538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:26:03.989010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"27'9","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.701473+0000","last_change":"2024-04-27T03:30:21.415884+0000","last_active":"2024-04-27T03:33:20.701473+0000","last_peered":"2024-04-27T03:33:20.701473+0000","last_clean":"2024-04-27T03:33:20.701473+0000","last_became_active":"2024-04-27T03:30:21.415494+0000","last_became_peered":"2024-04-27T03:30:21.415494+0000","last_unstale":"2024-04-27T03:33:20.701473+0000","last_undegraded":"2024-04-27T03:33:20.701473+0000","last_fullsized":"2024-04-27T03:33:20.701473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:34.804682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'6","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009826+0000","last_change":"2024-04-27T03:30:13.388008+0000","last_active":"2024-04-27T03:33:21.009826+0000","last_peered":"2024-04-27T03:33:21.009826+0000","last_clean":"2024-04-27T03:33:21.009826+0000","last_became_active":"2024-04-27T03:30:13.387533+0000","last_became_peered":"2024-04-27T03:30:13.387533+0000","last_unstale":"2024-04-27T03:33:21.009826+0000","last_undegraded":"2024-04-27T03:33:21.009826+0000","last_fullsized":"2024-04-27T03:33:21.009826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:53.142322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437632+0000","last_change":"2024-04-27T03:30:11.373706+0000","last_active":"2024-04-27T03:31:01.437632+0000","last_peered":"2024-04-27T03:31:01.437632+0000","last_clean":"2024-04-27T03:31:01.437632+0000","last_became_active":"2024-04-27T03:30:11.373440+0000","last_became_peered":"2024-04-27T03:30:11.373440+0000","last_unstale":"2024-04-27T03:31:01.437632+0000","last_undegraded":"2024-04-27T03:31:01.437632+0000","last_fullsized":"2024-04-27T03:31:01.437632+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:41:06.626342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432793+0000","last_change":"2024-04-27T03:30:15.383722+0000","last_active":"2024-04-27T03:31:01.432793+0000","last_peered":"2024-04-27T03:31:01.432793+0000","last_clean":"2024-04-27T03:31:01.432793+0000","last_became_active":"2024-04-27T03:30:15.382913+0000","last_became_peered":"2024-04-27T03:30:15.382913+0000","last_unstale":"2024-04-27T03:31:01.432793+0000","last_undegraded":"2024-04-27T03:31:01.432793+0000","last_fullsized":"2024-04-27T03:31:01.432793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:59:19.478810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"27'36","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.928891+0000","last_change":"2024-04-27T03:30:17.382348+0000","last_active":"2024-04-27T03:32:18.928891+0000","last_peered":"2024-04-27T03:32:18.928891+0000","last_clean":"2024-04-27T03:32:18.928891+0000","last_became_active":"2024-04-27T03:30:17.381106+0000","last_became_peered":"2024-04-27T03:30:17.381106+0000","last_unstale":"2024-04-27T03:32:18.928891+0000","last_undegraded":"2024-04-27T03:32:18.928891+0000","last_fullsized":"2024-04-27T03:32:18.928891+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:36:29.105227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"27'282","reported_seq":403,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.124909+0000","last_change":"2024-04-27T03:30:19.403728+0000","last_active":"2024-04-27T03:33:21.124909+0000","last_peered":"2024-04-27T03:33:21.124909+0000","last_clean":"2024-04-27T03:33:21.124909+0000","last_became_active":"2024-04-27T03:30:19.403413+0000","last_became_peered":"2024-04-27T03:30:19.403413+0000","last_unstale":"2024-04-27T03:33:21.124909+0000","last_undegraded":"2024-04-27T03:33:21.124909+0000","last_fullsized":"2024-04-27T03:33:21.124909+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.087236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"27'14","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.289774+0000","last_change":"2024-04-27T03:30:21.403250+0000","last_active":"2024-04-27T03:32:29.289774+0000","last_peered":"2024-04-27T03:32:29.289774+0000","last_clean":"2024-04-27T03:32:29.289774+0000","last_became_active":"2024-04-27T03:30:21.402102+0000","last_became_peered":"2024-04-27T03:30:21.402102+0000","last_unstale":"2024-04-27T03:32:29.289774+0000","last_undegraded":"2024-04-27T03:32:29.289774+0000","last_fullsized":"2024-04-27T03:32:29.289774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:00.181478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":23,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'12","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.795005+0000","last_change":"2024-04-27T03:30:13.383203+0000","last_active":"2024-04-27T03:33:20.795005+0000","last_peered":"2024-04-27T03:33:20.795005+0000","last_clean":"2024-04-27T03:33:20.795005+0000","last_became_active":"2024-04-27T03:30:13.382899+0000","last_became_peered":"2024-04-27T03:30:13.382899+0000","last_unstale":"2024-04-27T03:33:20.795005+0000","last_undegraded":"2024-04-27T03:33:20.795005+0000","last_fullsized":"2024-04-27T03:33:20.795005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:57.593400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'4","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.924480+0000","last_change":"2024-04-27T03:31:01.444153+0000","last_active":"2024-04-27T03:33:20.924480+0000","last_peered":"2024-04-27T03:33:20.924480+0000","last_clean":"2024-04-27T03:33:20.924480+0000","last_became_active":"2024-04-27T03:31:01.439776+0000","last_became_peered":"2024-04-27T03:31:01.439776+0000","last_unstale":"2024-04-27T03:33:20.924480+0000","last_undegraded":"2024-04-27T03:33:20.924480+0000","last_fullsized":"2024-04-27T03:33:20.924480+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:15.239233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"27'3","reported_seq":413,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.994698+0000","last_change":"2024-04-27T03:30:15.382936+0000","last_active":"2024-04-27T03:32:19.994698+0000","last_peered":"2024-04-27T03:32:19.994698+0000","last_clean":"2024-04-27T03:32:19.994698+0000","last_became_active":"2024-04-27T03:30:15.382502+0000","last_became_peered":"2024-04-27T03:30:15.382502+0000","last_unstale":"2024-04-27T03:32:19.994698+0000","last_undegraded":"2024-04-27T03:32:19.994698+0000","last_fullsized":"2024-04-27T03:32:19.994698+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:17.198784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":381,"num_read_kb":381,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"25'10","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438996+0000","last_change":"2024-04-27T03:30:17.389548+0000","last_active":"2024-04-27T03:31:01.438996+0000","last_peered":"2024-04-27T03:31:01.438996+0000","last_clean":"2024-04-27T03:31:01.438996+0000","last_became_active":"2024-04-27T03:30:17.387739+0000","last_became_peered":"2024-04-27T03:30:17.387739+0000","last_unstale":"2024-04-27T03:31:01.438996+0000","last_undegraded":"2024-04-27T03:31:01.438996+0000","last_fullsized":"2024-04-27T03:31:01.438996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:03:39.120700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"27'282","reported_seq":510,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.076259+0000","last_change":"2024-04-27T03:30:19.402853+0000","last_active":"2024-04-27T03:33:21.076259+0000","last_peered":"2024-04-27T03:33:21.076259+0000","last_clean":"2024-04-27T03:33:21.076259+0000","last_became_active":"2024-04-27T03:30:19.402484+0000","last_became_peered":"2024-04-27T03:30:19.402484+0000","last_unstale":"2024-04-27T03:33:21.076259+0000","last_undegraded":"2024-04-27T03:33:21.076259+0000","last_fullsized":"2024-04-27T03:33:21.076259+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:31.846278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"27'4","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.132546+0000","last_change":"2024-04-27T03:30:21.408801+0000","last_active":"2024-04-27T03:32:20.132546+0000","last_peered":"2024-04-27T03:32:20.132546+0000","last_clean":"2024-04-27T03:32:20.132546+0000","last_became_active":"2024-04-27T03:30:21.408512+0000","last_became_peered":"2024-04-27T03:30:21.408512+0000","last_unstale":"2024-04-27T03:32:20.132546+0000","last_undegraded":"2024-04-27T03:32:20.132546+0000","last_fullsized":"2024-04-27T03:32:20.132546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:49:39.647798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'14","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.792438+0000","last_change":"2024-04-27T03:30:13.381639+0000","last_active":"2024-04-27T03:33:20.792438+0000","last_peered":"2024-04-27T03:33:20.792438+0000","last_clean":"2024-04-27T03:33:20.792438+0000","last_became_active":"2024-04-27T03:30:13.380540+0000","last_became_peered":"2024-04-27T03:30:13.380540+0000","last_unstale":"2024-04-27T03:33:20.792438+0000","last_undegraded":"2024-04-27T03:33:20.792438+0000","last_fullsized":"2024-04-27T03:33:20.792438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:44.819638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.190061+0000","last_change":"2024-04-27T03:30:11.372470+0000","last_active":"2024-04-27T03:32:20.190061+0000","last_peered":"2024-04-27T03:32:20.190061+0000","last_clean":"2024-04-27T03:32:20.190061+0000","last_became_active":"2024-04-27T03:30:11.372193+0000","last_became_peered":"2024-04-27T03:30:11.372193+0000","last_unstale":"2024-04-27T03:32:20.190061+0000","last_undegraded":"2024-04-27T03:32:20.190061+0000","last_fullsized":"2024-04-27T03:32:20.190061+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:16.294699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438746+0000","last_change":"2024-04-27T03:30:15.385100+0000","last_active":"2024-04-27T03:31:01.438746+0000","last_peered":"2024-04-27T03:31:01.438746+0000","last_clean":"2024-04-27T03:31:01.438746+0000","last_became_active":"2024-04-27T03:30:15.383924+0000","last_became_peered":"2024-04-27T03:30:15.383924+0000","last_unstale":"2024-04-27T03:31:01.438746+0000","last_undegraded":"2024-04-27T03:31:01.438746+0000","last_fullsized":"2024-04-27T03:31:01.438746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:08.259240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"27'38","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.915730+0000","last_change":"2024-04-27T03:30:17.388297+0000","last_active":"2024-04-27T03:32:18.915730+0000","last_peered":"2024-04-27T03:32:18.915730+0000","last_clean":"2024-04-27T03:32:18.915730+0000","last_became_active":"2024-04-27T03:30:17.387578+0000","last_became_peered":"2024-04-27T03:30:17.387578+0000","last_unstale":"2024-04-27T03:32:18.915730+0000","last_undegraded":"2024-04-27T03:32:18.915730+0000","last_fullsized":"2024-04-27T03:32:18.915730+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:26.963495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":41,"num_read_kb":31,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433032+0000","last_change":"2024-04-27T03:30:19.405992+0000","last_active":"2024-04-27T03:31:01.433032+0000","last_peered":"2024-04-27T03:31:01.433032+0000","last_clean":"2024-04-27T03:31:01.433032+0000","last_became_active":"2024-04-27T03:30:19.405512+0000","last_became_peered":"2024-04-27T03:30:19.405512+0000","last_unstale":"2024-04-27T03:31:01.433032+0000","last_undegraded":"2024-04-27T03:31:01.433032+0000","last_fullsized":"2024-04-27T03:31:01.433032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:15.297931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"27'14","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.046419+0000","last_change":"2024-04-27T03:30:21.407207+0000","last_active":"2024-04-27T03:33:21.046419+0000","last_peered":"2024-04-27T03:33:21.046419+0000","last_clean":"2024-04-27T03:33:21.046419+0000","last_became_active":"2024-04-27T03:30:21.406883+0000","last_became_peered":"2024-04-27T03:30:21.406883+0000","last_unstale":"2024-04-27T03:33:21.046419+0000","last_undegraded":"2024-04-27T03:33:21.046419+0000","last_fullsized":"2024-04-27T03:33:21.046419+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:00.181478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":59,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'20","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933899+0000","last_change":"2024-04-27T03:30:13.387655+0000","last_active":"2024-04-27T03:33:20.933899+0000","last_peered":"2024-04-27T03:33:20.933899+0000","last_clean":"2024-04-27T03:33:20.933899+0000","last_became_active":"2024-04-27T03:30:13.387180+0000","last_became_peered":"2024-04-27T03:30:13.387180+0000","last_unstale":"2024-04-27T03:33:20.933899+0000","last_undegraded":"2024-04-27T03:33:20.933899+0000","last_fullsized":"2024-04-27T03:33:20.933899+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:24.510289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431660+0000","last_change":"2024-04-27T03:30:06.104151+0000","last_active":"2024-04-27T03:31:01.431660+0000","last_peered":"2024-04-27T03:31:01.431660+0000","last_clean":"2024-04-27T03:31:01.431660+0000","last_became_active":"2024-04-27T03:30:06.102893+0000","last_became_peered":"2024-04-27T03:30:06.102893+0000","last_unstale":"2024-04-27T03:31:01.431660+0000","last_undegraded":"2024-04-27T03:31:01.431660+0000","last_fullsized":"2024-04-27T03:31:01.431660+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:27:47.408864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439201+0000","last_change":"2024-04-27T03:30:06.106875+0000","last_active":"2024-04-27T03:31:01.439201+0000","last_peered":"2024-04-27T03:31:01.439201+0000","last_clean":"2024-04-27T03:31:01.439201+0000","last_became_active":"2024-04-27T03:30:06.106511+0000","last_became_peered":"2024-04-27T03:30:06.106511+0000","last_unstale":"2024-04-27T03:31:01.439201+0000","last_undegraded":"2024-04-27T03:31:01.439201+0000","last_fullsized":"2024-04-27T03:31:01.439201+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:56:59.055441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.005690+0000","last_change":"2024-04-27T03:30:11.362699+0000","last_active":"2024-04-27T03:33:21.005690+0000","last_peered":"2024-04-27T03:33:21.005690+0000","last_clean":"2024-04-27T03:33:21.005690+0000","last_became_active":"2024-04-27T03:30:11.362445+0000","last_became_peered":"2024-04-27T03:30:11.362445+0000","last_unstale":"2024-04-27T03:33:21.005690+0000","last_undegraded":"2024-04-27T03:33:21.005690+0000","last_fullsized":"2024-04-27T03:33:21.005690+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:11:24.757438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.441975+0000","last_change":"2024-04-27T03:31:01.441027+0000","last_active":"2024-04-27T03:31:01.441975+0000","last_peered":"2024-04-27T03:31:01.441975+0000","last_clean":"2024-04-27T03:31:01.441975+0000","last_became_active":"2024-04-27T03:31:01.440678+0000","last_became_peered":"2024-04-27T03:31:01.440678+0000","last_unstale":"2024-04-27T03:31:01.441975+0000","last_undegraded":"2024-04-27T03:31:01.441975+0000","last_fullsized":"2024-04-27T03:31:01.441975+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:22.897973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"27'83","reported_seq":158,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.927035+0000","last_change":"2024-04-27T03:30:17.388976+0000","last_active":"2024-04-27T03:32:18.927035+0000","last_peered":"2024-04-27T03:32:18.927035+0000","last_clean":"2024-04-27T03:32:18.927035+0000","last_became_active":"2024-04-27T03:30:17.387707+0000","last_became_peered":"2024-04-27T03:30:17.387707+0000","last_unstale":"2024-04-27T03:32:18.927035+0000","last_undegraded":"2024-04-27T03:32:18.927035+0000","last_fullsized":"2024-04-27T03:32:18.927035+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":83,"log_dups_size":0,"ondisk_log_size":83,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:03.008955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":75,"num_read_kb":63,"num_write":28,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433316+0000","last_change":"2024-04-27T03:30:19.405693+0000","last_active":"2024-04-27T03:31:01.433316+0000","last_peered":"2024-04-27T03:31:01.433316+0000","last_clean":"2024-04-27T03:31:01.433316+0000","last_became_active":"2024-04-27T03:30:19.405359+0000","last_became_peered":"2024-04-27T03:30:19.405359+0000","last_unstale":"2024-04-27T03:31:01.433316+0000","last_undegraded":"2024-04-27T03:31:01.433316+0000","last_fullsized":"2024-04-27T03:31:01.433316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:36.634879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"27'81","reported_seq":162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.022864+0000","last_change":"2024-04-27T03:30:21.404603+0000","last_active":"2024-04-27T03:33:21.022864+0000","last_peered":"2024-04-27T03:33:21.022864+0000","last_clean":"2024-04-27T03:33:21.022864+0000","last_became_active":"2024-04-27T03:30:21.404300+0000","last_became_peered":"2024-04-27T03:30:21.404300+0000","last_unstale":"2024-04-27T03:33:21.022864+0000","last_undegraded":"2024-04-27T03:33:21.022864+0000","last_fullsized":"2024-04-27T03:33:21.022864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:05.598061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439232+0000","last_change":"2024-04-27T03:30:13.378248+0000","last_active":"2024-04-27T03:31:01.439232+0000","last_peered":"2024-04-27T03:31:01.439232+0000","last_clean":"2024-04-27T03:31:01.439232+0000","last_became_active":"2024-04-27T03:30:13.377713+0000","last_became_peered":"2024-04-27T03:30:13.377713+0000","last_unstale":"2024-04-27T03:31:01.439232+0000","last_undegraded":"2024-04-27T03:31:01.439232+0000","last_fullsized":"2024-04-27T03:31:01.439232+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:09.267760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436259+0000","last_change":"2024-04-27T03:30:06.106890+0000","last_active":"2024-04-27T03:31:01.436259+0000","last_peered":"2024-04-27T03:31:01.436259+0000","last_clean":"2024-04-27T03:31:01.436259+0000","last_became_active":"2024-04-27T03:30:06.106474+0000","last_became_peered":"2024-04-27T03:30:06.106474+0000","last_unstale":"2024-04-27T03:31:01.436259+0000","last_undegraded":"2024-04-27T03:31:01.436259+0000","last_fullsized":"2024-04-27T03:31:01.436259+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:12.652758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.643945+0000","last_change":"2024-04-27T03:30:11.360679+0000","last_active":"2024-04-27T03:31:36.643945+0000","last_peered":"2024-04-27T03:31:36.643945+0000","last_clean":"2024-04-27T03:31:36.643945+0000","last_became_active":"2024-04-27T03:30:11.360244+0000","last_became_peered":"2024-04-27T03:30:11.360244+0000","last_unstale":"2024-04-27T03:31:36.643945+0000","last_undegraded":"2024-04-27T03:31:36.643945+0000","last_fullsized":"2024-04-27T03:31:36.643945+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.539212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440319+0000","last_change":"2024-04-27T03:30:15.379725+0000","last_active":"2024-04-27T03:31:01.440319+0000","last_peered":"2024-04-27T03:31:01.440319+0000","last_clean":"2024-04-27T03:31:01.440319+0000","last_became_active":"2024-04-27T03:30:15.379448+0000","last_became_peered":"2024-04-27T03:30:15.379448+0000","last_unstale":"2024-04-27T03:31:01.440319+0000","last_undegraded":"2024-04-27T03:31:01.440319+0000","last_fullsized":"2024-04-27T03:31:01.440319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:06.243654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"27'279","reported_seq":488,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462874+0000","last_change":"2024-04-27T03:30:17.391189+0000","last_active":"2024-04-27T03:33:20.462874+0000","last_peered":"2024-04-27T03:33:20.462874+0000","last_clean":"2024-04-27T03:33:20.462874+0000","last_became_active":"2024-04-27T03:30:17.390884+0000","last_became_peered":"2024-04-27T03:30:17.390884+0000","last_unstale":"2024-04-27T03:33:20.462874+0000","last_undegraded":"2024-04-27T03:33:20.462874+0000","last_fullsized":"2024-04-27T03:33:20.462874+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":279,"log_dups_size":0,"ondisk_log_size":279,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:57.164266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":366,"num_read_kb":350,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439611+0000","last_change":"2024-04-27T03:30:19.408904+0000","last_active":"2024-04-27T03:31:01.439611+0000","last_peered":"2024-04-27T03:31:01.439611+0000","last_clean":"2024-04-27T03:31:01.439611+0000","last_became_active":"2024-04-27T03:30:19.405777+0000","last_became_peered":"2024-04-27T03:30:19.405777+0000","last_unstale":"2024-04-27T03:31:01.439611+0000","last_undegraded":"2024-04-27T03:31:01.439611+0000","last_fullsized":"2024-04-27T03:31:01.439611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:57.389557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"27'7","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.968472+0000","last_change":"2024-04-27T03:30:21.409632+0000","last_active":"2024-04-27T03:33:20.968472+0000","last_peered":"2024-04-27T03:33:20.968472+0000","last_clean":"2024-04-27T03:33:20.968472+0000","last_became_active":"2024-04-27T03:30:21.409359+0000","last_became_peered":"2024-04-27T03:30:21.409359+0000","last_unstale":"2024-04-27T03:33:20.968472+0000","last_undegraded":"2024-04-27T03:33:20.968472+0000","last_fullsized":"2024-04-27T03:33:20.968472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:41.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"25'10","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433170+0000","last_change":"2024-04-27T03:30:13.385862+0000","last_active":"2024-04-27T03:31:01.433170+0000","last_peered":"2024-04-27T03:31:01.433170+0000","last_clean":"2024-04-27T03:31:01.433170+0000","last_became_active":"2024-04-27T03:30:13.385521+0000","last_became_peered":"2024-04-27T03:30:13.385521+0000","last_unstale":"2024-04-27T03:31:01.433170+0000","last_undegraded":"2024-04-27T03:31:01.433170+0000","last_fullsized":"2024-04-27T03:31:01.433170+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:38.976155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435195+0000","last_change":"2024-04-27T03:30:06.103722+0000","last_active":"2024-04-27T03:31:01.435195+0000","last_peered":"2024-04-27T03:31:01.435195+0000","last_clean":"2024-04-27T03:31:01.435195+0000","last_became_active":"2024-04-27T03:30:06.103441+0000","last_became_peered":"2024-04-27T03:30:06.103441+0000","last_unstale":"2024-04-27T03:31:01.435195+0000","last_undegraded":"2024-04-27T03:31:01.435195+0000","last_fullsized":"2024-04-27T03:31:01.435195+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:59.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430689+0000","last_change":"2024-04-27T03:30:11.360789+0000","last_active":"2024-04-27T03:31:01.430689+0000","last_peered":"2024-04-27T03:31:01.430689+0000","last_clean":"2024-04-27T03:31:01.430689+0000","last_became_active":"2024-04-27T03:30:11.360443+0000","last_became_peered":"2024-04-27T03:30:11.360443+0000","last_unstale":"2024-04-27T03:31:01.430689+0000","last_undegraded":"2024-04-27T03:31:01.430689+0000","last_fullsized":"2024-04-27T03:31:01.430689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.863119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"18'1","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437997+0000","last_change":"2024-04-27T03:30:15.383569+0000","last_active":"2024-04-27T03:31:01.437997+0000","last_peered":"2024-04-27T03:31:01.437997+0000","last_clean":"2024-04-27T03:31:01.437997+0000","last_became_active":"2024-04-27T03:30:15.383213+0000","last_became_peered":"2024-04-27T03:30:15.383213+0000","last_unstale":"2024-04-27T03:31:01.437997+0000","last_undegraded":"2024-04-27T03:31:01.437997+0000","last_fullsized":"2024-04-27T03:31:01.437997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:48.217027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"27'16","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462975+0000","last_change":"2024-04-27T03:30:17.388886+0000","last_active":"2024-04-27T03:33:20.462975+0000","last_peered":"2024-04-27T03:33:20.462975+0000","last_clean":"2024-04-27T03:33:20.462975+0000","last_became_active":"2024-04-27T03:30:17.388588+0000","last_became_peered":"2024-04-27T03:30:17.388588+0000","last_unstale":"2024-04-27T03:33:20.462975+0000","last_undegraded":"2024-04-27T03:33:20.462975+0000","last_fullsized":"2024-04-27T03:33:20.462975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:59:21.489327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431871+0000","last_change":"2024-04-27T03:30:19.407292+0000","last_active":"2024-04-27T03:31:01.431871+0000","last_peered":"2024-04-27T03:31:01.431871+0000","last_clean":"2024-04-27T03:31:01.431871+0000","last_became_active":"2024-04-27T03:30:19.407004+0000","last_became_peered":"2024-04-27T03:30:19.407004+0000","last_unstale":"2024-04-27T03:31:01.431871+0000","last_undegraded":"2024-04-27T03:31:01.431871+0000","last_fullsized":"2024-04-27T03:31:01.431871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:10.262399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"25'2","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438400+0000","last_change":"2024-04-27T03:30:21.417022+0000","last_active":"2024-04-27T03:31:01.438400+0000","last_peered":"2024-04-27T03:31:01.438400+0000","last_clean":"2024-04-27T03:31:01.438400+0000","last_became_active":"2024-04-27T03:30:21.416389+0000","last_became_peered":"2024-04-27T03:30:21.416389+0000","last_unstale":"2024-04-27T03:31:01.438400+0000","last_undegraded":"2024-04-27T03:31:01.438400+0000","last_fullsized":"2024-04-27T03:31:01.438400+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:11:27.333113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'8","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.696826+0000","last_change":"2024-04-27T03:30:13.369982+0000","last_active":"2024-04-27T03:33:20.696826+0000","last_peered":"2024-04-27T03:33:20.696826+0000","last_clean":"2024-04-27T03:33:20.696826+0000","last_became_active":"2024-04-27T03:30:13.369638+0000","last_became_peered":"2024-04-27T03:30:13.369638+0000","last_unstale":"2024-04-27T03:33:20.696826+0000","last_undegraded":"2024-04-27T03:33:20.696826+0000","last_fullsized":"2024-04-27T03:33:20.696826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:57.593400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436751+0000","last_change":"2024-04-27T03:30:06.107772+0000","last_active":"2024-04-27T03:31:01.436751+0000","last_peered":"2024-04-27T03:31:01.436751+0000","last_clean":"2024-04-27T03:31:01.436751+0000","last_became_active":"2024-04-27T03:30:06.107356+0000","last_became_peered":"2024-04-27T03:30:06.107356+0000","last_unstale":"2024-04-27T03:31:01.436751+0000","last_undegraded":"2024-04-27T03:31:01.436751+0000","last_fullsized":"2024-04-27T03:31:01.436751+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:05:28.414626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436437+0000","last_change":"2024-04-27T03:30:11.357072+0000","last_active":"2024-04-27T03:31:01.436437+0000","last_peered":"2024-04-27T03:31:01.436437+0000","last_clean":"2024-04-27T03:31:01.436437+0000","last_became_active":"2024-04-27T03:30:11.356685+0000","last_became_peered":"2024-04-27T03:30:11.356685+0000","last_unstale":"2024-04-27T03:31:01.436437+0000","last_undegraded":"2024-04-27T03:31:01.436437+0000","last_fullsized":"2024-04-27T03:31:01.436437+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.863119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432748+0000","last_change":"2024-04-27T03:30:15.382673+0000","last_active":"2024-04-27T03:31:01.432748+0000","last_peered":"2024-04-27T03:31:01.432748+0000","last_clean":"2024-04-27T03:31:01.432748+0000","last_became_active":"2024-04-27T03:30:15.382326+0000","last_became_peered":"2024-04-27T03:30:15.382326+0000","last_unstale":"2024-04-27T03:31:01.432748+0000","last_undegraded":"2024-04-27T03:31:01.432748+0000","last_fullsized":"2024-04-27T03:31:01.432748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:37.500097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"27'13","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.319621+0000","last_change":"2024-04-27T03:30:17.390316+0000","last_active":"2024-04-27T03:33:20.319621+0000","last_peered":"2024-04-27T03:33:20.319621+0000","last_clean":"2024-04-27T03:33:20.319621+0000","last_became_active":"2024-04-27T03:30:17.389373+0000","last_became_peered":"2024-04-27T03:30:17.389373+0000","last_unstale":"2024-04-27T03:33:20.319621+0000","last_undegraded":"2024-04-27T03:33:20.319621+0000","last_fullsized":"2024-04-27T03:33:20.319621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:08.896469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438561+0000","last_change":"2024-04-27T03:30:19.406634+0000","last_active":"2024-04-27T03:31:01.438561+0000","last_peered":"2024-04-27T03:31:01.438561+0000","last_clean":"2024-04-27T03:31:01.438561+0000","last_became_active":"2024-04-27T03:30:19.405953+0000","last_became_peered":"2024-04-27T03:30:19.405953+0000","last_unstale":"2024-04-27T03:31:01.438561+0000","last_undegraded":"2024-04-27T03:31:01.438561+0000","last_fullsized":"2024-04-27T03:31:01.438561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:02.743749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"27'6","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.073585+0000","last_change":"2024-04-27T03:30:21.413040+0000","last_active":"2024-04-27T03:33:21.073585+0000","last_peered":"2024-04-27T03:33:21.073585+0000","last_clean":"2024-04-27T03:33:21.073585+0000","last_became_active":"2024-04-27T03:30:21.412790+0000","last_became_peered":"2024-04-27T03:30:21.412790+0000","last_unstale":"2024-04-27T03:33:21.073585+0000","last_undegraded":"2024-04-27T03:33:21.073585+0000","last_fullsized":"2024-04-27T03:33:21.073585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:58.583851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'18","reported_seq":86,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933802+0000","last_change":"2024-04-27T03:30:13.381418+0000","last_active":"2024-04-27T03:33:20.933802+0000","last_peered":"2024-04-27T03:33:20.933802+0000","last_clean":"2024-04-27T03:33:20.933802+0000","last_became_active":"2024-04-27T03:30:13.380888+0000","last_became_peered":"2024-04-27T03:30:13.380888+0000","last_unstale":"2024-04-27T03:33:20.933802+0000","last_undegraded":"2024-04-27T03:33:20.933802+0000","last_fullsized":"2024-04-27T03:33:20.933802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:28.256872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":73,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433189+0000","last_change":"2024-04-27T03:30:11.362550+0000","last_active":"2024-04-27T03:31:01.433189+0000","last_peered":"2024-04-27T03:31:01.433189+0000","last_clean":"2024-04-27T03:31:01.433189+0000","last_became_active":"2024-04-27T03:30:11.361786+0000","last_became_peered":"2024-04-27T03:30:11.361786+0000","last_unstale":"2024-04-27T03:31:01.433189+0000","last_undegraded":"2024-04-27T03:31:01.433189+0000","last_fullsized":"2024-04-27T03:31:01.433189+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.539212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431862+0000","last_change":"2024-04-27T03:30:06.107861+0000","last_active":"2024-04-27T03:31:01.431862+0000","last_peered":"2024-04-27T03:31:01.431862+0000","last_clean":"2024-04-27T03:31:01.431862+0000","last_became_active":"2024-04-27T03:30:06.107594+0000","last_became_peered":"2024-04-27T03:30:06.107594+0000","last_unstale":"2024-04-27T03:31:01.431862+0000","last_undegraded":"2024-04-27T03:31:01.431862+0000","last_fullsized":"2024-04-27T03:31:01.431862+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:10.319152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"27'1","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.289051+0000","last_change":"2024-04-27T03:30:15.381770+0000","last_active":"2024-04-27T03:32:24.289051+0000","last_peered":"2024-04-27T03:32:24.289051+0000","last_clean":"2024-04-27T03:32:24.289051+0000","last_became_active":"2024-04-27T03:30:15.381392+0000","last_became_peered":"2024-04-27T03:30:15.381392+0000","last_unstale":"2024-04-27T03:32:24.289051+0000","last_undegraded":"2024-04-27T03:32:24.289051+0000","last_fullsized":"2024-04-27T03:32:24.289051+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:01.291555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"27'73","reported_seq":144,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288088+0000","last_change":"2024-04-27T03:30:17.390751+0000","last_active":"2024-04-27T03:32:19.288088+0000","last_peered":"2024-04-27T03:32:19.288088+0000","last_clean":"2024-04-27T03:32:19.288088+0000","last_became_active":"2024-04-27T03:30:17.390409+0000","last_became_peered":"2024-04-27T03:30:17.390409+0000","last_unstale":"2024-04-27T03:32:19.288088+0000","last_undegraded":"2024-04-27T03:32:19.288088+0000","last_fullsized":"2024-04-27T03:32:19.288088+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.209300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":77,"num_read_kb":63,"num_write":38,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435401+0000","last_change":"2024-04-27T03:30:19.403292+0000","last_active":"2024-04-27T03:31:01.435401+0000","last_peered":"2024-04-27T03:31:01.435401+0000","last_clean":"2024-04-27T03:31:01.435401+0000","last_became_active":"2024-04-27T03:30:19.402977+0000","last_became_peered":"2024-04-27T03:30:19.402977+0000","last_unstale":"2024-04-27T03:31:01.435401+0000","last_undegraded":"2024-04-27T03:31:01.435401+0000","last_fullsized":"2024-04-27T03:31:01.435401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:24.818605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"27'11","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.984330+0000","last_change":"2024-04-27T03:30:21.410763+0000","last_active":"2024-04-27T03:33:20.984330+0000","last_peered":"2024-04-27T03:33:20.984330+0000","last_clean":"2024-04-27T03:33:20.984330+0000","last_became_active":"2024-04-27T03:30:21.410511+0000","last_became_peered":"2024-04-27T03:30:21.410511+0000","last_unstale":"2024-04-27T03:33:20.984330+0000","last_undegraded":"2024-04-27T03:33:20.984330+0000","last_fullsized":"2024-04-27T03:33:20.984330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:58.583851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.019081+0000","last_change":"2024-04-27T03:30:13.380442+0000","last_active":"2024-04-27T03:33:21.019081+0000","last_peered":"2024-04-27T03:33:21.019081+0000","last_clean":"2024-04-27T03:33:21.019081+0000","last_became_active":"2024-04-27T03:30:13.379985+0000","last_became_peered":"2024-04-27T03:30:13.379985+0000","last_unstale":"2024-04-27T03:33:21.019081+0000","last_undegraded":"2024-04-27T03:33:21.019081+0000","last_fullsized":"2024-04-27T03:33:21.019081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:19.215823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288186+0000","last_change":"2024-04-27T03:30:11.361129+0000","last_active":"2024-04-27T03:32:19.288186+0000","last_peered":"2024-04-27T03:32:19.288186+0000","last_clean":"2024-04-27T03:32:19.288186+0000","last_became_active":"2024-04-27T03:30:11.360352+0000","last_became_peered":"2024-04-27T03:30:11.360352+0000","last_unstale":"2024-04-27T03:32:19.288186+0000","last_undegraded":"2024-04-27T03:32:19.288186+0000","last_fullsized":"2024-04-27T03:32:19.288186+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:22.798184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440363+0000","last_change":"2024-04-27T03:30:06.106816+0000","last_active":"2024-04-27T03:31:01.440363+0000","last_peered":"2024-04-27T03:31:01.440363+0000","last_clean":"2024-04-27T03:31:01.440363+0000","last_became_active":"2024-04-27T03:30:06.106475+0000","last_became_peered":"2024-04-27T03:30:06.106475+0000","last_unstale":"2024-04-27T03:31:01.440363+0000","last_undegraded":"2024-04-27T03:31:01.440363+0000","last_fullsized":"2024-04-27T03:31:01.440363+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:36.362210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437040+0000","last_change":"2024-04-27T03:30:15.380295+0000","last_active":"2024-04-27T03:31:01.437040+0000","last_peered":"2024-04-27T03:31:01.437040+0000","last_clean":"2024-04-27T03:31:01.437040+0000","last_became_active":"2024-04-27T03:30:15.380030+0000","last_became_peered":"2024-04-27T03:30:15.380030+0000","last_unstale":"2024-04-27T03:31:01.437040+0000","last_undegraded":"2024-04-27T03:31:01.437040+0000","last_fullsized":"2024-04-27T03:31:01.437040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:27.827534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"25'20","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433411+0000","last_change":"2024-04-27T03:30:17.393593+0000","last_active":"2024-04-27T03:31:01.433411+0000","last_peered":"2024-04-27T03:31:01.433411+0000","last_clean":"2024-04-27T03:31:01.433411+0000","last_became_active":"2024-04-27T03:30:17.393253+0000","last_became_peered":"2024-04-27T03:30:17.393253+0000","last_unstale":"2024-04-27T03:31:01.433411+0000","last_undegraded":"2024-04-27T03:31:01.433411+0000","last_fullsized":"2024-04-27T03:31:01.433411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:56.968218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"27'282","reported_seq":386,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462731+0000","last_change":"2024-04-27T03:30:19.402596+0000","last_active":"2024-04-27T03:33:20.462731+0000","last_peered":"2024-04-27T03:33:20.462731+0000","last_clean":"2024-04-27T03:33:20.462731+0000","last_became_active":"2024-04-27T03:30:19.402319+0000","last_became_peered":"2024-04-27T03:30:19.402319+0000","last_unstale":"2024-04-27T03:33:20.462731+0000","last_undegraded":"2024-04-27T03:33:20.462731+0000","last_fullsized":"2024-04-27T03:33:20.462731+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:21.528207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438342+0000","last_change":"2024-04-27T03:30:21.414388+0000","last_active":"2024-04-27T03:31:01.438342+0000","last_peered":"2024-04-27T03:31:01.438342+0000","last_clean":"2024-04-27T03:31:01.438342+0000","last_became_active":"2024-04-27T03:30:21.414109+0000","last_became_peered":"2024-04-27T03:30:21.414109+0000","last_unstale":"2024-04-27T03:31:01.438342+0000","last_undegraded":"2024-04-27T03:31:01.438342+0000","last_fullsized":"2024-04-27T03:31:01.438342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:34.636870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436753+0000","last_change":"2024-04-27T03:30:13.388070+0000","last_active":"2024-04-27T03:31:01.436753+0000","last_peered":"2024-04-27T03:31:01.436753+0000","last_clean":"2024-04-27T03:31:01.436753+0000","last_became_active":"2024-04-27T03:30:13.387764+0000","last_became_peered":"2024-04-27T03:30:13.387764+0000","last_unstale":"2024-04-27T03:31:01.436753+0000","last_undegraded":"2024-04-27T03:31:01.436753+0000","last_fullsized":"2024-04-27T03:31:01.436753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:16.285485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.918891+0000","last_change":"2024-04-27T03:30:11.356555+0000","last_active":"2024-04-27T03:33:20.918891+0000","last_peered":"2024-04-27T03:33:20.918891+0000","last_clean":"2024-04-27T03:33:20.918891+0000","last_became_active":"2024-04-27T03:30:11.355704+0000","last_became_peered":"2024-04-27T03:30:11.355704+0000","last_unstale":"2024-04-27T03:33:20.918891+0000","last_undegraded":"2024-04-27T03:33:20.918891+0000","last_fullsized":"2024-04-27T03:33:20.918891+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:35.764527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432650+0000","last_change":"2024-04-27T03:30:06.104067+0000","last_active":"2024-04-27T03:31:01.432650+0000","last_peered":"2024-04-27T03:31:01.432650+0000","last_clean":"2024-04-27T03:31:01.432650+0000","last_became_active":"2024-04-27T03:30:06.103759+0000","last_became_peered":"2024-04-27T03:30:06.103759+0000","last_unstale":"2024-04-27T03:31:01.432650+0000","last_undegraded":"2024-04-27T03:31:01.432650+0000","last_fullsized":"2024-04-27T03:31:01.432650+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:27:47.408864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"27'4","reported_seq":226,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.984492+0000","last_change":"2024-04-27T03:30:15.384765+0000","last_active":"2024-04-27T03:32:19.984492+0000","last_peered":"2024-04-27T03:32:19.984492+0000","last_clean":"2024-04-27T03:32:19.984492+0000","last_became_active":"2024-04-27T03:30:15.384372+0000","last_became_peered":"2024-04-27T03:30:15.384372+0000","last_unstale":"2024-04-27T03:32:19.984492+0000","last_undegraded":"2024-04-27T03:32:19.984492+0000","last_fullsized":"2024-04-27T03:32:19.984492+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:55.971988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":298,"num_read_kb":298,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'20","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295153+0000","last_change":"2024-04-27T03:30:17.389511+0000","last_active":"2024-04-27T03:32:24.295153+0000","last_peered":"2024-04-27T03:32:24.295153+0000","last_clean":"2024-04-27T03:32:24.295153+0000","last_became_active":"2024-04-27T03:30:17.389154+0000","last_became_peered":"2024-04-27T03:30:17.389154+0000","last_unstale":"2024-04-27T03:32:24.295153+0000","last_undegraded":"2024-04-27T03:32:24.295153+0000","last_fullsized":"2024-04-27T03:32:24.295153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:41:46.387083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"27'282","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.121400+0000","last_change":"2024-04-27T03:30:19.405158+0000","last_active":"2024-04-27T03:33:21.121400+0000","last_peered":"2024-04-27T03:33:21.121400+0000","last_clean":"2024-04-27T03:33:21.121400+0000","last_became_active":"2024-04-27T03:30:19.404867+0000","last_became_peered":"2024-04-27T03:30:19.404867+0000","last_unstale":"2024-04-27T03:33:21.121400+0000","last_undegraded":"2024-04-27T03:33:21.121400+0000","last_fullsized":"2024-04-27T03:33:21.121400+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:12.277984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"25'3","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434777+0000","last_change":"2024-04-27T03:30:21.413577+0000","last_active":"2024-04-27T03:31:01.434777+0000","last_peered":"2024-04-27T03:31:01.434777+0000","last_clean":"2024-04-27T03:31:01.434777+0000","last_became_active":"2024-04-27T03:30:21.413264+0000","last_became_peered":"2024-04-27T03:30:21.413264+0000","last_unstale":"2024-04-27T03:31:01.434777+0000","last_undegraded":"2024-04-27T03:31:01.434777+0000","last_fullsized":"2024-04-27T03:31:01.434777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:23.492592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"27'4","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.696091+0000","last_change":"2024-04-27T03:30:13.383496+0000","last_active":"2024-04-27T03:33:20.696091+0000","last_peered":"2024-04-27T03:33:20.696091+0000","last_clean":"2024-04-27T03:33:20.696091+0000","last_became_active":"2024-04-27T03:30:13.383208+0000","last_became_peered":"2024-04-27T03:30:13.383208+0000","last_unstale":"2024-04-27T03:33:20.696091+0000","last_undegraded":"2024-04-27T03:33:20.696091+0000","last_fullsized":"2024-04-27T03:33:20.696091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:13.288239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436165+0000","last_change":"2024-04-27T03:30:11.370648+0000","last_active":"2024-04-27T03:31:01.436165+0000","last_peered":"2024-04-27T03:31:01.436165+0000","last_clean":"2024-04-27T03:31:01.436165+0000","last_became_active":"2024-04-27T03:30:11.369568+0000","last_became_peered":"2024-04-27T03:30:11.369568+0000","last_unstale":"2024-04-27T03:31:01.436165+0000","last_undegraded":"2024-04-27T03:31:01.436165+0000","last_fullsized":"2024-04-27T03:31:01.436165+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:51.203886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438979+0000","last_change":"2024-04-27T03:30:06.107954+0000","last_active":"2024-04-27T03:31:01.438979+0000","last_peered":"2024-04-27T03:31:01.438979+0000","last_clean":"2024-04-27T03:31:01.438979+0000","last_became_active":"2024-04-27T03:30:06.107600+0000","last_became_peered":"2024-04-27T03:30:06.107600+0000","last_unstale":"2024-04-27T03:31:01.438979+0000","last_undegraded":"2024-04-27T03:31:01.438979+0000","last_fullsized":"2024-04-27T03:31:01.438979+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:44.876194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437548+0000","last_change":"2024-04-27T03:30:15.380881+0000","last_active":"2024-04-27T03:31:01.437548+0000","last_peered":"2024-04-27T03:31:01.437548+0000","last_clean":"2024-04-27T03:31:01.437548+0000","last_became_active":"2024-04-27T03:30:15.380598+0000","last_became_peered":"2024-04-27T03:30:15.380598+0000","last_unstale":"2024-04-27T03:31:01.437548+0000","last_undegraded":"2024-04-27T03:31:01.437548+0000","last_fullsized":"2024-04-27T03:31:01.437548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:10.183004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"27'89","reported_seq":167,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294574+0000","last_change":"2024-04-27T03:30:17.391634+0000","last_active":"2024-04-27T03:32:24.294574+0000","last_peered":"2024-04-27T03:32:24.294574+0000","last_clean":"2024-04-27T03:32:24.294574+0000","last_became_active":"2024-04-27T03:30:17.391191+0000","last_became_peered":"2024-04-27T03:30:17.391191+0000","last_unstale":"2024-04-27T03:32:24.294574+0000","last_undegraded":"2024-04-27T03:32:24.294574+0000","last_fullsized":"2024-04-27T03:32:24.294574+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":89,"log_dups_size":0,"ondisk_log_size":89,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:12.065352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":84,"num_read_kb":69,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431584+0000","last_change":"2024-04-27T03:30:19.404491+0000","last_active":"2024-04-27T03:31:01.431584+0000","last_peered":"2024-04-27T03:31:01.431584+0000","last_clean":"2024-04-27T03:31:01.431584+0000","last_became_active":"2024-04-27T03:30:19.404187+0000","last_became_peered":"2024-04-27T03:30:19.404187+0000","last_unstale":"2024-04-27T03:31:01.431584+0000","last_undegraded":"2024-04-27T03:31:01.431584+0000","last_fullsized":"2024-04-27T03:31:01.431584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:21.528207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432393+0000","last_change":"2024-04-27T03:30:21.408168+0000","last_active":"2024-04-27T03:31:01.432393+0000","last_peered":"2024-04-27T03:31:01.432393+0000","last_clean":"2024-04-27T03:31:01.432393+0000","last_became_active":"2024-04-27T03:30:21.407857+0000","last_became_peered":"2024-04-27T03:30:21.407857+0000","last_unstale":"2024-04-27T03:31:01.432393+0000","last_undegraded":"2024-04-27T03:31:01.432393+0000","last_fullsized":"2024-04-27T03:31:01.432393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:28.740636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"25'2","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432483+0000","last_change":"2024-04-27T03:30:21.411358+0000","last_active":"2024-04-27T03:31:01.432483+0000","last_peered":"2024-04-27T03:31:01.432483+0000","last_clean":"2024-04-27T03:31:01.432483+0000","last_became_active":"2024-04-27T03:30:21.411054+0000","last_became_peered":"2024-04-27T03:30:21.411054+0000","last_unstale":"2024-04-27T03:31:01.432483+0000","last_undegraded":"2024-04-27T03:31:01.432483+0000","last_fullsized":"2024-04-27T03:31:01.432483+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:44:47.262023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.653891+0000","last_change":"2024-04-27T03:30:13.383003+0000","last_active":"2024-04-27T03:31:36.653891+0000","last_peered":"2024-04-27T03:31:36.653891+0000","last_clean":"2024-04-27T03:31:36.653891+0000","last_became_active":"2024-04-27T03:30:13.382675+0000","last_became_peered":"2024-04-27T03:30:13.382675+0000","last_unstale":"2024-04-27T03:31:36.653891+0000","last_undegraded":"2024-04-27T03:31:36.653891+0000","last_fullsized":"2024-04-27T03:31:36.653891+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:21.911691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438857+0000","last_change":"2024-04-27T03:30:11.359531+0000","last_active":"2024-04-27T03:31:01.438857+0000","last_peered":"2024-04-27T03:31:01.438857+0000","last_clean":"2024-04-27T03:31:01.438857+0000","last_became_active":"2024-04-27T03:30:11.359195+0000","last_became_peered":"2024-04-27T03:30:11.359195+0000","last_unstale":"2024-04-27T03:31:01.438857+0000","last_undegraded":"2024-04-27T03:31:01.438857+0000","last_fullsized":"2024-04-27T03:31:01.438857+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:52.318456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434515+0000","last_change":"2024-04-27T03:30:15.379854+0000","last_active":"2024-04-27T03:31:01.434515+0000","last_peered":"2024-04-27T03:31:01.434515+0000","last_clean":"2024-04-27T03:31:01.434515+0000","last_became_active":"2024-04-27T03:30:15.379541+0000","last_became_peered":"2024-04-27T03:30:15.379541+0000","last_unstale":"2024-04-27T03:31:01.434515+0000","last_undegraded":"2024-04-27T03:31:01.434515+0000","last_fullsized":"2024-04-27T03:31:01.434515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:32:11.127449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"27'38","reported_seq":87,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.919746+0000","last_change":"2024-04-27T03:30:17.386926+0000","last_active":"2024-04-27T03:32:18.919746+0000","last_peered":"2024-04-27T03:32:18.919746+0000","last_clean":"2024-04-27T03:32:18.919746+0000","last_became_active":"2024-04-27T03:30:17.386496+0000","last_became_peered":"2024-04-27T03:30:17.386496+0000","last_unstale":"2024-04-27T03:32:18.919746+0000","last_undegraded":"2024-04-27T03:32:18.919746+0000","last_fullsized":"2024-04-27T03:32:18.919746+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:56.661166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":41,"num_read_kb":33,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437333+0000","last_change":"2024-04-27T03:30:19.405864+0000","last_active":"2024-04-27T03:31:01.437333+0000","last_peered":"2024-04-27T03:31:01.437333+0000","last_clean":"2024-04-27T03:31:01.437333+0000","last_became_active":"2024-04-27T03:30:19.405189+0000","last_became_peered":"2024-04-27T03:30:19.405189+0000","last_unstale":"2024-04-27T03:31:01.437333+0000","last_undegraded":"2024-04-27T03:31:01.437333+0000","last_fullsized":"2024-04-27T03:31:01.437333+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:23.134716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"27'3","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.594114+0000","last_change":"2024-04-27T03:30:21.414203+0000","last_active":"2024-04-27T03:32:18.594114+0000","last_peered":"2024-04-27T03:32:18.594114+0000","last_clean":"2024-04-27T03:32:18.594114+0000","last_became_active":"2024-04-27T03:30:21.413920+0000","last_became_peered":"2024-04-27T03:30:21.413920+0000","last_unstale":"2024-04-27T03:32:18.594114+0000","last_undegraded":"2024-04-27T03:32:18.594114+0000","last_fullsized":"2024-04-27T03:32:18.594114+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:40.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"27'16","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981976+0000","last_change":"2024-04-27T03:30:13.381187+0000","last_active":"2024-04-27T03:33:20.981976+0000","last_peered":"2024-04-27T03:33:20.981976+0000","last_clean":"2024-04-27T03:33:20.981976+0000","last_became_active":"2024-04-27T03:30:13.380240+0000","last_became_peered":"2024-04-27T03:30:13.380240+0000","last_unstale":"2024-04-27T03:33:20.981976+0000","last_undegraded":"2024-04-27T03:33:20.981976+0000","last_fullsized":"2024-04-27T03:33:20.981976+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:56.024650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":31,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433005+0000","last_change":"2024-04-27T03:30:11.361106+0000","last_active":"2024-04-27T03:31:01.433005+0000","last_peered":"2024-04-27T03:31:01.433005+0000","last_clean":"2024-04-27T03:31:01.433005+0000","last_became_active":"2024-04-27T03:30:11.360217+0000","last_became_peered":"2024-04-27T03:30:11.360217+0000","last_unstale":"2024-04-27T03:31:01.433005+0000","last_undegraded":"2024-04-27T03:31:01.433005+0000","last_fullsized":"2024-04-27T03:31:01.433005+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:51.203886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439951+0000","last_change":"2024-04-27T03:30:15.381885+0000","last_active":"2024-04-27T03:31:01.439951+0000","last_peered":"2024-04-27T03:31:01.439951+0000","last_clean":"2024-04-27T03:31:01.439951+0000","last_became_active":"2024-04-27T03:30:15.381507+0000","last_became_peered":"2024-04-27T03:30:15.381507+0000","last_unstale":"2024-04-27T03:31:01.439951+0000","last_undegraded":"2024-04-27T03:31:01.439951+0000","last_fullsized":"2024-04-27T03:31:01.439951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:31:37.867070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.324334+0000","last_change":"2024-04-27T03:30:17.385578+0000","last_active":"2024-04-27T03:33:20.324334+0000","last_peered":"2024-04-27T03:33:20.324334+0000","last_clean":"2024-04-27T03:33:20.324334+0000","last_became_active":"2024-04-27T03:30:17.385318+0000","last_became_peered":"2024-04-27T03:30:17.385318+0000","last_unstale":"2024-04-27T03:33:20.324334+0000","last_undegraded":"2024-04-27T03:33:20.324334+0000","last_fullsized":"2024-04-27T03:33:20.324334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:47.385488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435327+0000","last_change":"2024-04-27T03:30:19.407084+0000","last_active":"2024-04-27T03:31:01.435327+0000","last_peered":"2024-04-27T03:31:01.435327+0000","last_clean":"2024-04-27T03:31:01.435327+0000","last_became_active":"2024-04-27T03:30:19.406736+0000","last_became_peered":"2024-04-27T03:30:19.406736+0000","last_unstale":"2024-04-27T03:31:01.435327+0000","last_undegraded":"2024-04-27T03:31:01.435327+0000","last_fullsized":"2024-04-27T03:31:01.435327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:21.217528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"27'4","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.798144+0000","last_change":"2024-04-27T03:30:21.408419+0000","last_active":"2024-04-27T03:33:20.798144+0000","last_peered":"2024-04-27T03:33:20.798144+0000","last_clean":"2024-04-27T03:33:20.798144+0000","last_became_active":"2024-04-27T03:30:21.408079+0000","last_became_peered":"2024-04-27T03:30:21.408079+0000","last_unstale":"2024-04-27T03:33:20.798144+0000","last_undegraded":"2024-04-27T03:33:20.798144+0000","last_fullsized":"2024-04-27T03:33:20.798144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.646754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009922+0000","last_change":"2024-04-27T03:30:13.375656+0000","last_active":"2024-04-27T03:33:21.009922+0000","last_peered":"2024-04-27T03:33:21.009922+0000","last_clean":"2024-04-27T03:33:21.009922+0000","last_became_active":"2024-04-27T03:30:13.375350+0000","last_became_peered":"2024-04-27T03:30:13.375350+0000","last_unstale":"2024-04-27T03:33:21.009922+0000","last_undegraded":"2024-04-27T03:33:21.009922+0000","last_fullsized":"2024-04-27T03:33:21.009922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:13.288239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434851+0000","last_change":"2024-04-27T03:30:11.361187+0000","last_active":"2024-04-27T03:31:01.434851+0000","last_peered":"2024-04-27T03:31:01.434851+0000","last_clean":"2024-04-27T03:31:01.434851+0000","last_became_active":"2024-04-27T03:30:11.360938+0000","last_became_peered":"2024-04-27T03:30:11.360938+0000","last_unstale":"2024-04-27T03:31:01.434851+0000","last_undegraded":"2024-04-27T03:31:01.434851+0000","last_fullsized":"2024-04-27T03:31:01.434851+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:11.138117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431691+0000","last_change":"2024-04-27T03:30:15.382918+0000","last_active":"2024-04-27T03:31:01.431691+0000","last_peered":"2024-04-27T03:31:01.431691+0000","last_clean":"2024-04-27T03:31:01.431691+0000","last_became_active":"2024-04-27T03:30:15.382287+0000","last_became_peered":"2024-04-27T03:30:15.382287+0000","last_unstale":"2024-04-27T03:31:01.431691+0000","last_undegraded":"2024-04-27T03:31:01.431691+0000","last_fullsized":"2024-04-27T03:31:01.431691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:21.227251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"27'40","reported_seq":99,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.385027+0000","last_change":"2024-04-27T03:30:17.388554+0000","last_active":"2024-04-27T03:33:20.385027+0000","last_peered":"2024-04-27T03:33:20.385027+0000","last_clean":"2024-04-27T03:33:20.385027+0000","last_became_active":"2024-04-27T03:30:17.388260+0000","last_became_peered":"2024-04-27T03:30:17.388260+0000","last_unstale":"2024-04-27T03:33:20.385027+0000","last_undegraded":"2024-04-27T03:33:20.385027+0000","last_fullsized":"2024-04-27T03:33:20.385027+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:22.810377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":64,"num_write":46,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432676+0000","last_change":"2024-04-27T03:30:19.402729+0000","last_active":"2024-04-27T03:31:01.432676+0000","last_peered":"2024-04-27T03:31:01.432676+0000","last_clean":"2024-04-27T03:31:01.432676+0000","last_became_active":"2024-04-27T03:30:19.402430+0000","last_became_peered":"2024-04-27T03:30:19.402430+0000","last_unstale":"2024-04-27T03:31:01.432676+0000","last_undegraded":"2024-04-27T03:31:01.432676+0000","last_fullsized":"2024-04-27T03:31:01.432676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:59.540815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438223+0000","last_change":"2024-04-27T03:30:21.416316+0000","last_active":"2024-04-27T03:31:01.438223+0000","last_peered":"2024-04-27T03:31:01.438223+0000","last_clean":"2024-04-27T03:31:01.438223+0000","last_became_active":"2024-04-27T03:30:21.415984+0000","last_became_peered":"2024-04-27T03:30:21.415984+0000","last_unstale":"2024-04-27T03:31:01.438223+0000","last_undegraded":"2024-04-27T03:31:01.438223+0000","last_fullsized":"2024-04-27T03:31:01.438223+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:30.270957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988543+0000","last_change":"2024-04-27T03:30:13.387737+0000","last_active":"2024-04-27T03:33:20.988543+0000","last_peered":"2024-04-27T03:33:20.988543+0000","last_clean":"2024-04-27T03:33:20.988543+0000","last_became_active":"2024-04-27T03:30:13.386985+0000","last_became_peered":"2024-04-27T03:30:13.386985+0000","last_unstale":"2024-04-27T03:33:20.988543+0000","last_undegraded":"2024-04-27T03:33:20.988543+0000","last_fullsized":"2024-04-27T03:33:20.988543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:43.363544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436347+0000","last_change":"2024-04-27T03:30:11.370182+0000","last_active":"2024-04-27T03:31:01.436347+0000","last_peered":"2024-04-27T03:31:01.436347+0000","last_clean":"2024-04-27T03:31:01.436347+0000","last_became_active":"2024-04-27T03:30:11.369824+0000","last_became_peered":"2024-04-27T03:30:11.369824+0000","last_unstale":"2024-04-27T03:31:01.436347+0000","last_undegraded":"2024-04-27T03:31:01.436347+0000","last_fullsized":"2024-04-27T03:31:01.436347+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:11.138117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438418+0000","last_change":"2024-04-27T03:30:15.384051+0000","last_active":"2024-04-27T03:31:01.438418+0000","last_peered":"2024-04-27T03:31:01.438418+0000","last_clean":"2024-04-27T03:31:01.438418+0000","last_became_active":"2024-04-27T03:30:15.383771+0000","last_became_peered":"2024-04-27T03:30:15.383771+0000","last_unstale":"2024-04-27T03:31:01.438418+0000","last_undegraded":"2024-04-27T03:31:01.438418+0000","last_fullsized":"2024-04-27T03:31:01.438418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:32.616135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"27'30","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.904671+0000","last_change":"2024-04-27T03:30:17.385866+0000","last_active":"2024-04-27T03:32:18.904671+0000","last_peered":"2024-04-27T03:32:18.904671+0000","last_clean":"2024-04-27T03:32:18.904671+0000","last_became_active":"2024-04-27T03:30:17.385488+0000","last_became_peered":"2024-04-27T03:30:17.385488+0000","last_unstale":"2024-04-27T03:32:18.904671+0000","last_undegraded":"2024-04-27T03:32:18.904671+0000","last_fullsized":"2024-04-27T03:32:18.904671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:39.510614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":29,"num_read_kb":23,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437192+0000","last_change":"2024-04-27T03:30:19.398792+0000","last_active":"2024-04-27T03:31:01.437192+0000","last_peered":"2024-04-27T03:31:01.437192+0000","last_clean":"2024-04-27T03:31:01.437192+0000","last_became_active":"2024-04-27T03:30:19.398484+0000","last_became_peered":"2024-04-27T03:30:19.398484+0000","last_unstale":"2024-04-27T03:31:01.437192+0000","last_undegraded":"2024-04-27T03:31:01.437192+0000","last_fullsized":"2024-04-27T03:31:01.437192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:16.023756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"25'2","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440063+0000","last_change":"2024-04-27T03:30:13.377494+0000","last_active":"2024-04-27T03:31:01.440063+0000","last_peered":"2024-04-27T03:31:01.440063+0000","last_clean":"2024-04-27T03:31:01.440063+0000","last_became_active":"2024-04-27T03:30:13.377217+0000","last_became_peered":"2024-04-27T03:30:13.377217+0000","last_unstale":"2024-04-27T03:31:01.440063+0000","last_undegraded":"2024-04-27T03:31:01.440063+0000","last_fullsized":"2024-04-27T03:31:01.440063+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:18.788433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"25'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436454+0000","last_change":"2024-04-27T03:30:11.370820+0000","last_active":"2024-04-27T03:31:01.436454+0000","last_peered":"2024-04-27T03:31:01.436454+0000","last_clean":"2024-04-27T03:31:01.436454+0000","last_became_active":"2024-04-27T03:30:11.370547+0000","last_became_peered":"2024-04-27T03:30:11.370547+0000","last_unstale":"2024-04-27T03:31:01.436454+0000","last_undegraded":"2024-04-27T03:31:01.436454+0000","last_fullsized":"2024-04-27T03:31:01.436454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:34.092880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433313+0000","last_change":"2024-04-27T03:30:15.382292+0000","last_active":"2024-04-27T03:31:01.433313+0000","last_peered":"2024-04-27T03:31:01.433313+0000","last_clean":"2024-04-27T03:31:01.433313+0000","last_became_active":"2024-04-27T03:30:15.381956+0000","last_became_peered":"2024-04-27T03:30:15.381956+0000","last_unstale":"2024-04-27T03:31:01.433313+0000","last_undegraded":"2024-04-27T03:31:01.433313+0000","last_fullsized":"2024-04-27T03:31:01.433313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:10.054835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"27'60","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.925854+0000","last_change":"2024-04-27T03:30:17.383982+0000","last_active":"2024-04-27T03:32:18.925854+0000","last_peered":"2024-04-27T03:32:18.925854+0000","last_clean":"2024-04-27T03:32:18.925854+0000","last_became_active":"2024-04-27T03:30:17.383178+0000","last_became_peered":"2024-04-27T03:30:17.383178+0000","last_unstale":"2024-04-27T03:32:18.925854+0000","last_undegraded":"2024-04-27T03:32:18.925854+0000","last_fullsized":"2024-04-27T03:32:18.925854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:14.015528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":58,"num_read_kb":48,"num_write":24,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433851+0000","last_change":"2024-04-27T03:30:19.403087+0000","last_active":"2024-04-27T03:31:01.433851+0000","last_peered":"2024-04-27T03:31:01.433851+0000","last_clean":"2024-04-27T03:31:01.433851+0000","last_became_active":"2024-04-27T03:30:19.402781+0000","last_became_peered":"2024-04-27T03:30:19.402781+0000","last_unstale":"2024-04-27T03:31:01.433851+0000","last_undegraded":"2024-04-27T03:31:01.433851+0000","last_fullsized":"2024-04-27T03:31:01.433851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:30.540461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"27'16","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.240928+0000","last_change":"2024-04-27T03:30:21.408415+0000","last_active":"2024-04-27T03:32:20.240928+0000","last_peered":"2024-04-27T03:32:20.240928+0000","last_clean":"2024-04-27T03:32:20.240928+0000","last_became_active":"2024-04-27T03:30:21.408049+0000","last_became_peered":"2024-04-27T03:30:21.408049+0000","last_unstale":"2024-04-27T03:32:20.240928+0000","last_undegraded":"2024-04-27T03:32:20.240928+0000","last_fullsized":"2024-04-27T03:32:20.240928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:09.753400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":21,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432877+0000","last_change":"2024-04-27T03:30:13.373552+0000","last_active":"2024-04-27T03:31:01.432877+0000","last_peered":"2024-04-27T03:31:01.432877+0000","last_clean":"2024-04-27T03:31:01.432877+0000","last_became_active":"2024-04-27T03:30:13.373257+0000","last_became_peered":"2024-04-27T03:30:13.373257+0000","last_unstale":"2024-04-27T03:31:01.432877+0000","last_undegraded":"2024-04-27T03:31:01.432877+0000","last_fullsized":"2024-04-27T03:31:01.432877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:07:51.011104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432526+0000","last_change":"2024-04-27T03:30:11.371431+0000","last_active":"2024-04-27T03:31:01.432526+0000","last_peered":"2024-04-27T03:31:01.432526+0000","last_clean":"2024-04-27T03:31:01.432526+0000","last_became_active":"2024-04-27T03:30:11.370091+0000","last_became_peered":"2024-04-27T03:30:11.370091+0000","last_unstale":"2024-04-27T03:31:01.432526+0000","last_undegraded":"2024-04-27T03:31:01.432526+0000","last_fullsized":"2024-04-27T03:31:01.432526+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.746108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432766+0000","last_change":"2024-04-27T03:30:15.380494+0000","last_active":"2024-04-27T03:31:01.432766+0000","last_peered":"2024-04-27T03:31:01.432766+0000","last_clean":"2024-04-27T03:31:01.432766+0000","last_became_active":"2024-04-27T03:30:15.379800+0000","last_became_peered":"2024-04-27T03:30:15.379800+0000","last_unstale":"2024-04-27T03:31:01.432766+0000","last_undegraded":"2024-04-27T03:31:01.432766+0000","last_fullsized":"2024-04-27T03:31:01.432766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:58.036077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"27'36","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.905630+0000","last_change":"2024-04-27T03:30:17.389821+0000","last_active":"2024-04-27T03:32:18.905630+0000","last_peered":"2024-04-27T03:32:18.905630+0000","last_clean":"2024-04-27T03:32:18.905630+0000","last_became_active":"2024-04-27T03:30:17.388246+0000","last_became_peered":"2024-04-27T03:30:17.388246+0000","last_unstale":"2024-04-27T03:32:18.905630+0000","last_undegraded":"2024-04-27T03:32:18.905630+0000","last_fullsized":"2024-04-27T03:32:18.905630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:10:57.956151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":38,"num_read_kb":29,"num_write":18,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437582+0000","last_change":"2024-04-27T03:30:19.407011+0000","last_active":"2024-04-27T03:31:01.437582+0000","last_peered":"2024-04-27T03:31:01.437582+0000","last_clean":"2024-04-27T03:31:01.437582+0000","last_became_active":"2024-04-27T03:30:19.406707+0000","last_became_peered":"2024-04-27T03:30:19.406707+0000","last_unstale":"2024-04-27T03:31:01.437582+0000","last_undegraded":"2024-04-27T03:31:01.437582+0000","last_fullsized":"2024-04-27T03:31:01.437582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:59.990733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436390+0000","last_change":"2024-04-27T03:30:21.415821+0000","last_active":"2024-04-27T03:31:01.436390+0000","last_peered":"2024-04-27T03:31:01.436390+0000","last_clean":"2024-04-27T03:31:01.436390+0000","last_became_active":"2024-04-27T03:30:21.415441+0000","last_became_peered":"2024-04-27T03:30:21.415441+0000","last_unstale":"2024-04-27T03:31:01.436390+0000","last_undegraded":"2024-04-27T03:31:01.436390+0000","last_fullsized":"2024-04-27T03:31:01.436390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:03.042667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933527+0000","last_change":"2024-04-27T03:30:13.388229+0000","last_active":"2024-04-27T03:33:20.933527+0000","last_peered":"2024-04-27T03:33:20.933527+0000","last_clean":"2024-04-27T03:33:20.933527+0000","last_became_active":"2024-04-27T03:30:13.387397+0000","last_became_peered":"2024-04-27T03:30:13.387397+0000","last_unstale":"2024-04-27T03:33:20.933527+0000","last_undegraded":"2024-04-27T03:33:20.933527+0000","last_fullsized":"2024-04-27T03:33:20.933527+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:45:47.437394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432931+0000","last_change":"2024-04-27T03:30:11.370706+0000","last_active":"2024-04-27T03:31:01.432931+0000","last_peered":"2024-04-27T03:31:01.432931+0000","last_clean":"2024-04-27T03:31:01.432931+0000","last_became_active":"2024-04-27T03:30:11.369799+0000","last_became_peered":"2024-04-27T03:30:11.369799+0000","last_unstale":"2024-04-27T03:31:01.432931+0000","last_undegraded":"2024-04-27T03:31:01.432931+0000","last_fullsized":"2024-04-27T03:31:01.432931+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.539212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433610+0000","last_change":"2024-04-27T03:30:15.380214+0000","last_active":"2024-04-27T03:31:01.433610+0000","last_peered":"2024-04-27T03:31:01.433610+0000","last_clean":"2024-04-27T03:31:01.433610+0000","last_became_active":"2024-04-27T03:30:15.379873+0000","last_became_peered":"2024-04-27T03:30:15.379873+0000","last_unstale":"2024-04-27T03:31:01.433610+0000","last_undegraded":"2024-04-27T03:31:01.433610+0000","last_fullsized":"2024-04-27T03:31:01.433610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:53.446024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"27'40","reported_seq":89,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.923662+0000","last_change":"2024-04-27T03:30:17.387440+0000","last_active":"2024-04-27T03:32:18.923662+0000","last_peered":"2024-04-27T03:32:18.923662+0000","last_clean":"2024-04-27T03:32:18.923662+0000","last_became_active":"2024-04-27T03:30:17.387205+0000","last_became_peered":"2024-04-27T03:30:17.387205+0000","last_unstale":"2024-04-27T03:32:18.923662+0000","last_undegraded":"2024-04-27T03:32:18.923662+0000","last_fullsized":"2024-04-27T03:32:18.923662+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":40,"log_dups_size":0,"ondisk_log_size":40,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:26.079008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":54,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432503+0000","last_change":"2024-04-27T03:30:19.405759+0000","last_active":"2024-04-27T03:31:01.432503+0000","last_peered":"2024-04-27T03:31:01.432503+0000","last_clean":"2024-04-27T03:31:01.432503+0000","last_became_active":"2024-04-27T03:30:19.405462+0000","last_became_peered":"2024-04-27T03:30:19.405462+0000","last_unstale":"2024-04-27T03:31:01.432503+0000","last_undegraded":"2024-04-27T03:31:01.432503+0000","last_fullsized":"2024-04-27T03:31:01.432503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:59.990733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"27'7","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.302334+0000","last_change":"2024-04-27T03:30:21.416894+0000","last_active":"2024-04-27T03:32:20.302334+0000","last_peered":"2024-04-27T03:32:20.302334+0000","last_clean":"2024-04-27T03:32:20.302334+0000","last_became_active":"2024-04-27T03:30:21.416592+0000","last_became_peered":"2024-04-27T03:30:21.416592+0000","last_unstale":"2024-04-27T03:32:20.302334+0000","last_undegraded":"2024-04-27T03:32:20.302334+0000","last_fullsized":"2024-04-27T03:32:20.302334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:24:21.730546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"27'6","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.304764+0000","last_change":"2024-04-27T03:30:13.370253+0000","last_active":"2024-04-27T03:32:20.304764+0000","last_peered":"2024-04-27T03:32:20.304764+0000","last_clean":"2024-04-27T03:32:20.304764+0000","last_became_active":"2024-04-27T03:30:13.369643+0000","last_became_peered":"2024-04-27T03:30:13.369643+0000","last_unstale":"2024-04-27T03:32:20.304764+0000","last_undegraded":"2024-04-27T03:32:20.304764+0000","last_fullsized":"2024-04-27T03:32:20.304764+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:28.256872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298559+0000","last_change":"2024-04-27T03:30:11.372867+0000","last_active":"2024-04-27T03:32:44.298559+0000","last_peered":"2024-04-27T03:32:44.298559+0000","last_clean":"2024-04-27T03:32:44.298559+0000","last_became_active":"2024-04-27T03:30:11.372220+0000","last_became_peered":"2024-04-27T03:30:11.372220+0000","last_unstale":"2024-04-27T03:32:44.298559+0000","last_undegraded":"2024-04-27T03:32:44.298559+0000","last_fullsized":"2024-04-27T03:32:44.298559+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:43.207328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":639872,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437382+0000","last_change":"2024-04-27T03:30:15.380858+0000","last_active":"2024-04-27T03:31:01.437382+0000","last_peered":"2024-04-27T03:31:01.437382+0000","last_clean":"2024-04-27T03:31:01.437382+0000","last_became_active":"2024-04-27T03:30:15.380573+0000","last_became_peered":"2024-04-27T03:30:15.380573+0000","last_unstale":"2024-04-27T03:31:01.437382+0000","last_undegraded":"2024-04-27T03:31:01.437382+0000","last_fullsized":"2024-04-27T03:31:01.437382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:45.374972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"25'16","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438994+0000","last_change":"2024-04-27T03:30:17.389436+0000","last_active":"2024-04-27T03:31:01.438994+0000","last_peered":"2024-04-27T03:31:01.438994+0000","last_clean":"2024-04-27T03:31:01.438994+0000","last_became_active":"2024-04-27T03:30:17.388289+0000","last_became_peered":"2024-04-27T03:30:17.388289+0000","last_unstale":"2024-04-27T03:31:01.438994+0000","last_undegraded":"2024-04-27T03:31:01.438994+0000","last_fullsized":"2024-04-27T03:31:01.438994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:02:33.022377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"27'282","reported_seq":372,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462759+0000","last_change":"2024-04-27T03:30:19.407449+0000","last_active":"2024-04-27T03:33:20.462759+0000","last_peered":"2024-04-27T03:33:20.462759+0000","last_clean":"2024-04-27T03:33:20.462759+0000","last_became_active":"2024-04-27T03:30:19.406854+0000","last_became_peered":"2024-04-27T03:30:19.406854+0000","last_unstale":"2024-04-27T03:33:20.462759+0000","last_undegraded":"2024-04-27T03:33:20.462759+0000","last_fullsized":"2024-04-27T03:33:20.462759+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":282,"log_dups_size":0,"ondisk_log_size":282,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:55.112238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435927+0000","last_change":"2024-04-27T03:30:21.415992+0000","last_active":"2024-04-27T03:31:01.435927+0000","last_peered":"2024-04-27T03:31:01.435927+0000","last_clean":"2024-04-27T03:31:01.435927+0000","last_became_active":"2024-04-27T03:30:21.415534+0000","last_became_peered":"2024-04-27T03:30:21.415534+0000","last_unstale":"2024-04-27T03:31:01.435927+0000","last_undegraded":"2024-04-27T03:31:01.435927+0000","last_fullsized":"2024-04-27T03:31:01.435927+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:16.401751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"27'24","reported_seq":102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981535+0000","last_change":"2024-04-27T03:30:13.371535+0000","last_active":"2024-04-27T03:33:20.981535+0000","last_peered":"2024-04-27T03:33:20.981535+0000","last_clean":"2024-04-27T03:33:20.981535+0000","last_became_active":"2024-04-27T03:30:13.371158+0000","last_became_peered":"2024-04-27T03:30:13.371158+0000","last_unstale":"2024-04-27T03:33:20.981535+0000","last_undegraded":"2024-04-27T03:33:20.981535+0000","last_fullsized":"2024-04-27T03:33:20.981535+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:11:26.763810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":88,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435110+0000","last_change":"2024-04-27T03:30:11.359205+0000","last_active":"2024-04-27T03:31:01.435110+0000","last_peered":"2024-04-27T03:31:01.435110+0000","last_clean":"2024-04-27T03:31:01.435110+0000","last_became_active":"2024-04-27T03:30:11.358893+0000","last_became_peered":"2024-04-27T03:30:11.358893+0000","last_unstale":"2024-04-27T03:31:01.435110+0000","last_undegraded":"2024-04-27T03:31:01.435110+0000","last_fullsized":"2024-04-27T03:31:01.435110+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:52:31.047433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436411+0000","last_change":"2024-04-27T03:30:15.381535+0000","last_active":"2024-04-27T03:31:01.436411+0000","last_peered":"2024-04-27T03:31:01.436411+0000","last_clean":"2024-04-27T03:31:01.436411+0000","last_became_active":"2024-04-27T03:30:15.381175+0000","last_became_peered":"2024-04-27T03:30:15.381175+0000","last_unstale":"2024-04-27T03:31:01.436411+0000","last_undegraded":"2024-04-27T03:31:01.436411+0000","last_fullsized":"2024-04-27T03:31:01.436411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:21:29.464234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"27'60","reported_seq":129,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.896020+0000","last_change":"2024-04-27T03:30:17.386967+0000","last_active":"2024-04-27T03:32:18.896020+0000","last_peered":"2024-04-27T03:32:18.896020+0000","last_clean":"2024-04-27T03:32:18.896020+0000","last_became_active":"2024-04-27T03:30:17.386673+0000","last_became_peered":"2024-04-27T03:30:17.386673+0000","last_unstale":"2024-04-27T03:32:18.896020+0000","last_undegraded":"2024-04-27T03:32:18.896020+0000","last_fullsized":"2024-04-27T03:32:18.896020+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":60,"log_dups_size":0,"ondisk_log_size":60,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:54:14.640233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":74,"num_read_kb":61,"num_write":42,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430575+0000","last_change":"2024-04-27T03:30:19.407289+0000","last_active":"2024-04-27T03:31:01.430575+0000","last_peered":"2024-04-27T03:31:01.430575+0000","last_clean":"2024-04-27T03:31:01.430575+0000","last_became_active":"2024-04-27T03:30:19.406990+0000","last_became_peered":"2024-04-27T03:30:19.406990+0000","last_unstale":"2024-04-27T03:31:01.430575+0000","last_undegraded":"2024-04-27T03:31:01.430575+0000","last_fullsized":"2024-04-27T03:31:01.430575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:56:49.549144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433183+0000","last_change":"2024-04-27T03:30:21.410357+0000","last_active":"2024-04-27T03:31:01.433183+0000","last_peered":"2024-04-27T03:31:01.433183+0000","last_clean":"2024-04-27T03:31:01.433183+0000","last_became_active":"2024-04-27T03:30:21.410060+0000","last_became_peered":"2024-04-27T03:30:21.410060+0000","last_unstale":"2024-04-27T03:31:01.433183+0000","last_undegraded":"2024-04-27T03:31:01.433183+0000","last_fullsized":"2024-04-27T03:31:01.433183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:40.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"27'10","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009762+0000","last_change":"2024-04-27T03:30:13.372186+0000","last_active":"2024-04-27T03:33:21.009762+0000","last_peered":"2024-04-27T03:33:21.009762+0000","last_clean":"2024-04-27T03:33:21.009762+0000","last_became_active":"2024-04-27T03:30:13.371866+0000","last_became_peered":"2024-04-27T03:30:13.371866+0000","last_unstale":"2024-04-27T03:33:21.009762+0000","last_undegraded":"2024-04-27T03:33:21.009762+0000","last_fullsized":"2024-04-27T03:33:21.009762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:45.213699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.929513+0000","last_change":"2024-04-27T03:30:11.362991+0000","last_active":"2024-04-27T03:33:20.929513+0000","last_peered":"2024-04-27T03:33:20.929513+0000","last_clean":"2024-04-27T03:33:20.929513+0000","last_became_active":"2024-04-27T03:30:11.362420+0000","last_became_peered":"2024-04-27T03:30:11.362420+0000","last_unstale":"2024-04-27T03:33:20.929513+0000","last_undegraded":"2024-04-27T03:33:20.929513+0000","last_fullsized":"2024-04-27T03:33:20.929513+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:16.411097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437111+0000","last_change":"2024-04-27T03:30:15.382620+0000","last_active":"2024-04-27T03:31:01.437111+0000","last_peered":"2024-04-27T03:31:01.437111+0000","last_clean":"2024-04-27T03:31:01.437111+0000","last_became_active":"2024-04-27T03:30:15.382305+0000","last_became_peered":"2024-04-27T03:30:15.382305+0000","last_unstale":"2024-04-27T03:31:01.437111+0000","last_undegraded":"2024-04-27T03:31:01.437111+0000","last_fullsized":"2024-04-27T03:31:01.437111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:04.836869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"27'38","reported_seq":86,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.292067+0000","last_change":"2024-04-27T03:30:17.390602+0000","last_active":"2024-04-27T03:32:44.292067+0000","last_peered":"2024-04-27T03:32:44.292067+0000","last_clean":"2024-04-27T03:32:44.292067+0000","last_became_active":"2024-04-27T03:30:17.390281+0000","last_became_peered":"2024-04-27T03:30:17.390281+0000","last_unstale":"2024-04-27T03:32:44.292067+0000","last_undegraded":"2024-04-27T03:32:44.292067+0000","last_fullsized":"2024-04-27T03:32:44.292067+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":38,"log_dups_size":0,"ondisk_log_size":38,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:20.307428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":5807,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":40,"num_read_kb":33,"num_write":20,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437067+0000","last_change":"2024-04-27T03:30:19.406218+0000","last_active":"2024-04-27T03:31:01.437067+0000","last_peered":"2024-04-27T03:31:01.437067+0000","last_clean":"2024-04-27T03:31:01.437067+0000","last_became_active":"2024-04-27T03:30:19.405353+0000","last_became_peered":"2024-04-27T03:30:19.405353+0000","last_unstale":"2024-04-27T03:31:01.437067+0000","last_undegraded":"2024-04-27T03:31:01.437067+0000","last_fullsized":"2024-04-27T03:31:01.437067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:21.451111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438395+0000","last_change":"2024-04-27T03:30:21.416876+0000","last_active":"2024-04-27T03:31:01.438395+0000","last_peered":"2024-04-27T03:31:01.438395+0000","last_clean":"2024-04-27T03:31:01.438395+0000","last_became_active":"2024-04-27T03:30:21.416276+0000","last_became_peered":"2024-04-27T03:30:21.416276+0000","last_unstale":"2024-04-27T03:31:01.438395+0000","last_undegraded":"2024-04-27T03:31:01.438395+0000","last_fullsized":"2024-04-27T03:31:01.438395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:58.245817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'24","reported_seq":101,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.697833+0000","last_change":"2024-04-27T03:30:13.377820+0000","last_active":"2024-04-27T03:33:20.697833+0000","last_peered":"2024-04-27T03:33:20.697833+0000","last_clean":"2024-04-27T03:33:20.697833+0000","last_became_active":"2024-04-27T03:30:13.376415+0000","last_became_peered":"2024-04-27T03:30:13.376415+0000","last_unstale":"2024-04-27T03:33:20.697833+0000","last_undegraded":"2024-04-27T03:33:20.697833+0000","last_fullsized":"2024-04-27T03:33:20.697833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:02.028024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.287615+0000","last_change":"2024-04-27T03:30:11.361140+0000","last_active":"2024-04-27T03:32:14.287615+0000","last_peered":"2024-04-27T03:32:14.287615+0000","last_clean":"2024-04-27T03:32:14.287615+0000","last_became_active":"2024-04-27T03:30:11.360590+0000","last_became_peered":"2024-04-27T03:30:11.360590+0000","last_unstale":"2024-04-27T03:32:14.287615+0000","last_undegraded":"2024-04-27T03:32:14.287615+0000","last_fullsized":"2024-04-27T03:32:14.287615+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:52:58.316121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434401+0000","last_change":"2024-04-27T03:30:15.385912+0000","last_active":"2024-04-27T03:31:01.434401+0000","last_peered":"2024-04-27T03:31:01.434401+0000","last_clean":"2024-04-27T03:31:01.434401+0000","last_became_active":"2024-04-27T03:30:15.385614+0000","last_became_peered":"2024-04-27T03:30:15.385614+0000","last_unstale":"2024-04-27T03:31:01.434401+0000","last_undegraded":"2024-04-27T03:31:01.434401+0000","last_fullsized":"2024-04-27T03:31:01.434401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:54:32.462460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"27'32","reported_seq":78,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.921601+0000","last_change":"2024-04-27T03:30:17.387854+0000","last_active":"2024-04-27T03:32:18.921601+0000","last_peered":"2024-04-27T03:32:18.921601+0000","last_clean":"2024-04-27T03:32:18.921601+0000","last_became_active":"2024-04-27T03:30:17.386983+0000","last_became_peered":"2024-04-27T03:30:17.386983+0000","last_unstale":"2024-04-27T03:32:18.921601+0000","last_undegraded":"2024-04-27T03:32:18.921601+0000","last_fullsized":"2024-04-27T03:32:18.921601+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":32,"log_dups_size":0,"ondisk_log_size":32,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:38.038360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":32,"num_read_kb":27,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436657+0000","last_change":"2024-04-27T03:30:19.402931+0000","last_active":"2024-04-27T03:31:01.436657+0000","last_peered":"2024-04-27T03:31:01.436657+0000","last_clean":"2024-04-27T03:31:01.436657+0000","last_became_active":"2024-04-27T03:30:19.402652+0000","last_became_peered":"2024-04-27T03:30:19.402652+0000","last_unstale":"2024-04-27T03:31:01.436657+0000","last_undegraded":"2024-04-27T03:31:01.436657+0000","last_fullsized":"2024-04-27T03:31:01.436657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:24.199405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"27'6","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.720254+0000","last_change":"2024-04-27T03:30:21.420988+0000","last_active":"2024-04-27T03:33:20.720254+0000","last_peered":"2024-04-27T03:33:20.720254+0000","last_clean":"2024-04-27T03:33:20.720254+0000","last_became_active":"2024-04-27T03:30:21.420725+0000","last_became_peered":"2024-04-27T03:30:21.420725+0000","last_unstale":"2024-04-27T03:33:20.720254+0000","last_undegraded":"2024-04-27T03:33:20.720254+0000","last_fullsized":"2024-04-27T03:33:20.720254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:02.117649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432186+0000","last_change":"2024-04-27T03:30:13.375384+0000","last_active":"2024-04-27T03:31:01.432186+0000","last_peered":"2024-04-27T03:31:01.432186+0000","last_clean":"2024-04-27T03:31:01.432186+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:31:01.432186+0000","last_undegraded":"2024-04-27T03:31:01.432186+0000","last_fullsized":"2024-04-27T03:31:01.432186+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:40:10.176600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436421+0000","last_change":"2024-04-27T03:30:11.370518+0000","last_active":"2024-04-27T03:31:01.436421+0000","last_peered":"2024-04-27T03:31:01.436421+0000","last_clean":"2024-04-27T03:31:01.436421+0000","last_became_active":"2024-04-27T03:30:11.369877+0000","last_became_peered":"2024-04-27T03:30:11.369877+0000","last_unstale":"2024-04-27T03:31:01.436421+0000","last_undegraded":"2024-04-27T03:31:01.436421+0000","last_fullsized":"2024-04-27T03:31:01.436421+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.059046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"27'2","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.550532+0000","last_change":"2024-04-27T03:30:15.382111+0000","last_active":"2024-04-27T03:32:17.550532+0000","last_peered":"2024-04-27T03:32:17.550532+0000","last_clean":"2024-04-27T03:32:17.550532+0000","last_became_active":"2024-04-27T03:30:15.381574+0000","last_became_peered":"2024-04-27T03:30:15.381574+0000","last_unstale":"2024-04-27T03:32:17.550532+0000","last_undegraded":"2024-04-27T03:32:17.550532+0000","last_fullsized":"2024-04-27T03:32:17.550532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:04.039451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"27'44","reported_seq":100,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.325275+0000","last_change":"2024-04-27T03:30:17.391410+0000","last_active":"2024-04-27T03:33:20.325275+0000","last_peered":"2024-04-27T03:33:20.325275+0000","last_clean":"2024-04-27T03:33:20.325275+0000","last_became_active":"2024-04-27T03:30:17.388277+0000","last_became_peered":"2024-04-27T03:30:17.388277+0000","last_unstale":"2024-04-27T03:33:20.325275+0000","last_undegraded":"2024-04-27T03:33:20.325275+0000","last_fullsized":"2024-04-27T03:33:20.325275+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":44,"log_dups_size":0,"ondisk_log_size":44,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:05.729566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":52,"num_read_kb":46,"num_write":30,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431300+0000","last_change":"2024-04-27T03:30:19.406905+0000","last_active":"2024-04-27T03:31:01.431300+0000","last_peered":"2024-04-27T03:31:01.431300+0000","last_clean":"2024-04-27T03:31:01.431300+0000","last_became_active":"2024-04-27T03:30:19.406565+0000","last_became_peered":"2024-04-27T03:30:19.406565+0000","last_unstale":"2024-04-27T03:31:01.431300+0000","last_undegraded":"2024-04-27T03:31:01.431300+0000","last_fullsized":"2024-04-27T03:31:01.431300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:31.846278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"27'4","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.934155+0000","last_change":"2024-04-27T03:30:21.414808+0000","last_active":"2024-04-27T03:32:18.934155+0000","last_peered":"2024-04-27T03:32:18.934155+0000","last_clean":"2024-04-27T03:32:18.934155+0000","last_became_active":"2024-04-27T03:30:21.414510+0000","last_became_peered":"2024-04-27T03:30:21.414510+0000","last_unstale":"2024-04-27T03:32:18.934155+0000","last_undegraded":"2024-04-27T03:32:18.934155+0000","last_fullsized":"2024-04-27T03:32:18.934155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:07.942157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"25'8","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436998+0000","last_change":"2024-04-27T03:30:13.387657+0000","last_active":"2024-04-27T03:31:01.436998+0000","last_peered":"2024-04-27T03:31:01.436998+0000","last_clean":"2024-04-27T03:31:01.436998+0000","last_became_active":"2024-04-27T03:30:13.387149+0000","last_became_peered":"2024-04-27T03:30:13.387149+0000","last_unstale":"2024-04-27T03:31:01.436998+0000","last_undegraded":"2024-04-27T03:31:01.436998+0000","last_fullsized":"2024-04-27T03:31:01.436998+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:00.325582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"25'6","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433385+0000","last_change":"2024-04-27T03:30:11.358174+0000","last_active":"2024-04-27T03:31:01.433385+0000","last_peered":"2024-04-27T03:31:01.433385+0000","last_clean":"2024-04-27T03:31:01.433385+0000","last_became_active":"2024-04-27T03:30:11.357878+0000","last_became_peered":"2024-04-27T03:30:11.357878+0000","last_unstale":"2024-04-27T03:31:01.433385+0000","last_undegraded":"2024-04-27T03:31:01.433385+0000","last_fullsized":"2024-04-27T03:31:01.433385+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:17.981757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.304915+0000","last_change":"2024-04-27T03:30:13.369979+0000","last_active":"2024-04-27T03:32:20.304915+0000","last_peered":"2024-04-27T03:32:20.304915+0000","last_clean":"2024-04-27T03:32:20.304915+0000","last_became_active":"2024-04-27T03:30:13.369681+0000","last_became_peered":"2024-04-27T03:30:13.369681+0000","last_unstale":"2024-04-27T03:32:20.304915+0000","last_undegraded":"2024-04-27T03:32:20.304915+0000","last_fullsized":"2024-04-27T03:32:20.304915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:37.770898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"25'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430839+0000","last_change":"2024-04-27T03:30:11.373422+0000","last_active":"2024-04-27T03:31:01.430839+0000","last_peered":"2024-04-27T03:31:01.430839+0000","last_clean":"2024-04-27T03:31:01.430839+0000","last_became_active":"2024-04-27T03:30:11.373186+0000","last_became_peered":"2024-04-27T03:30:11.373186+0000","last_unstale":"2024-04-27T03:31:01.430839+0000","last_undegraded":"2024-04-27T03:31:01.430839+0000","last_fullsized":"2024-04-27T03:31:01.430839+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:22.719424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933550+0000","last_change":"2024-04-27T03:30:13.382330+0000","last_active":"2024-04-27T03:33:20.933550+0000","last_peered":"2024-04-27T03:33:20.933550+0000","last_clean":"2024-04-27T03:33:20.933550+0000","last_became_active":"2024-04-27T03:30:13.382069+0000","last_became_peered":"2024-04-27T03:30:13.382069+0000","last_unstale":"2024-04-27T03:33:20.933550+0000","last_undegraded":"2024-04-27T03:33:20.933550+0000","last_fullsized":"2024-04-27T03:33:20.933550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:22.941552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"27'3","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.420503+0000","last_change":"2024-04-27T03:30:11.350547+0000","last_active":"2024-04-27T03:31:36.420503+0000","last_peered":"2024-04-27T03:31:36.420503+0000","last_clean":"2024-04-27T03:31:36.420503+0000","last_became_active":"2024-04-27T03:30:11.350295+0000","last_became_peered":"2024-04-27T03:30:11.350295+0000","last_unstale":"2024-04-27T03:31:36.420503+0000","last_undegraded":"2024-04-27T03:31:36.420503+0000","last_fullsized":"2024-04-27T03:31:36.420503+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:45:58.123163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933497+0000","last_change":"2024-04-27T03:30:13.370393+0000","last_active":"2024-04-27T03:33:20.933497+0000","last_peered":"2024-04-27T03:33:20.933497+0000","last_clean":"2024-04-27T03:33:20.933497+0000","last_became_active":"2024-04-27T03:30:13.370066+0000","last_became_peered":"2024-04-27T03:30:13.370066+0000","last_unstale":"2024-04-27T03:33:20.933497+0000","last_undegraded":"2024-04-27T03:33:20.933497+0000","last_fullsized":"2024-04-27T03:33:20.933497+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:33.811635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"27'3","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288156+0000","last_change":"2024-04-27T03:30:11.360775+0000","last_active":"2024-04-27T03:32:19.288156+0000","last_peered":"2024-04-27T03:32:19.288156+0000","last_clean":"2024-04-27T03:32:19.288156+0000","last_became_active":"2024-04-27T03:30:11.360376+0000","last_became_peered":"2024-04-27T03:30:11.360376+0000","last_unstale":"2024-04-27T03:32:19.288156+0000","last_undegraded":"2024-04-27T03:32:19.288156+0000","last_fullsized":"2024-04-27T03:32:19.288156+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:00.825717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":3,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.792398+0000","last_change":"2024-04-27T03:30:13.378662+0000","last_active":"2024-04-27T03:33:20.792398+0000","last_peered":"2024-04-27T03:33:20.792398+0000","last_clean":"2024-04-27T03:33:20.792398+0000","last_became_active":"2024-04-27T03:30:13.378368+0000","last_became_peered":"2024-04-27T03:30:13.378368+0000","last_unstale":"2024-04-27T03:33:20.792398+0000","last_undegraded":"2024-04-27T03:33:20.792398+0000","last_fullsized":"2024-04-27T03:33:20.792398+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:56.906775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432139+0000","last_change":"2024-04-27T03:30:11.371839+0000","last_active":"2024-04-27T03:31:01.432139+0000","last_peered":"2024-04-27T03:31:01.432139+0000","last_clean":"2024-04-27T03:31:01.432139+0000","last_became_active":"2024-04-27T03:30:11.371553+0000","last_became_peered":"2024-04-27T03:30:11.371553+0000","last_unstale":"2024-04-27T03:31:01.432139+0000","last_undegraded":"2024-04-27T03:31:01.432139+0000","last_fullsized":"2024-04-27T03:31:01.432139+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.059046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"27'10","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009740+0000","last_change":"2024-04-27T03:31:01.440910+0000","last_active":"2024-04-27T03:33:21.009740+0000","last_peered":"2024-04-27T03:33:21.009740+0000","last_clean":"2024-04-27T03:33:21.009740+0000","last_became_active":"2024-04-27T03:31:01.440576+0000","last_became_peered":"2024-04-27T03:31:01.440576+0000","last_unstale":"2024-04-27T03:33:21.009740+0000","last_undegraded":"2024-04-27T03:33:21.009740+0000","last_fullsized":"2024-04-27T03:33:21.009740+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:04.874525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.213761+0000","last_change":"2024-04-27T03:30:11.371034+0000","last_active":"2024-04-27T03:32:20.213761+0000","last_peered":"2024-04-27T03:32:20.213761+0000","last_clean":"2024-04-27T03:32:20.213761+0000","last_became_active":"2024-04-27T03:30:11.369517+0000","last_became_peered":"2024-04-27T03:30:11.369517+0000","last_unstale":"2024-04-27T03:32:20.213761+0000","last_undegraded":"2024-04-27T03:32:20.213761+0000","last_fullsized":"2024-04-27T03:32:20.213761+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:22.719424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.107522+0000","last_change":"2024-04-27T03:30:13.374070+0000","last_active":"2024-04-27T03:32:17.107522+0000","last_peered":"2024-04-27T03:32:17.107522+0000","last_clean":"2024-04-27T03:32:17.107522+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:32:17.107522+0000","last_undegraded":"2024-04-27T03:32:17.107522+0000","last_fullsized":"2024-04-27T03:32:17.107522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:54.324828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.442163+0000","last_change":"2024-04-27T03:31:01.441506+0000","last_active":"2024-04-27T03:31:01.442163+0000","last_peered":"2024-04-27T03:31:01.442163+0000","last_clean":"2024-04-27T03:31:01.442163+0000","last_became_active":"2024-04-27T03:31:01.441159+0000","last_became_peered":"2024-04-27T03:31:01.441159+0000","last_unstale":"2024-04-27T03:31:01.442163+0000","last_undegraded":"2024-04-27T03:31:01.442163+0000","last_fullsized":"2024-04-27T03:31:01.442163+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:11:09.529188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"27'3","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.802333+0000","last_change":"2024-04-27T03:30:13.378083+0000","last_active":"2024-04-27T03:33:20.802333+0000","last_peered":"2024-04-27T03:33:20.802333+0000","last_clean":"2024-04-27T03:33:20.802333+0000","last_became_active":"2024-04-27T03:30:13.377821+0000","last_became_peered":"2024-04-27T03:30:13.377821+0000","last_unstale":"2024-04-27T03:33:20.802333+0000","last_undegraded":"2024-04-27T03:33:20.802333+0000","last_fullsized":"2024-04-27T03:33:20.802333+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:18.417469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432873+0000","last_change":"2024-04-27T03:30:11.371906+0000","last_active":"2024-04-27T03:31:01.432873+0000","last_peered":"2024-04-27T03:31:01.432873+0000","last_clean":"2024-04-27T03:31:01.432873+0000","last_became_active":"2024-04-27T03:30:11.371604+0000","last_became_peered":"2024-04-27T03:30:11.371604+0000","last_unstale":"2024-04-27T03:31:01.432873+0000","last_undegraded":"2024-04-27T03:31:01.432873+0000","last_fullsized":"2024-04-27T03:31:01.432873+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:33:05.923239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"27'8","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981663+0000","last_change":"2024-04-27T03:30:13.388711+0000","last_active":"2024-04-27T03:33:20.981663+0000","last_peered":"2024-04-27T03:33:20.981663+0000","last_clean":"2024-04-27T03:33:20.981663+0000","last_became_active":"2024-04-27T03:30:13.388421+0000","last_became_peered":"2024-04-27T03:30:13.388421+0000","last_unstale":"2024-04-27T03:33:20.981663+0000","last_undegraded":"2024-04-27T03:33:20.981663+0000","last_fullsized":"2024-04-27T03:33:20.981663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:35.526991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.678119+0000","last_change":"2024-04-27T03:30:11.355453+0000","last_active":"2024-04-27T03:33:20.678119+0000","last_peered":"2024-04-27T03:33:20.678119+0000","last_clean":"2024-04-27T03:33:20.678119+0000","last_became_active":"2024-04-27T03:30:11.355183+0000","last_became_peered":"2024-04-27T03:30:11.355183+0000","last_unstale":"2024-04-27T03:33:20.678119+0000","last_undegraded":"2024-04-27T03:33:20.678119+0000","last_fullsized":"2024-04-27T03:33:20.678119+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:12.667203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"27'4","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.107011+0000","last_change":"2024-04-27T03:30:13.377155+0000","last_active":"2024-04-27T03:32:17.107011+0000","last_peered":"2024-04-27T03:32:17.107011+0000","last_clean":"2024-04-27T03:32:17.107011+0000","last_became_active":"2024-04-27T03:30:13.376875+0000","last_became_peered":"2024-04-27T03:30:13.376875+0000","last_unstale":"2024-04-27T03:32:17.107011+0000","last_undegraded":"2024-04-27T03:32:17.107011+0000","last_fullsized":"2024-04-27T03:32:17.107011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:09.993585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431158+0000","last_change":"2024-04-27T03:30:11.361373+0000","last_active":"2024-04-27T03:31:01.431158+0000","last_peered":"2024-04-27T03:31:01.431158+0000","last_clean":"2024-04-27T03:31:01.431158+0000","last_became_active":"2024-04-27T03:30:11.360299+0000","last_became_peered":"2024-04-27T03:30:11.360299+0000","last_unstale":"2024-04-27T03:31:01.431158+0000","last_undegraded":"2024-04-27T03:31:01.431158+0000","last_fullsized":"2024-04-27T03:31:01.431158+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.746108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"27'10","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981744+0000","last_change":"2024-04-27T03:30:13.376269+0000","last_active":"2024-04-27T03:33:20.981744+0000","last_peered":"2024-04-27T03:33:20.981744+0000","last_clean":"2024-04-27T03:33:20.981744+0000","last_became_active":"2024-04-27T03:30:13.376087+0000","last_became_peered":"2024-04-27T03:30:13.376087+0000","last_unstale":"2024-04-27T03:33:20.981744+0000","last_undegraded":"2024-04-27T03:33:20.981744+0000","last_fullsized":"2024-04-27T03:33:20.981744+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:18.301070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432834+0000","last_change":"2024-04-27T03:30:11.371237+0000","last_active":"2024-04-27T03:31:01.432834+0000","last_peered":"2024-04-27T03:31:01.432834+0000","last_clean":"2024-04-27T03:31:01.432834+0000","last_became_active":"2024-04-27T03:30:11.370837+0000","last_became_peered":"2024-04-27T03:30:11.370837+0000","last_unstale":"2024-04-27T03:31:01.432834+0000","last_undegraded":"2024-04-27T03:31:01.432834+0000","last_fullsized":"2024-04-27T03:31:01.432834+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.863119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"27'6","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.697535+0000","last_change":"2024-04-27T03:30:13.376130+0000","last_active":"2024-04-27T03:33:20.697535+0000","last_peered":"2024-04-27T03:33:20.697535+0000","last_clean":"2024-04-27T03:33:20.697535+0000","last_became_active":"2024-04-27T03:30:13.373511+0000","last_became_peered":"2024-04-27T03:30:13.373511+0000","last_unstale":"2024-04-27T03:33:20.697535+0000","last_undegraded":"2024-04-27T03:33:20.697535+0000","last_fullsized":"2024-04-27T03:33:20.697535+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:51.434596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.257092+0000","last_change":"2024-04-27T03:30:11.370821+0000","last_active":"2024-04-27T03:32:20.257092+0000","last_peered":"2024-04-27T03:32:20.257092+0000","last_clean":"2024-04-27T03:32:20.257092+0000","last_became_active":"2024-04-27T03:30:11.370283+0000","last_became_peered":"2024-04-27T03:30:11.370283+0000","last_unstale":"2024-04-27T03:32:20.257092+0000","last_undegraded":"2024-04-27T03:32:20.257092+0000","last_fullsized":"2024-04-27T03:32:20.257092+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:17.981757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"27'11","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.016376+0000","last_change":"2024-04-27T03:30:13.380862+0000","last_active":"2024-04-27T03:33:21.016376+0000","last_peered":"2024-04-27T03:33:21.016376+0000","last_clean":"2024-04-27T03:33:21.016376+0000","last_became_active":"2024-04-27T03:30:13.380404+0000","last_became_peered":"2024-04-27T03:30:13.380404+0000","last_unstale":"2024-04-27T03:33:21.016376+0000","last_undegraded":"2024-04-27T03:33:21.016376+0000","last_fullsized":"2024-04-27T03:33:21.016376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:14.849083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439493+0000","last_change":"2024-04-27T03:30:11.373282+0000","last_active":"2024-04-27T03:31:01.439493+0000","last_peered":"2024-04-27T03:31:01.439493+0000","last_clean":"2024-04-27T03:31:01.439493+0000","last_became_active":"2024-04-27T03:30:11.373026+0000","last_became_peered":"2024-04-27T03:30:11.373026+0000","last_unstale":"2024-04-27T03:31:01.439493+0000","last_undegraded":"2024-04-27T03:31:01.439493+0000","last_fullsized":"2024-04-27T03:31:01.439493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:40:08.170228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"27'1","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295321+0000","last_change":"2024-04-27T03:30:13.386073+0000","last_active":"2024-04-27T03:32:29.295321+0000","last_peered":"2024-04-27T03:32:29.295321+0000","last_clean":"2024-04-27T03:32:29.295321+0000","last_became_active":"2024-04-27T03:30:13.385791+0000","last_became_peered":"2024-04-27T03:30:13.385791+0000","last_unstale":"2024-04-27T03:32:29.295321+0000","last_undegraded":"2024-04-27T03:32:29.295321+0000","last_fullsized":"2024-04-27T03:32:29.295321+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:18.301070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.441318+0000","last_change":"2024-04-27T03:30:11.358239+0000","last_active":"2024-04-27T03:31:01.441318+0000","last_peered":"2024-04-27T03:31:01.441318+0000","last_clean":"2024-04-27T03:31:01.441318+0000","last_became_active":"2024-04-27T03:30:11.357898+0000","last_became_peered":"2024-04-27T03:30:11.357898+0000","last_unstale":"2024-04-27T03:31:01.441318+0000","last_undegraded":"2024-04-27T03:31:01.441318+0000","last_fullsized":"2024-04-27T03:31:01.441318+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:26.250500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"27'5","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:19.303737+0000","last_change":"2024-04-27T03:31:01.444873+0000","last_active":"2024-04-27T03:33:19.303737+0000","last_peered":"2024-04-27T03:33:19.303737+0000","last_clean":"2024-04-27T03:33:19.303737+0000","last_became_active":"2024-04-27T03:31:01.440820+0000","last_became_peered":"2024-04-27T03:31:01.440820+0000","last_unstale":"2024-04-27T03:33:19.303737+0000","last_undegraded":"2024-04-27T03:33:19.303737+0000","last_fullsized":"2024-04-27T03:33:19.303737+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:15.446814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435323+0000","last_change":"2024-04-27T03:30:11.363676+0000","last_active":"2024-04-27T03:31:01.435323+0000","last_peered":"2024-04-27T03:31:01.435323+0000","last_clean":"2024-04-27T03:31:01.435323+0000","last_became_active":"2024-04-27T03:30:11.362788+0000","last_became_peered":"2024-04-27T03:30:11.362788+0000","last_unstale":"2024-04-27T03:31:01.435323+0000","last_undegraded":"2024-04-27T03:31:01.435323+0000","last_fullsized":"2024-04-27T03:31:01.435323+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:00.825717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"25'2","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436660+0000","last_change":"2024-04-27T03:30:13.378124+0000","last_active":"2024-04-27T03:31:01.436660+0000","last_peered":"2024-04-27T03:31:01.436660+0000","last_clean":"2024-04-27T03:31:01.436660+0000","last_became_active":"2024-04-27T03:30:13.377845+0000","last_became_peered":"2024-04-27T03:30:13.377845+0000","last_unstale":"2024-04-27T03:31:01.436660+0000","last_undegraded":"2024-04-27T03:31:01.436660+0000","last_fullsized":"2024-04-27T03:31:01.436660+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:37.770898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'1","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296913+0000","last_change":"2024-04-27T03:30:11.362917+0000","last_active":"2024-04-27T03:32:34.296913+0000","last_peered":"2024-04-27T03:32:34.296913+0000","last_clean":"2024-04-27T03:32:34.296913+0000","last_became_active":"2024-04-27T03:30:11.362559+0000","last_became_peered":"2024-04-27T03:30:11.362559+0000","last_unstale":"2024-04-27T03:32:34.296913+0000","last_undegraded":"2024-04-27T03:32:34.296913+0000","last_fullsized":"2024-04-27T03:32:34.296913+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:55.587028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436858+0000","last_change":"2024-04-27T03:30:11.363675+0000","last_active":"2024-04-27T03:31:01.436858+0000","last_peered":"2024-04-27T03:31:01.436858+0000","last_clean":"2024-04-27T03:31:01.436858+0000","last_became_active":"2024-04-27T03:30:11.363340+0000","last_became_peered":"2024-04-27T03:30:11.363340+0000","last_unstale":"2024-04-27T03:31:01.436858+0000","last_undegraded":"2024-04-27T03:31:01.436858+0000","last_fullsized":"2024-04-27T03:31:01.436858+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:26.857275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.107465+0000","last_change":"2024-04-27T03:30:13.386029+0000","last_active":"2024-04-27T03:32:17.107465+0000","last_peered":"2024-04-27T03:32:17.107465+0000","last_clean":"2024-04-27T03:32:17.107465+0000","last_became_active":"2024-04-27T03:30:13.385705+0000","last_became_peered":"2024-04-27T03:30:13.385705+0000","last_unstale":"2024-04-27T03:32:17.107465+0000","last_undegraded":"2024-04-27T03:32:17.107465+0000","last_fullsized":"2024-04-27T03:32:17.107465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:18.417469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430983+0000","last_change":"2024-04-27T03:30:11.360980+0000","last_active":"2024-04-27T03:31:01.430983+0000","last_peered":"2024-04-27T03:31:01.430983+0000","last_clean":"2024-04-27T03:31:01.430983+0000","last_became_active":"2024-04-27T03:30:11.360667+0000","last_became_peered":"2024-04-27T03:30:11.360667+0000","last_unstale":"2024-04-27T03:31:01.430983+0000","last_undegraded":"2024-04-27T03:31:01.430983+0000","last_fullsized":"2024-04-27T03:31:01.430983+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:51.203886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"27'14","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.305196+0000","last_change":"2024-04-27T03:30:13.380806+0000","last_active":"2024-04-27T03:32:20.305196+0000","last_peered":"2024-04-27T03:32:20.305196+0000","last_clean":"2024-04-27T03:32:20.305196+0000","last_became_active":"2024-04-27T03:30:13.380347+0000","last_became_peered":"2024-04-27T03:30:13.380347+0000","last_unstale":"2024-04-27T03:32:20.305196+0000","last_undegraded":"2024-04-27T03:32:20.305196+0000","last_fullsized":"2024-04-27T03:32:20.305196+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:48.533060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2256,"ondisk_log_size":2256,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":13385,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":1690,"num_read_kb":1444,"num_write":723,"num_write_kb":71,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":286720,"data_stored":10826,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1470,"ondisk_log_size":1470,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":961,"num_read_kb":961,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":25805696,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":128,"num_read_kb":82,"num_write":309,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":51617792,"data_stored":51611392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":76,"ondisk_log_size":76,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705706,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":44568,"kb_used_data":9232,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94327272,"statfs":{"total":96636764160,"available":96591126528,"internally_reserved":0,"allocated":9453568,"data_stored":9151139,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705707,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60832,"kb_used_data":25496,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94311008,"statfs":{"total":96636764160,"available":96574472192,"internally_reserved":0,"allocated":26107904,"data_stored":25873899,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705706,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36912,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94334928,"statfs":{"total":96636764160,"available":96598966272,"internally_reserved":0,"allocated":1613824,"data_stored":1348704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705708,"num_pgs":134,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56256,"kb_used_data":16760,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94315584,"statfs":{"total":96636764160,"available":96579158016,"internally_reserved":0,"allocated":17162240,"data_stored":16898458,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16777216,"data_stored":16777216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":643072,"data_stored":639872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":25165824,"data_stored":25165824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":9031680,"data_stored":9028480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":3361,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":4269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T03:33:28.749 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-27T03:33:28.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.867+0000 7f7f4666a640 1 Processor -- start 2024-04-27T03:33:28.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.867+0000 7f7f4666a640 1 -- start start 2024-04-27T03:33:28.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.867+0000 7f7f4666a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7f40127ce0 0x7f7f401280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:28.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.867+0000 7f7f4666a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f7f40127b00 con 0x7f7f40070100 2024-04-27T03:33:28.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.867+0000 7f7f4666a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f7f400708e0 con 0x7f7f40127ce0 2024-04-27T03:33:28.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.867+0000 7f7f3ffff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f7f40127ce0 0x7f7f401280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:28.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.867+0000 7f7f3f7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f7f40070100 0x7f7f4012b000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48024/0 (socket says 172.21.15.89:48024) 2024-04-27T03:33:28.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.867+0000 7f7f3f7fe640 1 -- 172.21.15.89:0/3574608210 learned_addr learned my addr 172.21.15.89:0/3574608210 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:28.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.867+0000 7f7f3ffff640 1 -- 172.21.15.89:0/3574608210 >> v1:172.21.15.89:6789/0 conn(0x7f7f40070100 legacy=0x7f7f4012b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:28.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.867+0000 7f7f3ffff640 1 -- 172.21.15.89:0/3574608210 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f40098800 con 0x7f7f40127ce0 2024-04-27T03:33:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.868+0000 7f7f3ffff640 1 --2- 172.21.15.89:0/3574608210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f40127ce0 0x7f7f401280c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009720 tx=0x7f7f3002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af0bbe7d1be1c7e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:28.876 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.868+0000 7f7f3effd640 1 -- 172.21.15.89:0/3574608210 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f300316d0 con 0x7f7f40127ce0 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.868+0000 7f7f3effd640 1 -- 172.21.15.89:0/3574608210 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f30037070 con 0x7f7f40127ce0 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.868+0000 7f7f3effd640 1 -- 172.21.15.89:0/3574608210 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f300319c0 con 0x7f7f40127ce0 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.869+0000 7f7f4666a640 1 -- 172.21.15.89:0/3574608210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f40127ce0 msgr2=0x7f7f401280c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.869+0000 7f7f4666a640 1 --2- 172.21.15.89:0/3574608210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f40127ce0 0x7f7f401280c0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009720 tx=0x7f7f3002b840 comp rx=0 tx=0).stop 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.869+0000 7f7f4666a640 1 -- 172.21.15.89:0/3574608210 shutdown_connections 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.869+0000 7f7f4666a640 1 --2- 172.21.15.89:0/3574608210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f40127ce0 0x7f7f401280c0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.869+0000 7f7f4666a640 1 -- 172.21.15.89:0/3574608210 >> 172.21.15.89:0/3574608210 conn(0x7f7f40099680 msgr2=0x7f7f40099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.869+0000 7f7f4666a640 1 -- 172.21.15.89:0/3574608210 shutdown_connections 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.869+0000 7f7f4666a640 1 -- 172.21.15.89:0/3574608210 wait complete. 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.870+0000 7f7f4666a640 1 Processor -- start 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.870+0000 7f7f4666a640 1 -- start start 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.870+0000 7f7f4666a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f40070100 0x7f7f4006d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.870+0000 7f7f4666a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f7f40128780 con 0x7f7f40070100 2024-04-27T03:33:28.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.870+0000 7f7f3ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f40070100 0x7f7f4006d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.870+0000 7f7f3ffff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f40070100 0x7f7f4006d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:60254/0 (socket says 172.21.15.89:60254) 2024-04-27T03:33:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.870+0000 7f7f3ffff640 1 -- 172.21.15.89:0/3612084202 learned_addr learned my addr 172.21.15.89:0/3612084202 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.870+0000 7f7f3ffff640 1 -- 172.21.15.89:0/3612084202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f4008a2d0 con 0x7f7f40070100 2024-04-27T03:33:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.871+0000 7f7f3ffff640 1 --2- 172.21.15.89:0/3612084202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f40070100 0x7f7f4006d420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009720 tx=0x7f7f300046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.871+0000 7f7f3cff9640 1 -- 172.21.15.89:0/3612084202 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f30032770 con 0x7f7f40070100 2024-04-27T03:33:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.871+0000 7f7f3cff9640 1 -- 172.21.15.89:0/3612084202 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f7f30032910 con 0x7f7f40070100 2024-04-27T03:33:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.871+0000 7f7f4666a640 1 -- 172.21.15.89:0/3612084202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f4008af60 con 0x7f7f40070100 2024-04-27T03:33:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.871+0000 7f7f3cff9640 1 -- 172.21.15.89:0/3612084202 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7f30032bf0 con 0x7f7f40070100 2024-04-27T03:33:28.878 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.871+0000 7f7f4666a640 1 -- 172.21.15.89:0/3612084202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f40125080 con 0x7f7f40070100 2024-04-27T03:33:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.872+0000 7f7f3cff9640 1 -- 172.21.15.89:0/3612084202 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f7f3004d3f0 con 0x7f7f40070100 2024-04-27T03:33:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.872+0000 7f7f3cff9640 1 --2- 172.21.15.89:0/3612084202 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7f2c038060 0x7f7f2c03a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.872+0000 7f7f4666a640 1 -- 172.21.15.89:0/3612084202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f4012b1f0 con 0x7f7f40070100 2024-04-27T03:33:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.872+0000 7f7f3cff9640 1 -- 172.21.15.89:0/3612084202 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f7f3002f050 con 0x7f7f40070100 2024-04-27T03:33:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.872+0000 7f7f3f7fe640 1 --2- 172.21.15.89:0/3612084202 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7f2c038060 0x7f7f2c03a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:28.879 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.873+0000 7f7f3f7fe640 1 --2- 172.21.15.89:0/3612084202 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7f2c038060 0x7f7f2c03a520 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7f34007fd0 tx=0x7f7f34007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:28.882 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:28.876+0000 7f7f3cff9640 1 -- 172.21.15.89:0/3612084202 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f7f4012b1f0 con 0x7f7f40070100 2024-04-27T03:33:29.054 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:29.049+0000 7f7f4666a640 1 -- 172.21.15.89:0/3612084202 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7f40062da0 con 0x7f7f40070100 2024-04-27T03:33:29.056 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:29.051+0000 7f7f3cff9640 1 -- 172.21.15.89:0/3612084202 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v27) ==== 74+0+21012 (secure 0 0 0) 0x7f7f30040070 con 0x7f7f40070100 2024-04-27T03:33:29.056 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:33:29.057 INFO:teuthology.orchestra.run.smithi089.stdout:{"epoch":27,"fsid":"7a1794da-6463-461c-a75b-269448ecfc86","created":"2024-04-27T03:29:55.910567+0000","modified":"2024-04-27T03:31:01.425660+0000","last_up_change":"2024-04-27T03:30:04.092854+0000","last_in_change":"2024-04-27T03:29:56.997311+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":8,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-27T03:30:04.239623+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2024-04-27T03:30:04.766593+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"12","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"default.rgw.buckets.data","create_time":"2024-04-27T03:30:10.238597+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"default.rgw.buckets.index","create_time":"2024-04-27T03:30:11.673338+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":".rgw.root","create_time":"2024-04-27T03:30:13.651729+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.1299999952316284,"score_stable":1.1299999952316284,"optimal_score":1,"raw_score_acting":1.1299999952316284,"raw_score_stable":1.1299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"default.rgw.log","create_time":"2024-04-27T03:30:15.462883+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"default.rgw.control","create_time":"2024-04-27T03:30:17.411033+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"default.rgw.meta","create_time":"2024-04-27T03:30:19.503471+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"25","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4},"application_metadata":{"rgw":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.25,"score_stable":1.25,"optimal_score":1,"raw_score_acting":1.25,"raw_score_stable":1.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"2643cf41-510f-4fc8-9611-7260d319e5f8","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6800","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6801","nonce":2353668432}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6802","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6803","nonce":2353668432}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6806","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6807","nonce":2353668432}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.89:6804","nonce":2353668432},{"type":"v1","addr":"172.21.15.89:6805","nonce":2353668432}]},"public_addr":"172.21.15.89:6801/2353668432","cluster_addr":"172.21.15.89:6803/2353668432","heartbeat_back_addr":"172.21.15.89:6807/2353668432","heartbeat_front_addr":"172.21.15.89:6805/2353668432","state":["exists","up"]},{"osd":1,"uuid":"6d14d21a-3c2d-47bb-a0d9-96f58d57d612","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6800","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6801","nonce":79879041}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6802","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6803","nonce":79879041}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6806","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6807","nonce":79879041}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6804","nonce":79879041},{"type":"v1","addr":"172.21.15.165:6805","nonce":79879041}]},"public_addr":"172.21.15.165:6801/79879041","cluster_addr":"172.21.15.165:6803/79879041","heartbeat_back_addr":"172.21.15.165:6807/79879041","heartbeat_front_addr":"172.21.15.165:6805/79879041","state":["exists","up"]},{"osd":2,"uuid":"12198ef3-e304-4085-992a-2f3563d60ebc","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":26,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6816","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6817","nonce":975457786}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6818","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6819","nonce":975457786}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6822","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6823","nonce":975457786}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6820","nonce":975457786},{"type":"v1","addr":"172.21.15.165:6821","nonce":975457786}]},"public_addr":"172.21.15.165:6817/975457786","cluster_addr":"172.21.15.165:6819/975457786","heartbeat_back_addr":"172.21.15.165:6823/975457786","heartbeat_front_addr":"172.21.15.165:6821/975457786","state":["exists","up"]},{"osd":3,"uuid":"5ecd16bd-fe0b-4b0d-8028-00b613ff7ec7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":23,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6808","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6809","nonce":2380918637}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6810","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6811","nonce":2380918637}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6814","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6815","nonce":2380918637}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.165:6812","nonce":2380918637},{"type":"v1","addr":"172.21.15.165:6813","nonce":2380918637}]},"public_addr":"172.21.15.165:6809/2380918637","cluster_addr":"172.21.15.165:6811/2380918637","heartbeat_back_addr":"172.21.15.165:6815/2380918637","heartbeat_front_addr":"172.21.15.165:6813/2380918637","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T03:30:02.036887+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T03:30:02.229251+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T03:30:02.245310+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-27T03:30:02.141804+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.9","mappings":[{"from":3,"to":1}]},{"pgid":"3.26","mappings":[{"from":3,"to":1}]},{"pgid":"4.22","mappings":[{"from":3,"to":2}]},{"pgid":"4.29","mappings":[{"from":3,"to":2}]},{"pgid":"4.33","mappings":[{"from":3,"to":2}]},{"pgid":"5.4","mappings":[{"from":0,"to":2}]},{"pgid":"5.8","mappings":[{"from":0,"to":2}]},{"pgid":"5.a","mappings":[{"from":0,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-27T03:33:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:29.052+0000 7f7f4666a640 1 -- 172.21.15.89:0/3612084202 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7f2c038060 msgr2=0x7f7f2c03a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:29.058 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:29.052+0000 7f7f4666a640 1 --2- 172.21.15.89:0/3612084202 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7f2c038060 0x7f7f2c03a520 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f7f34007fd0 tx=0x7f7f34007ac0 comp rx=0 tx=0).stop 2024-04-27T03:33:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:29.053+0000 7f7f4666a640 1 -- 172.21.15.89:0/3612084202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f40070100 msgr2=0x7f7f4006d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:29.053+0000 7f7f4666a640 1 --2- 172.21.15.89:0/3612084202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f40070100 0x7f7f4006d420 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7f7f30009720 tx=0x7f7f300046c0 comp rx=0 tx=0).stop 2024-04-27T03:33:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:29.053+0000 7f7f4666a640 1 -- 172.21.15.89:0/3612084202 shutdown_connections 2024-04-27T03:33:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:29.053+0000 7f7f4666a640 1 --2- 172.21.15.89:0/3612084202 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f7f2c038060 0x7f7f2c03a520 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:29.053+0000 7f7f4666a640 1 --2- 172.21.15.89:0/3612084202 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f7f40070100 0x7f7f4006d420 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:29.053+0000 7f7f4666a640 1 -- 172.21.15.89:0/3612084202 >> 172.21.15.89:0/3612084202 conn(0x7f7f40099680 msgr2=0x7f7f4006e530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:29.053+0000 7f7f4666a640 1 -- 172.21.15.89:0/3612084202 shutdown_connections 2024-04-27T03:33:29.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:29.053+0000 7f7f4666a640 1 -- 172.21.15.89:0/3612084202 wait complete. 2024-04-27T03:33:30.070 INFO:tasks.ceph:Scrubbing osd.0 2024-04-27T03:33:30.070 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2024-04-27T03:33:30.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.176+0000 7fd74040f640 1 Processor -- start 2024-04-27T03:33:30.181 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.176+0000 7fd74040f640 1 -- start start 2024-04-27T03:33:30.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.176+0000 7fd74040f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd738113160 0x7fd7381313a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.182 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.176+0000 7fd74040f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fd73812c1a0 con 0x7fd73812c3a0 2024-04-27T03:33:30.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.176+0000 7fd74040f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fd73806ffe0 con 0x7fd738113160 2024-04-27T03:33:30.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.176+0000 7fd73d9aa640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fd738113160 0x7fd7381313a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.176+0000 7fd73e1ab640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fd73812c3a0 0x7fd73812c780 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51770/0 (socket says 172.21.15.89:51770) 2024-04-27T03:33:30.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.176+0000 7fd73e1ab640 1 -- 172.21.15.89:0/3447721387 learned_addr learned my addr 172.21.15.89:0/3447721387 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:30.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.177+0000 7fd73d9aa640 1 -- 172.21.15.89:0/3447721387 >> v1:172.21.15.89:6789/0 conn(0x7fd73812c3a0 legacy=0x7fd73812c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.177+0000 7fd73d9aa640 1 -- 172.21.15.89:0/3447721387 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd738072670 con 0x7fd738113160 2024-04-27T03:33:30.183 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.177+0000 7fd73d1a9640 1 -- 172.21.15.89:0/3447721387 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4126625569 0 0) 0x7fd73812c1a0 con 0x7fd73812c3a0 2024-04-27T03:33:30.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.178+0000 7fd73d9aa640 1 --2- 172.21.15.89:0/3447721387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738113160 0x7fd7381313a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd734009720 tx=0x7fd73402b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d30b6a38d8ab4872 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.184 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.178+0000 7fd73d1a9640 1 -- 172.21.15.89:0/3447721387 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7340316d0 con 0x7fd738113160 2024-04-27T03:33:30.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.178+0000 7fd73d1a9640 1 -- 172.21.15.89:0/3447721387 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd734037070 con 0x7fd738113160 2024-04-27T03:33:30.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.178+0000 7fd73d1a9640 1 -- 172.21.15.89:0/3447721387 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd7340319c0 con 0x7fd738113160 2024-04-27T03:33:30.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.178+0000 7fd74040f640 1 -- 172.21.15.89:0/3447721387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738113160 msgr2=0x7fd7381313a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.178+0000 7fd74040f640 1 --2- 172.21.15.89:0/3447721387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738113160 0x7fd7381313a0 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x7fd734009720 tx=0x7fd73402b840 comp rx=0 tx=0).stop 2024-04-27T03:33:30.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.178+0000 7fd74040f640 1 -- 172.21.15.89:0/3447721387 shutdown_connections 2024-04-27T03:33:30.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.178+0000 7fd74040f640 1 --2- 172.21.15.89:0/3447721387 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738113160 0x7fd7381313a0 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.178+0000 7fd74040f640 1 -- 172.21.15.89:0/3447721387 >> 172.21.15.89:0/3447721387 conn(0x7fd738099510 msgr2=0x7fd738099920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:30.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.178+0000 7fd74040f640 1 -- 172.21.15.89:0/3447721387 shutdown_connections 2024-04-27T03:33:30.185 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.178+0000 7fd74040f640 1 -- 172.21.15.89:0/3447721387 wait complete. 2024-04-27T03:33:30.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.179+0000 7fd74040f640 1 Processor -- start 2024-04-27T03:33:30.186 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.179+0000 7fd74040f640 1 -- start start 2024-04-27T03:33:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.179+0000 7fd74040f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738113160 0x7fd738122630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.179+0000 7fd74040f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fd738113910 con 0x7fd738113160 2024-04-27T03:33:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.180+0000 7fd73e1ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738113160 0x7fd738122630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.180+0000 7fd73e1ab640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738113160 0x7fd738122630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49680/0 (socket says 172.21.15.89:49680) 2024-04-27T03:33:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.180+0000 7fd73e1ab640 1 -- 172.21.15.89:0/1561937436 learned_addr learned my addr 172.21.15.89:0/1561937436 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.180+0000 7fd73e1ab640 1 -- 172.21.15.89:0/1561937436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd738071ca0 con 0x7fd738113160 2024-04-27T03:33:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.180+0000 7fd73e1ab640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738113160 0x7fd738122630 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd72800c430 tx=0x7fd72800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.180+0000 7fd72effd640 1 -- 172.21.15.89:0/1561937436 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd728005cd0 con 0x7fd738113160 2024-04-27T03:33:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.180+0000 7fd72effd640 1 -- 172.21.15.89:0/1561937436 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fd728007070 con 0x7fd738113160 2024-04-27T03:33:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.180+0000 7fd74040f640 1 -- 172.21.15.89:0/1561937436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd73812f7e0 con 0x7fd738113160 2024-04-27T03:33:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.180+0000 7fd72effd640 1 -- 172.21.15.89:0/1561937436 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd728007370 con 0x7fd738113160 2024-04-27T03:33:30.189 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.180+0000 7fd74040f640 1 -- 172.21.15.89:0/1561937436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd738071430 con 0x7fd738113160 2024-04-27T03:33:30.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.181+0000 7fd74040f640 1 -- 172.21.15.89:0/1561937436 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fd73812fac0 con 0x7fd738113160 2024-04-27T03:33:30.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.181+0000 7fd72effd640 1 -- 172.21.15.89:0/1561937436 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fd728007510 con 0x7fd738113160 2024-04-27T03:33:30.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.181+0000 7fd72effd640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd71c038060 0x7fd71c03a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.181+0000 7fd72effd640 1 -- 172.21.15.89:0/1561937436 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fd72804b530 con 0x7fd738113160 2024-04-27T03:33:30.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.181+0000 7fd72effd640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd71c043e40 0x7fd71c0462e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.181+0000 7fd72effd640 1 -- 172.21.15.89:0/1561937436 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fd72801de70 con 0x7fd71c043e40 2024-04-27T03:33:30.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.181+0000 7fd72effd640 1 -- 172.21.15.89:0/1561937436 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fd728052050 con 0x7fd738113160 2024-04-27T03:33:30.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.181+0000 7fd73d9aa640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd71c038060 0x7fd71c03a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.181+0000 7fd73e9ac640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd71c043e40 0x7fd71c0462e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.182+0000 7fd73e9ac640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd71c043e40 0x7fd71c0462e0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.182+0000 7fd73d9aa640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd71c038060 0x7fd71c03a520 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd734036000 tx=0x7fd734008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.190 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.182+0000 7fd72effd640 1 -- 172.21.15.89:0/1561937436 <== osd.0 v2:172.21.15.89:6800/2353668432 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fd72801de70 con 0x7fd71c043e40 2024-04-27T03:33:30.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.195+0000 7fd74040f640 1 -- 172.21.15.89:0/1561937436 --> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fd7381302f0 con 0x7fd71c043e40 2024-04-27T03:33:30.201 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd72effd640 1 -- 172.21.15.89:0/1561937436 <== osd.0 v2:172.21.15.89:6800/2353668432 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fd7381302f0 con 0x7fd71c043e40 2024-04-27T03:33:30.201 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-04-27T03:33:30.201 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T03:33:30.202 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-04-27T03:33:30.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 -- 172.21.15.89:0/1561937436 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd71c043e40 msgr2=0x7fd71c0462e0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd71c043e40 0x7fd71c0462e0 crc :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 -- 172.21.15.89:0/1561937436 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd71c038060 msgr2=0x7fd71c03a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd71c038060 0x7fd71c03a520 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd734036000 tx=0x7fd734008030 comp rx=0 tx=0).stop 2024-04-27T03:33:30.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 -- 172.21.15.89:0/1561937436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738113160 msgr2=0x7fd738122630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738113160 0x7fd738122630 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7fd72800c430 tx=0x7fd72800c8f0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.203 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 -- 172.21.15.89:0/1561937436 shutdown_connections 2024-04-27T03:33:30.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.89:6800/2353668432,v1:172.21.15.89:6801/2353668432] conn(0x7fd71c043e40 0x7fd71c0462e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fd71c038060 0x7fd71c03a520 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 --2- 172.21.15.89:0/1561937436 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fd738113160 0x7fd738122630 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 -- 172.21.15.89:0/1561937436 >> 172.21.15.89:0/1561937436 conn(0x7fd738099510 msgr2=0x7fd73809a4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:30.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 -- 172.21.15.89:0/1561937436 shutdown_connections 2024-04-27T03:33:30.204 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.196+0000 7fd74040f640 1 -- 172.21.15.89:0/1561937436 wait complete. 2024-04-27T03:33:30.212 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2024-04-27T03:33:30.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.303+0000 7f06a0c8c640 1 Processor -- start 2024-04-27T03:33:30.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.304+0000 7f06a0c8c640 1 -- start start 2024-04-27T03:33:30.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.304+0000 7f06a0c8c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f069c096430 0x7f069c096810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.304+0000 7f06a0c8c640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f069c098800 con 0x7f069c071090 2024-04-27T03:33:30.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.304+0000 7f06a0c8c640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f069c071930 con 0x7f069c096430 2024-04-27T03:33:30.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.304+0000 7f069a59c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f069c096430 0x7f069c096810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.304+0000 7f0699d9b640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f069c071090 0x7f069c06d420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51782/0 (socket says 172.21.15.89:51782) 2024-04-27T03:33:30.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.304+0000 7f0699d9b640 1 -- 172.21.15.89:0/1508913726 learned_addr learned my addr 172.21.15.89:0/1508913726 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:30.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.305+0000 7f069a59c640 1 -- 172.21.15.89:0/1508913726 >> v1:172.21.15.89:6789/0 conn(0x7f069c071090 legacy=0x7f069c06d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.305+0000 7f069a59c640 1 -- 172.21.15.89:0/1508913726 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c071f50 con 0x7f069c096430 2024-04-27T03:33:30.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.305+0000 7f069a59c640 1 --2- 172.21.15.89:0/1508913726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c096430 0x7f069c096810 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0690009720 tx=0x7f069002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=952b3074b84a4560 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.305+0000 7f069959a640 1 -- 172.21.15.89:0/1508913726 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06900316d0 con 0x7f069c096430 2024-04-27T03:33:30.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.305+0000 7f069959a640 1 -- 172.21.15.89:0/1508913726 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0690037070 con 0x7f069c096430 2024-04-27T03:33:30.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.305+0000 7f069959a640 1 -- 172.21.15.89:0/1508913726 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f06900319c0 con 0x7f069c096430 2024-04-27T03:33:30.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.306+0000 7f06a0c8c640 1 -- 172.21.15.89:0/1508913726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c096430 msgr2=0x7f069c096810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.306+0000 7f06a0c8c640 1 --2- 172.21.15.89:0/1508913726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c096430 0x7f069c096810 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7f0690009720 tx=0x7f069002b840 comp rx=0 tx=0).stop 2024-04-27T03:33:30.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.306+0000 7f06a0c8c640 1 -- 172.21.15.89:0/1508913726 shutdown_connections 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.306+0000 7f06a0c8c640 1 --2- 172.21.15.89:0/1508913726 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c096430 0x7f069c096810 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.306+0000 7f06a0c8c640 1 -- 172.21.15.89:0/1508913726 >> 172.21.15.89:0/1508913726 conn(0x7f069c099680 msgr2=0x7f069c099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.306+0000 7f06a0c8c640 1 -- 172.21.15.89:0/1508913726 shutdown_connections 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.306+0000 7f06a0c8c640 1 -- 172.21.15.89:0/1508913726 wait complete. 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.307+0000 7f06a0c8c640 1 Processor -- start 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.307+0000 7f06a0c8c640 1 -- start start 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.307+0000 7f06a0c8c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c071090 0x7f069c089f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.307+0000 7f06a0c8c640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f069c096d50 con 0x7f069c071090 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.307+0000 7f069a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c071090 0x7f069c089f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.307+0000 7f069a59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c071090 0x7f069c089f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49702/0 (socket says 172.21.15.89:49702) 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.307+0000 7f069a59c640 1 -- 172.21.15.89:0/3885873450 learned_addr learned my addr 172.21.15.89:0/3885873450 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.308+0000 7f069a59c640 1 -- 172.21.15.89:0/3885873450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f069c08b420 con 0x7f069c071090 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.308+0000 7f069a59c640 1 --2- 172.21.15.89:0/3885873450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c071090 0x7f069c089f90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0690034040 tx=0x7f06900046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.315 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.308+0000 7f068affd640 1 -- 172.21.15.89:0/3885873450 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0690032770 con 0x7f069c071090 2024-04-27T03:33:30.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.308+0000 7f068affd640 1 -- 172.21.15.89:0/3885873450 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0690032930 con 0x7f069c071090 2024-04-27T03:33:30.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.308+0000 7f06a0c8c640 1 -- 172.21.15.89:0/3885873450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f069c08b8a0 con 0x7f069c071090 2024-04-27T03:33:30.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.308+0000 7f068affd640 1 -- 172.21.15.89:0/3885873450 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0690032c50 con 0x7f069c071090 2024-04-27T03:33:30.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.308+0000 7f06a0c8c640 1 -- 172.21.15.89:0/3885873450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f069c08c070 con 0x7f069c071090 2024-04-27T03:33:30.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.309+0000 7f068affd640 1 -- 172.21.15.89:0/3885873450 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f069004c060 con 0x7f069c071090 2024-04-27T03:33:30.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.309+0000 7f06a0c8c640 1 -- 172.21.15.89:0/3885873450 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f069c08d040 con 0x7f069c071090 2024-04-27T03:33:30.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.309+0000 7f068affd640 1 --2- 172.21.15.89:0/3885873450 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f067c038060 0x7f067c03a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.309+0000 7f068affd640 1 -- 172.21.15.89:0/3885873450 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f069002f050 con 0x7f069c071090 2024-04-27T03:33:30.316 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.309+0000 7f0699d9b640 1 --2- 172.21.15.89:0/3885873450 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f067c038060 0x7f067c03a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.317 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.310+0000 7f0699d9b640 1 --2- 172.21.15.89:0/3885873450 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f067c038060 0x7f067c03a520 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f068c007fd0 tx=0x7f068c007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.319 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.313+0000 7f068affd640 1 -- 172.21.15.89:0/3885873450 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0690040070 con 0x7f069c071090 2024-04-27T03:33:30.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.484+0000 7f06a0c8c640 1 -- 172.21.15.89:0/3885873450 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) -- 0x7f069c062da0 con 0x7f067c038060 2024-04-27T03:33:30.490 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.485+0000 7f068affd640 1 -- 172.21.15.89:0/3885873450 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f069c062da0 con 0x7f067c038060 2024-04-27T03:33:30.490 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 0 to deep-scrub 2024-04-27T03:33:30.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.487+0000 7f06a0c8c640 1 -- 172.21.15.89:0/3885873450 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f067c038060 msgr2=0x7f067c03a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.487+0000 7f06a0c8c640 1 --2- 172.21.15.89:0/3885873450 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f067c038060 0x7f067c03a520 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f068c007fd0 tx=0x7f068c007ac0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.487+0000 7f06a0c8c640 1 -- 172.21.15.89:0/3885873450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c071090 msgr2=0x7f069c089f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.487+0000 7f06a0c8c640 1 --2- 172.21.15.89:0/3885873450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c071090 0x7f069c089f90 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f0690034040 tx=0x7f06900046c0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.487+0000 7f06a0c8c640 1 -- 172.21.15.89:0/3885873450 shutdown_connections 2024-04-27T03:33:30.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.487+0000 7f06a0c8c640 1 --2- 172.21.15.89:0/3885873450 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f067c038060 0x7f067c03a520 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.487+0000 7f06a0c8c640 1 --2- 172.21.15.89:0/3885873450 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f069c071090 0x7f069c089f90 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.493 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.487+0000 7f06a0c8c640 1 -- 172.21.15.89:0/3885873450 >> 172.21.15.89:0/3885873450 conn(0x7f069c099680 msgr2=0x7f069c09b4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:30.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.487+0000 7f06a0c8c640 1 -- 172.21.15.89:0/3885873450 shutdown_connections 2024-04-27T03:33:30.494 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.487+0000 7f06a0c8c640 1 -- 172.21.15.89:0/3885873450 wait complete. 2024-04-27T03:33:30.503 INFO:tasks.ceph:Scrubbing osd.1 2024-04-27T03:33:30.503 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2024-04-27T03:33:30.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.597+0000 7f692ff52640 1 Processor -- start 2024-04-27T03:33:30.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.597+0000 7f692ff52640 1 -- start start 2024-04-27T03:33:30.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.598+0000 7f692ff52640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6928098610 0x7f69280989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.598+0000 7f692ff52640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f6928099640 con 0x7f6928099d70 2024-04-27T03:33:30.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.598+0000 7f692ff52640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f69280734b0 con 0x7f6928098610 2024-04-27T03:33:30.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.598+0000 7f692dcee640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f6928098610 0x7f69280989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.598+0000 7f692d4ed640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f6928099d70 0x7f6928098f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51792/0 (socket says 172.21.15.89:51792) 2024-04-27T03:33:30.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.598+0000 7f692d4ed640 1 -- 172.21.15.89:0/2661304495 learned_addr learned my addr 172.21.15.89:0/2661304495 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:30.604 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.598+0000 7f692dcee640 1 -- 172.21.15.89:0/2661304495 >> v1:172.21.15.89:6789/0 conn(0x7f6928099d70 legacy=0x7f6928098f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.598+0000 7f692dcee640 1 -- 172.21.15.89:0/2661304495 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6928134ab0 con 0x7f6928098610 2024-04-27T03:33:30.605 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.598+0000 7f692ccec640 1 -- 172.21.15.89:0/2661304495 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3701508248 0 0) 0x7f6928099640 con 0x7f6928099d70 2024-04-27T03:33:30.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.599+0000 7f692dcee640 1 --2- 172.21.15.89:0/2661304495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6928098610 0x7f69280989f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6924009d90 tx=0x7f692402beb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6219296ebb7b324f server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.599+0000 7f692ccec640 1 -- 172.21.15.89:0/2661304495 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6924039070 con 0x7f6928098610 2024-04-27T03:33:30.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.599+0000 7f692ccec640 1 -- 172.21.15.89:0/2661304495 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692402cb10 con 0x7f6928098610 2024-04-27T03:33:30.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.599+0000 7f692ccec640 1 -- 172.21.15.89:0/2661304495 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f692402ce10 con 0x7f6928098610 2024-04-27T03:33:30.607 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.599+0000 7f692ff52640 1 -- 172.21.15.89:0/2661304495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6928098610 msgr2=0x7f69280989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.599+0000 7f692ff52640 1 --2- 172.21.15.89:0/2661304495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6928098610 0x7f69280989f0 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f6924009d90 tx=0x7f692402beb0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.599+0000 7f692ff52640 1 -- 172.21.15.89:0/2661304495 shutdown_connections 2024-04-27T03:33:30.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.599+0000 7f692ff52640 1 --2- 172.21.15.89:0/2661304495 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6928098610 0x7f69280989f0 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.599+0000 7f692ff52640 1 -- 172.21.15.89:0/2661304495 >> 172.21.15.89:0/2661304495 conn(0x7f692809ec10 msgr2=0x7f692809f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:30.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.600+0000 7f692ff52640 1 -- 172.21.15.89:0/2661304495 shutdown_connections 2024-04-27T03:33:30.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.600+0000 7f692ff52640 1 -- 172.21.15.89:0/2661304495 wait complete. 2024-04-27T03:33:30.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.600+0000 7f692ff52640 1 Processor -- start 2024-04-27T03:33:30.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.600+0000 7f692ff52640 1 -- start start 2024-04-27T03:33:30.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.601+0000 7f692ff52640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6928098610 0x7f6928132fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.608 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.601+0000 7f692ff52640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f6928133e30 con 0x7f6928098610 2024-04-27T03:33:30.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.601+0000 7f692dcee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6928098610 0x7f6928132fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.601+0000 7f692dcee640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6928098610 0x7f6928132fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49730/0 (socket says 172.21.15.89:49730) 2024-04-27T03:33:30.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.601+0000 7f692dcee640 1 -- 172.21.15.89:0/2344851685 learned_addr learned my addr 172.21.15.89:0/2344851685 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:30.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.601+0000 7f692dcee640 1 -- 172.21.15.89:0/2344851685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6928092370 con 0x7f6928098610 2024-04-27T03:33:30.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.601+0000 7f692dcee640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6928098610 0x7f6928132fe0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6924037040 tx=0x7f6924004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.601+0000 7f691e7fc640 1 -- 172.21.15.89:0/2344851685 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6924044030 con 0x7f6928098610 2024-04-27T03:33:30.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.601+0000 7f691e7fc640 1 -- 172.21.15.89:0/2344851685 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f692403f5a0 con 0x7f6928098610 2024-04-27T03:33:30.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.601+0000 7f692ff52640 1 -- 172.21.15.89:0/2344851685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6928093c10 con 0x7f6928098610 2024-04-27T03:33:30.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.601+0000 7f691e7fc640 1 -- 172.21.15.89:0/2344851685 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6924039040 con 0x7f6928098610 2024-04-27T03:33:30.609 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.601+0000 7f692ff52640 1 -- 172.21.15.89:0/2344851685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6928133520 con 0x7f6928098610 2024-04-27T03:33:30.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.602+0000 7f692ff52640 1 -- 172.21.15.89:0/2344851685 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69280973d0 con 0x7f6928098610 2024-04-27T03:33:30.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.602+0000 7f691e7fc640 1 -- 172.21.15.89:0/2344851685 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f6924047070 con 0x7f6928098610 2024-04-27T03:33:30.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.602+0000 7f691e7fc640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f690c038060 0x7f690c03a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.602+0000 7f691e7fc640 1 -- 172.21.15.89:0/2344851685 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f6924045070 con 0x7f6928098610 2024-04-27T03:33:30.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.602+0000 7f691e7fc640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x7f690c043de0 0x7f690c046280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.602+0000 7f691e7fc640 1 -- 172.21.15.89:0/2344851685 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f6924036e90 con 0x7f690c043de0 2024-04-27T03:33:30.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.602+0000 7f691e7fc640 1 -- 172.21.15.89:0/2344851685 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f6924075050 con 0x7f6928098610 2024-04-27T03:33:30.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.603+0000 7f692d4ed640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f690c038060 0x7f690c03a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.603+0000 7f692e4ef640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x7f690c043de0 0x7f690c046280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.603+0000 7f692d4ed640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f690c038060 0x7f690c03a520 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6918000c00 tx=0x7f6918007970 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.603+0000 7f692e4ef640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x7f690c043de0 0x7f690c046280 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.611 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.604+0000 7f691e7fc640 1 -- 172.21.15.89:0/2344851685 <== osd.1 v2:172.21.15.165:6800/79879041 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f6924036e90 con 0x7f690c043de0 2024-04-27T03:33:30.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.617+0000 7f692ff52640 1 -- 172.21.15.89:0/2344851685 --> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f6928062da0 con 0x7f690c043de0 2024-04-27T03:33:30.623 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.617+0000 7f691e7fc640 1 -- 172.21.15.89:0/2344851685 <== osd.1 v2:172.21.15.165:6800/79879041 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f6928062da0 con 0x7f690c043de0 2024-04-27T03:33:30.623 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-04-27T03:33:30.623 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T03:33:30.623 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-04-27T03:33:30.624 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 -- 172.21.15.89:0/2344851685 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x7f690c043de0 msgr2=0x7f690c046280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x7f690c043de0 0x7f690c046280 crc :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 -- 172.21.15.89:0/2344851685 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f690c038060 msgr2=0x7f690c03a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f690c038060 0x7f690c03a520 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7f6918000c00 tx=0x7f6918007970 comp rx=0 tx=0).stop 2024-04-27T03:33:30.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 -- 172.21.15.89:0/2344851685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6928098610 msgr2=0x7f6928132fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6928098610 0x7f6928132fe0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f6924037040 tx=0x7f6924004a40 comp rx=0 tx=0).stop 2024-04-27T03:33:30.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 -- 172.21.15.89:0/2344851685 shutdown_connections 2024-04-27T03:33:30.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.165:6800/79879041,v1:172.21.15.165:6801/79879041] conn(0x7f690c043de0 0x7f690c046280 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f690c038060 0x7f690c03a520 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 --2- 172.21.15.89:0/2344851685 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f6928098610 0x7f6928132fe0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 -- 172.21.15.89:0/2344851685 >> 172.21.15.89:0/2344851685 conn(0x7f692809ec10 msgr2=0x7f692806d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:30.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 -- 172.21.15.89:0/2344851685 shutdown_connections 2024-04-27T03:33:30.625 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.618+0000 7f692ff52640 1 -- 172.21.15.89:0/2344851685 wait complete. 2024-04-27T03:33:30.634 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2024-04-27T03:33:30.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.724+0000 7f2166dbd640 1 Processor -- start 2024-04-27T03:33:30.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.724+0000 7f2166dbd640 1 -- start start 2024-04-27T03:33:30.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.724+0000 7f2166dbd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2160127ce0 0x7f21601280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.724+0000 7f2166dbd640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f2160127b00 con 0x7f2160070100 2024-04-27T03:33:30.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.724+0000 7f2166dbd640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f21600708e0 con 0x7f2160127ce0 2024-04-27T03:33:30.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.724+0000 7f2164b59640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f2160127ce0 0x7f21601280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.731 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.724+0000 7f215ffff640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f2160070100 0x7f216012b000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51808/0 (socket says 172.21.15.89:51808) 2024-04-27T03:33:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.724+0000 7f215ffff640 1 -- 172.21.15.89:0/3257970985 learned_addr learned my addr 172.21.15.89:0/3257970985 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.725+0000 7f2164b59640 1 -- 172.21.15.89:0/3257970985 >> v1:172.21.15.89:6789/0 conn(0x7f2160070100 legacy=0x7f216012b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.725+0000 7f2164b59640 1 -- 172.21.15.89:0/3257970985 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2160098800 con 0x7f2160127ce0 2024-04-27T03:33:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.725+0000 7f2164b59640 1 --2- 172.21.15.89:0/3257970985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2160127ce0 0x7f21601280c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f215400b9c0 tx=0x7f215402e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c35442cb148c8887 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.725+0000 7f215f7fe640 1 -- 172.21.15.89:0/3257970985 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f215403b070 con 0x7f2160127ce0 2024-04-27T03:33:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.725+0000 7f215f7fe640 1 -- 172.21.15.89:0/3257970985 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f215402ecf0 con 0x7f2160127ce0 2024-04-27T03:33:30.732 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.725+0000 7f215f7fe640 1 -- 172.21.15.89:0/3257970985 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2154036440 con 0x7f2160127ce0 2024-04-27T03:33:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.726+0000 7f2166dbd640 1 -- 172.21.15.89:0/3257970985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2160127ce0 msgr2=0x7f21601280c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.726+0000 7f2166dbd640 1 --2- 172.21.15.89:0/3257970985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2160127ce0 0x7f21601280c0 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f215400b9c0 tx=0x7f215402e010 comp rx=0 tx=0).stop 2024-04-27T03:33:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.726+0000 7f2166dbd640 1 -- 172.21.15.89:0/3257970985 shutdown_connections 2024-04-27T03:33:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.726+0000 7f2166dbd640 1 --2- 172.21.15.89:0/3257970985 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2160127ce0 0x7f21601280c0 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.726+0000 7f2166dbd640 1 -- 172.21.15.89:0/3257970985 >> 172.21.15.89:0/3257970985 conn(0x7f2160099680 msgr2=0x7f2160099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.726+0000 7f2166dbd640 1 -- 172.21.15.89:0/3257970985 shutdown_connections 2024-04-27T03:33:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.726+0000 7f2166dbd640 1 -- 172.21.15.89:0/3257970985 wait complete. 2024-04-27T03:33:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.727+0000 7f2166dbd640 1 Processor -- start 2024-04-27T03:33:30.735 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.727+0000 7f2166dbd640 1 -- start start 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.727+0000 7f2166dbd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2160070100 0x7f216008e3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.727+0000 7f2166dbd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f2160128780 con 0x7f2160070100 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.727+0000 7f2164b59640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2160070100 0x7f216008e3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.727+0000 7f2164b59640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2160070100 0x7f216008e3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49754/0 (socket says 172.21.15.89:49754) 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.727+0000 7f2164b59640 1 -- 172.21.15.89:0/2720736260 learned_addr learned my addr 172.21.15.89:0/2720736260 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.728+0000 7f2164b59640 1 -- 172.21.15.89:0/2720736260 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2160090870 con 0x7f2160070100 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.728+0000 7f2164b59640 1 --2- 172.21.15.89:0/2720736260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2160070100 0x7f216008e3a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f215402e050 tx=0x7f21540046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.728+0000 7f215d7fa640 1 -- 172.21.15.89:0/2720736260 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2154046070 con 0x7f2160070100 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.728+0000 7f215d7fa640 1 -- 172.21.15.89:0/2720736260 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f2154041560 con 0x7f2160070100 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.728+0000 7f215d7fa640 1 -- 172.21.15.89:0/2720736260 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f215403b040 con 0x7f2160070100 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.728+0000 7f2166dbd640 1 -- 172.21.15.89:0/2720736260 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2160125450 con 0x7f2160070100 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.728+0000 7f2166dbd640 1 -- 172.21.15.89:0/2720736260 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2160090260 con 0x7f2160070100 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.729+0000 7f215d7fa640 1 -- 172.21.15.89:0/2720736260 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f2154049070 con 0x7f2160070100 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.729+0000 7f215d7fa640 1 --2- 172.21.15.89:0/2720736260 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2144038010 0x7f214403a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.729+0000 7f2166dbd640 1 -- 172.21.15.89:0/2720736260 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2160090a40 con 0x7f2160070100 2024-04-27T03:33:30.736 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.729+0000 7f215d7fa640 1 -- 172.21.15.89:0/2720736260 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f215406d6f0 con 0x7f2160070100 2024-04-27T03:33:30.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.729+0000 7f215ffff640 1 --2- 172.21.15.89:0/2720736260 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2144038010 0x7f214403a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:30.737 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.730+0000 7f215ffff640 1 --2- 172.21.15.89:0/2720736260 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2144038010 0x7f214403a4d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f214c000f60 tx=0x7f214c007a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:30.739 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.733+0000 7f215d7fa640 1 -- 172.21.15.89:0/2720736260 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f21540355f0 con 0x7f2160070100 2024-04-27T03:33:30.914 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.909+0000 7f2166dbd640 1 -- 172.21.15.89:0/2720736260 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) -- 0x7f2160062da0 con 0x7f2144038010 2024-04-27T03:33:30.915 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.910+0000 7f215d7fa640 1 -- 172.21.15.89:0/2720736260 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f2160062da0 con 0x7f2144038010 2024-04-27T03:33:30.915 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 1 to deep-scrub 2024-04-27T03:33:30.917 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.911+0000 7f2166dbd640 1 -- 172.21.15.89:0/2720736260 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2144038010 msgr2=0x7f214403a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.911+0000 7f2166dbd640 1 --2- 172.21.15.89:0/2720736260 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2144038010 0x7f214403a4d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f214c000f60 tx=0x7f214c007a80 comp rx=0 tx=0).stop 2024-04-27T03:33:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.911+0000 7f2166dbd640 1 -- 172.21.15.89:0/2720736260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2160070100 msgr2=0x7f216008e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.911+0000 7f2166dbd640 1 --2- 172.21.15.89:0/2720736260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2160070100 0x7f216008e3a0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f215402e050 tx=0x7f21540046c0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.911+0000 7f2166dbd640 1 -- 172.21.15.89:0/2720736260 shutdown_connections 2024-04-27T03:33:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.911+0000 7f2166dbd640 1 --2- 172.21.15.89:0/2720736260 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f2144038010 0x7f214403a4d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.911+0000 7f2166dbd640 1 --2- 172.21.15.89:0/2720736260 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f2160070100 0x7f216008e3a0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.911+0000 7f2166dbd640 1 -- 172.21.15.89:0/2720736260 >> 172.21.15.89:0/2720736260 conn(0x7f2160099680 msgr2=0x7f2160126300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.912+0000 7f2166dbd640 1 -- 172.21.15.89:0/2720736260 shutdown_connections 2024-04-27T03:33:30.918 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:30.912+0000 7f2166dbd640 1 -- 172.21.15.89:0/2720736260 wait complete. 2024-04-27T03:33:30.928 INFO:tasks.ceph:Scrubbing osd.2 2024-04-27T03:33:30.928 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2024-04-27T03:33:31.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.021+0000 7f0d521db640 1 Processor -- start 2024-04-27T03:33:31.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.021+0000 7f0d521db640 1 -- start start 2024-04-27T03:33:31.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.022+0000 7f0d521db640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d4c099d70 0x7f0d4c099100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.022+0000 7f0d521db640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0d4c099640 con 0x7f0d4c098610 2024-04-27T03:33:31.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.022+0000 7f0d521db640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f0d4c0734b0 con 0x7f0d4c099d70 2024-04-27T03:33:31.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.022+0000 7f0d4affd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0d4c099d70 0x7f0d4c099100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.028 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.022+0000 7f0d4b7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0d4c098610 0x7f0d4c0989f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51812/0 (socket says 172.21.15.89:51812) 2024-04-27T03:33:31.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.022+0000 7f0d4b7fe640 1 -- 172.21.15.89:0/692183055 learned_addr learned my addr 172.21.15.89:0/692183055 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:31.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.022+0000 7f0d4affd640 1 -- 172.21.15.89:0/692183055 >> v1:172.21.15.89:6789/0 conn(0x7f0d4c098610 legacy=0x7f0d4c0989f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.022+0000 7f0d4affd640 1 -- 172.21.15.89:0/692183055 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c134ab0 con 0x7f0d4c099d70 2024-04-27T03:33:31.029 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.022+0000 7f0d4a7fc640 1 -- 172.21.15.89:0/692183055 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3278241670 0 0) 0x7f0d4c099640 con 0x7f0d4c098610 2024-04-27T03:33:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.023+0000 7f0d4affd640 1 --2- 172.21.15.89:0/692183055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d4c099d70 0x7f0d4c099100 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009720 tx=0x7f0d4002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbdbdb0b0ea76430 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.023+0000 7f0d4a7fc640 1 -- 172.21.15.89:0/692183055 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d400316d0 con 0x7f0d4c099d70 2024-04-27T03:33:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.023+0000 7f0d4a7fc640 1 -- 172.21.15.89:0/692183055 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d40037070 con 0x7f0d4c099d70 2024-04-27T03:33:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.023+0000 7f0d4a7fc640 1 -- 172.21.15.89:0/692183055 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d400319c0 con 0x7f0d4c099d70 2024-04-27T03:33:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.023+0000 7f0d521db640 1 -- 172.21.15.89:0/692183055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d4c099d70 msgr2=0x7f0d4c099100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.023+0000 7f0d521db640 1 --2- 172.21.15.89:0/692183055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d4c099d70 0x7f0d4c099100 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f0d40009720 tx=0x7f0d4002b840 comp rx=0 tx=0).stop 2024-04-27T03:33:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.023+0000 7f0d521db640 1 -- 172.21.15.89:0/692183055 shutdown_connections 2024-04-27T03:33:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.023+0000 7f0d521db640 1 --2- 172.21.15.89:0/692183055 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d4c099d70 0x7f0d4c099100 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.023+0000 7f0d521db640 1 -- 172.21.15.89:0/692183055 >> 172.21.15.89:0/692183055 conn(0x7f0d4c09ec10 msgr2=0x7f0d4c09f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.024+0000 7f0d521db640 1 -- 172.21.15.89:0/692183055 shutdown_connections 2024-04-27T03:33:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.024+0000 7f0d521db640 1 -- 172.21.15.89:0/692183055 wait complete. 2024-04-27T03:33:31.033 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.024+0000 7f0d521db640 1 Processor -- start 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.024+0000 7f0d521db640 1 -- start start 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.025+0000 7f0d521db640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d4c098610 0x7f0d4c0904d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.025+0000 7f0d521db640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0d4c133e30 con 0x7f0d4c098610 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.025+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d4c098610 0x7f0d4c0904d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.025+0000 7f0d4b7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d4c098610 0x7f0d4c0904d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49776/0 (socket says 172.21.15.89:49776) 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.025+0000 7f0d4b7fe640 1 -- 172.21.15.89:0/3153253794 learned_addr learned my addr 172.21.15.89:0/3153253794 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.025+0000 7f0d4b7fe640 1 -- 172.21.15.89:0/3153253794 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4c093ee0 con 0x7f0d4c098610 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.025+0000 7f0d4b7fe640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d4c098610 0x7f0d4c0904d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00c420 tx=0x7f0d3c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.025+0000 7f0d33fff640 1 -- 172.21.15.89:0/3153253794 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c018070 con 0x7f0d4c098610 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.025+0000 7f0d33fff640 1 -- 172.21.15.89:0/3153253794 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0d3c0074d0 con 0x7f0d4c098610 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.025+0000 7f0d521db640 1 -- 172.21.15.89:0/3153253794 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4c092390 con 0x7f0d4c098610 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.025+0000 7f0d33fff640 1 -- 172.21.15.89:0/3153253794 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0d3c004950 con 0x7f0d4c098610 2024-04-27T03:33:31.034 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.025+0000 7f0d521db640 1 -- 172.21.15.89:0/3153253794 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d4c093020 con 0x7f0d4c098610 2024-04-27T03:33:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.026+0000 7f0d521db640 1 -- 172.21.15.89:0/3153253794 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f0d4c062da0 con 0x7f0d4c098610 2024-04-27T03:33:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.026+0000 7f0d33fff640 1 -- 172.21.15.89:0/3153253794 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0d3c01b070 con 0x7f0d4c098610 2024-04-27T03:33:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.026+0000 7f0d33fff640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0d38038060 0x7f0d3803a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.026+0000 7f0d33fff640 1 -- 172.21.15.89:0/3153253794 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0d3c04ac60 con 0x7f0d4c098610 2024-04-27T03:33:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.027+0000 7f0d33fff640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d38043de0 0x7f0d38046280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.027+0000 7f0d33fff640 1 -- 172.21.15.89:0/3153253794 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f0d38046950 con 0x7f0d38043de0 2024-04-27T03:33:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.027+0000 7f0d33fff640 1 -- 172.21.15.89:0/3153253794 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7f0d3c0502a0 con 0x7f0d4c098610 2024-04-27T03:33:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.027+0000 7f0d4affd640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0d38038060 0x7f0d3803a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.027+0000 7f0d4bfff640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d38043de0 0x7f0d38046280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.035 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.027+0000 7f0d4affd640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0d38038060 0x7f0d3803a520 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0d40036000 tx=0x7f0d40008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.027+0000 7f0d4bfff640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d38043de0 0x7f0d38046280 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.036 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.028+0000 7f0d33fff640 1 -- 172.21.15.89:0/3153253794 <== osd.2 v2:172.21.15.165:6816/975457786 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f0d38046950 con 0x7f0d38043de0 2024-04-27T03:33:31.046 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.041+0000 7f0d521db640 1 -- 172.21.15.89:0/3153253794 --> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7f0d4c097090 con 0x7f0d38043de0 2024-04-27T03:33:31.047 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.041+0000 7f0d33fff640 1 -- 172.21.15.89:0/3153253794 <== osd.2 v2:172.21.15.165:6816/975457786 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7f0d4c097090 con 0x7f0d38043de0 2024-04-27T03:33:31.047 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-04-27T03:33:31.047 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T03:33:31.047 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-04-27T03:33:31.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 -- 172.21.15.89:0/3153253794 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d38043de0 msgr2=0x7f0d38046280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d38043de0 0x7f0d38046280 crc :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.048 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 -- 172.21.15.89:0/3153253794 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0d38038060 msgr2=0x7f0d3803a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0d38038060 0x7f0d3803a520 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f0d40036000 tx=0x7f0d40008030 comp rx=0 tx=0).stop 2024-04-27T03:33:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 -- 172.21.15.89:0/3153253794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d4c098610 msgr2=0x7f0d4c0904d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d4c098610 0x7f0d4c0904d0 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00c420 tx=0x7f0d3c00c8e0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 -- 172.21.15.89:0/3153253794 shutdown_connections 2024-04-27T03:33:31.049 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.165:6816/975457786,v1:172.21.15.165:6817/975457786] conn(0x7f0d38043de0 0x7f0d38046280 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0d38038060 0x7f0d3803a520 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 --2- 172.21.15.89:0/3153253794 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0d4c098610 0x7f0d4c0904d0 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 -- 172.21.15.89:0/3153253794 >> 172.21.15.89:0/3153253794 conn(0x7f0d4c09ec10 msgr2=0x7f0d4c06d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:31.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 -- 172.21.15.89:0/3153253794 shutdown_connections 2024-04-27T03:33:31.050 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.042+0000 7f0d521db640 1 -- 172.21.15.89:0/3153253794 wait complete. 2024-04-27T03:33:31.057 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2024-04-27T03:33:31.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.148+0000 7f5b0d1f4640 1 Processor -- start 2024-04-27T03:33:31.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.148+0000 7f5b0d1f4640 1 -- start start 2024-04-27T03:33:31.155 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.148+0000 7f5b0d1f4640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5b0806fd50 0x7f5b08133860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.149+0000 7f5b0d1f4640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f5b08133da0 con 0x7f5b0806f260 2024-04-27T03:33:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.149+0000 7f5b0d1f4640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f5b08070bc0 con 0x7f5b0806fd50 2024-04-27T03:33:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.149+0000 7f5b0659c640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f5b0806fd50 0x7f5b08133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.149+0000 7f5b06d9d640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f5b0806f260 0x7f5b0806f640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51816/0 (socket says 172.21.15.89:51816) 2024-04-27T03:33:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.149+0000 7f5b06d9d640 1 -- 172.21.15.89:0/540308367 learned_addr learned my addr 172.21.15.89:0/540308367 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.149+0000 7f5b0659c640 1 -- 172.21.15.89:0/540308367 >> v1:172.21.15.89:6789/0 conn(0x7f5b0806f260 legacy=0x7f5b0806f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.149+0000 7f5b05d9b640 1 -- 172.21.15.89:0/540308367 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1735900482 0 0) 0x7f5b08133da0 con 0x7f5b0806f260 2024-04-27T03:33:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.149+0000 7f5b0659c640 1 -- 172.21.15.89:0/540308367 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b08134f80 con 0x7f5b0806fd50 2024-04-27T03:33:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.150+0000 7f5b0659c640 1 --2- 172.21.15.89:0/540308367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b0806fd50 0x7f5b08133860 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009720 tx=0x7f5afc02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db7134085c75b57 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.150+0000 7f5b05d9b640 1 -- 172.21.15.89:0/540308367 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5afc0316d0 con 0x7f5b0806fd50 2024-04-27T03:33:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.150+0000 7f5b05d9b640 1 -- 172.21.15.89:0/540308367 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5afc037070 con 0x7f5b0806fd50 2024-04-27T03:33:31.156 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.150+0000 7f5b05d9b640 1 -- 172.21.15.89:0/540308367 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5afc0319c0 con 0x7f5b0806fd50 2024-04-27T03:33:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.150+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/540308367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b0806fd50 msgr2=0x7f5b08133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.150+0000 7f5b0d1f4640 1 --2- 172.21.15.89:0/540308367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b0806fd50 0x7f5b08133860 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009720 tx=0x7f5afc02b840 comp rx=0 tx=0).stop 2024-04-27T03:33:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.150+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/540308367 shutdown_connections 2024-04-27T03:33:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.150+0000 7f5b0d1f4640 1 --2- 172.21.15.89:0/540308367 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b0806fd50 0x7f5b08133860 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.150+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/540308367 >> 172.21.15.89:0/540308367 conn(0x7f5b0809e170 msgr2=0x7f5b0806d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:31.159 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.150+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/540308367 shutdown_connections 2024-04-27T03:33:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.150+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/540308367 wait complete. 2024-04-27T03:33:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.151+0000 7f5b0d1f4640 1 Processor -- start 2024-04-27T03:33:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.151+0000 7f5b0d1f4640 1 -- start start 2024-04-27T03:33:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.152+0000 7f5b0d1f4640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b0806f260 0x7f5b08124320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.152+0000 7f5b0d1f4640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f5b08134300 con 0x7f5b0806f260 2024-04-27T03:33:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.152+0000 7f5b06d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b0806f260 0x7f5b08124320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.152+0000 7f5b06d9d640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b0806f260 0x7f5b08124320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49800/0 (socket says 172.21.15.89:49800) 2024-04-27T03:33:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.152+0000 7f5b06d9d640 1 -- 172.21.15.89:0/2572847175 learned_addr learned my addr 172.21.15.89:0/2572847175 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.152+0000 7f5b06d9d640 1 -- 172.21.15.89:0/2572847175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b08130bf0 con 0x7f5b0806f260 2024-04-27T03:33:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.152+0000 7f5b06d9d640 1 --2- 172.21.15.89:0/2572847175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b0806f260 0x7f5b08124320 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5af800c420 tx=0x7f5af800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.160 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.152+0000 7f5aef7fe640 1 -- 172.21.15.89:0/2572847175 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af8018070 con 0x7f5b0806f260 2024-04-27T03:33:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.153+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/2572847175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b081333a0 con 0x7f5b0806f260 2024-04-27T03:33:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.153+0000 7f5aef7fe640 1 -- 172.21.15.89:0/2572847175 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f5af80074d0 con 0x7f5b0806f260 2024-04-27T03:33:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.153+0000 7f5aef7fe640 1 -- 172.21.15.89:0/2572847175 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5af8004950 con 0x7f5b0806f260 2024-04-27T03:33:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.153+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/2572847175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b081315d0 con 0x7f5b0806f260 2024-04-27T03:33:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.153+0000 7f5aef7fe640 1 -- 172.21.15.89:0/2572847175 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f5af801b070 con 0x7f5b0806f260 2024-04-27T03:33:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.153+0000 7f5aef7fe640 1 --2- 172.21.15.89:0/2572847175 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5af4038010 0x7f5af403a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.153+0000 7f5aef7fe640 1 -- 172.21.15.89:0/2572847175 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f5af804acb0 con 0x7f5b0806f260 2024-04-27T03:33:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.154+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/2572847175 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5b080a18a0 con 0x7f5b0806f260 2024-04-27T03:33:31.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.154+0000 7f5b0659c640 1 --2- 172.21.15.89:0/2572847175 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5af4038010 0x7f5af403a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.162 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.154+0000 7f5b0659c640 1 --2- 172.21.15.89:0/2572847175 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5af4038010 0x7f5af403a4d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009720 tx=0x7f5afc008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.164 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.158+0000 7f5aef7fe640 1 -- 172.21.15.89:0/2572847175 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f5b080a18a0 con 0x7f5b0806f260 2024-04-27T03:33:31.336 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.330+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/2572847175 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) -- 0x7f5b08062da0 con 0x7f5af4038010 2024-04-27T03:33:31.337 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.331+0000 7f5aef7fe640 1 -- 172.21.15.89:0/2572847175 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f5b08062da0 con 0x7f5af4038010 2024-04-27T03:33:31.337 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 2 to deep-scrub 2024-04-27T03:33:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.333+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/2572847175 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5af4038010 msgr2=0x7f5af403a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.333+0000 7f5b0d1f4640 1 --2- 172.21.15.89:0/2572847175 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5af4038010 0x7f5af403a4d0 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f5afc009720 tx=0x7f5afc008030 comp rx=0 tx=0).stop 2024-04-27T03:33:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.333+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/2572847175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b0806f260 msgr2=0x7f5b08124320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.333+0000 7f5b0d1f4640 1 --2- 172.21.15.89:0/2572847175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b0806f260 0x7f5b08124320 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f5af800c420 tx=0x7f5af800c8e0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.333+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/2572847175 shutdown_connections 2024-04-27T03:33:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.333+0000 7f5b0d1f4640 1 --2- 172.21.15.89:0/2572847175 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f5af4038010 0x7f5af403a4d0 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.333+0000 7f5b0d1f4640 1 --2- 172.21.15.89:0/2572847175 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f5b0806f260 0x7f5b08124320 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.339 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.333+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/2572847175 >> 172.21.15.89:0/2572847175 conn(0x7f5b0809e170 msgr2=0x7f5b0809e790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.333+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/2572847175 shutdown_connections 2024-04-27T03:33:31.340 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.333+0000 7f5b0d1f4640 1 -- 172.21.15.89:0/2572847175 wait complete. 2024-04-27T03:33:31.349 INFO:tasks.ceph:Scrubbing osd.3 2024-04-27T03:33:31.349 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2024-04-27T03:33:31.452 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.445+0000 7fc9c5c82640 1 Processor -- start 2024-04-27T03:33:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.445+0000 7fc9c5c82640 1 -- start start 2024-04-27T03:33:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.446+0000 7fc9c5c82640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc9c0098610 0x7fc9c00989f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.446+0000 7fc9c5c82640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fc9c0099640 con 0x7fc9c0099d70 2024-04-27T03:33:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.446+0000 7fc9c5c82640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fc9c00734b0 con 0x7fc9c0098610 2024-04-27T03:33:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.446+0000 7fc9bf7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc9c0098610 0x7fc9c00989f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.446+0000 7fc9beffd640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fc9c0099d70 0x7fc9c0098f30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51828/0 (socket says 172.21.15.89:51828) 2024-04-27T03:33:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.446+0000 7fc9bf7fe640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fc9c0098610 0x7fc9c00989f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49808/0 (socket says 172.21.15.89:49808) 2024-04-27T03:33:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.446+0000 7fc9beffd640 1 -- 172.21.15.89:0/406274832 learned_addr learned my addr 172.21.15.89:0/406274832 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.446+0000 7fc9bf7fe640 1 -- 172.21.15.89:0/406274832 >> v1:172.21.15.89:6789/0 conn(0x7fc9c0099d70 legacy=0x7fc9c0098f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.453 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.446+0000 7fc9bf7fe640 1 -- 172.21.15.89:0/406274832 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c0134ab0 con 0x7fc9c0098610 2024-04-27T03:33:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.447+0000 7fc9bf7fe640 1 --2- 172.21.15.89:0/406274832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9c0098610 0x7fc9c00989f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009720 tx=0x7fc9b002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0da9674fa312603 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.447+0000 7fc9be7fc640 1 -- 172.21.15.89:0/406274832 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b00316d0 con 0x7fc9c0098610 2024-04-27T03:33:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.447+0000 7fc9be7fc640 1 -- 172.21.15.89:0/406274832 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b0037070 con 0x7fc9c0098610 2024-04-27T03:33:31.454 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.447+0000 7fc9be7fc640 1 -- 172.21.15.89:0/406274832 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b00319c0 con 0x7fc9c0098610 2024-04-27T03:33:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.447+0000 7fc9c5c82640 1 -- 172.21.15.89:0/406274832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9c0098610 msgr2=0x7fc9c00989f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.447+0000 7fc9c5c82640 1 --2- 172.21.15.89:0/406274832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9c0098610 0x7fc9c00989f0 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0009720 tx=0x7fc9b002b840 comp rx=0 tx=0).stop 2024-04-27T03:33:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.447+0000 7fc9c5c82640 1 -- 172.21.15.89:0/406274832 shutdown_connections 2024-04-27T03:33:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.447+0000 7fc9c5c82640 1 --2- 172.21.15.89:0/406274832 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9c0098610 0x7fc9c00989f0 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.447+0000 7fc9c5c82640 1 -- 172.21.15.89:0/406274832 >> 172.21.15.89:0/406274832 conn(0x7fc9c009ec10 msgr2=0x7fc9c009f020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.448+0000 7fc9c5c82640 1 -- 172.21.15.89:0/406274832 shutdown_connections 2024-04-27T03:33:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.448+0000 7fc9c5c82640 1 -- 172.21.15.89:0/406274832 wait complete. 2024-04-27T03:33:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.448+0000 7fc9c5c82640 1 Processor -- start 2024-04-27T03:33:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.448+0000 7fc9c5c82640 1 -- start start 2024-04-27T03:33:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.449+0000 7fc9c5c82640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9c0098610 0x7fc9c0090370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.458 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.449+0000 7fc9c5c82640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fc9c0133e30 con 0x7fc9c0098610 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.449+0000 7fc9bf7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9c0098610 0x7fc9c0090370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.449+0000 7fc9bf7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9c0098610 0x7fc9c0090370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49820/0 (socket says 172.21.15.89:49820) 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.449+0000 7fc9bf7fe640 1 -- 172.21.15.89:0/1668680210 learned_addr learned my addr 172.21.15.89:0/1668680210 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.449+0000 7fc9bf7fe640 1 -- 172.21.15.89:0/1668680210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9c0093ea0 con 0x7fc9c0098610 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.449+0000 7fc9bf7fe640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9c0098610 0x7fc9c0090370 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0034040 tx=0x7fc9b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.449+0000 7fc9a7fff640 1 -- 172.21.15.89:0/1668680210 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b0032770 con 0x7fc9c0098610 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.449+0000 7fc9a7fff640 1 -- 172.21.15.89:0/1668680210 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fc9b0032910 con 0x7fc9c0098610 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.449+0000 7fc9c5c82640 1 -- 172.21.15.89:0/1668680210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9c00932c0 con 0x7fc9c0098610 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.449+0000 7fc9a7fff640 1 -- 172.21.15.89:0/1668680210 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc9b0032c30 con 0x7fc9c0098610 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.449+0000 7fc9c5c82640 1 -- 172.21.15.89:0/1668680210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9c01332a0 con 0x7fc9c0098610 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.450+0000 7fc9c5c82640 1 -- 172.21.15.89:0/1668680210 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc9c0097680 con 0x7fc9c0098610 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.450+0000 7fc9a7fff640 1 -- 172.21.15.89:0/1668680210 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fc9b004c060 con 0x7fc9c0098610 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.450+0000 7fc9a7fff640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc9ac038060 0x7fc9ac03a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.459 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.450+0000 7fc9a7fff640 1 -- 172.21.15.89:0/1668680210 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fc9b002f050 con 0x7fc9c0098610 2024-04-27T03:33:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.451+0000 7fc9a7fff640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x7fc9ac043de0 0x7fc9ac046280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.451+0000 7fc9a7fff640 1 -- 172.21.15.89:0/1668680210 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc9b004fe50 con 0x7fc9ac043de0 2024-04-27T03:33:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.451+0000 7fc9a7fff640 1 -- 172.21.15.89:0/1668680210 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_get_version_reply(handle=1 version=27) ==== 24+0+0 (secure 0 0 0) 0x7fc9b0074050 con 0x7fc9c0098610 2024-04-27T03:33:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.451+0000 7fc9beffd640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc9ac038060 0x7fc9ac03a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.451+0000 7fc9bffff640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x7fc9ac043de0 0x7fc9ac046280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.451+0000 7fc9bffff640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x7fc9ac043de0 0x7fc9ac046280 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.451+0000 7fc9beffd640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc9ac038060 0x7fc9ac03a520 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4005880 tx=0x7fc9b4007a80 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.460 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.452+0000 7fc9a7fff640 1 -- 172.21.15.89:0/1668680210 <== osd.3 v2:172.21.15.165:6808/2380918637 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc9b004fe50 con 0x7fc9ac043de0 2024-04-27T03:33:31.470 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.465+0000 7fc9c5c82640 1 -- 172.21.15.89:0/1668680210 --> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) -- 0x7fc9c0062da0 con 0x7fc9ac043de0 2024-04-27T03:33:31.471 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9a7fff640 1 -- 172.21.15.89:0/1668680210 <== osd.3 v2:172.21.15.165:6808/2380918637 2 ==== command_reply(tid 2: 0 ) ==== 8+0+97 (crc 0 0 0) 0x7fc9c0062da0 con 0x7fc9ac043de0 2024-04-27T03:33:31.472 INFO:teuthology.orchestra.run.smithi089.stdout:{ 2024-04-27T03:33:31.472 INFO:teuthology.orchestra.run.smithi089.stdout: "success": "osd_debug_deep_scrub_sleep = '' (not observed, change may require restart) " 2024-04-27T03:33:31.472 INFO:teuthology.orchestra.run.smithi089.stdout:} 2024-04-27T03:33:31.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 -- 172.21.15.89:0/1668680210 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x7fc9ac043de0 msgr2=0x7fc9ac046280 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x7fc9ac043de0 0x7fc9ac046280 crc :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 -- 172.21.15.89:0/1668680210 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc9ac038060 msgr2=0x7fc9ac03a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc9ac038060 0x7fc9ac03a520 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fc9b4005880 tx=0x7fc9b4007a80 comp rx=0 tx=0).stop 2024-04-27T03:33:31.473 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 -- 172.21.15.89:0/1668680210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9c0098610 msgr2=0x7fc9c0090370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9c0098610 0x7fc9c0090370 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7fc9b0034040 tx=0x7fc9b00046c0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 -- 172.21.15.89:0/1668680210 shutdown_connections 2024-04-27T03:33:31.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.165:6808/2380918637,v1:172.21.15.165:6809/2380918637] conn(0x7fc9ac043de0 0x7fc9ac046280 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fc9ac038060 0x7fc9ac03a520 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 --2- 172.21.15.89:0/1668680210 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fc9c0098610 0x7fc9c0090370 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 -- 172.21.15.89:0/1668680210 >> 172.21.15.89:0/1668680210 conn(0x7fc9c009ec10 msgr2=0x7fc9c006d4d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:31.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 -- 172.21.15.89:0/1668680210 shutdown_connections 2024-04-27T03:33:31.474 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.466+0000 7fc9c5c82640 1 -- 172.21.15.89:0/1668680210 wait complete. 2024-04-27T03:33:31.482 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2024-04-27T03:33:31.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.573+0000 7f4c2c7ec640 1 Processor -- start 2024-04-27T03:33:31.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.573+0000 7f4c2c7ec640 1 -- start start 2024-04-27T03:33:31.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.573+0000 7f4c2c7ec640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c24127ce0 0x7f4c241280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.579 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.573+0000 7f4c2c7ec640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f4c24127b00 con 0x7f4c24070100 2024-04-27T03:33:31.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.573+0000 7f4c2c7ec640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f4c240708e0 con 0x7f4c24127ce0 2024-04-27T03:33:31.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.573+0000 7f4c2a588640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f4c24127ce0 0x7f4c241280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.573+0000 7f4c29d87640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f4c24070100 0x7f4c2412b000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51832/0 (socket says 172.21.15.89:51832) 2024-04-27T03:33:31.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.573+0000 7f4c29d87640 1 -- 172.21.15.89:0/2429661426 learned_addr learned my addr 172.21.15.89:0/2429661426 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:31.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.574+0000 7f4c2a588640 1 -- 172.21.15.89:0/2429661426 >> v1:172.21.15.89:6789/0 conn(0x7f4c24070100 legacy=0x7f4c2412b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.574+0000 7f4c2a588640 1 -- 172.21.15.89:0/2429661426 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c24098800 con 0x7f4c24127ce0 2024-04-27T03:33:31.580 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.574+0000 7f4c29586640 1 -- 172.21.15.89:0/2429661426 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 1752024012 0 0) 0x7f4c14001110 con 0x7f4c24070100 2024-04-27T03:33:31.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.574+0000 7f4c2a588640 1 --2- 172.21.15.89:0/2429661426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c24127ce0 0x7f4c241280c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009720 tx=0x7f4c2002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a180e2d4a35701c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.574+0000 7f4c29586640 1 -- 172.21.15.89:0/2429661426 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c200316d0 con 0x7f4c24127ce0 2024-04-27T03:33:31.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.574+0000 7f4c29586640 1 -- 172.21.15.89:0/2429661426 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c20037070 con 0x7f4c24127ce0 2024-04-27T03:33:31.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.574+0000 7f4c29586640 1 -- 172.21.15.89:0/2429661426 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c200319c0 con 0x7f4c24127ce0 2024-04-27T03:33:31.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.575+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/2429661426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c24127ce0 msgr2=0x7f4c241280c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.575+0000 7f4c2c7ec640 1 --2- 172.21.15.89:0/2429661426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c24127ce0 0x7f4c241280c0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009720 tx=0x7f4c2002b840 comp rx=0 tx=0).stop 2024-04-27T03:33:31.583 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.575+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/2429661426 shutdown_connections 2024-04-27T03:33:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.575+0000 7f4c2c7ec640 1 --2- 172.21.15.89:0/2429661426 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c24127ce0 0x7f4c241280c0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.575+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/2429661426 >> 172.21.15.89:0/2429661426 conn(0x7f4c24099680 msgr2=0x7f4c24099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.575+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/2429661426 shutdown_connections 2024-04-27T03:33:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.575+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/2429661426 wait complete. 2024-04-27T03:33:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.576+0000 7f4c2c7ec640 1 Processor -- start 2024-04-27T03:33:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.576+0000 7f4c2c7ec640 1 -- start start 2024-04-27T03:33:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.576+0000 7f4c2c7ec640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c24070100 0x7f4c241c2260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.576+0000 7f4c2c7ec640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f4c24128780 con 0x7f4c24070100 2024-04-27T03:33:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.576+0000 7f4c2a588640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c24070100 0x7f4c241c2260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.576+0000 7f4c2a588640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c24070100 0x7f4c241c2260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49830/0 (socket says 172.21.15.89:49830) 2024-04-27T03:33:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.576+0000 7f4c2a588640 1 -- 172.21.15.89:0/753246887 learned_addr learned my addr 172.21.15.89:0/753246887 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:31.584 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.577+0000 7f4c2a588640 1 -- 172.21.15.89:0/753246887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c2408eb20 con 0x7f4c24070100 2024-04-27T03:33:31.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.577+0000 7f4c2a588640 1 --2- 172.21.15.89:0/753246887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c24070100 0x7f4c241c2260 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009840 tx=0x7f4c200046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.577+0000 7f4c1affd640 1 -- 172.21.15.89:0/753246887 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c20032770 con 0x7f4c24070100 2024-04-27T03:33:31.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.577+0000 7f4c1affd640 1 -- 172.21.15.89:0/753246887 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f4c20032910 con 0x7f4c24070100 2024-04-27T03:33:31.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.577+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/753246887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c2412a330 con 0x7f4c24070100 2024-04-27T03:33:31.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.577+0000 7f4c1affd640 1 -- 172.21.15.89:0/753246887 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4c20032c30 con 0x7f4c24070100 2024-04-27T03:33:31.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.577+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/753246887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c2409ba00 con 0x7f4c24070100 2024-04-27T03:33:31.585 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.578+0000 7f4c1affd640 1 -- 172.21.15.89:0/753246887 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f4c2004d3f0 con 0x7f4c24070100 2024-04-27T03:33:31.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.578+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/753246887 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c2408ef40 con 0x7f4c24070100 2024-04-27T03:33:31.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.578+0000 7f4c1affd640 1 --2- 172.21.15.89:0/753246887 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4c08038060 0x7f4c0803a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.578+0000 7f4c1affd640 1 -- 172.21.15.89:0/753246887 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f4c2002f050 con 0x7f4c24070100 2024-04-27T03:33:31.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.578+0000 7f4c29d87640 1 --2- 172.21.15.89:0/753246887 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4c08038060 0x7f4c0803a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.586 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.579+0000 7f4c29d87640 1 --2- 172.21.15.89:0/753246887 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4c08038060 0x7f4c0803a520 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4c14000c00 tx=0x7f4c14001310 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.588 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.582+0000 7f4c1affd640 1 -- 172.21.15.89:0/753246887 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f4c2408ef40 con 0x7f4c24070100 2024-04-27T03:33:31.759 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.753+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/753246887 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) -- 0x7f4c24062da0 con 0x7f4c08038060 2024-04-27T03:33:31.760 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.754+0000 7f4c1affd640 1 -- 172.21.15.89:0/753246887 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) ==== 41+0+0 (secure 0 0 0) 0x7f4c24062da0 con 0x7f4c08038060 2024-04-27T03:33:31.760 INFO:teuthology.orchestra.run.smithi089.stderr:instructed osd(s) 3 to deep-scrub 2024-04-27T03:33:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.756+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/753246887 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4c08038060 msgr2=0x7f4c0803a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.756+0000 7f4c2c7ec640 1 --2- 172.21.15.89:0/753246887 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4c08038060 0x7f4c0803a520 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f4c14000c00 tx=0x7f4c14001310 comp rx=0 tx=0).stop 2024-04-27T03:33:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.756+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/753246887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c24070100 msgr2=0x7f4c241c2260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.762 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.756+0000 7f4c2c7ec640 1 --2- 172.21.15.89:0/753246887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c24070100 0x7f4c241c2260 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f4c20009840 tx=0x7f4c200046c0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.756+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/753246887 shutdown_connections 2024-04-27T03:33:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.756+0000 7f4c2c7ec640 1 --2- 172.21.15.89:0/753246887 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f4c08038060 0x7f4c0803a520 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.756+0000 7f4c2c7ec640 1 --2- 172.21.15.89:0/753246887 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f4c24070100 0x7f4c241c2260 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.756+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/753246887 >> 172.21.15.89:0/753246887 conn(0x7f4c24099680 msgr2=0x7f4c240967e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.756+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/753246887 shutdown_connections 2024-04-27T03:33:31.763 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.756+0000 7f4c2c7ec640 1 -- 172.21.15.89:0/753246887 wait complete. 2024-04-27T03:33:31.772 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T03:33:31.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.861+0000 7fed6069a640 1 Processor -- start 2024-04-27T03:33:31.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.861+0000 7fed6069a640 1 -- start start 2024-04-27T03:33:31.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.862+0000 7fed6069a640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fed58071090 0x7fed58131430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.862+0000 7fed6069a640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7fed58098800 con 0x7fed58096430 2024-04-27T03:33:31.868 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.862+0000 7fed6069a640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7fed58071930 con 0x7fed58071090 2024-04-27T03:33:31.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.862+0000 7fed5dc35640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7fed58071090 0x7fed58131430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.862+0000 7fed5e436640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7fed58096430 0x7fed58096810 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:51840/0 (socket says 172.21.15.89:51840) 2024-04-27T03:33:31.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.862+0000 7fed5e436640 1 -- 172.21.15.89:0/4135226893 learned_addr learned my addr 172.21.15.89:0/4135226893 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:31.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.862+0000 7fed5dc35640 1 -- 172.21.15.89:0/4135226893 >> v1:172.21.15.89:6789/0 conn(0x7fed58096430 legacy=0x7fed58096810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.862+0000 7fed5dc35640 1 -- 172.21.15.89:0/4135226893 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed58071f50 con 0x7fed58071090 2024-04-27T03:33:31.869 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.862+0000 7fed5d434640 1 -- 172.21.15.89:0/4135226893 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 4200958497 0 0) 0x7fed58098800 con 0x7fed58096430 2024-04-27T03:33:31.872 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.863+0000 7fed5dc35640 1 --2- 172.21.15.89:0/4135226893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed58071090 0x7fed58131430 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fed48009720 tx=0x7fed4802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181c6dfc84805adf server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.863+0000 7fed5d434640 1 -- 172.21.15.89:0/4135226893 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed480316d0 con 0x7fed58071090 2024-04-27T03:33:31.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.863+0000 7fed5d434640 1 -- 172.21.15.89:0/4135226893 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed48037070 con 0x7fed58071090 2024-04-27T03:33:31.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.863+0000 7fed5d434640 1 -- 172.21.15.89:0/4135226893 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed480319c0 con 0x7fed58071090 2024-04-27T03:33:31.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.864+0000 7fed6069a640 1 -- 172.21.15.89:0/4135226893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed58071090 msgr2=0x7fed58131430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:31.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.864+0000 7fed6069a640 1 --2- 172.21.15.89:0/4135226893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed58071090 0x7fed58131430 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7fed48009720 tx=0x7fed4802b840 comp rx=0 tx=0).stop 2024-04-27T03:33:31.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.864+0000 7fed6069a640 1 -- 172.21.15.89:0/4135226893 shutdown_connections 2024-04-27T03:33:31.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.864+0000 7fed6069a640 1 --2- 172.21.15.89:0/4135226893 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed58071090 0x7fed58131430 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:31.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.864+0000 7fed6069a640 1 -- 172.21.15.89:0/4135226893 >> 172.21.15.89:0/4135226893 conn(0x7fed58099680 msgr2=0x7fed58099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:31.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.864+0000 7fed6069a640 1 -- 172.21.15.89:0/4135226893 shutdown_connections 2024-04-27T03:33:31.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.864+0000 7fed6069a640 1 -- 172.21.15.89:0/4135226893 wait complete. 2024-04-27T03:33:31.873 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.865+0000 7fed6069a640 1 Processor -- start 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.865+0000 7fed6069a640 1 -- start start 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.865+0000 7fed6069a640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed58071090 0x7fed58121a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.865+0000 7fed6069a640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7fed58131af0 con 0x7fed58071090 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.865+0000 7fed5e436640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed58071090 0x7fed58121a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.865+0000 7fed5e436640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed58071090 0x7fed58121a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:49838/0 (socket says 172.21.15.89:49838) 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.865+0000 7fed5e436640 1 -- 172.21.15.89:0/2698005443 learned_addr learned my addr 172.21.15.89:0/2698005443 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.865+0000 7fed5e436640 1 -- 172.21.15.89:0/2698005443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed581242f0 con 0x7fed58071090 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.866+0000 7fed5e436640 1 --2- 172.21.15.89:0/2698005443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed58071090 0x7fed58121a90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fed5400c450 tx=0x7fed5400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.866+0000 7fed4effd640 1 -- 172.21.15.89:0/2698005443 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed54018070 con 0x7fed58071090 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.866+0000 7fed4effd640 1 -- 172.21.15.89:0/2698005443 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fed54007410 con 0x7fed58071090 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.866+0000 7fed6069a640 1 -- 172.21.15.89:0/2698005443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed58122e70 con 0x7fed58071090 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.866+0000 7fed4effd640 1 -- 172.21.15.89:0/2698005443 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fed540047e0 con 0x7fed58071090 2024-04-27T03:33:31.874 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.866+0000 7fed6069a640 1 -- 172.21.15.89:0/2698005443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed58124b50 con 0x7fed58071090 2024-04-27T03:33:31.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.867+0000 7fed4effd640 1 -- 172.21.15.89:0/2698005443 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7fed5401b070 con 0x7fed58071090 2024-04-27T03:33:31.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.867+0000 7fed6069a640 1 -- 172.21.15.89:0/2698005443 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed58062da0 con 0x7fed58071090 2024-04-27T03:33:31.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.867+0000 7fed4effd640 1 --2- 172.21.15.89:0/2698005443 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fed3c038060 0x7fed3c03a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:31.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.867+0000 7fed4effd640 1 -- 172.21.15.89:0/2698005443 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7fed5404ac10 con 0x7fed58071090 2024-04-27T03:33:31.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.867+0000 7fed5dc35640 1 --2- 172.21.15.89:0/2698005443 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fed3c038060 0x7fed3c03a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:31.875 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.868+0000 7fed5dc35640 1 --2- 172.21.15.89:0/2698005443 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fed3c038060 0x7fed3c03a520 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fed48036000 tx=0x7fed48008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:31.877 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:31.871+0000 7fed4effd640 1 -- 172.21.15.89:0/2698005443 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7fed54004980 con 0x7fed58071090 2024-04-27T03:33:31.963 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.rgw.client.0 has been restored 2024-04-27T03:33:32.044 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:32.039+0000 7fed6069a640 1 -- 172.21.15.89:0/2698005443 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fed58122b80 con 0x7fed3c038060 2024-04-27T03:33:32.059 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:32.054+0000 7fed4effd640 1 -- 172.21.15.89:0/2698005443 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665288 (secure 0 0 0) 0x7fed48037040 con 0x7fed3c038060 2024-04-27T03:33:32.067 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:33:32.072 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-04-27T03:33:32.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:32.058+0000 7fed6069a640 1 -- 172.21.15.89:0/2698005443 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fed3c038060 msgr2=0x7fed3c03a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:32.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:32.058+0000 7fed6069a640 1 --2- 172.21.15.89:0/2698005443 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fed3c038060 0x7fed3c03a520 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fed48036000 tx=0x7fed48008030 comp rx=0 tx=0).stop 2024-04-27T03:33:32.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:32.058+0000 7fed6069a640 1 -- 172.21.15.89:0/2698005443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed58071090 msgr2=0x7fed58121a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:32.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:32.058+0000 7fed6069a640 1 --2- 172.21.15.89:0/2698005443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed58071090 0x7fed58121a90 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fed5400c450 tx=0x7fed5400c910 comp rx=0 tx=0).stop 2024-04-27T03:33:32.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:32.059+0000 7fed6069a640 1 -- 172.21.15.89:0/2698005443 shutdown_connections 2024-04-27T03:33:32.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:32.059+0000 7fed6069a640 1 --2- 172.21.15.89:0/2698005443 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7fed3c038060 0x7fed3c03a520 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:32.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:32.059+0000 7fed6069a640 1 --2- 172.21.15.89:0/2698005443 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7fed58071090 0x7fed58121a90 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:32.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:32.059+0000 7fed6069a640 1 -- 172.21.15.89:0/2698005443 >> 172.21.15.89:0/2698005443 conn(0x7fed58099680 msgr2=0x7fed58129ef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:32.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:32.059+0000 7fed6069a640 1 -- 172.21.15.89:0/2698005443 shutdown_connections 2024-04-27T03:33:32.075 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:32.059+0000 7fed6069a640 1 -- 172.21.15.89:0/2698005443 wait complete. 2024-04-27T03:33:32.076 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":127,"stamp":"2024-04-27T03:33:30.271811+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":252,"num_object_clones":0,"num_object_copies":504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":252,"num_whiteouts":0,"num_read":5378,"num_read_kb":4932,"num_write":2369,"num_write_kb":61373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4801,"ondisk_log_size":4801,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":148156,"kb_used_data":2648,"kb_used_omap":6,"kb_used_meta":145465,"kb_avail":377339204,"statfs":{"total":386547056640,"available":386395344896,"internally_reserved":0,"allocated":2711552,"data_stored":1657386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":148956950},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":4281,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":713,"num_read_kb":672,"num_write":457,"num_write_kb":29372,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003820"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433166+0000","last_change":"2024-04-27T03:30:11.362820+0000","last_active":"2024-04-27T03:31:01.433166+0000","last_peered":"2024-04-27T03:31:01.433166+0000","last_clean":"2024-04-27T03:31:01.433166+0000","last_became_active":"2024-04-27T03:30:11.362574+0000","last_became_peered":"2024-04-27T03:30:11.362574+0000","last_unstale":"2024-04-27T03:31:01.433166+0000","last_undegraded":"2024-04-27T03:31:01.433166+0000","last_fullsized":"2024-04-27T03:31:01.433166+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.746108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933798+0000","last_change":"2024-04-27T03:30:13.384819+0000","last_active":"2024-04-27T03:33:20.933798+0000","last_peered":"2024-04-27T03:33:20.933798+0000","last_clean":"2024-04-27T03:33:20.933798+0000","last_became_active":"2024-04-27T03:30:13.384580+0000","last_became_peered":"2024-04-27T03:30:13.384580+0000","last_unstale":"2024-04-27T03:33:20.933798+0000","last_undegraded":"2024-04-27T03:33:20.933798+0000","last_fullsized":"2024-04-27T03:33:20.933798+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:07:51.011104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.921795+0000","last_change":"2024-04-27T03:30:11.361366+0000","last_active":"2024-04-27T03:33:20.921795+0000","last_peered":"2024-04-27T03:33:20.921795+0000","last_clean":"2024-04-27T03:33:20.921795+0000","last_became_active":"2024-04-27T03:30:11.360857+0000","last_became_peered":"2024-04-27T03:30:11.360857+0000","last_unstale":"2024-04-27T03:33:20.921795+0000","last_undegraded":"2024-04-27T03:33:20.921795+0000","last_fullsized":"2024-04-27T03:33:20.921795+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:11.138117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"27'12","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.695008+0000","last_change":"2024-04-27T03:30:13.376545+0000","last_active":"2024-04-27T03:33:20.695008+0000","last_peered":"2024-04-27T03:33:20.695008+0000","last_clean":"2024-04-27T03:33:20.695008+0000","last_became_active":"2024-04-27T03:30:13.376221+0000","last_became_peered":"2024-04-27T03:30:13.376221+0000","last_unstale":"2024-04-27T03:33:20.695008+0000","last_undegraded":"2024-04-27T03:33:20.695008+0000","last_fullsized":"2024-04-27T03:33:20.695008+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:08.985512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440740+0000","last_change":"2024-04-27T03:30:11.363460+0000","last_active":"2024-04-27T03:31:01.440740+0000","last_peered":"2024-04-27T03:31:01.440740+0000","last_clean":"2024-04-27T03:31:01.440740+0000","last_became_active":"2024-04-27T03:30:11.362739+0000","last_became_peered":"2024-04-27T03:30:11.362739+0000","last_unstale":"2024-04-27T03:31:01.440740+0000","last_undegraded":"2024-04-27T03:31:01.440740+0000","last_fullsized":"2024-04-27T03:31:01.440740+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:12.842711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"27'7","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298536+0000","last_change":"2024-04-27T03:30:13.376819+0000","last_active":"2024-04-27T03:32:44.298536+0000","last_peered":"2024-04-27T03:32:44.298536+0000","last_clean":"2024-04-27T03:32:44.298536+0000","last_became_active":"2024-04-27T03:30:13.376524+0000","last_became_peered":"2024-04-27T03:30:13.376524+0000","last_unstale":"2024-04-27T03:32:44.298536+0000","last_undegraded":"2024-04-27T03:32:44.298536+0000","last_fullsized":"2024-04-27T03:32:44.298536+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:54:30.451033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.610122+0000","last_change":"2024-04-27T03:30:11.361039+0000","last_active":"2024-04-27T03:33:20.610122+0000","last_peered":"2024-04-27T03:33:20.610122+0000","last_clean":"2024-04-27T03:33:20.610122+0000","last_became_active":"2024-04-27T03:30:11.360188+0000","last_became_peered":"2024-04-27T03:30:11.360188+0000","last_unstale":"2024-04-27T03:33:20.610122+0000","last_undegraded":"2024-04-27T03:33:20.610122+0000","last_fullsized":"2024-04-27T03:33:20.610122+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:22.719424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"27'16","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009953+0000","last_change":"2024-04-27T03:30:13.376574+0000","last_active":"2024-04-27T03:33:21.009953+0000","last_peered":"2024-04-27T03:33:21.009953+0000","last_clean":"2024-04-27T03:33:21.009953+0000","last_became_active":"2024-04-27T03:30:13.376288+0000","last_became_peered":"2024-04-27T03:30:13.376288+0000","last_unstale":"2024-04-27T03:33:21.009953+0000","last_undegraded":"2024-04-27T03:33:21.009953+0000","last_fullsized":"2024-04-27T03:33:21.009953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:15.187356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439707+0000","last_change":"2024-04-27T03:30:11.355274+0000","last_active":"2024-04-27T03:31:01.439707+0000","last_peered":"2024-04-27T03:31:01.439707+0000","last_clean":"2024-04-27T03:31:01.439707+0000","last_became_active":"2024-04-27T03:30:11.354989+0000","last_became_peered":"2024-04-27T03:30:11.354989+0000","last_unstale":"2024-04-27T03:31:01.439707+0000","last_undegraded":"2024-04-27T03:31:01.439707+0000","last_fullsized":"2024-04-27T03:31:01.439707+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:11.281867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"27'6","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933479+0000","last_change":"2024-04-27T03:30:13.374182+0000","last_active":"2024-04-27T03:33:20.933479+0000","last_peered":"2024-04-27T03:33:20.933479+0000","last_clean":"2024-04-27T03:33:20.933479+0000","last_became_active":"2024-04-27T03:30:13.373424+0000","last_became_peered":"2024-04-27T03:30:13.373424+0000","last_unstale":"2024-04-27T03:33:20.933479+0000","last_undegraded":"2024-04-27T03:33:20.933479+0000","last_fullsized":"2024-04-27T03:33:20.933479+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:02.825441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.657187+0000","last_change":"2024-04-27T03:30:11.361019+0000","last_active":"2024-04-27T03:33:20.657187+0000","last_peered":"2024-04-27T03:33:20.657187+0000","last_clean":"2024-04-27T03:33:20.657187+0000","last_became_active":"2024-04-27T03:30:11.360219+0000","last_became_peered":"2024-04-27T03:30:11.360219+0000","last_unstale":"2024-04-27T03:33:20.657187+0000","last_undegraded":"2024-04-27T03:33:20.657187+0000","last_fullsized":"2024-04-27T03:33:20.657187+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:45.376863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"25'8","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440191+0000","last_change":"2024-04-27T03:30:13.385912+0000","last_active":"2024-04-27T03:31:01.440191+0000","last_peered":"2024-04-27T03:31:01.440191+0000","last_clean":"2024-04-27T03:31:01.440191+0000","last_became_active":"2024-04-27T03:30:13.385665+0000","last_became_peered":"2024-04-27T03:30:13.385665+0000","last_unstale":"2024-04-27T03:31:01.440191+0000","last_undegraded":"2024-04-27T03:31:01.440191+0000","last_fullsized":"2024-04-27T03:31:01.440191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:49.082066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437953+0000","last_change":"2024-04-27T03:30:11.373049+0000","last_active":"2024-04-27T03:31:01.437953+0000","last_peered":"2024-04-27T03:31:01.437953+0000","last_clean":"2024-04-27T03:31:01.437953+0000","last_became_active":"2024-04-27T03:30:11.372704+0000","last_became_peered":"2024-04-27T03:30:11.372704+0000","last_unstale":"2024-04-27T03:31:01.437953+0000","last_undegraded":"2024-04-27T03:31:01.437953+0000","last_fullsized":"2024-04-27T03:31:01.437953+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:03.639873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"27'4","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981936+0000","last_change":"2024-04-27T03:30:13.374520+0000","last_active":"2024-04-27T03:33:20.981936+0000","last_peered":"2024-04-27T03:33:20.981936+0000","last_clean":"2024-04-27T03:33:20.981936+0000","last_became_active":"2024-04-27T03:30:13.374199+0000","last_became_peered":"2024-04-27T03:30:13.374199+0000","last_unstale":"2024-04-27T03:33:20.981936+0000","last_undegraded":"2024-04-27T03:33:20.981936+0000","last_fullsized":"2024-04-27T03:33:20.981936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:33.811635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.926966+0000","last_change":"2024-04-27T03:30:11.358470+0000","last_active":"2024-04-27T03:33:20.926966+0000","last_peered":"2024-04-27T03:33:20.926966+0000","last_clean":"2024-04-27T03:33:20.926966+0000","last_became_active":"2024-04-27T03:30:11.358181+0000","last_became_peered":"2024-04-27T03:30:11.358181+0000","last_unstale":"2024-04-27T03:33:20.926966+0000","last_undegraded":"2024-04-27T03:33:20.926966+0000","last_fullsized":"2024-04-27T03:33:20.926966+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.921690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.794975+0000","last_change":"2024-04-27T03:30:13.379341+0000","last_active":"2024-04-27T03:33:20.794975+0000","last_peered":"2024-04-27T03:33:20.794975+0000","last_clean":"2024-04-27T03:33:20.794975+0000","last_became_active":"2024-04-27T03:30:13.379059+0000","last_became_peered":"2024-04-27T03:30:13.379059+0000","last_unstale":"2024-04-27T03:33:20.794975+0000","last_undegraded":"2024-04-27T03:33:20.794975+0000","last_fullsized":"2024-04-27T03:33:20.794975+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:56.906775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"27'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.010139+0000","last_change":"2024-04-27T03:30:13.383971+0000","last_active":"2024-04-27T03:33:21.010139+0000","last_peered":"2024-04-27T03:33:21.010139+0000","last_clean":"2024-04-27T03:33:21.010139+0000","last_became_active":"2024-04-27T03:30:13.383694+0000","last_became_peered":"2024-04-27T03:30:13.383694+0000","last_unstale":"2024-04-27T03:33:21.010139+0000","last_undegraded":"2024-04-27T03:33:21.010139+0000","last_fullsized":"2024-04-27T03:33:21.010139+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:36:25.083284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436571+0000","last_change":"2024-04-27T03:30:11.370455+0000","last_active":"2024-04-27T03:31:01.436571+0000","last_peered":"2024-04-27T03:31:01.436571+0000","last_clean":"2024-04-27T03:31:01.436571+0000","last_became_active":"2024-04-27T03:30:11.370157+0000","last_became_peered":"2024-04-27T03:30:11.370157+0000","last_unstale":"2024-04-27T03:31:01.436571+0000","last_undegraded":"2024-04-27T03:31:01.436571+0000","last_fullsized":"2024-04-27T03:31:01.436571+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:06.887756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"27'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009774+0000","last_change":"2024-04-27T03:30:13.373269+0000","last_active":"2024-04-27T03:33:21.009774+0000","last_peered":"2024-04-27T03:33:21.009774+0000","last_clean":"2024-04-27T03:33:21.009774+0000","last_became_active":"2024-04-27T03:30:13.372973+0000","last_became_peered":"2024-04-27T03:30:13.372973+0000","last_unstale":"2024-04-27T03:33:21.009774+0000","last_undegraded":"2024-04-27T03:33:21.009774+0000","last_fullsized":"2024-04-27T03:33:21.009774+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:57.593400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"27'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.571577+0000","last_change":"2024-04-27T03:30:11.363764+0000","last_active":"2024-04-27T03:33:20.571577+0000","last_peered":"2024-04-27T03:33:20.571577+0000","last_clean":"2024-04-27T03:33:20.571577+0000","last_became_active":"2024-04-27T03:30:11.363109+0000","last_became_peered":"2024-04-27T03:30:11.363109+0000","last_unstale":"2024-04-27T03:33:20.571577+0000","last_undegraded":"2024-04-27T03:33:20.571577+0000","last_fullsized":"2024-04-27T03:33:20.571577+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:52:31.047433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"27'9","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988406+0000","last_change":"2024-04-27T03:30:13.384110+0000","last_active":"2024-04-27T03:33:20.988406+0000","last_peered":"2024-04-27T03:33:20.988406+0000","last_clean":"2024-04-27T03:33:20.988406+0000","last_became_active":"2024-04-27T03:30:13.383742+0000","last_became_peered":"2024-04-27T03:30:13.383742+0000","last_unstale":"2024-04-27T03:33:20.988406+0000","last_undegraded":"2024-04-27T03:33:20.988406+0000","last_fullsized":"2024-04-27T03:33:20.988406+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:14.799339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"25'4","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436806+0000","last_change":"2024-04-27T03:30:11.370635+0000","last_active":"2024-04-27T03:31:01.436806+0000","last_peered":"2024-04-27T03:31:01.436806+0000","last_clean":"2024-04-27T03:31:01.436806+0000","last_became_active":"2024-04-27T03:30:11.370380+0000","last_became_peered":"2024-04-27T03:30:11.370380+0000","last_unstale":"2024-04-27T03:31:01.436806+0000","last_undegraded":"2024-04-27T03:31:01.436806+0000","last_fullsized":"2024-04-27T03:31:01.436806+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:45.376863+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"27'6","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009766+0000","last_change":"2024-04-27T03:31:01.446266+0000","last_active":"2024-04-27T03:33:21.009766+0000","last_peered":"2024-04-27T03:33:21.009766+0000","last_clean":"2024-04-27T03:33:21.009766+0000","last_became_active":"2024-04-27T03:31:01.440762+0000","last_became_peered":"2024-04-27T03:31:01.440762+0000","last_unstale":"2024-04-27T03:33:21.009766+0000","last_undegraded":"2024-04-27T03:33:21.009766+0000","last_fullsized":"2024-04-27T03:33:21.009766+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:42:58.987011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.279469+0000","last_change":"2024-04-27T03:30:11.369834+0000","last_active":"2024-04-27T03:32:20.279469+0000","last_peered":"2024-04-27T03:32:20.279469+0000","last_clean":"2024-04-27T03:32:20.279469+0000","last_became_active":"2024-04-27T03:30:11.369551+0000","last_became_peered":"2024-04-27T03:30:11.369551+0000","last_unstale":"2024-04-27T03:32:20.279469+0000","last_undegraded":"2024-04-27T03:32:20.279469+0000","last_fullsized":"2024-04-27T03:32:20.279469+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.921690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"27'2","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981818+0000","last_change":"2024-04-27T03:30:13.382150+0000","last_active":"2024-04-27T03:33:20.981818+0000","last_peered":"2024-04-27T03:33:20.981818+0000","last_clean":"2024-04-27T03:33:20.981818+0000","last_became_active":"2024-04-27T03:30:13.381901+0000","last_became_peered":"2024-04-27T03:30:13.381901+0000","last_unstale":"2024-04-27T03:33:20.981818+0000","last_undegraded":"2024-04-27T03:33:20.981818+0000","last_fullsized":"2024-04-27T03:33:20.981818+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:05.646245+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"25'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431969+0000","last_change":"2024-04-27T03:30:11.361129+0000","last_active":"2024-04-27T03:31:01.431969+0000","last_peered":"2024-04-27T03:31:01.431969+0000","last_clean":"2024-04-27T03:31:01.431969+0000","last_became_active":"2024-04-27T03:30:11.360515+0000","last_became_peered":"2024-04-27T03:30:11.360515+0000","last_unstale":"2024-04-27T03:31:01.431969+0000","last_undegraded":"2024-04-27T03:31:01.431969+0000","last_fullsized":"2024-04-27T03:31:01.431969+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:52:31.047433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"27'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.307097+0000","last_change":"2024-04-27T03:30:13.382249+0000","last_active":"2024-04-27T03:32:20.307097+0000","last_peered":"2024-04-27T03:32:20.307097+0000","last_clean":"2024-04-27T03:32:20.307097+0000","last_became_active":"2024-04-27T03:30:13.378932+0000","last_became_peered":"2024-04-27T03:30:13.378932+0000","last_unstale":"2024-04-27T03:32:20.307097+0000","last_undegraded":"2024-04-27T03:32:20.307097+0000","last_fullsized":"2024-04-27T03:32:20.307097+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:00:35.273219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437123+0000","last_change":"2024-04-27T03:30:11.372964+0000","last_active":"2024-04-27T03:31:01.437123+0000","last_peered":"2024-04-27T03:31:01.437123+0000","last_clean":"2024-04-27T03:31:01.437123+0000","last_became_active":"2024-04-27T03:30:11.372673+0000","last_became_peered":"2024-04-27T03:30:11.372673+0000","last_unstale":"2024-04-27T03:31:01.437123+0000","last_undegraded":"2024-04-27T03:31:01.437123+0000","last_fullsized":"2024-04-27T03:31:01.437123+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:08:10.593535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"27'35","reported_seq":99,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.957246+0000","last_change":"2024-04-27T03:30:21.413597+0000","last_active":"2024-04-27T03:33:20.957246+0000","last_peered":"2024-04-27T03:33:20.957246+0000","last_clean":"2024-04-27T03:33:20.957246+0000","last_became_active":"2024-04-27T03:30:21.413255+0000","last_became_peered":"2024-04-27T03:30:21.413255+0000","last_unstale":"2024-04-27T03:33:20.957246+0000","last_undegraded":"2024-04-27T03:33:20.957246+0000","last_fullsized":"2024-04-27T03:33:20.957246+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:11:19.581338+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"27'3","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296833+0000","last_change":"2024-04-27T03:30:13.377382+0000","last_active":"2024-04-27T03:32:34.296833+0000","last_peered":"2024-04-27T03:32:34.296833+0000","last_clean":"2024-04-27T03:32:34.296833+0000","last_became_active":"2024-04-27T03:30:13.377110+0000","last_became_peered":"2024-04-27T03:30:13.377110+0000","last_unstale":"2024-04-27T03:32:34.296833+0000","last_undegraded":"2024-04-27T03:32:34.296833+0000","last_fullsized":"2024-04-27T03:32:34.296833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:21:27.452807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439438+0000","last_change":"2024-04-27T03:30:11.360152+0000","last_active":"2024-04-27T03:31:01.439438+0000","last_peered":"2024-04-27T03:31:01.439438+0000","last_clean":"2024-04-27T03:31:01.439438+0000","last_became_active":"2024-04-27T03:30:11.359892+0000","last_became_peered":"2024-04-27T03:30:11.359892+0000","last_unstale":"2024-04-27T03:31:01.439438+0000","last_undegraded":"2024-04-27T03:31:01.439438+0000","last_fullsized":"2024-04-27T03:31:01.439438+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:07:49.004733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440221+0000","last_change":"2024-04-27T03:30:15.381168+0000","last_active":"2024-04-27T03:31:01.440221+0000","last_peered":"2024-04-27T03:31:01.440221+0000","last_clean":"2024-04-27T03:31:01.440221+0000","last_became_active":"2024-04-27T03:30:15.380864+0000","last_became_peered":"2024-04-27T03:30:15.380864+0000","last_unstale":"2024-04-27T03:31:01.440221+0000","last_undegraded":"2024-04-27T03:31:01.440221+0000","last_fullsized":"2024-04-27T03:31:01.440221+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:57:52.741876+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"27'45","reported_seq":103,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.636261+0000","last_change":"2024-04-27T03:30:17.388325+0000","last_active":"2024-04-27T03:33:27.636261+0000","last_peered":"2024-04-27T03:33:27.636261+0000","last_clean":"2024-04-27T03:33:27.636261+0000","last_became_active":"2024-04-27T03:30:17.387391+0000","last_became_peered":"2024-04-27T03:30:17.387391+0000","last_unstale":"2024-04-27T03:33:27.636261+0000","last_undegraded":"2024-04-27T03:33:27.636261+0000","last_fullsized":"2024-04-27T03:33:27.636261+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:02:27.282043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":45,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433271+0000","last_change":"2024-04-27T03:30:19.407954+0000","last_active":"2024-04-27T03:31:01.433271+0000","last_peered":"2024-04-27T03:31:01.433271+0000","last_clean":"2024-04-27T03:31:01.433271+0000","last_became_active":"2024-04-27T03:30:19.407637+0000","last_became_peered":"2024-04-27T03:30:19.407637+0000","last_unstale":"2024-04-27T03:31:01.433271+0000","last_undegraded":"2024-04-27T03:31:01.433271+0000","last_fullsized":"2024-04-27T03:31:01.433271+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:12.277984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"27'9","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.436568+0000","last_change":"2024-04-27T03:30:21.414223+0000","last_active":"2024-04-27T03:31:36.436568+0000","last_peered":"2024-04-27T03:31:36.436568+0000","last_clean":"2024-04-27T03:31:36.436568+0000","last_became_active":"2024-04-27T03:30:21.413940+0000","last_became_peered":"2024-04-27T03:30:21.413940+0000","last_unstale":"2024-04-27T03:31:36.436568+0000","last_undegraded":"2024-04-27T03:31:36.436568+0000","last_fullsized":"2024-04-27T03:31:36.436568+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:44.051317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"27'16","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981635+0000","last_change":"2024-04-27T03:30:13.381748+0000","last_active":"2024-04-27T03:33:20.981635+0000","last_peered":"2024-04-27T03:33:20.981635+0000","last_clean":"2024-04-27T03:33:20.981635+0000","last_became_active":"2024-04-27T03:30:13.381487+0000","last_became_peered":"2024-04-27T03:30:13.381487+0000","last_unstale":"2024-04-27T03:33:20.981635+0000","last_undegraded":"2024-04-27T03:33:20.981635+0000","last_fullsized":"2024-04-27T03:33:20.981635+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:52.946275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435664+0000","last_change":"2024-04-27T03:30:11.359690+0000","last_active":"2024-04-27T03:31:01.435664+0000","last_peered":"2024-04-27T03:31:01.435664+0000","last_clean":"2024-04-27T03:31:01.435664+0000","last_became_active":"2024-04-27T03:30:11.359446+0000","last_became_peered":"2024-04-27T03:30:11.359446+0000","last_unstale":"2024-04-27T03:31:01.435664+0000","last_undegraded":"2024-04-27T03:31:01.435664+0000","last_fullsized":"2024-04-27T03:31:01.435664+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:25:12.921690+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431730+0000","last_change":"2024-04-27T03:30:15.380566+0000","last_active":"2024-04-27T03:31:01.431730+0000","last_peered":"2024-04-27T03:31:01.431730+0000","last_clean":"2024-04-27T03:31:01.431730+0000","last_became_active":"2024-04-27T03:30:15.380313+0000","last_became_peered":"2024-04-27T03:30:15.380313+0000","last_unstale":"2024-04-27T03:31:01.431730+0000","last_undegraded":"2024-04-27T03:31:01.431730+0000","last_fullsized":"2024-04-27T03:31:01.431730+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:45:16.754705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"25'12","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433370+0000","last_change":"2024-04-27T03:30:17.389399+0000","last_active":"2024-04-27T03:31:01.433370+0000","last_peered":"2024-04-27T03:31:01.433370+0000","last_clean":"2024-04-27T03:31:01.433370+0000","last_became_active":"2024-04-27T03:30:17.389126+0000","last_became_peered":"2024-04-27T03:30:17.389126+0000","last_unstale":"2024-04-27T03:31:01.433370+0000","last_undegraded":"2024-04-27T03:31:01.433370+0000","last_fullsized":"2024-04-27T03:31:01.433370+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:42:03.150440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432915+0000","last_change":"2024-04-27T03:30:19.405456+0000","last_active":"2024-04-27T03:31:01.432915+0000","last_peered":"2024-04-27T03:31:01.432915+0000","last_clean":"2024-04-27T03:31:01.432915+0000","last_became_active":"2024-04-27T03:30:19.405170+0000","last_became_peered":"2024-04-27T03:30:19.405170+0000","last_unstale":"2024-04-27T03:31:01.432915+0000","last_undegraded":"2024-04-27T03:31:01.432915+0000","last_fullsized":"2024-04-27T03:31:01.432915+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:31:41.885815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'6","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.427217+0000","last_change":"2024-04-27T03:30:21.408308+0000","last_active":"2024-04-27T03:31:36.427217+0000","last_peered":"2024-04-27T03:31:36.427217+0000","last_clean":"2024-04-27T03:31:36.427217+0000","last_became_active":"2024-04-27T03:30:21.407917+0000","last_became_peered":"2024-04-27T03:30:21.407917+0000","last_unstale":"2024-04-27T03:31:36.427217+0000","last_undegraded":"2024-04-27T03:31:36.427217+0000","last_fullsized":"2024-04-27T03:31:36.427217+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:23.492592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":95,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.792066+0000","last_change":"2024-04-27T03:30:13.378024+0000","last_active":"2024-04-27T03:33:20.792066+0000","last_peered":"2024-04-27T03:33:20.792066+0000","last_clean":"2024-04-27T03:33:20.792066+0000","last_became_active":"2024-04-27T03:30:13.377423+0000","last_became_peered":"2024-04-27T03:30:13.377423+0000","last_unstale":"2024-04-27T03:33:20.792066+0000","last_undegraded":"2024-04-27T03:33:20.792066+0000","last_fullsized":"2024-04-27T03:33:20.792066+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:22.057065+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437475+0000","last_change":"2024-04-27T03:30:11.359542+0000","last_active":"2024-04-27T03:31:01.437475+0000","last_peered":"2024-04-27T03:31:01.437475+0000","last_clean":"2024-04-27T03:31:01.437475+0000","last_became_active":"2024-04-27T03:30:11.359261+0000","last_became_peered":"2024-04-27T03:30:11.359261+0000","last_unstale":"2024-04-27T03:31:01.437475+0000","last_undegraded":"2024-04-27T03:31:01.437475+0000","last_fullsized":"2024-04-27T03:31:01.437475+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:19.905319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433488+0000","last_change":"2024-04-27T03:30:15.383902+0000","last_active":"2024-04-27T03:31:01.433488+0000","last_peered":"2024-04-27T03:31:01.433488+0000","last_clean":"2024-04-27T03:31:01.433488+0000","last_became_active":"2024-04-27T03:30:15.383618+0000","last_became_peered":"2024-04-27T03:30:15.383618+0000","last_unstale":"2024-04-27T03:31:01.433488+0000","last_undegraded":"2024-04-27T03:31:01.433488+0000","last_fullsized":"2024-04-27T03:31:01.433488+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:54.227106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"25'23","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433761+0000","last_change":"2024-04-27T03:30:17.392488+0000","last_active":"2024-04-27T03:31:01.433761+0000","last_peered":"2024-04-27T03:31:01.433761+0000","last_clean":"2024-04-27T03:31:01.433761+0000","last_became_active":"2024-04-27T03:30:17.392223+0000","last_became_peered":"2024-04-27T03:30:17.392223+0000","last_unstale":"2024-04-27T03:31:01.433761+0000","last_undegraded":"2024-04-27T03:31:01.433761+0000","last_fullsized":"2024-04-27T03:31:01.433761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:56.237623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"27'285","reported_seq":384,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.559488+0000","last_change":"2024-04-27T03:30:19.406264+0000","last_active":"2024-04-27T03:33:27.559488+0000","last_peered":"2024-04-27T03:33:27.559488+0000","last_clean":"2024-04-27T03:33:27.559488+0000","last_became_active":"2024-04-27T03:30:19.405730+0000","last_became_peered":"2024-04-27T03:30:19.405730+0000","last_unstale":"2024-04-27T03:33:27.559488+0000","last_undegraded":"2024-04-27T03:33:27.559488+0000","last_fullsized":"2024-04-27T03:33:27.559488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:34:24.551790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436546+0000","last_change":"2024-04-27T03:30:21.409739+0000","last_active":"2024-04-27T03:31:01.436546+0000","last_peered":"2024-04-27T03:31:01.436546+0000","last_clean":"2024-04-27T03:31:01.436546+0000","last_became_active":"2024-04-27T03:30:21.409156+0000","last_became_peered":"2024-04-27T03:30:21.409156+0000","last_unstale":"2024-04-27T03:31:01.436546+0000","last_undegraded":"2024-04-27T03:31:01.436546+0000","last_fullsized":"2024-04-27T03:31:01.436546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:56:51.564750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.802359+0000","last_change":"2024-04-27T03:30:13.385803+0000","last_active":"2024-04-27T03:33:20.802359+0000","last_peered":"2024-04-27T03:33:20.802359+0000","last_clean":"2024-04-27T03:33:20.802359+0000","last_became_active":"2024-04-27T03:30:13.385491+0000","last_became_peered":"2024-04-27T03:30:13.385491+0000","last_unstale":"2024-04-27T03:33:20.802359+0000","last_undegraded":"2024-04-27T03:33:20.802359+0000","last_fullsized":"2024-04-27T03:33:20.802359+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:14.849083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431754+0000","last_change":"2024-04-27T03:30:11.370401+0000","last_active":"2024-04-27T03:31:01.431754+0000","last_peered":"2024-04-27T03:31:01.431754+0000","last_clean":"2024-04-27T03:31:01.431754+0000","last_became_active":"2024-04-27T03:30:11.369939+0000","last_became_peered":"2024-04-27T03:30:11.369939+0000","last_unstale":"2024-04-27T03:31:01.431754+0000","last_undegraded":"2024-04-27T03:31:01.431754+0000","last_fullsized":"2024-04-27T03:31:01.431754+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:00.825717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439146+0000","last_change":"2024-04-27T03:30:15.384389+0000","last_active":"2024-04-27T03:31:01.439146+0000","last_peered":"2024-04-27T03:31:01.439146+0000","last_clean":"2024-04-27T03:31:01.439146+0000","last_became_active":"2024-04-27T03:30:15.384114+0000","last_became_peered":"2024-04-27T03:30:15.384114+0000","last_unstale":"2024-04-27T03:31:01.439146+0000","last_undegraded":"2024-04-27T03:31:01.439146+0000","last_fullsized":"2024-04-27T03:31:01.439146+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:17.509463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"27'39","reported_seq":91,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.620517+0000","last_change":"2024-04-27T03:30:17.392377+0000","last_active":"2024-04-27T03:33:27.620517+0000","last_peered":"2024-04-27T03:33:27.620517+0000","last_clean":"2024-04-27T03:33:27.620517+0000","last_became_active":"2024-04-27T03:30:17.391880+0000","last_became_peered":"2024-04-27T03:30:17.391880+0000","last_unstale":"2024-04-27T03:33:27.620517+0000","last_undegraded":"2024-04-27T03:33:27.620517+0000","last_fullsized":"2024-04-27T03:33:27.620517+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:17:13.007456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"27'285","reported_seq":408,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.564212+0000","last_change":"2024-04-27T03:30:19.405073+0000","last_active":"2024-04-27T03:33:27.564212+0000","last_peered":"2024-04-27T03:33:27.564212+0000","last_clean":"2024-04-27T03:33:27.564212+0000","last_became_active":"2024-04-27T03:30:19.404744+0000","last_became_peered":"2024-04-27T03:30:19.404744+0000","last_unstale":"2024-04-27T03:33:27.564212+0000","last_undegraded":"2024-04-27T03:33:27.564212+0000","last_fullsized":"2024-04-27T03:33:27.564212+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:31:41.885815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'7","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.706493+0000","last_change":"2024-04-27T03:30:13.387974+0000","last_active":"2024-04-27T03:33:20.706493+0000","last_peered":"2024-04-27T03:33:20.706493+0000","last_clean":"2024-04-27T03:33:20.706493+0000","last_became_active":"2024-04-27T03:30:13.387335+0000","last_became_peered":"2024-04-27T03:30:13.387335+0000","last_unstale":"2024-04-27T03:33:20.706493+0000","last_undegraded":"2024-04-27T03:33:20.706493+0000","last_fullsized":"2024-04-27T03:33:20.706493+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:32:09.116022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433282+0000","last_change":"2024-04-27T03:30:11.372061+0000","last_active":"2024-04-27T03:31:01.433282+0000","last_peered":"2024-04-27T03:31:01.433282+0000","last_clean":"2024-04-27T03:31:01.433282+0000","last_became_active":"2024-04-27T03:30:11.371600+0000","last_became_peered":"2024-04-27T03:30:11.371600+0000","last_unstale":"2024-04-27T03:31:01.433282+0000","last_undegraded":"2024-04-27T03:31:01.433282+0000","last_fullsized":"2024-04-27T03:31:01.433282+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.059046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433141+0000","last_change":"2024-04-27T03:30:15.381767+0000","last_active":"2024-04-27T03:31:01.433141+0000","last_peered":"2024-04-27T03:31:01.433141+0000","last_clean":"2024-04-27T03:31:01.433141+0000","last_became_active":"2024-04-27T03:30:15.381359+0000","last_became_peered":"2024-04-27T03:30:15.381359+0000","last_unstale":"2024-04-27T03:31:01.433141+0000","last_undegraded":"2024-04-27T03:31:01.433141+0000","last_fullsized":"2024-04-27T03:31:01.433141+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:19:12.341976+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"27'49","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.648502+0000","last_change":"2024-04-27T03:30:17.390288+0000","last_active":"2024-04-27T03:33:27.648502+0000","last_peered":"2024-04-27T03:33:27.648502+0000","last_clean":"2024-04-27T03:33:27.648502+0000","last_became_active":"2024-04-27T03:30:17.390021+0000","last_became_peered":"2024-04-27T03:30:17.390021+0000","last_unstale":"2024-04-27T03:33:27.648502+0000","last_undegraded":"2024-04-27T03:33:27.648502+0000","last_fullsized":"2024-04-27T03:33:27.648502+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":49,"log_dups_size":0,"ondisk_log_size":49,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:58.407158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":30,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431506+0000","last_change":"2024-04-27T03:30:19.405394+0000","last_active":"2024-04-27T03:31:01.431506+0000","last_peered":"2024-04-27T03:31:01.431506+0000","last_clean":"2024-04-27T03:31:01.431506+0000","last_became_active":"2024-04-27T03:30:19.404948+0000","last_became_peered":"2024-04-27T03:30:19.404948+0000","last_unstale":"2024-04-27T03:31:01.431506+0000","last_undegraded":"2024-04-27T03:31:01.431506+0000","last_fullsized":"2024-04-27T03:31:01.431506+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:59.540815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"27'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.438491+0000","last_change":"2024-04-27T03:30:21.412598+0000","last_active":"2024-04-27T03:31:36.438491+0000","last_peered":"2024-04-27T03:31:36.438491+0000","last_clean":"2024-04-27T03:31:36.438491+0000","last_became_active":"2024-04-27T03:30:21.412331+0000","last_became_peered":"2024-04-27T03:30:21.412331+0000","last_unstale":"2024-04-27T03:31:36.438491+0000","last_undegraded":"2024-04-27T03:31:36.438491+0000","last_fullsized":"2024-04-27T03:31:36.438491+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:21.686877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"25'6","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432638+0000","last_change":"2024-04-27T03:30:13.373532+0000","last_active":"2024-04-27T03:31:01.432638+0000","last_peered":"2024-04-27T03:31:01.432638+0000","last_clean":"2024-04-27T03:31:01.432638+0000","last_became_active":"2024-04-27T03:30:13.373226+0000","last_became_peered":"2024-04-27T03:30:13.373226+0000","last_unstale":"2024-04-27T03:31:01.432638+0000","last_undegraded":"2024-04-27T03:31:01.432638+0000","last_fullsized":"2024-04-27T03:31:01.432638+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:38.976155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437884+0000","last_change":"2024-04-27T03:30:11.373710+0000","last_active":"2024-04-27T03:31:01.437884+0000","last_peered":"2024-04-27T03:31:01.437884+0000","last_clean":"2024-04-27T03:31:01.437884+0000","last_became_active":"2024-04-27T03:30:11.373430+0000","last_became_peered":"2024-04-27T03:30:11.373430+0000","last_unstale":"2024-04-27T03:31:01.437884+0000","last_undegraded":"2024-04-27T03:31:01.437884+0000","last_fullsized":"2024-04-27T03:31:01.437884+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:52:58.316121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"18'1","reported_seq":80,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.531430+0000","last_change":"2024-04-27T03:31:01.447513+0000","last_active":"2024-04-27T03:33:27.531430+0000","last_peered":"2024-04-27T03:33:27.531430+0000","last_clean":"2024-04-27T03:33:27.531430+0000","last_became_active":"2024-04-27T03:31:01.443259+0000","last_became_peered":"2024-04-27T03:31:01.443259+0000","last_unstale":"2024-04-27T03:33:27.531430+0000","last_undegraded":"2024-04-27T03:33:27.531430+0000","last_fullsized":"2024-04-27T03:33:27.531430+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:30:54.147127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"27'42","reported_seq":94,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.693344+0000","last_change":"2024-04-27T03:30:17.389610+0000","last_active":"2024-04-27T03:33:27.693344+0000","last_peered":"2024-04-27T03:33:27.693344+0000","last_clean":"2024-04-27T03:33:27.693344+0000","last_became_active":"2024-04-27T03:30:17.388994+0000","last_became_peered":"2024-04-27T03:30:17.388994+0000","last_unstale":"2024-04-27T03:33:27.693344+0000","last_undegraded":"2024-04-27T03:33:27.693344+0000","last_fullsized":"2024-04-27T03:33:27.693344+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:01:03.772535+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431125+0000","last_change":"2024-04-27T03:30:19.405695+0000","last_active":"2024-04-27T03:31:01.431125+0000","last_peered":"2024-04-27T03:31:01.431125+0000","last_clean":"2024-04-27T03:31:01.431125+0000","last_became_active":"2024-04-27T03:30:19.405334+0000","last_became_peered":"2024-04-27T03:30:19.405334+0000","last_unstale":"2024-04-27T03:31:01.431125+0000","last_undegraded":"2024-04-27T03:31:01.431125+0000","last_fullsized":"2024-04-27T03:31:01.431125+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:44:45.246417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"27'3","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:30.912296+0000","last_change":"2024-04-27T03:30:21.410221+0000","last_active":"2024-04-27T03:31:30.912296+0000","last_peered":"2024-04-27T03:31:30.912296+0000","last_clean":"2024-04-27T03:31:30.912296+0000","last_became_active":"2024-04-27T03:30:21.409922+0000","last_became_peered":"2024-04-27T03:30:21.409922+0000","last_unstale":"2024-04-27T03:31:30.912296+0000","last_undegraded":"2024-04-27T03:31:30.912296+0000","last_fullsized":"2024-04-27T03:31:30.912296+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.646754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"27'6","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.010033+0000","last_change":"2024-04-27T03:30:13.381521+0000","last_active":"2024-04-27T03:33:21.010033+0000","last_peered":"2024-04-27T03:33:21.010033+0000","last_clean":"2024-04-27T03:33:21.010033+0000","last_became_active":"2024-04-27T03:30:13.381270+0000","last_became_peered":"2024-04-27T03:30:13.381270+0000","last_unstale":"2024-04-27T03:33:21.010033+0000","last_undegraded":"2024-04-27T03:33:21.010033+0000","last_fullsized":"2024-04-27T03:33:21.010033+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:58:48.554677+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440331+0000","last_change":"2024-04-27T03:30:11.372966+0000","last_active":"2024-04-27T03:31:01.440331+0000","last_peered":"2024-04-27T03:31:01.440331+0000","last_clean":"2024-04-27T03:31:01.440331+0000","last_became_active":"2024-04-27T03:30:11.372583+0000","last_became_peered":"2024-04-27T03:30:11.372583+0000","last_unstale":"2024-04-27T03:31:01.440331+0000","last_undegraded":"2024-04-27T03:31:01.440331+0000","last_fullsized":"2024-04-27T03:31:01.440331+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:54.900404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433260+0000","last_change":"2024-04-27T03:30:15.382571+0000","last_active":"2024-04-27T03:31:01.433260+0000","last_peered":"2024-04-27T03:31:01.433260+0000","last_clean":"2024-04-27T03:31:01.433260+0000","last_became_active":"2024-04-27T03:30:15.382207+0000","last_became_peered":"2024-04-27T03:30:15.382207+0000","last_unstale":"2024-04-27T03:31:01.433260+0000","last_undegraded":"2024-04-27T03:31:01.433260+0000","last_fullsized":"2024-04-27T03:31:01.433260+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:41:56.396642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"27'52","reported_seq":113,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.654227+0000","last_change":"2024-04-27T03:30:17.390418+0000","last_active":"2024-04-27T03:33:27.654227+0000","last_peered":"2024-04-27T03:33:27.654227+0000","last_clean":"2024-04-27T03:33:27.654227+0000","last_became_active":"2024-04-27T03:30:17.389242+0000","last_became_peered":"2024-04-27T03:30:17.389242+0000","last_unstale":"2024-04-27T03:33:27.654227+0000","last_undegraded":"2024-04-27T03:33:27.654227+0000","last_fullsized":"2024-04-27T03:33:27.654227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:58:59.857055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":62,"num_read_kb":51,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"27'285","reported_seq":351,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.565341+0000","last_change":"2024-04-27T03:30:19.406718+0000","last_active":"2024-04-27T03:33:27.565341+0000","last_peered":"2024-04-27T03:33:27.565341+0000","last_clean":"2024-04-27T03:33:27.565341+0000","last_became_active":"2024-04-27T03:30:19.406364+0000","last_became_peered":"2024-04-27T03:30:19.406364+0000","last_unstale":"2024-04-27T03:33:27.565341+0000","last_undegraded":"2024-04-27T03:33:27.565341+0000","last_fullsized":"2024-04-27T03:33:27.565341+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:01:19.671271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432341+0000","last_change":"2024-04-27T03:30:21.408481+0000","last_active":"2024-04-27T03:31:01.432341+0000","last_peered":"2024-04-27T03:31:01.432341+0000","last_clean":"2024-04-27T03:31:01.432341+0000","last_became_active":"2024-04-27T03:30:21.408168+0000","last_became_peered":"2024-04-27T03:30:21.408168+0000","last_unstale":"2024-04-27T03:31:01.432341+0000","last_undegraded":"2024-04-27T03:31:01.432341+0000","last_fullsized":"2024-04-27T03:31:01.432341+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:31:56.856006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"27'14","reported_seq":101,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.703564+0000","last_change":"2024-04-27T03:30:13.371773+0000","last_active":"2024-04-27T03:33:20.703564+0000","last_peered":"2024-04-27T03:33:20.703564+0000","last_clean":"2024-04-27T03:33:20.703564+0000","last_became_active":"2024-04-27T03:30:13.371503+0000","last_became_peered":"2024-04-27T03:30:13.371503+0000","last_unstale":"2024-04-27T03:33:20.703564+0000","last_undegraded":"2024-04-27T03:33:20.703564+0000","last_fullsized":"2024-04-27T03:33:20.703564+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:54.324828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438871+0000","last_change":"2024-04-27T03:30:11.357185+0000","last_active":"2024-04-27T03:31:01.438871+0000","last_peered":"2024-04-27T03:31:01.438871+0000","last_clean":"2024-04-27T03:31:01.438871+0000","last_became_active":"2024-04-27T03:30:11.356739+0000","last_became_peered":"2024-04-27T03:30:11.356739+0000","last_unstale":"2024-04-27T03:31:01.438871+0000","last_undegraded":"2024-04-27T03:31:01.438871+0000","last_fullsized":"2024-04-27T03:31:01.438871+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:31.805263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":11,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:04.282986+0000","last_change":"2024-04-27T03:31:01.443432+0000","last_active":"2024-04-27T03:31:04.282986+0000","last_peered":"2024-04-27T03:31:04.282986+0000","last_clean":"2024-04-27T03:31:04.282986+0000","last_became_active":"2024-04-27T03:31:01.443118+0000","last_became_peered":"2024-04-27T03:31:01.443118+0000","last_unstale":"2024-04-27T03:31:04.282986+0000","last_undegraded":"2024-04-27T03:31:04.282986+0000","last_fullsized":"2024-04-27T03:31:04.282986+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:53:38.016966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"27'74","reported_seq":150,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.702374+0000","last_change":"2024-04-27T03:30:17.392542+0000","last_active":"2024-04-27T03:33:27.702374+0000","last_peered":"2024-04-27T03:33:27.702374+0000","last_clean":"2024-04-27T03:33:27.702374+0000","last_became_active":"2024-04-27T03:30:17.387318+0000","last_became_peered":"2024-04-27T03:30:17.387318+0000","last_unstale":"2024-04-27T03:33:27.702374+0000","last_undegraded":"2024-04-27T03:33:27.702374+0000","last_fullsized":"2024-04-27T03:33:27.702374+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:36.902404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":79,"num_read_kb":68,"num_write":34,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430991+0000","last_change":"2024-04-27T03:30:19.404488+0000","last_active":"2024-04-27T03:31:01.430991+0000","last_peered":"2024-04-27T03:31:01.430991+0000","last_clean":"2024-04-27T03:31:01.430991+0000","last_became_active":"2024-04-27T03:30:19.404265+0000","last_became_peered":"2024-04-27T03:30:19.404265+0000","last_unstale":"2024-04-27T03:31:01.430991+0000","last_undegraded":"2024-04-27T03:31:01.430991+0000","last_fullsized":"2024-04-27T03:31:01.430991+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:26.725030+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"27'18","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.120745+0000","last_change":"2024-04-27T03:30:21.414109+0000","last_active":"2024-04-27T03:33:21.120745+0000","last_peered":"2024-04-27T03:33:21.120745+0000","last_clean":"2024-04-27T03:33:21.120745+0000","last_became_active":"2024-04-27T03:30:21.413757+0000","last_became_peered":"2024-04-27T03:30:21.413757+0000","last_unstale":"2024-04-27T03:33:21.120745+0000","last_undegraded":"2024-04-27T03:33:21.120745+0000","last_fullsized":"2024-04-27T03:33:21.120745+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:41.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981981+0000","last_change":"2024-04-27T03:30:13.380233+0000","last_active":"2024-04-27T03:33:20.981981+0000","last_peered":"2024-04-27T03:33:20.981981+0000","last_clean":"2024-04-27T03:33:20.981981+0000","last_became_active":"2024-04-27T03:30:13.379861+0000","last_became_peered":"2024-04-27T03:30:13.379861+0000","last_unstale":"2024-04-27T03:33:20.981981+0000","last_undegraded":"2024-04-27T03:33:20.981981+0000","last_fullsized":"2024-04-27T03:33:20.981981+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:46.205600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431725+0000","last_change":"2024-04-27T03:30:11.357503+0000","last_active":"2024-04-27T03:31:01.431725+0000","last_peered":"2024-04-27T03:31:01.431725+0000","last_clean":"2024-04-27T03:31:01.431725+0000","last_became_active":"2024-04-27T03:30:11.357176+0000","last_became_peered":"2024-04-27T03:30:11.357176+0000","last_unstale":"2024-04-27T03:31:01.431725+0000","last_undegraded":"2024-04-27T03:31:01.431725+0000","last_fullsized":"2024-04-27T03:31:01.431725+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:06.887756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432802+0000","last_change":"2024-04-27T03:30:15.382614+0000","last_active":"2024-04-27T03:31:01.432802+0000","last_peered":"2024-04-27T03:31:01.432802+0000","last_clean":"2024-04-27T03:31:01.432802+0000","last_became_active":"2024-04-27T03:30:15.382290+0000","last_became_peered":"2024-04-27T03:30:15.382290+0000","last_unstale":"2024-04-27T03:31:01.432802+0000","last_undegraded":"2024-04-27T03:31:01.432802+0000","last_fullsized":"2024-04-27T03:31:01.432802+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:16.810766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"27'41","reported_seq":94,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.706044+0000","last_change":"2024-04-27T03:30:17.386194+0000","last_active":"2024-04-27T03:33:27.706044+0000","last_peered":"2024-04-27T03:33:27.706044+0000","last_clean":"2024-04-27T03:33:27.706044+0000","last_became_active":"2024-04-27T03:30:17.385902+0000","last_became_peered":"2024-04-27T03:30:17.385902+0000","last_unstale":"2024-04-27T03:33:27.706044+0000","last_undegraded":"2024-04-27T03:33:27.706044+0000","last_fullsized":"2024-04-27T03:33:27.706044+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:45:51.459337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438538+0000","last_change":"2024-04-27T03:30:19.406061+0000","last_active":"2024-04-27T03:31:01.438538+0000","last_peered":"2024-04-27T03:31:01.438538+0000","last_clean":"2024-04-27T03:31:01.438538+0000","last_became_active":"2024-04-27T03:30:19.405450+0000","last_became_peered":"2024-04-27T03:30:19.405450+0000","last_unstale":"2024-04-27T03:31:01.438538+0000","last_undegraded":"2024-04-27T03:31:01.438538+0000","last_fullsized":"2024-04-27T03:31:01.438538+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:26:03.989010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"27'9","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.701473+0000","last_change":"2024-04-27T03:30:21.415884+0000","last_active":"2024-04-27T03:33:20.701473+0000","last_peered":"2024-04-27T03:33:20.701473+0000","last_clean":"2024-04-27T03:33:20.701473+0000","last_became_active":"2024-04-27T03:30:21.415494+0000","last_became_peered":"2024-04-27T03:30:21.415494+0000","last_unstale":"2024-04-27T03:33:20.701473+0000","last_undegraded":"2024-04-27T03:33:20.701473+0000","last_fullsized":"2024-04-27T03:33:20.701473+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:13:34.804682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'6","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009826+0000","last_change":"2024-04-27T03:30:13.388008+0000","last_active":"2024-04-27T03:33:21.009826+0000","last_peered":"2024-04-27T03:33:21.009826+0000","last_clean":"2024-04-27T03:33:21.009826+0000","last_became_active":"2024-04-27T03:30:13.387533+0000","last_became_peered":"2024-04-27T03:30:13.387533+0000","last_unstale":"2024-04-27T03:33:21.009826+0000","last_undegraded":"2024-04-27T03:33:21.009826+0000","last_fullsized":"2024-04-27T03:33:21.009826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:53.142322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437632+0000","last_change":"2024-04-27T03:30:11.373706+0000","last_active":"2024-04-27T03:31:01.437632+0000","last_peered":"2024-04-27T03:31:01.437632+0000","last_clean":"2024-04-27T03:31:01.437632+0000","last_became_active":"2024-04-27T03:30:11.373440+0000","last_became_peered":"2024-04-27T03:30:11.373440+0000","last_unstale":"2024-04-27T03:31:01.437632+0000","last_undegraded":"2024-04-27T03:31:01.437632+0000","last_fullsized":"2024-04-27T03:31:01.437632+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:41:06.626342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432793+0000","last_change":"2024-04-27T03:30:15.383722+0000","last_active":"2024-04-27T03:31:01.432793+0000","last_peered":"2024-04-27T03:31:01.432793+0000","last_clean":"2024-04-27T03:31:01.432793+0000","last_became_active":"2024-04-27T03:30:15.382913+0000","last_became_peered":"2024-04-27T03:30:15.382913+0000","last_unstale":"2024-04-27T03:31:01.432793+0000","last_undegraded":"2024-04-27T03:31:01.432793+0000","last_fullsized":"2024-04-27T03:31:01.432793+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:59:19.478810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"27'39","reported_seq":88,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.679362+0000","last_change":"2024-04-27T03:30:17.382348+0000","last_active":"2024-04-27T03:33:27.679362+0000","last_peered":"2024-04-27T03:33:27.679362+0000","last_clean":"2024-04-27T03:33:27.679362+0000","last_became_active":"2024-04-27T03:30:17.381106+0000","last_became_peered":"2024-04-27T03:30:17.381106+0000","last_unstale":"2024-04-27T03:33:27.679362+0000","last_undegraded":"2024-04-27T03:33:27.679362+0000","last_fullsized":"2024-04-27T03:33:27.679362+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:36:29.105227+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"27'285","reported_seq":406,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.560574+0000","last_change":"2024-04-27T03:30:19.403728+0000","last_active":"2024-04-27T03:33:27.560574+0000","last_peered":"2024-04-27T03:33:27.560574+0000","last_clean":"2024-04-27T03:33:27.560574+0000","last_became_active":"2024-04-27T03:30:19.403413+0000","last_became_peered":"2024-04-27T03:30:19.403413+0000","last_unstale":"2024-04-27T03:33:27.560574+0000","last_undegraded":"2024-04-27T03:33:27.560574+0000","last_fullsized":"2024-04-27T03:33:27.560574+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:28.087236+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"27'14","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.289774+0000","last_change":"2024-04-27T03:30:21.403250+0000","last_active":"2024-04-27T03:32:29.289774+0000","last_peered":"2024-04-27T03:32:29.289774+0000","last_clean":"2024-04-27T03:32:29.289774+0000","last_became_active":"2024-04-27T03:30:21.402102+0000","last_became_peered":"2024-04-27T03:30:21.402102+0000","last_unstale":"2024-04-27T03:32:29.289774+0000","last_undegraded":"2024-04-27T03:32:29.289774+0000","last_fullsized":"2024-04-27T03:32:29.289774+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:00.181478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":23,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'12","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.795005+0000","last_change":"2024-04-27T03:30:13.383203+0000","last_active":"2024-04-27T03:33:20.795005+0000","last_peered":"2024-04-27T03:33:20.795005+0000","last_clean":"2024-04-27T03:33:20.795005+0000","last_became_active":"2024-04-27T03:30:13.382899+0000","last_became_peered":"2024-04-27T03:30:13.382899+0000","last_unstale":"2024-04-27T03:33:20.795005+0000","last_undegraded":"2024-04-27T03:33:20.795005+0000","last_fullsized":"2024-04-27T03:33:20.795005+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:57.593400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'4","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.924480+0000","last_change":"2024-04-27T03:31:01.444153+0000","last_active":"2024-04-27T03:33:20.924480+0000","last_peered":"2024-04-27T03:33:20.924480+0000","last_clean":"2024-04-27T03:33:20.924480+0000","last_became_active":"2024-04-27T03:31:01.439776+0000","last_became_peered":"2024-04-27T03:31:01.439776+0000","last_unstale":"2024-04-27T03:33:20.924480+0000","last_undegraded":"2024-04-27T03:33:20.924480+0000","last_fullsized":"2024-04-27T03:33:20.924480+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:18:15.239233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"27'3","reported_seq":417,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.540677+0000","last_change":"2024-04-27T03:30:15.382936+0000","last_active":"2024-04-27T03:33:27.540677+0000","last_peered":"2024-04-27T03:33:27.540677+0000","last_clean":"2024-04-27T03:33:27.540677+0000","last_became_active":"2024-04-27T03:30:15.382502+0000","last_became_peered":"2024-04-27T03:30:15.382502+0000","last_unstale":"2024-04-27T03:33:27.540677+0000","last_undegraded":"2024-04-27T03:33:27.540677+0000","last_fullsized":"2024-04-27T03:33:27.540677+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:17.198784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"25'10","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438996+0000","last_change":"2024-04-27T03:30:17.389548+0000","last_active":"2024-04-27T03:31:01.438996+0000","last_peered":"2024-04-27T03:31:01.438996+0000","last_clean":"2024-04-27T03:31:01.438996+0000","last_became_active":"2024-04-27T03:30:17.387739+0000","last_became_peered":"2024-04-27T03:30:17.387739+0000","last_unstale":"2024-04-27T03:31:01.438996+0000","last_undegraded":"2024-04-27T03:31:01.438996+0000","last_fullsized":"2024-04-27T03:31:01.438996+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:03:39.120700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"27'285","reported_seq":513,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.561666+0000","last_change":"2024-04-27T03:30:19.402853+0000","last_active":"2024-04-27T03:33:27.561666+0000","last_peered":"2024-04-27T03:33:27.561666+0000","last_clean":"2024-04-27T03:33:27.561666+0000","last_became_active":"2024-04-27T03:30:19.402484+0000","last_became_peered":"2024-04-27T03:30:19.402484+0000","last_unstale":"2024-04-27T03:33:27.561666+0000","last_undegraded":"2024-04-27T03:33:27.561666+0000","last_fullsized":"2024-04-27T03:33:27.561666+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:31.846278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"27'4","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.132546+0000","last_change":"2024-04-27T03:30:21.408801+0000","last_active":"2024-04-27T03:32:20.132546+0000","last_peered":"2024-04-27T03:32:20.132546+0000","last_clean":"2024-04-27T03:32:20.132546+0000","last_became_active":"2024-04-27T03:30:21.408512+0000","last_became_peered":"2024-04-27T03:30:21.408512+0000","last_unstale":"2024-04-27T03:32:20.132546+0000","last_undegraded":"2024-04-27T03:32:20.132546+0000","last_fullsized":"2024-04-27T03:32:20.132546+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:49:39.647798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'14","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.792438+0000","last_change":"2024-04-27T03:30:13.381639+0000","last_active":"2024-04-27T03:33:20.792438+0000","last_peered":"2024-04-27T03:33:20.792438+0000","last_clean":"2024-04-27T03:33:20.792438+0000","last_became_active":"2024-04-27T03:30:13.380540+0000","last_became_peered":"2024-04-27T03:30:13.380540+0000","last_unstale":"2024-04-27T03:33:20.792438+0000","last_undegraded":"2024-04-27T03:33:20.792438+0000","last_fullsized":"2024-04-27T03:33:20.792438+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:37:44.819638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.190061+0000","last_change":"2024-04-27T03:30:11.372470+0000","last_active":"2024-04-27T03:32:20.190061+0000","last_peered":"2024-04-27T03:32:20.190061+0000","last_clean":"2024-04-27T03:32:20.190061+0000","last_became_active":"2024-04-27T03:30:11.372193+0000","last_became_peered":"2024-04-27T03:30:11.372193+0000","last_unstale":"2024-04-27T03:32:20.190061+0000","last_undegraded":"2024-04-27T03:32:20.190061+0000","last_fullsized":"2024-04-27T03:32:20.190061+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:16.294699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438746+0000","last_change":"2024-04-27T03:30:15.385100+0000","last_active":"2024-04-27T03:31:01.438746+0000","last_peered":"2024-04-27T03:31:01.438746+0000","last_clean":"2024-04-27T03:31:01.438746+0000","last_became_active":"2024-04-27T03:30:15.383924+0000","last_became_peered":"2024-04-27T03:30:15.383924+0000","last_unstale":"2024-04-27T03:31:01.438746+0000","last_undegraded":"2024-04-27T03:31:01.438746+0000","last_fullsized":"2024-04-27T03:31:01.438746+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:08.259240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"27'41","reported_seq":91,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.639600+0000","last_change":"2024-04-27T03:30:17.388297+0000","last_active":"2024-04-27T03:33:27.639600+0000","last_peered":"2024-04-27T03:33:27.639600+0000","last_clean":"2024-04-27T03:33:27.639600+0000","last_became_active":"2024-04-27T03:30:17.387578+0000","last_became_peered":"2024-04-27T03:30:17.387578+0000","last_unstale":"2024-04-27T03:33:27.639600+0000","last_undegraded":"2024-04-27T03:33:27.639600+0000","last_fullsized":"2024-04-27T03:33:27.639600+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:26.963495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433032+0000","last_change":"2024-04-27T03:30:19.405992+0000","last_active":"2024-04-27T03:31:01.433032+0000","last_peered":"2024-04-27T03:31:01.433032+0000","last_clean":"2024-04-27T03:31:01.433032+0000","last_became_active":"2024-04-27T03:30:19.405512+0000","last_became_peered":"2024-04-27T03:30:19.405512+0000","last_unstale":"2024-04-27T03:31:01.433032+0000","last_undegraded":"2024-04-27T03:31:01.433032+0000","last_fullsized":"2024-04-27T03:31:01.433032+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:15.297931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"27'14","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.046419+0000","last_change":"2024-04-27T03:30:21.407207+0000","last_active":"2024-04-27T03:33:21.046419+0000","last_peered":"2024-04-27T03:33:21.046419+0000","last_clean":"2024-04-27T03:33:21.046419+0000","last_became_active":"2024-04-27T03:30:21.406883+0000","last_became_peered":"2024-04-27T03:30:21.406883+0000","last_unstale":"2024-04-27T03:33:21.046419+0000","last_undegraded":"2024-04-27T03:33:21.046419+0000","last_fullsized":"2024-04-27T03:33:21.046419+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:31:00.181478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":59,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'20","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933899+0000","last_change":"2024-04-27T03:30:13.387655+0000","last_active":"2024-04-27T03:33:20.933899+0000","last_peered":"2024-04-27T03:33:20.933899+0000","last_clean":"2024-04-27T03:33:20.933899+0000","last_became_active":"2024-04-27T03:30:13.387180+0000","last_became_peered":"2024-04-27T03:30:13.387180+0000","last_unstale":"2024-04-27T03:33:20.933899+0000","last_undegraded":"2024-04-27T03:33:20.933899+0000","last_fullsized":"2024-04-27T03:33:20.933899+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:24.510289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":193,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431660+0000","last_change":"2024-04-27T03:30:06.104151+0000","last_active":"2024-04-27T03:31:01.431660+0000","last_peered":"2024-04-27T03:31:01.431660+0000","last_clean":"2024-04-27T03:31:01.431660+0000","last_became_active":"2024-04-27T03:30:06.102893+0000","last_became_peered":"2024-04-27T03:30:06.102893+0000","last_unstale":"2024-04-27T03:31:01.431660+0000","last_undegraded":"2024-04-27T03:31:01.431660+0000","last_fullsized":"2024-04-27T03:31:01.431660+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:27:47.408864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439201+0000","last_change":"2024-04-27T03:30:06.106875+0000","last_active":"2024-04-27T03:31:01.439201+0000","last_peered":"2024-04-27T03:31:01.439201+0000","last_clean":"2024-04-27T03:31:01.439201+0000","last_became_active":"2024-04-27T03:30:06.106511+0000","last_became_peered":"2024-04-27T03:30:06.106511+0000","last_unstale":"2024-04-27T03:31:01.439201+0000","last_undegraded":"2024-04-27T03:31:01.439201+0000","last_fullsized":"2024-04-27T03:31:01.439201+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:56:59.055441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.005690+0000","last_change":"2024-04-27T03:30:11.362699+0000","last_active":"2024-04-27T03:33:21.005690+0000","last_peered":"2024-04-27T03:33:21.005690+0000","last_clean":"2024-04-27T03:33:21.005690+0000","last_became_active":"2024-04-27T03:30:11.362445+0000","last_became_peered":"2024-04-27T03:30:11.362445+0000","last_unstale":"2024-04-27T03:33:21.005690+0000","last_undegraded":"2024-04-27T03:33:21.005690+0000","last_fullsized":"2024-04-27T03:33:21.005690+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:11:24.757438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.441975+0000","last_change":"2024-04-27T03:31:01.441027+0000","last_active":"2024-04-27T03:31:01.441975+0000","last_peered":"2024-04-27T03:31:01.441975+0000","last_clean":"2024-04-27T03:31:01.441975+0000","last_became_active":"2024-04-27T03:31:01.440678+0000","last_became_peered":"2024-04-27T03:31:01.440678+0000","last_unstale":"2024-04-27T03:31:01.441975+0000","last_undegraded":"2024-04-27T03:31:01.441975+0000","last_fullsized":"2024-04-27T03:31:01.441975+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:50:22.897973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"27'92","reported_seq":179,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.672699+0000","last_change":"2024-04-27T03:30:17.388976+0000","last_active":"2024-04-27T03:33:27.672699+0000","last_peered":"2024-04-27T03:33:27.672699+0000","last_clean":"2024-04-27T03:33:27.672699+0000","last_became_active":"2024-04-27T03:30:17.387707+0000","last_became_peered":"2024-04-27T03:30:17.387707+0000","last_unstale":"2024-04-27T03:33:27.672699+0000","last_undegraded":"2024-04-27T03:33:27.672699+0000","last_fullsized":"2024-04-27T03:33:27.672699+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":92,"log_dups_size":0,"ondisk_log_size":92,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:43:03.008955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":93,"num_read_kb":81,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433316+0000","last_change":"2024-04-27T03:30:19.405693+0000","last_active":"2024-04-27T03:31:01.433316+0000","last_peered":"2024-04-27T03:31:01.433316+0000","last_clean":"2024-04-27T03:31:01.433316+0000","last_became_active":"2024-04-27T03:30:19.405359+0000","last_became_peered":"2024-04-27T03:30:19.405359+0000","last_unstale":"2024-04-27T03:31:01.433316+0000","last_undegraded":"2024-04-27T03:31:01.433316+0000","last_fullsized":"2024-04-27T03:31:01.433316+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:36.634879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"27'81","reported_seq":162,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.022864+0000","last_change":"2024-04-27T03:30:21.404603+0000","last_active":"2024-04-27T03:33:21.022864+0000","last_peered":"2024-04-27T03:33:21.022864+0000","last_clean":"2024-04-27T03:33:21.022864+0000","last_became_active":"2024-04-27T03:30:21.404300+0000","last_became_peered":"2024-04-27T03:30:21.404300+0000","last_unstale":"2024-04-27T03:33:21.022864+0000","last_undegraded":"2024-04-27T03:33:21.022864+0000","last_fullsized":"2024-04-27T03:33:21.022864+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:05.598061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439232+0000","last_change":"2024-04-27T03:30:13.378248+0000","last_active":"2024-04-27T03:31:01.439232+0000","last_peered":"2024-04-27T03:31:01.439232+0000","last_clean":"2024-04-27T03:31:01.439232+0000","last_became_active":"2024-04-27T03:30:13.377713+0000","last_became_peered":"2024-04-27T03:30:13.377713+0000","last_unstale":"2024-04-27T03:31:01.439232+0000","last_undegraded":"2024-04-27T03:31:01.439232+0000","last_fullsized":"2024-04-27T03:31:01.439232+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:59:09.267760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436259+0000","last_change":"2024-04-27T03:30:06.106890+0000","last_active":"2024-04-27T03:31:01.436259+0000","last_peered":"2024-04-27T03:31:01.436259+0000","last_clean":"2024-04-27T03:31:01.436259+0000","last_became_active":"2024-04-27T03:30:06.106474+0000","last_became_peered":"2024-04-27T03:30:06.106474+0000","last_unstale":"2024-04-27T03:31:01.436259+0000","last_undegraded":"2024-04-27T03:31:01.436259+0000","last_fullsized":"2024-04-27T03:31:01.436259+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:40:12.652758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"27'4","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.643945+0000","last_change":"2024-04-27T03:30:11.360679+0000","last_active":"2024-04-27T03:31:36.643945+0000","last_peered":"2024-04-27T03:31:36.643945+0000","last_clean":"2024-04-27T03:31:36.643945+0000","last_became_active":"2024-04-27T03:30:11.360244+0000","last_became_peered":"2024-04-27T03:30:11.360244+0000","last_unstale":"2024-04-27T03:31:36.643945+0000","last_undegraded":"2024-04-27T03:31:36.643945+0000","last_fullsized":"2024-04-27T03:31:36.643945+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.539212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440319+0000","last_change":"2024-04-27T03:30:15.379725+0000","last_active":"2024-04-27T03:31:01.440319+0000","last_peered":"2024-04-27T03:31:01.440319+0000","last_clean":"2024-04-27T03:31:01.440319+0000","last_became_active":"2024-04-27T03:30:15.379448+0000","last_became_peered":"2024-04-27T03:30:15.379448+0000","last_unstale":"2024-04-27T03:31:01.440319+0000","last_undegraded":"2024-04-27T03:31:01.440319+0000","last_fullsized":"2024-04-27T03:31:01.440319+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:06.243654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"27'290","reported_seq":512,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.697223+0000","last_change":"2024-04-27T03:30:17.391189+0000","last_active":"2024-04-27T03:33:27.697223+0000","last_peered":"2024-04-27T03:33:27.697223+0000","last_clean":"2024-04-27T03:33:27.697223+0000","last_became_active":"2024-04-27T03:30:17.390884+0000","last_became_peered":"2024-04-27T03:30:17.390884+0000","last_unstale":"2024-04-27T03:33:27.697223+0000","last_undegraded":"2024-04-27T03:33:27.697223+0000","last_fullsized":"2024-04-27T03:33:27.697223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":290,"log_dups_size":0,"ondisk_log_size":290,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:07:57.164266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":387,"num_read_kb":371,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439611+0000","last_change":"2024-04-27T03:30:19.408904+0000","last_active":"2024-04-27T03:31:01.439611+0000","last_peered":"2024-04-27T03:31:01.439611+0000","last_clean":"2024-04-27T03:31:01.439611+0000","last_became_active":"2024-04-27T03:30:19.405777+0000","last_became_peered":"2024-04-27T03:30:19.405777+0000","last_unstale":"2024-04-27T03:31:01.439611+0000","last_undegraded":"2024-04-27T03:31:01.439611+0000","last_fullsized":"2024-04-27T03:31:01.439611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:57.389557+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"27'7","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.968472+0000","last_change":"2024-04-27T03:30:21.409632+0000","last_active":"2024-04-27T03:33:20.968472+0000","last_peered":"2024-04-27T03:33:20.968472+0000","last_clean":"2024-04-27T03:33:20.968472+0000","last_became_active":"2024-04-27T03:30:21.409359+0000","last_became_peered":"2024-04-27T03:30:21.409359+0000","last_unstale":"2024-04-27T03:33:20.968472+0000","last_undegraded":"2024-04-27T03:33:20.968472+0000","last_fullsized":"2024-04-27T03:33:20.968472+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:41.429115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"25'10","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433170+0000","last_change":"2024-04-27T03:30:13.385862+0000","last_active":"2024-04-27T03:31:01.433170+0000","last_peered":"2024-04-27T03:31:01.433170+0000","last_clean":"2024-04-27T03:31:01.433170+0000","last_became_active":"2024-04-27T03:30:13.385521+0000","last_became_peered":"2024-04-27T03:30:13.385521+0000","last_unstale":"2024-04-27T03:31:01.433170+0000","last_undegraded":"2024-04-27T03:31:01.433170+0000","last_fullsized":"2024-04-27T03:31:01.433170+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:32:38.976155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435195+0000","last_change":"2024-04-27T03:30:06.103722+0000","last_active":"2024-04-27T03:31:01.435195+0000","last_peered":"2024-04-27T03:31:01.435195+0000","last_clean":"2024-04-27T03:31:01.435195+0000","last_became_active":"2024-04-27T03:30:06.103441+0000","last_became_peered":"2024-04-27T03:30:06.103441+0000","last_unstale":"2024-04-27T03:31:01.435195+0000","last_undegraded":"2024-04-27T03:31:01.435195+0000","last_fullsized":"2024-04-27T03:31:01.435195+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:57:59.881443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430689+0000","last_change":"2024-04-27T03:30:11.360789+0000","last_active":"2024-04-27T03:31:01.430689+0000","last_peered":"2024-04-27T03:31:01.430689+0000","last_clean":"2024-04-27T03:31:01.430689+0000","last_became_active":"2024-04-27T03:30:11.360443+0000","last_became_peered":"2024-04-27T03:30:11.360443+0000","last_unstale":"2024-04-27T03:31:01.430689+0000","last_undegraded":"2024-04-27T03:31:01.430689+0000","last_fullsized":"2024-04-27T03:31:01.430689+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.863119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"18'1","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437997+0000","last_change":"2024-04-27T03:30:15.383569+0000","last_active":"2024-04-27T03:31:01.437997+0000","last_peered":"2024-04-27T03:31:01.437997+0000","last_clean":"2024-04-27T03:31:01.437997+0000","last_became_active":"2024-04-27T03:30:15.383213+0000","last_became_peered":"2024-04-27T03:30:15.383213+0000","last_unstale":"2024-04-27T03:31:01.437997+0000","last_undegraded":"2024-04-27T03:31:01.437997+0000","last_fullsized":"2024-04-27T03:31:01.437997+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:42:48.217027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"27'16","reported_seq":84,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.462975+0000","last_change":"2024-04-27T03:30:17.388886+0000","last_active":"2024-04-27T03:33:20.462975+0000","last_peered":"2024-04-27T03:33:20.462975+0000","last_clean":"2024-04-27T03:33:20.462975+0000","last_became_active":"2024-04-27T03:30:17.388588+0000","last_became_peered":"2024-04-27T03:30:17.388588+0000","last_unstale":"2024-04-27T03:33:20.462975+0000","last_undegraded":"2024-04-27T03:33:20.462975+0000","last_fullsized":"2024-04-27T03:33:20.462975+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:59:21.489327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431871+0000","last_change":"2024-04-27T03:30:19.407292+0000","last_active":"2024-04-27T03:31:01.431871+0000","last_peered":"2024-04-27T03:31:01.431871+0000","last_clean":"2024-04-27T03:31:01.431871+0000","last_became_active":"2024-04-27T03:30:19.407004+0000","last_became_peered":"2024-04-27T03:30:19.407004+0000","last_unstale":"2024-04-27T03:31:01.431871+0000","last_undegraded":"2024-04-27T03:31:01.431871+0000","last_fullsized":"2024-04-27T03:31:01.431871+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:18:10.262399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"25'2","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438400+0000","last_change":"2024-04-27T03:30:21.417022+0000","last_active":"2024-04-27T03:31:01.438400+0000","last_peered":"2024-04-27T03:31:01.438400+0000","last_clean":"2024-04-27T03:31:01.438400+0000","last_became_active":"2024-04-27T03:30:21.416389+0000","last_became_peered":"2024-04-27T03:30:21.416389+0000","last_unstale":"2024-04-27T03:31:01.438400+0000","last_undegraded":"2024-04-27T03:31:01.438400+0000","last_fullsized":"2024-04-27T03:31:01.438400+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:11:27.333113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'8","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.696826+0000","last_change":"2024-04-27T03:30:13.369982+0000","last_active":"2024-04-27T03:33:20.696826+0000","last_peered":"2024-04-27T03:33:20.696826+0000","last_clean":"2024-04-27T03:33:20.696826+0000","last_became_active":"2024-04-27T03:30:13.369638+0000","last_became_peered":"2024-04-27T03:30:13.369638+0000","last_unstale":"2024-04-27T03:33:20.696826+0000","last_undegraded":"2024-04-27T03:33:20.696826+0000","last_fullsized":"2024-04-27T03:33:20.696826+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:57.593400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436751+0000","last_change":"2024-04-27T03:30:06.107772+0000","last_active":"2024-04-27T03:31:01.436751+0000","last_peered":"2024-04-27T03:31:01.436751+0000","last_clean":"2024-04-27T03:31:01.436751+0000","last_became_active":"2024-04-27T03:30:06.107356+0000","last_became_peered":"2024-04-27T03:30:06.107356+0000","last_unstale":"2024-04-27T03:31:01.436751+0000","last_undegraded":"2024-04-27T03:31:01.436751+0000","last_fullsized":"2024-04-27T03:31:01.436751+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:05:28.414626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436437+0000","last_change":"2024-04-27T03:30:11.357072+0000","last_active":"2024-04-27T03:31:01.436437+0000","last_peered":"2024-04-27T03:31:01.436437+0000","last_clean":"2024-04-27T03:31:01.436437+0000","last_became_active":"2024-04-27T03:30:11.356685+0000","last_became_peered":"2024-04-27T03:30:11.356685+0000","last_unstale":"2024-04-27T03:31:01.436437+0000","last_undegraded":"2024-04-27T03:31:01.436437+0000","last_fullsized":"2024-04-27T03:31:01.436437+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.863119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432748+0000","last_change":"2024-04-27T03:30:15.382673+0000","last_active":"2024-04-27T03:31:01.432748+0000","last_peered":"2024-04-27T03:31:01.432748+0000","last_clean":"2024-04-27T03:31:01.432748+0000","last_became_active":"2024-04-27T03:30:15.382326+0000","last_became_peered":"2024-04-27T03:30:15.382326+0000","last_unstale":"2024-04-27T03:31:01.432748+0000","last_undegraded":"2024-04-27T03:31:01.432748+0000","last_fullsized":"2024-04-27T03:31:01.432748+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:37.500097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"27'13","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.319621+0000","last_change":"2024-04-27T03:30:17.390316+0000","last_active":"2024-04-27T03:33:20.319621+0000","last_peered":"2024-04-27T03:33:20.319621+0000","last_clean":"2024-04-27T03:33:20.319621+0000","last_became_active":"2024-04-27T03:30:17.389373+0000","last_became_peered":"2024-04-27T03:30:17.389373+0000","last_unstale":"2024-04-27T03:33:20.319621+0000","last_undegraded":"2024-04-27T03:33:20.319621+0000","last_fullsized":"2024-04-27T03:33:20.319621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:08.896469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438561+0000","last_change":"2024-04-27T03:30:19.406634+0000","last_active":"2024-04-27T03:31:01.438561+0000","last_peered":"2024-04-27T03:31:01.438561+0000","last_clean":"2024-04-27T03:31:01.438561+0000","last_became_active":"2024-04-27T03:30:19.405953+0000","last_became_peered":"2024-04-27T03:30:19.405953+0000","last_unstale":"2024-04-27T03:31:01.438561+0000","last_undegraded":"2024-04-27T03:31:01.438561+0000","last_fullsized":"2024-04-27T03:31:01.438561+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:02.743749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"27'6","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.073585+0000","last_change":"2024-04-27T03:30:21.413040+0000","last_active":"2024-04-27T03:33:21.073585+0000","last_peered":"2024-04-27T03:33:21.073585+0000","last_clean":"2024-04-27T03:33:21.073585+0000","last_became_active":"2024-04-27T03:30:21.412790+0000","last_became_peered":"2024-04-27T03:30:21.412790+0000","last_unstale":"2024-04-27T03:33:21.073585+0000","last_undegraded":"2024-04-27T03:33:21.073585+0000","last_fullsized":"2024-04-27T03:33:21.073585+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:58.583851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'18","reported_seq":86,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933802+0000","last_change":"2024-04-27T03:30:13.381418+0000","last_active":"2024-04-27T03:33:20.933802+0000","last_peered":"2024-04-27T03:33:20.933802+0000","last_clean":"2024-04-27T03:33:20.933802+0000","last_became_active":"2024-04-27T03:30:13.380888+0000","last_became_peered":"2024-04-27T03:30:13.380888+0000","last_unstale":"2024-04-27T03:33:20.933802+0000","last_undegraded":"2024-04-27T03:33:20.933802+0000","last_fullsized":"2024-04-27T03:33:20.933802+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:28.256872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":73,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433189+0000","last_change":"2024-04-27T03:30:11.362550+0000","last_active":"2024-04-27T03:31:01.433189+0000","last_peered":"2024-04-27T03:31:01.433189+0000","last_clean":"2024-04-27T03:31:01.433189+0000","last_became_active":"2024-04-27T03:30:11.361786+0000","last_became_peered":"2024-04-27T03:30:11.361786+0000","last_unstale":"2024-04-27T03:31:01.433189+0000","last_undegraded":"2024-04-27T03:31:01.433189+0000","last_fullsized":"2024-04-27T03:31:01.433189+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.539212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431862+0000","last_change":"2024-04-27T03:30:06.107861+0000","last_active":"2024-04-27T03:31:01.431862+0000","last_peered":"2024-04-27T03:31:01.431862+0000","last_clean":"2024-04-27T03:31:01.431862+0000","last_became_active":"2024-04-27T03:30:06.107594+0000","last_became_peered":"2024-04-27T03:30:06.107594+0000","last_unstale":"2024-04-27T03:31:01.431862+0000","last_undegraded":"2024-04-27T03:31:01.431862+0000","last_fullsized":"2024-04-27T03:31:01.431862+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:59:10.319152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"27'1","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.147450+0000","last_change":"2024-04-27T03:30:15.381770+0000","last_active":"2024-04-27T03:33:21.147450+0000","last_peered":"2024-04-27T03:33:21.147450+0000","last_clean":"2024-04-27T03:33:21.147450+0000","last_became_active":"2024-04-27T03:30:15.381392+0000","last_became_peered":"2024-04-27T03:30:15.381392+0000","last_unstale":"2024-04-27T03:33:21.147450+0000","last_undegraded":"2024-04-27T03:33:21.147450+0000","last_fullsized":"2024-04-27T03:33:21.147450+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:01.291555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"27'79","reported_seq":158,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.675889+0000","last_change":"2024-04-27T03:30:17.390751+0000","last_active":"2024-04-27T03:33:27.675889+0000","last_peered":"2024-04-27T03:33:27.675889+0000","last_clean":"2024-04-27T03:33:27.675889+0000","last_became_active":"2024-04-27T03:30:17.390409+0000","last_became_peered":"2024-04-27T03:30:17.390409+0000","last_unstale":"2024-04-27T03:33:27.675889+0000","last_undegraded":"2024-04-27T03:33:27.675889+0000","last_fullsized":"2024-04-27T03:33:27.675889+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:19.209300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":89,"num_read_kb":75,"num_write":42,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435401+0000","last_change":"2024-04-27T03:30:19.403292+0000","last_active":"2024-04-27T03:31:01.435401+0000","last_peered":"2024-04-27T03:31:01.435401+0000","last_clean":"2024-04-27T03:31:01.435401+0000","last_became_active":"2024-04-27T03:30:19.402977+0000","last_became_peered":"2024-04-27T03:30:19.402977+0000","last_unstale":"2024-04-27T03:31:01.435401+0000","last_undegraded":"2024-04-27T03:31:01.435401+0000","last_fullsized":"2024-04-27T03:31:01.435401+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:24.818605+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"27'11","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.984330+0000","last_change":"2024-04-27T03:30:21.410763+0000","last_active":"2024-04-27T03:33:20.984330+0000","last_peered":"2024-04-27T03:33:20.984330+0000","last_clean":"2024-04-27T03:33:20.984330+0000","last_became_active":"2024-04-27T03:30:21.410511+0000","last_became_peered":"2024-04-27T03:30:21.410511+0000","last_unstale":"2024-04-27T03:33:20.984330+0000","last_undegraded":"2024-04-27T03:33:20.984330+0000","last_fullsized":"2024-04-27T03:33:20.984330+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:07:58.583851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"27'3","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.019081+0000","last_change":"2024-04-27T03:30:13.380442+0000","last_active":"2024-04-27T03:33:21.019081+0000","last_peered":"2024-04-27T03:33:21.019081+0000","last_clean":"2024-04-27T03:33:21.019081+0000","last_became_active":"2024-04-27T03:30:13.379985+0000","last_became_peered":"2024-04-27T03:30:13.379985+0000","last_unstale":"2024-04-27T03:33:21.019081+0000","last_undegraded":"2024-04-27T03:33:21.019081+0000","last_fullsized":"2024-04-27T03:33:21.019081+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:19.215823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"27'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.596854+0000","last_change":"2024-04-27T03:30:11.361129+0000","last_active":"2024-04-27T03:33:20.596854+0000","last_peered":"2024-04-27T03:33:20.596854+0000","last_clean":"2024-04-27T03:33:20.596854+0000","last_became_active":"2024-04-27T03:30:11.360352+0000","last_became_peered":"2024-04-27T03:30:11.360352+0000","last_unstale":"2024-04-27T03:33:20.596854+0000","last_undegraded":"2024-04-27T03:33:20.596854+0000","last_fullsized":"2024-04-27T03:33:20.596854+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:06:22.798184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440363+0000","last_change":"2024-04-27T03:30:06.106816+0000","last_active":"2024-04-27T03:31:01.440363+0000","last_peered":"2024-04-27T03:31:01.440363+0000","last_clean":"2024-04-27T03:31:01.440363+0000","last_became_active":"2024-04-27T03:30:06.106475+0000","last_became_peered":"2024-04-27T03:30:06.106475+0000","last_unstale":"2024-04-27T03:31:01.440363+0000","last_undegraded":"2024-04-27T03:31:01.440363+0000","last_fullsized":"2024-04-27T03:31:01.440363+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:18:36.362210+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437040+0000","last_change":"2024-04-27T03:30:15.380295+0000","last_active":"2024-04-27T03:31:01.437040+0000","last_peered":"2024-04-27T03:31:01.437040+0000","last_clean":"2024-04-27T03:31:01.437040+0000","last_became_active":"2024-04-27T03:30:15.380030+0000","last_became_peered":"2024-04-27T03:30:15.380030+0000","last_unstale":"2024-04-27T03:31:01.437040+0000","last_undegraded":"2024-04-27T03:31:01.437040+0000","last_fullsized":"2024-04-27T03:31:01.437040+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:27.827534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"25'20","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433411+0000","last_change":"2024-04-27T03:30:17.393593+0000","last_active":"2024-04-27T03:31:01.433411+0000","last_peered":"2024-04-27T03:31:01.433411+0000","last_clean":"2024-04-27T03:31:01.433411+0000","last_became_active":"2024-04-27T03:30:17.393253+0000","last_became_peered":"2024-04-27T03:30:17.393253+0000","last_unstale":"2024-04-27T03:31:01.433411+0000","last_undegraded":"2024-04-27T03:31:01.433411+0000","last_fullsized":"2024-04-27T03:31:01.433411+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:49:56.968218+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"27'285","reported_seq":389,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.558096+0000","last_change":"2024-04-27T03:30:19.402596+0000","last_active":"2024-04-27T03:33:27.558096+0000","last_peered":"2024-04-27T03:33:27.558096+0000","last_clean":"2024-04-27T03:33:27.558096+0000","last_became_active":"2024-04-27T03:30:19.402319+0000","last_became_peered":"2024-04-27T03:30:19.402319+0000","last_unstale":"2024-04-27T03:33:27.558096+0000","last_undegraded":"2024-04-27T03:33:27.558096+0000","last_fullsized":"2024-04-27T03:33:27.558096+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:21.528207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438342+0000","last_change":"2024-04-27T03:30:21.414388+0000","last_active":"2024-04-27T03:31:01.438342+0000","last_peered":"2024-04-27T03:31:01.438342+0000","last_clean":"2024-04-27T03:31:01.438342+0000","last_became_active":"2024-04-27T03:30:21.414109+0000","last_became_peered":"2024-04-27T03:30:21.414109+0000","last_unstale":"2024-04-27T03:31:01.438342+0000","last_undegraded":"2024-04-27T03:31:01.438342+0000","last_fullsized":"2024-04-27T03:31:01.438342+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:34.636870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436753+0000","last_change":"2024-04-27T03:30:13.388070+0000","last_active":"2024-04-27T03:31:01.436753+0000","last_peered":"2024-04-27T03:31:01.436753+0000","last_clean":"2024-04-27T03:31:01.436753+0000","last_became_active":"2024-04-27T03:30:13.387764+0000","last_became_peered":"2024-04-27T03:30:13.387764+0000","last_unstale":"2024-04-27T03:31:01.436753+0000","last_undegraded":"2024-04-27T03:31:01.436753+0000","last_fullsized":"2024-04-27T03:31:01.436753+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:16.285485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.918891+0000","last_change":"2024-04-27T03:30:11.356555+0000","last_active":"2024-04-27T03:33:20.918891+0000","last_peered":"2024-04-27T03:33:20.918891+0000","last_clean":"2024-04-27T03:33:20.918891+0000","last_became_active":"2024-04-27T03:30:11.355704+0000","last_became_peered":"2024-04-27T03:30:11.355704+0000","last_unstale":"2024-04-27T03:33:20.918891+0000","last_undegraded":"2024-04-27T03:33:20.918891+0000","last_fullsized":"2024-04-27T03:33:20.918891+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:35.764527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432650+0000","last_change":"2024-04-27T03:30:06.104067+0000","last_active":"2024-04-27T03:31:01.432650+0000","last_peered":"2024-04-27T03:31:01.432650+0000","last_clean":"2024-04-27T03:31:01.432650+0000","last_became_active":"2024-04-27T03:30:06.103759+0000","last_became_peered":"2024-04-27T03:30:06.103759+0000","last_unstale":"2024-04-27T03:31:01.432650+0000","last_undegraded":"2024-04-27T03:31:01.432650+0000","last_fullsized":"2024-04-27T03:31:01.432650+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:27:47.408864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"27'4","reported_seq":228,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.530409+0000","last_change":"2024-04-27T03:30:15.384765+0000","last_active":"2024-04-27T03:33:27.530409+0000","last_peered":"2024-04-27T03:33:27.530409+0000","last_clean":"2024-04-27T03:33:27.530409+0000","last_became_active":"2024-04-27T03:30:15.384372+0000","last_became_peered":"2024-04-27T03:30:15.384372+0000","last_unstale":"2024-04-27T03:33:27.530409+0000","last_undegraded":"2024-04-27T03:33:27.530409+0000","last_fullsized":"2024-04-27T03:33:27.530409+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:55.971988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'20","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295153+0000","last_change":"2024-04-27T03:30:17.389511+0000","last_active":"2024-04-27T03:32:24.295153+0000","last_peered":"2024-04-27T03:32:24.295153+0000","last_clean":"2024-04-27T03:32:24.295153+0000","last_became_active":"2024-04-27T03:30:17.389154+0000","last_became_peered":"2024-04-27T03:30:17.389154+0000","last_unstale":"2024-04-27T03:32:24.295153+0000","last_undegraded":"2024-04-27T03:32:24.295153+0000","last_fullsized":"2024-04-27T03:32:24.295153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:41:46.387083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"27'285","reported_seq":401,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.562832+0000","last_change":"2024-04-27T03:30:19.405158+0000","last_active":"2024-04-27T03:33:27.562832+0000","last_peered":"2024-04-27T03:33:27.562832+0000","last_clean":"2024-04-27T03:33:27.562832+0000","last_became_active":"2024-04-27T03:30:19.404867+0000","last_became_peered":"2024-04-27T03:30:19.404867+0000","last_unstale":"2024-04-27T03:33:27.562832+0000","last_undegraded":"2024-04-27T03:33:27.562832+0000","last_fullsized":"2024-04-27T03:33:27.562832+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:12.277984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"25'3","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434777+0000","last_change":"2024-04-27T03:30:21.413577+0000","last_active":"2024-04-27T03:31:01.434777+0000","last_peered":"2024-04-27T03:31:01.434777+0000","last_clean":"2024-04-27T03:31:01.434777+0000","last_became_active":"2024-04-27T03:30:21.413264+0000","last_became_peered":"2024-04-27T03:30:21.413264+0000","last_unstale":"2024-04-27T03:31:01.434777+0000","last_undegraded":"2024-04-27T03:31:01.434777+0000","last_fullsized":"2024-04-27T03:31:01.434777+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:23.492592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"27'4","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.696091+0000","last_change":"2024-04-27T03:30:13.383496+0000","last_active":"2024-04-27T03:33:20.696091+0000","last_peered":"2024-04-27T03:33:20.696091+0000","last_clean":"2024-04-27T03:33:20.696091+0000","last_became_active":"2024-04-27T03:30:13.383208+0000","last_became_peered":"2024-04-27T03:30:13.383208+0000","last_unstale":"2024-04-27T03:33:20.696091+0000","last_undegraded":"2024-04-27T03:33:20.696091+0000","last_fullsized":"2024-04-27T03:33:20.696091+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:13.288239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436165+0000","last_change":"2024-04-27T03:30:11.370648+0000","last_active":"2024-04-27T03:31:01.436165+0000","last_peered":"2024-04-27T03:31:01.436165+0000","last_clean":"2024-04-27T03:31:01.436165+0000","last_became_active":"2024-04-27T03:30:11.369568+0000","last_became_peered":"2024-04-27T03:30:11.369568+0000","last_unstale":"2024-04-27T03:31:01.436165+0000","last_undegraded":"2024-04-27T03:31:01.436165+0000","last_fullsized":"2024-04-27T03:31:01.436165+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:51.203886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438979+0000","last_change":"2024-04-27T03:30:06.107954+0000","last_active":"2024-04-27T03:31:01.438979+0000","last_peered":"2024-04-27T03:31:01.438979+0000","last_clean":"2024-04-27T03:31:01.438979+0000","last_became_active":"2024-04-27T03:30:06.107600+0000","last_became_peered":"2024-04-27T03:30:06.107600+0000","last_unstale":"2024-04-27T03:31:01.438979+0000","last_undegraded":"2024-04-27T03:31:01.438979+0000","last_fullsized":"2024-04-27T03:31:01.438979+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:25:44.876194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437548+0000","last_change":"2024-04-27T03:30:15.380881+0000","last_active":"2024-04-27T03:31:01.437548+0000","last_peered":"2024-04-27T03:31:01.437548+0000","last_clean":"2024-04-27T03:31:01.437548+0000","last_became_active":"2024-04-27T03:30:15.380598+0000","last_became_peered":"2024-04-27T03:30:15.380598+0000","last_unstale":"2024-04-27T03:31:01.437548+0000","last_undegraded":"2024-04-27T03:31:01.437548+0000","last_fullsized":"2024-04-27T03:31:01.437548+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:12:10.183004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"27'98","reported_seq":188,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.699205+0000","last_change":"2024-04-27T03:30:17.391634+0000","last_active":"2024-04-27T03:33:27.699205+0000","last_peered":"2024-04-27T03:33:27.699205+0000","last_clean":"2024-04-27T03:33:27.699205+0000","last_became_active":"2024-04-27T03:30:17.391191+0000","last_became_peered":"2024-04-27T03:30:17.391191+0000","last_unstale":"2024-04-27T03:33:27.699205+0000","last_undegraded":"2024-04-27T03:33:27.699205+0000","last_fullsized":"2024-04-27T03:33:27.699205+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":98,"log_dups_size":0,"ondisk_log_size":98,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:12.065352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":102,"num_read_kb":87,"num_write":40,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431584+0000","last_change":"2024-04-27T03:30:19.404491+0000","last_active":"2024-04-27T03:31:01.431584+0000","last_peered":"2024-04-27T03:31:01.431584+0000","last_clean":"2024-04-27T03:31:01.431584+0000","last_became_active":"2024-04-27T03:30:19.404187+0000","last_became_peered":"2024-04-27T03:30:19.404187+0000","last_unstale":"2024-04-27T03:31:01.431584+0000","last_undegraded":"2024-04-27T03:31:01.431584+0000","last_fullsized":"2024-04-27T03:31:01.431584+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:21.528207+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432393+0000","last_change":"2024-04-27T03:30:21.408168+0000","last_active":"2024-04-27T03:31:01.432393+0000","last_peered":"2024-04-27T03:31:01.432393+0000","last_clean":"2024-04-27T03:31:01.432393+0000","last_became_active":"2024-04-27T03:30:21.407857+0000","last_became_peered":"2024-04-27T03:30:21.407857+0000","last_unstale":"2024-04-27T03:31:01.432393+0000","last_undegraded":"2024-04-27T03:31:01.432393+0000","last_fullsized":"2024-04-27T03:31:01.432393+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:28.740636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"25'2","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432483+0000","last_change":"2024-04-27T03:30:21.411358+0000","last_active":"2024-04-27T03:31:01.432483+0000","last_peered":"2024-04-27T03:31:01.432483+0000","last_clean":"2024-04-27T03:31:01.432483+0000","last_became_active":"2024-04-27T03:30:21.411054+0000","last_became_peered":"2024-04-27T03:30:21.411054+0000","last_unstale":"2024-04-27T03:31:01.432483+0000","last_undegraded":"2024-04-27T03:31:01.432483+0000","last_fullsized":"2024-04-27T03:31:01.432483+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:44:47.262023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.653891+0000","last_change":"2024-04-27T03:30:13.383003+0000","last_active":"2024-04-27T03:31:36.653891+0000","last_peered":"2024-04-27T03:31:36.653891+0000","last_clean":"2024-04-27T03:31:36.653891+0000","last_became_active":"2024-04-27T03:30:13.382675+0000","last_became_peered":"2024-04-27T03:30:13.382675+0000","last_unstale":"2024-04-27T03:31:36.653891+0000","last_undegraded":"2024-04-27T03:31:36.653891+0000","last_fullsized":"2024-04-27T03:31:36.653891+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:34:21.911691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438857+0000","last_change":"2024-04-27T03:30:11.359531+0000","last_active":"2024-04-27T03:31:01.438857+0000","last_peered":"2024-04-27T03:31:01.438857+0000","last_clean":"2024-04-27T03:31:01.438857+0000","last_became_active":"2024-04-27T03:30:11.359195+0000","last_became_peered":"2024-04-27T03:30:11.359195+0000","last_unstale":"2024-04-27T03:31:01.438857+0000","last_undegraded":"2024-04-27T03:31:01.438857+0000","last_fullsized":"2024-04-27T03:31:01.438857+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:52.318456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434515+0000","last_change":"2024-04-27T03:30:15.379854+0000","last_active":"2024-04-27T03:31:01.434515+0000","last_peered":"2024-04-27T03:31:01.434515+0000","last_clean":"2024-04-27T03:31:01.434515+0000","last_became_active":"2024-04-27T03:30:15.379541+0000","last_became_peered":"2024-04-27T03:30:15.379541+0000","last_unstale":"2024-04-27T03:31:01.434515+0000","last_undegraded":"2024-04-27T03:31:01.434515+0000","last_fullsized":"2024-04-27T03:31:01.434515+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:32:11.127449+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"27'41","reported_seq":94,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.651419+0000","last_change":"2024-04-27T03:30:17.386926+0000","last_active":"2024-04-27T03:33:27.651419+0000","last_peered":"2024-04-27T03:33:27.651419+0000","last_clean":"2024-04-27T03:33:27.651419+0000","last_became_active":"2024-04-27T03:30:17.386496+0000","last_became_peered":"2024-04-27T03:30:17.386496+0000","last_unstale":"2024-04-27T03:33:27.651419+0000","last_undegraded":"2024-04-27T03:33:27.651419+0000","last_fullsized":"2024-04-27T03:33:27.651419+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:39:56.661166+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437333+0000","last_change":"2024-04-27T03:30:19.405864+0000","last_active":"2024-04-27T03:31:01.437333+0000","last_peered":"2024-04-27T03:31:01.437333+0000","last_clean":"2024-04-27T03:31:01.437333+0000","last_became_active":"2024-04-27T03:30:19.405189+0000","last_became_peered":"2024-04-27T03:30:19.405189+0000","last_unstale":"2024-04-27T03:31:01.437333+0000","last_undegraded":"2024-04-27T03:31:01.437333+0000","last_fullsized":"2024-04-27T03:31:01.437333+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:59:23.134716+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"27'3","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.594114+0000","last_change":"2024-04-27T03:30:21.414203+0000","last_active":"2024-04-27T03:32:18.594114+0000","last_peered":"2024-04-27T03:32:18.594114+0000","last_clean":"2024-04-27T03:32:18.594114+0000","last_became_active":"2024-04-27T03:30:21.413920+0000","last_became_peered":"2024-04-27T03:30:21.413920+0000","last_unstale":"2024-04-27T03:32:18.594114+0000","last_undegraded":"2024-04-27T03:32:18.594114+0000","last_fullsized":"2024-04-27T03:32:18.594114+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:40.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"27'16","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981976+0000","last_change":"2024-04-27T03:30:13.381187+0000","last_active":"2024-04-27T03:33:20.981976+0000","last_peered":"2024-04-27T03:33:20.981976+0000","last_clean":"2024-04-27T03:33:20.981976+0000","last_became_active":"2024-04-27T03:30:13.380240+0000","last_became_peered":"2024-04-27T03:30:13.380240+0000","last_unstale":"2024-04-27T03:33:20.981976+0000","last_undegraded":"2024-04-27T03:33:20.981976+0000","last_fullsized":"2024-04-27T03:33:20.981976+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:56.024650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":31,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433005+0000","last_change":"2024-04-27T03:30:11.361106+0000","last_active":"2024-04-27T03:31:01.433005+0000","last_peered":"2024-04-27T03:31:01.433005+0000","last_clean":"2024-04-27T03:31:01.433005+0000","last_became_active":"2024-04-27T03:30:11.360217+0000","last_became_peered":"2024-04-27T03:30:11.360217+0000","last_unstale":"2024-04-27T03:31:01.433005+0000","last_undegraded":"2024-04-27T03:31:01.433005+0000","last_fullsized":"2024-04-27T03:31:01.433005+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:51.203886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439951+0000","last_change":"2024-04-27T03:30:15.381885+0000","last_active":"2024-04-27T03:31:01.439951+0000","last_peered":"2024-04-27T03:31:01.439951+0000","last_clean":"2024-04-27T03:31:01.439951+0000","last_became_active":"2024-04-27T03:30:15.381507+0000","last_became_peered":"2024-04-27T03:30:15.381507+0000","last_unstale":"2024-04-27T03:31:01.439951+0000","last_undegraded":"2024-04-27T03:31:01.439951+0000","last_fullsized":"2024-04-27T03:31:01.439951+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:31:37.867070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"27'6","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.324334+0000","last_change":"2024-04-27T03:30:17.385578+0000","last_active":"2024-04-27T03:33:20.324334+0000","last_peered":"2024-04-27T03:33:20.324334+0000","last_clean":"2024-04-27T03:33:20.324334+0000","last_became_active":"2024-04-27T03:30:17.385318+0000","last_became_peered":"2024-04-27T03:30:17.385318+0000","last_unstale":"2024-04-27T03:33:20.324334+0000","last_undegraded":"2024-04-27T03:33:20.324334+0000","last_fullsized":"2024-04-27T03:33:20.324334+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:47.385488+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435327+0000","last_change":"2024-04-27T03:30:19.407084+0000","last_active":"2024-04-27T03:31:01.435327+0000","last_peered":"2024-04-27T03:31:01.435327+0000","last_clean":"2024-04-27T03:31:01.435327+0000","last_became_active":"2024-04-27T03:30:19.406736+0000","last_became_peered":"2024-04-27T03:30:19.406736+0000","last_unstale":"2024-04-27T03:31:01.435327+0000","last_undegraded":"2024-04-27T03:31:01.435327+0000","last_fullsized":"2024-04-27T03:31:01.435327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:42:21.217528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"27'4","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.798144+0000","last_change":"2024-04-27T03:30:21.408419+0000","last_active":"2024-04-27T03:33:20.798144+0000","last_peered":"2024-04-27T03:33:20.798144+0000","last_clean":"2024-04-27T03:33:20.798144+0000","last_became_active":"2024-04-27T03:30:21.408079+0000","last_became_peered":"2024-04-27T03:30:21.408079+0000","last_unstale":"2024-04-27T03:33:20.798144+0000","last_undegraded":"2024-04-27T03:33:20.798144+0000","last_fullsized":"2024-04-27T03:33:20.798144+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:54.646754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009922+0000","last_change":"2024-04-27T03:30:13.375656+0000","last_active":"2024-04-27T03:33:21.009922+0000","last_peered":"2024-04-27T03:33:21.009922+0000","last_clean":"2024-04-27T03:33:21.009922+0000","last_became_active":"2024-04-27T03:30:13.375350+0000","last_became_peered":"2024-04-27T03:30:13.375350+0000","last_unstale":"2024-04-27T03:33:21.009922+0000","last_undegraded":"2024-04-27T03:33:21.009922+0000","last_fullsized":"2024-04-27T03:33:21.009922+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:29:13.288239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434851+0000","last_change":"2024-04-27T03:30:11.361187+0000","last_active":"2024-04-27T03:31:01.434851+0000","last_peered":"2024-04-27T03:31:01.434851+0000","last_clean":"2024-04-27T03:31:01.434851+0000","last_became_active":"2024-04-27T03:30:11.360938+0000","last_became_peered":"2024-04-27T03:30:11.360938+0000","last_unstale":"2024-04-27T03:31:01.434851+0000","last_undegraded":"2024-04-27T03:31:01.434851+0000","last_fullsized":"2024-04-27T03:31:01.434851+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:11.138117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431691+0000","last_change":"2024-04-27T03:30:15.382918+0000","last_active":"2024-04-27T03:31:01.431691+0000","last_peered":"2024-04-27T03:31:01.431691+0000","last_clean":"2024-04-27T03:31:01.431691+0000","last_became_active":"2024-04-27T03:30:15.382287+0000","last_became_peered":"2024-04-27T03:30:15.382287+0000","last_unstale":"2024-04-27T03:31:01.431691+0000","last_undegraded":"2024-04-27T03:31:01.431691+0000","last_fullsized":"2024-04-27T03:31:01.431691+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:15:21.227251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"27'43","reported_seq":106,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.617393+0000","last_change":"2024-04-27T03:30:17.388554+0000","last_active":"2024-04-27T03:33:27.617393+0000","last_peered":"2024-04-27T03:33:27.617393+0000","last_clean":"2024-04-27T03:33:27.617393+0000","last_became_active":"2024-04-27T03:30:17.388260+0000","last_became_peered":"2024-04-27T03:30:17.388260+0000","last_unstale":"2024-04-27T03:33:27.617393+0000","last_undegraded":"2024-04-27T03:33:27.617393+0000","last_fullsized":"2024-04-27T03:33:27.617393+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:22.810377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432676+0000","last_change":"2024-04-27T03:30:19.402729+0000","last_active":"2024-04-27T03:31:01.432676+0000","last_peered":"2024-04-27T03:31:01.432676+0000","last_clean":"2024-04-27T03:31:01.432676+0000","last_became_active":"2024-04-27T03:30:19.402430+0000","last_became_peered":"2024-04-27T03:30:19.402430+0000","last_unstale":"2024-04-27T03:31:01.432676+0000","last_undegraded":"2024-04-27T03:31:01.432676+0000","last_fullsized":"2024-04-27T03:31:01.432676+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:22:59.540815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438223+0000","last_change":"2024-04-27T03:30:21.416316+0000","last_active":"2024-04-27T03:31:01.438223+0000","last_peered":"2024-04-27T03:31:01.438223+0000","last_clean":"2024-04-27T03:31:01.438223+0000","last_became_active":"2024-04-27T03:30:21.415984+0000","last_became_peered":"2024-04-27T03:30:21.415984+0000","last_unstale":"2024-04-27T03:31:01.438223+0000","last_undegraded":"2024-04-27T03:31:01.438223+0000","last_fullsized":"2024-04-27T03:31:01.438223+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:30.270957+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"27'5","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988543+0000","last_change":"2024-04-27T03:30:13.387737+0000","last_active":"2024-04-27T03:33:20.988543+0000","last_peered":"2024-04-27T03:33:20.988543+0000","last_clean":"2024-04-27T03:33:20.988543+0000","last_became_active":"2024-04-27T03:30:13.386985+0000","last_became_peered":"2024-04-27T03:30:13.386985+0000","last_unstale":"2024-04-27T03:33:20.988543+0000","last_undegraded":"2024-04-27T03:33:20.988543+0000","last_fullsized":"2024-04-27T03:33:20.988543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:43.363544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436347+0000","last_change":"2024-04-27T03:30:11.370182+0000","last_active":"2024-04-27T03:31:01.436347+0000","last_peered":"2024-04-27T03:31:01.436347+0000","last_clean":"2024-04-27T03:31:01.436347+0000","last_became_active":"2024-04-27T03:30:11.369824+0000","last_became_peered":"2024-04-27T03:30:11.369824+0000","last_unstale":"2024-04-27T03:31:01.436347+0000","last_undegraded":"2024-04-27T03:31:01.436347+0000","last_fullsized":"2024-04-27T03:31:01.436347+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:05:11.138117+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438418+0000","last_change":"2024-04-27T03:30:15.384051+0000","last_active":"2024-04-27T03:31:01.438418+0000","last_peered":"2024-04-27T03:31:01.438418+0000","last_clean":"2024-04-27T03:31:01.438418+0000","last_became_active":"2024-04-27T03:30:15.383771+0000","last_became_peered":"2024-04-27T03:30:15.383771+0000","last_unstale":"2024-04-27T03:31:01.438418+0000","last_undegraded":"2024-04-27T03:31:01.438418+0000","last_fullsized":"2024-04-27T03:31:01.438418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:32.616135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"27'33","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.608415+0000","last_change":"2024-04-27T03:30:17.385866+0000","last_active":"2024-04-27T03:33:27.608415+0000","last_peered":"2024-04-27T03:33:27.608415+0000","last_clean":"2024-04-27T03:33:27.608415+0000","last_became_active":"2024-04-27T03:30:17.385488+0000","last_became_peered":"2024-04-27T03:30:17.385488+0000","last_unstale":"2024-04-27T03:33:27.608415+0000","last_undegraded":"2024-04-27T03:33:27.608415+0000","last_fullsized":"2024-04-27T03:33:27.608415+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":33,"log_dups_size":0,"ondisk_log_size":33,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:56:39.510614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437192+0000","last_change":"2024-04-27T03:30:19.398792+0000","last_active":"2024-04-27T03:31:01.437192+0000","last_peered":"2024-04-27T03:31:01.437192+0000","last_clean":"2024-04-27T03:31:01.437192+0000","last_became_active":"2024-04-27T03:30:19.398484+0000","last_became_peered":"2024-04-27T03:30:19.398484+0000","last_unstale":"2024-04-27T03:31:01.437192+0000","last_undegraded":"2024-04-27T03:31:01.437192+0000","last_fullsized":"2024-04-27T03:31:01.437192+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:16.023756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"25'2","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.440063+0000","last_change":"2024-04-27T03:30:13.377494+0000","last_active":"2024-04-27T03:31:01.440063+0000","last_peered":"2024-04-27T03:31:01.440063+0000","last_clean":"2024-04-27T03:31:01.440063+0000","last_became_active":"2024-04-27T03:30:13.377217+0000","last_became_peered":"2024-04-27T03:30:13.377217+0000","last_unstale":"2024-04-27T03:31:01.440063+0000","last_undegraded":"2024-04-27T03:31:01.440063+0000","last_fullsized":"2024-04-27T03:31:01.440063+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:18.788433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"25'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436454+0000","last_change":"2024-04-27T03:30:11.370820+0000","last_active":"2024-04-27T03:31:01.436454+0000","last_peered":"2024-04-27T03:31:01.436454+0000","last_clean":"2024-04-27T03:31:01.436454+0000","last_became_active":"2024-04-27T03:30:11.370547+0000","last_became_peered":"2024-04-27T03:30:11.370547+0000","last_unstale":"2024-04-27T03:31:01.436454+0000","last_undegraded":"2024-04-27T03:31:01.436454+0000","last_fullsized":"2024-04-27T03:31:01.436454+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:19:34.092880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433313+0000","last_change":"2024-04-27T03:30:15.382292+0000","last_active":"2024-04-27T03:31:01.433313+0000","last_peered":"2024-04-27T03:31:01.433313+0000","last_clean":"2024-04-27T03:31:01.433313+0000","last_became_active":"2024-04-27T03:30:15.381956+0000","last_became_peered":"2024-04-27T03:30:15.381956+0000","last_unstale":"2024-04-27T03:31:01.433313+0000","last_undegraded":"2024-04-27T03:31:01.433313+0000","last_fullsized":"2024-04-27T03:31:01.433313+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:10.054835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"27'66","reported_seq":136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.669312+0000","last_change":"2024-04-27T03:30:17.383982+0000","last_active":"2024-04-27T03:33:27.669312+0000","last_peered":"2024-04-27T03:33:27.669312+0000","last_clean":"2024-04-27T03:33:27.669312+0000","last_became_active":"2024-04-27T03:30:17.383178+0000","last_became_peered":"2024-04-27T03:30:17.383178+0000","last_unstale":"2024-04-27T03:33:27.669312+0000","last_undegraded":"2024-04-27T03:33:27.669312+0000","last_fullsized":"2024-04-27T03:33:27.669312+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":66,"log_dups_size":0,"ondisk_log_size":66,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:14.015528+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":28,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433851+0000","last_change":"2024-04-27T03:30:19.403087+0000","last_active":"2024-04-27T03:31:01.433851+0000","last_peered":"2024-04-27T03:31:01.433851+0000","last_clean":"2024-04-27T03:31:01.433851+0000","last_became_active":"2024-04-27T03:30:19.402781+0000","last_became_peered":"2024-04-27T03:30:19.402781+0000","last_unstale":"2024-04-27T03:31:01.433851+0000","last_undegraded":"2024-04-27T03:31:01.433851+0000","last_fullsized":"2024-04-27T03:31:01.433851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:15:30.540461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"27'16","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.240928+0000","last_change":"2024-04-27T03:30:21.408415+0000","last_active":"2024-04-27T03:32:20.240928+0000","last_peered":"2024-04-27T03:32:20.240928+0000","last_clean":"2024-04-27T03:32:20.240928+0000","last_became_active":"2024-04-27T03:30:21.408049+0000","last_became_peered":"2024-04-27T03:30:21.408049+0000","last_unstale":"2024-04-27T03:32:20.240928+0000","last_undegraded":"2024-04-27T03:32:20.240928+0000","last_fullsized":"2024-04-27T03:32:20.240928+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:09.753400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":21,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432877+0000","last_change":"2024-04-27T03:30:13.373552+0000","last_active":"2024-04-27T03:31:01.432877+0000","last_peered":"2024-04-27T03:31:01.432877+0000","last_clean":"2024-04-27T03:31:01.432877+0000","last_became_active":"2024-04-27T03:30:13.373257+0000","last_became_peered":"2024-04-27T03:30:13.373257+0000","last_unstale":"2024-04-27T03:31:01.432877+0000","last_undegraded":"2024-04-27T03:31:01.432877+0000","last_fullsized":"2024-04-27T03:31:01.432877+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:07:51.011104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432526+0000","last_change":"2024-04-27T03:30:11.371431+0000","last_active":"2024-04-27T03:31:01.432526+0000","last_peered":"2024-04-27T03:31:01.432526+0000","last_clean":"2024-04-27T03:31:01.432526+0000","last_became_active":"2024-04-27T03:30:11.370091+0000","last_became_peered":"2024-04-27T03:30:11.370091+0000","last_unstale":"2024-04-27T03:31:01.432526+0000","last_undegraded":"2024-04-27T03:31:01.432526+0000","last_fullsized":"2024-04-27T03:31:01.432526+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.746108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432766+0000","last_change":"2024-04-27T03:30:15.380494+0000","last_active":"2024-04-27T03:31:01.432766+0000","last_peered":"2024-04-27T03:31:01.432766+0000","last_clean":"2024-04-27T03:31:01.432766+0000","last_became_active":"2024-04-27T03:30:15.379800+0000","last_became_peered":"2024-04-27T03:30:15.379800+0000","last_unstale":"2024-04-27T03:31:01.432766+0000","last_undegraded":"2024-04-27T03:31:01.432766+0000","last_fullsized":"2024-04-27T03:31:01.432766+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:58.036077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"27'39","reported_seq":88,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.611519+0000","last_change":"2024-04-27T03:30:17.389821+0000","last_active":"2024-04-27T03:33:27.611519+0000","last_peered":"2024-04-27T03:33:27.611519+0000","last_clean":"2024-04-27T03:33:27.611519+0000","last_became_active":"2024-04-27T03:30:17.388246+0000","last_became_peered":"2024-04-27T03:30:17.388246+0000","last_unstale":"2024-04-27T03:33:27.611519+0000","last_undegraded":"2024-04-27T03:33:27.611519+0000","last_fullsized":"2024-04-27T03:33:27.611519+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:10:57.956151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437582+0000","last_change":"2024-04-27T03:30:19.407011+0000","last_active":"2024-04-27T03:31:01.437582+0000","last_peered":"2024-04-27T03:31:01.437582+0000","last_clean":"2024-04-27T03:31:01.437582+0000","last_became_active":"2024-04-27T03:30:19.406707+0000","last_became_peered":"2024-04-27T03:30:19.406707+0000","last_unstale":"2024-04-27T03:31:01.437582+0000","last_undegraded":"2024-04-27T03:31:01.437582+0000","last_fullsized":"2024-04-27T03:31:01.437582+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:59.990733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436390+0000","last_change":"2024-04-27T03:30:21.415821+0000","last_active":"2024-04-27T03:31:01.436390+0000","last_peered":"2024-04-27T03:31:01.436390+0000","last_clean":"2024-04-27T03:31:01.436390+0000","last_became_active":"2024-04-27T03:30:21.415441+0000","last_became_peered":"2024-04-27T03:30:21.415441+0000","last_unstale":"2024-04-27T03:31:01.436390+0000","last_undegraded":"2024-04-27T03:31:01.436390+0000","last_fullsized":"2024-04-27T03:31:01.436390+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:30:03.042667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933527+0000","last_change":"2024-04-27T03:30:13.388229+0000","last_active":"2024-04-27T03:33:20.933527+0000","last_peered":"2024-04-27T03:33:20.933527+0000","last_clean":"2024-04-27T03:33:20.933527+0000","last_became_active":"2024-04-27T03:30:13.387397+0000","last_became_peered":"2024-04-27T03:30:13.387397+0000","last_unstale":"2024-04-27T03:33:20.933527+0000","last_undegraded":"2024-04-27T03:33:20.933527+0000","last_fullsized":"2024-04-27T03:33:20.933527+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:45:47.437394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432931+0000","last_change":"2024-04-27T03:30:11.370706+0000","last_active":"2024-04-27T03:31:01.432931+0000","last_peered":"2024-04-27T03:31:01.432931+0000","last_clean":"2024-04-27T03:31:01.432931+0000","last_became_active":"2024-04-27T03:30:11.369799+0000","last_became_peered":"2024-04-27T03:30:11.369799+0000","last_unstale":"2024-04-27T03:31:01.432931+0000","last_undegraded":"2024-04-27T03:31:01.432931+0000","last_fullsized":"2024-04-27T03:31:01.432931+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:30:27.539212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433610+0000","last_change":"2024-04-27T03:30:15.380214+0000","last_active":"2024-04-27T03:31:01.433610+0000","last_peered":"2024-04-27T03:31:01.433610+0000","last_clean":"2024-04-27T03:31:01.433610+0000","last_became_active":"2024-04-27T03:30:15.379873+0000","last_became_peered":"2024-04-27T03:30:15.379873+0000","last_unstale":"2024-04-27T03:31:01.433610+0000","last_undegraded":"2024-04-27T03:31:01.433610+0000","last_fullsized":"2024-04-27T03:31:01.433610+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:53.446024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"27'43","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.663287+0000","last_change":"2024-04-27T03:30:17.387440+0000","last_active":"2024-04-27T03:33:27.663287+0000","last_peered":"2024-04-27T03:33:27.663287+0000","last_clean":"2024-04-27T03:33:27.663287+0000","last_became_active":"2024-04-27T03:30:17.387205+0000","last_became_peered":"2024-04-27T03:30:17.387205+0000","last_unstale":"2024-04-27T03:33:27.663287+0000","last_undegraded":"2024-04-27T03:33:27.663287+0000","last_fullsized":"2024-04-27T03:33:27.663287+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:27:26.079008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":63,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432503+0000","last_change":"2024-04-27T03:30:19.405759+0000","last_active":"2024-04-27T03:31:01.432503+0000","last_peered":"2024-04-27T03:31:01.432503+0000","last_clean":"2024-04-27T03:31:01.432503+0000","last_became_active":"2024-04-27T03:30:19.405462+0000","last_became_peered":"2024-04-27T03:30:19.405462+0000","last_unstale":"2024-04-27T03:31:01.432503+0000","last_undegraded":"2024-04-27T03:31:01.432503+0000","last_fullsized":"2024-04-27T03:31:01.432503+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:59.990733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"27'7","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.302334+0000","last_change":"2024-04-27T03:30:21.416894+0000","last_active":"2024-04-27T03:32:20.302334+0000","last_peered":"2024-04-27T03:32:20.302334+0000","last_clean":"2024-04-27T03:32:20.302334+0000","last_became_active":"2024-04-27T03:30:21.416592+0000","last_became_peered":"2024-04-27T03:30:21.416592+0000","last_unstale":"2024-04-27T03:32:20.302334+0000","last_undegraded":"2024-04-27T03:32:20.302334+0000","last_fullsized":"2024-04-27T03:32:20.302334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:24:21.730546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"27'6","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.304764+0000","last_change":"2024-04-27T03:30:13.370253+0000","last_active":"2024-04-27T03:32:20.304764+0000","last_peered":"2024-04-27T03:32:20.304764+0000","last_clean":"2024-04-27T03:32:20.304764+0000","last_became_active":"2024-04-27T03:30:13.369643+0000","last_became_peered":"2024-04-27T03:30:13.369643+0000","last_unstale":"2024-04-27T03:32:20.304764+0000","last_undegraded":"2024-04-27T03:32:20.304764+0000","last_fullsized":"2024-04-27T03:32:20.304764+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:28.256872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.631684+0000","last_change":"2024-04-27T03:30:11.372867+0000","last_active":"2024-04-27T03:33:20.631684+0000","last_peered":"2024-04-27T03:33:20.631684+0000","last_clean":"2024-04-27T03:33:20.631684+0000","last_became_active":"2024-04-27T03:30:11.372220+0000","last_became_peered":"2024-04-27T03:30:11.372220+0000","last_unstale":"2024-04-27T03:33:20.631684+0000","last_undegraded":"2024-04-27T03:33:20.631684+0000","last_fullsized":"2024-04-27T03:33:20.631684+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:43.207328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437382+0000","last_change":"2024-04-27T03:30:15.380858+0000","last_active":"2024-04-27T03:31:01.437382+0000","last_peered":"2024-04-27T03:31:01.437382+0000","last_clean":"2024-04-27T03:31:01.437382+0000","last_became_active":"2024-04-27T03:30:15.380573+0000","last_became_peered":"2024-04-27T03:30:15.380573+0000","last_unstale":"2024-04-27T03:31:01.437382+0000","last_undegraded":"2024-04-27T03:31:01.437382+0000","last_fullsized":"2024-04-27T03:31:01.437382+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:51:45.374972+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"25'16","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438994+0000","last_change":"2024-04-27T03:30:17.389436+0000","last_active":"2024-04-27T03:31:01.438994+0000","last_peered":"2024-04-27T03:31:01.438994+0000","last_clean":"2024-04-27T03:31:01.438994+0000","last_became_active":"2024-04-27T03:30:17.388289+0000","last_became_peered":"2024-04-27T03:30:17.388289+0000","last_unstale":"2024-04-27T03:31:01.438994+0000","last_undegraded":"2024-04-27T03:31:01.438994+0000","last_fullsized":"2024-04-27T03:31:01.438994+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:02:33.022377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"27'285","reported_seq":375,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.556281+0000","last_change":"2024-04-27T03:30:19.407449+0000","last_active":"2024-04-27T03:33:27.556281+0000","last_peered":"2024-04-27T03:33:27.556281+0000","last_clean":"2024-04-27T03:33:27.556281+0000","last_became_active":"2024-04-27T03:30:19.406854+0000","last_became_peered":"2024-04-27T03:30:19.406854+0000","last_unstale":"2024-04-27T03:33:27.556281+0000","last_undegraded":"2024-04-27T03:33:27.556281+0000","last_fullsized":"2024-04-27T03:33:27.556281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:32:55.112238+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435927+0000","last_change":"2024-04-27T03:30:21.415992+0000","last_active":"2024-04-27T03:31:01.435927+0000","last_peered":"2024-04-27T03:31:01.435927+0000","last_clean":"2024-04-27T03:31:01.435927+0000","last_became_active":"2024-04-27T03:30:21.415534+0000","last_became_peered":"2024-04-27T03:30:21.415534+0000","last_unstale":"2024-04-27T03:31:01.435927+0000","last_undegraded":"2024-04-27T03:31:01.435927+0000","last_fullsized":"2024-04-27T03:31:01.435927+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:17:16.401751+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"27'24","reported_seq":102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981535+0000","last_change":"2024-04-27T03:30:13.371535+0000","last_active":"2024-04-27T03:33:20.981535+0000","last_peered":"2024-04-27T03:33:20.981535+0000","last_clean":"2024-04-27T03:33:20.981535+0000","last_became_active":"2024-04-27T03:30:13.371158+0000","last_became_peered":"2024-04-27T03:30:13.371158+0000","last_unstale":"2024-04-27T03:33:20.981535+0000","last_undegraded":"2024-04-27T03:33:20.981535+0000","last_fullsized":"2024-04-27T03:33:20.981535+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:11:26.763810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":88,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435110+0000","last_change":"2024-04-27T03:30:11.359205+0000","last_active":"2024-04-27T03:31:01.435110+0000","last_peered":"2024-04-27T03:31:01.435110+0000","last_clean":"2024-04-27T03:31:01.435110+0000","last_became_active":"2024-04-27T03:30:11.358893+0000","last_became_peered":"2024-04-27T03:30:11.358893+0000","last_unstale":"2024-04-27T03:31:01.435110+0000","last_undegraded":"2024-04-27T03:31:01.435110+0000","last_fullsized":"2024-04-27T03:31:01.435110+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:52:31.047433+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436411+0000","last_change":"2024-04-27T03:30:15.381535+0000","last_active":"2024-04-27T03:31:01.436411+0000","last_peered":"2024-04-27T03:31:01.436411+0000","last_clean":"2024-04-27T03:31:01.436411+0000","last_became_active":"2024-04-27T03:30:15.381175+0000","last_became_peered":"2024-04-27T03:30:15.381175+0000","last_unstale":"2024-04-27T03:31:01.436411+0000","last_undegraded":"2024-04-27T03:31:01.436411+0000","last_fullsized":"2024-04-27T03:31:01.436411+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:21:29.464234+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"27'63","reported_seq":136,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.682686+0000","last_change":"2024-04-27T03:30:17.386967+0000","last_active":"2024-04-27T03:33:27.682686+0000","last_peered":"2024-04-27T03:33:27.682686+0000","last_clean":"2024-04-27T03:33:27.682686+0000","last_became_active":"2024-04-27T03:30:17.386673+0000","last_became_peered":"2024-04-27T03:30:17.386673+0000","last_unstale":"2024-04-27T03:33:27.682686+0000","last_undegraded":"2024-04-27T03:33:27.682686+0000","last_fullsized":"2024-04-27T03:33:27.682686+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":63,"log_dups_size":0,"ondisk_log_size":63,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:54:14.640233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":80,"num_read_kb":67,"num_write":44,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430575+0000","last_change":"2024-04-27T03:30:19.407289+0000","last_active":"2024-04-27T03:31:01.430575+0000","last_peered":"2024-04-27T03:31:01.430575+0000","last_clean":"2024-04-27T03:31:01.430575+0000","last_became_active":"2024-04-27T03:30:19.406990+0000","last_became_peered":"2024-04-27T03:30:19.406990+0000","last_unstale":"2024-04-27T03:31:01.430575+0000","last_undegraded":"2024-04-27T03:31:01.430575+0000","last_fullsized":"2024-04-27T03:31:01.430575+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:56:49.549144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433183+0000","last_change":"2024-04-27T03:30:21.410357+0000","last_active":"2024-04-27T03:31:01.433183+0000","last_peered":"2024-04-27T03:31:01.433183+0000","last_clean":"2024-04-27T03:31:01.433183+0000","last_became_active":"2024-04-27T03:30:21.410060+0000","last_became_peered":"2024-04-27T03:30:21.410060+0000","last_unstale":"2024-04-27T03:31:01.433183+0000","last_undegraded":"2024-04-27T03:31:01.433183+0000","last_fullsized":"2024-04-27T03:31:01.433183+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:08:40.977705+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"27'10","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009762+0000","last_change":"2024-04-27T03:30:13.372186+0000","last_active":"2024-04-27T03:33:21.009762+0000","last_peered":"2024-04-27T03:33:21.009762+0000","last_clean":"2024-04-27T03:33:21.009762+0000","last_became_active":"2024-04-27T03:30:13.371866+0000","last_became_peered":"2024-04-27T03:30:13.371866+0000","last_unstale":"2024-04-27T03:33:21.009762+0000","last_undegraded":"2024-04-27T03:33:21.009762+0000","last_fullsized":"2024-04-27T03:33:21.009762+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:14:45.213699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.929513+0000","last_change":"2024-04-27T03:30:11.362991+0000","last_active":"2024-04-27T03:33:20.929513+0000","last_peered":"2024-04-27T03:33:20.929513+0000","last_clean":"2024-04-27T03:33:20.929513+0000","last_became_active":"2024-04-27T03:30:11.362420+0000","last_became_peered":"2024-04-27T03:30:11.362420+0000","last_unstale":"2024-04-27T03:33:20.929513+0000","last_undegraded":"2024-04-27T03:33:20.929513+0000","last_fullsized":"2024-04-27T03:33:20.929513+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:16.411097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437111+0000","last_change":"2024-04-27T03:30:15.382620+0000","last_active":"2024-04-27T03:31:01.437111+0000","last_peered":"2024-04-27T03:31:01.437111+0000","last_clean":"2024-04-27T03:31:01.437111+0000","last_became_active":"2024-04-27T03:30:15.382305+0000","last_became_peered":"2024-04-27T03:30:15.382305+0000","last_unstale":"2024-04-27T03:31:01.437111+0000","last_undegraded":"2024-04-27T03:31:01.437111+0000","last_fullsized":"2024-04-27T03:31:01.437111+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:29:04.836869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"27'42","reported_seq":94,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.690352+0000","last_change":"2024-04-27T03:30:17.390602+0000","last_active":"2024-04-27T03:33:27.690352+0000","last_peered":"2024-04-27T03:33:27.690352+0000","last_clean":"2024-04-27T03:33:27.690352+0000","last_became_active":"2024-04-27T03:30:17.390281+0000","last_became_peered":"2024-04-27T03:30:17.390281+0000","last_unstale":"2024-04-27T03:33:27.690352+0000","last_undegraded":"2024-04-27T03:33:27.690352+0000","last_fullsized":"2024-04-27T03:33:27.690352+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:24:20.307428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":50,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.437067+0000","last_change":"2024-04-27T03:30:19.406218+0000","last_active":"2024-04-27T03:31:01.437067+0000","last_peered":"2024-04-27T03:31:01.437067+0000","last_clean":"2024-04-27T03:31:01.437067+0000","last_became_active":"2024-04-27T03:30:19.405353+0000","last_became_peered":"2024-04-27T03:30:19.405353+0000","last_unstale":"2024-04-27T03:31:01.437067+0000","last_undegraded":"2024-04-27T03:31:01.437067+0000","last_fullsized":"2024-04-27T03:31:01.437067+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:21.451111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":17,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.438395+0000","last_change":"2024-04-27T03:30:21.416876+0000","last_active":"2024-04-27T03:31:01.438395+0000","last_peered":"2024-04-27T03:31:01.438395+0000","last_clean":"2024-04-27T03:31:01.438395+0000","last_became_active":"2024-04-27T03:30:21.416276+0000","last_became_peered":"2024-04-27T03:30:21.416276+0000","last_unstale":"2024-04-27T03:31:01.438395+0000","last_undegraded":"2024-04-27T03:31:01.438395+0000","last_fullsized":"2024-04-27T03:31:01.438395+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:36:58.245817+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'24","reported_seq":101,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.697833+0000","last_change":"2024-04-27T03:30:13.377820+0000","last_active":"2024-04-27T03:33:20.697833+0000","last_peered":"2024-04-27T03:33:20.697833+0000","last_clean":"2024-04-27T03:33:20.697833+0000","last_became_active":"2024-04-27T03:30:13.376415+0000","last_became_peered":"2024-04-27T03:30:13.376415+0000","last_unstale":"2024-04-27T03:33:20.697833+0000","last_undegraded":"2024-04-27T03:33:20.697833+0000","last_fullsized":"2024-04-27T03:33:20.697833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:02.028024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.629553+0000","last_change":"2024-04-27T03:30:11.361140+0000","last_active":"2024-04-27T03:33:20.629553+0000","last_peered":"2024-04-27T03:33:20.629553+0000","last_clean":"2024-04-27T03:33:20.629553+0000","last_became_active":"2024-04-27T03:30:11.360590+0000","last_became_peered":"2024-04-27T03:30:11.360590+0000","last_unstale":"2024-04-27T03:33:20.629553+0000","last_undegraded":"2024-04-27T03:33:20.629553+0000","last_fullsized":"2024-04-27T03:33:20.629553+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:52:58.316121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.434401+0000","last_change":"2024-04-27T03:30:15.385912+0000","last_active":"2024-04-27T03:31:01.434401+0000","last_peered":"2024-04-27T03:31:01.434401+0000","last_clean":"2024-04-27T03:31:01.434401+0000","last_became_active":"2024-04-27T03:30:15.385614+0000","last_became_peered":"2024-04-27T03:30:15.385614+0000","last_unstale":"2024-04-27T03:31:01.434401+0000","last_undegraded":"2024-04-27T03:31:01.434401+0000","last_fullsized":"2024-04-27T03:31:01.434401+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:54:32.462460+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"27'35","reported_seq":85,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.657008+0000","last_change":"2024-04-27T03:30:17.387854+0000","last_active":"2024-04-27T03:33:27.657008+0000","last_peered":"2024-04-27T03:33:27.657008+0000","last_clean":"2024-04-27T03:33:27.657008+0000","last_became_active":"2024-04-27T03:30:17.386983+0000","last_became_peered":"2024-04-27T03:30:17.386983+0000","last_unstale":"2024-04-27T03:33:27.657008+0000","last_undegraded":"2024-04-27T03:33:27.657008+0000","last_fullsized":"2024-04-27T03:33:27.657008+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:38.038360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":33,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436657+0000","last_change":"2024-04-27T03:30:19.402931+0000","last_active":"2024-04-27T03:31:01.436657+0000","last_peered":"2024-04-27T03:31:01.436657+0000","last_clean":"2024-04-27T03:31:01.436657+0000","last_became_active":"2024-04-27T03:30:19.402652+0000","last_became_peered":"2024-04-27T03:30:19.402652+0000","last_unstale":"2024-04-27T03:31:01.436657+0000","last_undegraded":"2024-04-27T03:31:01.436657+0000","last_fullsized":"2024-04-27T03:31:01.436657+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:24.199405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"27'6","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.720254+0000","last_change":"2024-04-27T03:30:21.420988+0000","last_active":"2024-04-27T03:33:20.720254+0000","last_peered":"2024-04-27T03:33:20.720254+0000","last_clean":"2024-04-27T03:33:20.720254+0000","last_became_active":"2024-04-27T03:30:21.420725+0000","last_became_peered":"2024-04-27T03:30:21.420725+0000","last_unstale":"2024-04-27T03:33:20.720254+0000","last_undegraded":"2024-04-27T03:33:20.720254+0000","last_fullsized":"2024-04-27T03:33:20.720254+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:03:02.117649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432186+0000","last_change":"2024-04-27T03:30:13.375384+0000","last_active":"2024-04-27T03:31:01.432186+0000","last_peered":"2024-04-27T03:31:01.432186+0000","last_clean":"2024-04-27T03:31:01.432186+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:31:01.432186+0000","last_undegraded":"2024-04-27T03:31:01.432186+0000","last_fullsized":"2024-04-27T03:31:01.432186+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:40:10.176600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436421+0000","last_change":"2024-04-27T03:30:11.370518+0000","last_active":"2024-04-27T03:31:01.436421+0000","last_peered":"2024-04-27T03:31:01.436421+0000","last_clean":"2024-04-27T03:31:01.436421+0000","last_became_active":"2024-04-27T03:30:11.369877+0000","last_became_peered":"2024-04-27T03:30:11.369877+0000","last_unstale":"2024-04-27T03:31:01.436421+0000","last_undegraded":"2024-04-27T03:31:01.436421+0000","last_fullsized":"2024-04-27T03:31:01.436421+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.059046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"27'2","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.550532+0000","last_change":"2024-04-27T03:30:15.382111+0000","last_active":"2024-04-27T03:32:17.550532+0000","last_peered":"2024-04-27T03:32:17.550532+0000","last_clean":"2024-04-27T03:32:17.550532+0000","last_became_active":"2024-04-27T03:30:15.381574+0000","last_became_peered":"2024-04-27T03:30:15.381574+0000","last_unstale":"2024-04-27T03:32:17.550532+0000","last_undegraded":"2024-04-27T03:32:17.550532+0000","last_fullsized":"2024-04-27T03:32:17.550532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:57:04.039451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"27'47","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.666168+0000","last_change":"2024-04-27T03:30:17.391410+0000","last_active":"2024-04-27T03:33:27.666168+0000","last_peered":"2024-04-27T03:33:27.666168+0000","last_clean":"2024-04-27T03:33:27.666168+0000","last_became_active":"2024-04-27T03:30:17.388277+0000","last_became_peered":"2024-04-27T03:30:17.388277+0000","last_unstale":"2024-04-27T03:33:27.666168+0000","last_undegraded":"2024-04-27T03:33:27.666168+0000","last_fullsized":"2024-04-27T03:33:27.666168+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:20:05.729566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":52,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":21,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431300+0000","last_change":"2024-04-27T03:30:19.406905+0000","last_active":"2024-04-27T03:31:01.431300+0000","last_peered":"2024-04-27T03:31:01.431300+0000","last_clean":"2024-04-27T03:31:01.431300+0000","last_became_active":"2024-04-27T03:30:19.406565+0000","last_became_peered":"2024-04-27T03:30:19.406565+0000","last_unstale":"2024-04-27T03:31:01.431300+0000","last_undegraded":"2024-04-27T03:31:01.431300+0000","last_fullsized":"2024-04-27T03:31:01.431300+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:31.846278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"27'4","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:18.934155+0000","last_change":"2024-04-27T03:30:21.414808+0000","last_active":"2024-04-27T03:32:18.934155+0000","last_peered":"2024-04-27T03:32:18.934155+0000","last_clean":"2024-04-27T03:32:18.934155+0000","last_became_active":"2024-04-27T03:30:21.414510+0000","last_became_peered":"2024-04-27T03:30:21.414510+0000","last_unstale":"2024-04-27T03:32:18.934155+0000","last_undegraded":"2024-04-27T03:32:18.934155+0000","last_fullsized":"2024-04-27T03:32:18.934155+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:07.942157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"25'8","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436998+0000","last_change":"2024-04-27T03:30:13.387657+0000","last_active":"2024-04-27T03:31:01.436998+0000","last_peered":"2024-04-27T03:31:01.436998+0000","last_clean":"2024-04-27T03:31:01.436998+0000","last_became_active":"2024-04-27T03:30:13.387149+0000","last_became_peered":"2024-04-27T03:30:13.387149+0000","last_unstale":"2024-04-27T03:31:01.436998+0000","last_undegraded":"2024-04-27T03:31:01.436998+0000","last_fullsized":"2024-04-27T03:31:01.436998+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:04:00.325582+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"25'6","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.433385+0000","last_change":"2024-04-27T03:30:11.358174+0000","last_active":"2024-04-27T03:31:01.433385+0000","last_peered":"2024-04-27T03:31:01.433385+0000","last_clean":"2024-04-27T03:31:01.433385+0000","last_became_active":"2024-04-27T03:30:11.357878+0000","last_became_peered":"2024-04-27T03:30:11.357878+0000","last_unstale":"2024-04-27T03:31:01.433385+0000","last_undegraded":"2024-04-27T03:31:01.433385+0000","last_fullsized":"2024-04-27T03:31:01.433385+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:17.981757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.304915+0000","last_change":"2024-04-27T03:30:13.369979+0000","last_active":"2024-04-27T03:32:20.304915+0000","last_peered":"2024-04-27T03:32:20.304915+0000","last_clean":"2024-04-27T03:32:20.304915+0000","last_became_active":"2024-04-27T03:30:13.369681+0000","last_became_peered":"2024-04-27T03:30:13.369681+0000","last_unstale":"2024-04-27T03:32:20.304915+0000","last_undegraded":"2024-04-27T03:32:20.304915+0000","last_fullsized":"2024-04-27T03:32:20.304915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:37.770898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"25'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430839+0000","last_change":"2024-04-27T03:30:11.373422+0000","last_active":"2024-04-27T03:31:01.430839+0000","last_peered":"2024-04-27T03:31:01.430839+0000","last_clean":"2024-04-27T03:31:01.430839+0000","last_became_active":"2024-04-27T03:30:11.373186+0000","last_became_peered":"2024-04-27T03:30:11.373186+0000","last_unstale":"2024-04-27T03:31:01.430839+0000","last_undegraded":"2024-04-27T03:31:01.430839+0000","last_fullsized":"2024-04-27T03:31:01.430839+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:22.719424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"27'2","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933550+0000","last_change":"2024-04-27T03:30:13.382330+0000","last_active":"2024-04-27T03:33:20.933550+0000","last_peered":"2024-04-27T03:33:20.933550+0000","last_clean":"2024-04-27T03:33:20.933550+0000","last_became_active":"2024-04-27T03:30:13.382069+0000","last_became_peered":"2024-04-27T03:30:13.382069+0000","last_unstale":"2024-04-27T03:33:20.933550+0000","last_undegraded":"2024-04-27T03:33:20.933550+0000","last_fullsized":"2024-04-27T03:33:20.933550+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:45:22.941552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"27'3","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:36.420503+0000","last_change":"2024-04-27T03:30:11.350547+0000","last_active":"2024-04-27T03:31:36.420503+0000","last_peered":"2024-04-27T03:31:36.420503+0000","last_clean":"2024-04-27T03:31:36.420503+0000","last_became_active":"2024-04-27T03:30:11.350295+0000","last_became_peered":"2024-04-27T03:30:11.350295+0000","last_unstale":"2024-04-27T03:31:36.420503+0000","last_undegraded":"2024-04-27T03:31:36.420503+0000","last_fullsized":"2024-04-27T03:31:36.420503+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:45:58.123163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.933497+0000","last_change":"2024-04-27T03:30:13.370393+0000","last_active":"2024-04-27T03:33:20.933497+0000","last_peered":"2024-04-27T03:33:20.933497+0000","last_clean":"2024-04-27T03:33:20.933497+0000","last_became_active":"2024-04-27T03:30:13.370066+0000","last_became_peered":"2024-04-27T03:30:13.370066+0000","last_unstale":"2024-04-27T03:33:20.933497+0000","last_undegraded":"2024-04-27T03:33:20.933497+0000","last_fullsized":"2024-04-27T03:33:20.933497+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:33.811635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.585708+0000","last_change":"2024-04-27T03:30:11.360775+0000","last_active":"2024-04-27T03:33:20.585708+0000","last_peered":"2024-04-27T03:33:20.585708+0000","last_clean":"2024-04-27T03:33:20.585708+0000","last_became_active":"2024-04-27T03:30:11.360376+0000","last_became_peered":"2024-04-27T03:30:11.360376+0000","last_unstale":"2024-04-27T03:33:20.585708+0000","last_undegraded":"2024-04-27T03:33:20.585708+0000","last_fullsized":"2024-04-27T03:33:20.585708+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:00.825717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.792398+0000","last_change":"2024-04-27T03:30:13.378662+0000","last_active":"2024-04-27T03:33:20.792398+0000","last_peered":"2024-04-27T03:33:20.792398+0000","last_clean":"2024-04-27T03:33:20.792398+0000","last_became_active":"2024-04-27T03:30:13.378368+0000","last_became_peered":"2024-04-27T03:30:13.378368+0000","last_unstale":"2024-04-27T03:33:20.792398+0000","last_undegraded":"2024-04-27T03:33:20.792398+0000","last_fullsized":"2024-04-27T03:33:20.792398+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:56.906775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432139+0000","last_change":"2024-04-27T03:30:11.371839+0000","last_active":"2024-04-27T03:31:01.432139+0000","last_peered":"2024-04-27T03:31:01.432139+0000","last_clean":"2024-04-27T03:31:01.432139+0000","last_became_active":"2024-04-27T03:30:11.371553+0000","last_became_peered":"2024-04-27T03:30:11.371553+0000","last_unstale":"2024-04-27T03:31:01.432139+0000","last_undegraded":"2024-04-27T03:31:01.432139+0000","last_fullsized":"2024-04-27T03:31:01.432139+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:02:21.059046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"27'10","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.009740+0000","last_change":"2024-04-27T03:31:01.440910+0000","last_active":"2024-04-27T03:33:21.009740+0000","last_peered":"2024-04-27T03:33:21.009740+0000","last_clean":"2024-04-27T03:33:21.009740+0000","last_became_active":"2024-04-27T03:31:01.440576+0000","last_became_peered":"2024-04-27T03:31:01.440576+0000","last_unstale":"2024-04-27T03:33:21.009740+0000","last_undegraded":"2024-04-27T03:33:21.009740+0000","last_fullsized":"2024-04-27T03:33:21.009740+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:04.874525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.213761+0000","last_change":"2024-04-27T03:30:11.371034+0000","last_active":"2024-04-27T03:32:20.213761+0000","last_peered":"2024-04-27T03:32:20.213761+0000","last_clean":"2024-04-27T03:32:20.213761+0000","last_became_active":"2024-04-27T03:30:11.369517+0000","last_became_peered":"2024-04-27T03:30:11.369517+0000","last_unstale":"2024-04-27T03:32:20.213761+0000","last_undegraded":"2024-04-27T03:32:20.213761+0000","last_fullsized":"2024-04-27T03:32:20.213761+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:22.719424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.107522+0000","last_change":"2024-04-27T03:30:13.374070+0000","last_active":"2024-04-27T03:32:17.107522+0000","last_peered":"2024-04-27T03:32:17.107522+0000","last_clean":"2024-04-27T03:32:17.107522+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:32:17.107522+0000","last_undegraded":"2024-04-27T03:32:17.107522+0000","last_fullsized":"2024-04-27T03:32:17.107522+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:32:54.324828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.442163+0000","last_change":"2024-04-27T03:31:01.441506+0000","last_active":"2024-04-27T03:31:01.442163+0000","last_peered":"2024-04-27T03:31:01.442163+0000","last_clean":"2024-04-27T03:31:01.442163+0000","last_became_active":"2024-04-27T03:31:01.441159+0000","last_became_peered":"2024-04-27T03:31:01.441159+0000","last_unstale":"2024-04-27T03:31:01.442163+0000","last_undegraded":"2024-04-27T03:31:01.442163+0000","last_fullsized":"2024-04-27T03:31:01.442163+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:11:09.529188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"27'3","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.802333+0000","last_change":"2024-04-27T03:30:13.378083+0000","last_active":"2024-04-27T03:33:20.802333+0000","last_peered":"2024-04-27T03:33:20.802333+0000","last_clean":"2024-04-27T03:33:20.802333+0000","last_became_active":"2024-04-27T03:30:13.377821+0000","last_became_peered":"2024-04-27T03:30:13.377821+0000","last_unstale":"2024-04-27T03:33:20.802333+0000","last_undegraded":"2024-04-27T03:33:20.802333+0000","last_fullsized":"2024-04-27T03:33:20.802333+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:18.417469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432873+0000","last_change":"2024-04-27T03:30:11.371906+0000","last_active":"2024-04-27T03:31:01.432873+0000","last_peered":"2024-04-27T03:31:01.432873+0000","last_clean":"2024-04-27T03:31:01.432873+0000","last_became_active":"2024-04-27T03:30:11.371604+0000","last_became_peered":"2024-04-27T03:30:11.371604+0000","last_unstale":"2024-04-27T03:31:01.432873+0000","last_undegraded":"2024-04-27T03:31:01.432873+0000","last_fullsized":"2024-04-27T03:31:01.432873+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:33:05.923239+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"27'8","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981663+0000","last_change":"2024-04-27T03:30:13.388711+0000","last_active":"2024-04-27T03:33:20.981663+0000","last_peered":"2024-04-27T03:33:20.981663+0000","last_clean":"2024-04-27T03:33:20.981663+0000","last_became_active":"2024-04-27T03:30:13.388421+0000","last_became_peered":"2024-04-27T03:30:13.388421+0000","last_unstale":"2024-04-27T03:33:20.981663+0000","last_undegraded":"2024-04-27T03:33:20.981663+0000","last_fullsized":"2024-04-27T03:33:20.981663+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:38:35.526991+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.678119+0000","last_change":"2024-04-27T03:30:11.355453+0000","last_active":"2024-04-27T03:33:20.678119+0000","last_peered":"2024-04-27T03:33:20.678119+0000","last_clean":"2024-04-27T03:33:20.678119+0000","last_became_active":"2024-04-27T03:30:11.355183+0000","last_became_peered":"2024-04-27T03:30:11.355183+0000","last_unstale":"2024-04-27T03:33:20.678119+0000","last_undegraded":"2024-04-27T03:33:20.678119+0000","last_fullsized":"2024-04-27T03:33:20.678119+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:12.667203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"27'4","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.107011+0000","last_change":"2024-04-27T03:30:13.377155+0000","last_active":"2024-04-27T03:32:17.107011+0000","last_peered":"2024-04-27T03:32:17.107011+0000","last_clean":"2024-04-27T03:32:17.107011+0000","last_became_active":"2024-04-27T03:30:13.376875+0000","last_became_peered":"2024-04-27T03:30:13.376875+0000","last_unstale":"2024-04-27T03:32:17.107011+0000","last_undegraded":"2024-04-27T03:32:17.107011+0000","last_fullsized":"2024-04-27T03:32:17.107011+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:54:09.993585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.431158+0000","last_change":"2024-04-27T03:30:11.361373+0000","last_active":"2024-04-27T03:31:01.431158+0000","last_peered":"2024-04-27T03:31:01.431158+0000","last_clean":"2024-04-27T03:31:01.431158+0000","last_became_active":"2024-04-27T03:30:11.360299+0000","last_became_peered":"2024-04-27T03:30:11.360299+0000","last_unstale":"2024-04-27T03:31:01.431158+0000","last_undegraded":"2024-04-27T03:31:01.431158+0000","last_fullsized":"2024-04-27T03:31:01.431158+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:37.746108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"27'10","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.981744+0000","last_change":"2024-04-27T03:30:13.376269+0000","last_active":"2024-04-27T03:33:20.981744+0000","last_peered":"2024-04-27T03:33:20.981744+0000","last_clean":"2024-04-27T03:33:20.981744+0000","last_became_active":"2024-04-27T03:30:13.376087+0000","last_became_peered":"2024-04-27T03:30:13.376087+0000","last_unstale":"2024-04-27T03:33:20.981744+0000","last_undegraded":"2024-04-27T03:33:20.981744+0000","last_fullsized":"2024-04-27T03:33:20.981744+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:18.301070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.432834+0000","last_change":"2024-04-27T03:30:11.371237+0000","last_active":"2024-04-27T03:31:01.432834+0000","last_peered":"2024-04-27T03:31:01.432834+0000","last_clean":"2024-04-27T03:31:01.432834+0000","last_became_active":"2024-04-27T03:30:11.370837+0000","last_became_peered":"2024-04-27T03:30:11.370837+0000","last_unstale":"2024-04-27T03:31:01.432834+0000","last_undegraded":"2024-04-27T03:31:01.432834+0000","last_fullsized":"2024-04-27T03:31:01.432834+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:08:30.863119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"27'6","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.697535+0000","last_change":"2024-04-27T03:30:13.376130+0000","last_active":"2024-04-27T03:33:20.697535+0000","last_peered":"2024-04-27T03:33:20.697535+0000","last_clean":"2024-04-27T03:33:20.697535+0000","last_became_active":"2024-04-27T03:30:13.373511+0000","last_became_peered":"2024-04-27T03:30:13.373511+0000","last_unstale":"2024-04-27T03:33:20.697535+0000","last_undegraded":"2024-04-27T03:33:20.697535+0000","last_fullsized":"2024-04-27T03:33:20.697535+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:08:51.434596+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.257092+0000","last_change":"2024-04-27T03:30:11.370821+0000","last_active":"2024-04-27T03:32:20.257092+0000","last_peered":"2024-04-27T03:32:20.257092+0000","last_clean":"2024-04-27T03:32:20.257092+0000","last_became_active":"2024-04-27T03:30:11.370283+0000","last_became_peered":"2024-04-27T03:30:11.370283+0000","last_unstale":"2024-04-27T03:32:20.257092+0000","last_undegraded":"2024-04-27T03:32:20.257092+0000","last_fullsized":"2024-04-27T03:32:20.257092+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:17.981757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"27'11","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.016376+0000","last_change":"2024-04-27T03:30:13.380862+0000","last_active":"2024-04-27T03:33:21.016376+0000","last_peered":"2024-04-27T03:33:21.016376+0000","last_clean":"2024-04-27T03:33:21.016376+0000","last_became_active":"2024-04-27T03:30:13.380404+0000","last_became_peered":"2024-04-27T03:30:13.380404+0000","last_unstale":"2024-04-27T03:33:21.016376+0000","last_undegraded":"2024-04-27T03:33:21.016376+0000","last_fullsized":"2024-04-27T03:33:21.016376+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:14.849083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.439493+0000","last_change":"2024-04-27T03:30:11.373282+0000","last_active":"2024-04-27T03:31:01.439493+0000","last_peered":"2024-04-27T03:31:01.439493+0000","last_clean":"2024-04-27T03:31:01.439493+0000","last_became_active":"2024-04-27T03:30:11.373026+0000","last_became_peered":"2024-04-27T03:30:11.373026+0000","last_unstale":"2024-04-27T03:31:01.439493+0000","last_undegraded":"2024-04-27T03:31:01.439493+0000","last_fullsized":"2024-04-27T03:31:01.439493+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:40:08.170228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"27'1","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295321+0000","last_change":"2024-04-27T03:30:13.386073+0000","last_active":"2024-04-27T03:32:29.295321+0000","last_peered":"2024-04-27T03:32:29.295321+0000","last_clean":"2024-04-27T03:32:29.295321+0000","last_became_active":"2024-04-27T03:30:13.385791+0000","last_became_peered":"2024-04-27T03:30:13.385791+0000","last_unstale":"2024-04-27T03:32:29.295321+0000","last_undegraded":"2024-04-27T03:32:29.295321+0000","last_fullsized":"2024-04-27T03:32:29.295321+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:35:18.301070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.441318+0000","last_change":"2024-04-27T03:30:11.358239+0000","last_active":"2024-04-27T03:31:01.441318+0000","last_peered":"2024-04-27T03:31:01.441318+0000","last_clean":"2024-04-27T03:31:01.441318+0000","last_became_active":"2024-04-27T03:30:11.357898+0000","last_became_peered":"2024-04-27T03:30:11.357898+0000","last_unstale":"2024-04-27T03:31:01.441318+0000","last_undegraded":"2024-04-27T03:31:01.441318+0000","last_fullsized":"2024-04-27T03:31:01.441318+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:00:26.250500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"27'5","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:19.303737+0000","last_change":"2024-04-27T03:31:01.444873+0000","last_active":"2024-04-27T03:33:19.303737+0000","last_peered":"2024-04-27T03:33:19.303737+0000","last_clean":"2024-04-27T03:33:19.303737+0000","last_became_active":"2024-04-27T03:31:01.440820+0000","last_became_peered":"2024-04-27T03:31:01.440820+0000","last_unstale":"2024-04-27T03:33:19.303737+0000","last_undegraded":"2024-04-27T03:33:19.303737+0000","last_fullsized":"2024-04-27T03:33:19.303737+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:52:15.446814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.435323+0000","last_change":"2024-04-27T03:30:11.363676+0000","last_active":"2024-04-27T03:31:01.435323+0000","last_peered":"2024-04-27T03:31:01.435323+0000","last_clean":"2024-04-27T03:31:01.435323+0000","last_became_active":"2024-04-27T03:30:11.362788+0000","last_became_peered":"2024-04-27T03:30:11.362788+0000","last_unstale":"2024-04-27T03:31:01.435323+0000","last_undegraded":"2024-04-27T03:31:01.435323+0000","last_fullsized":"2024-04-27T03:31:01.435323+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:00.825717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"25'2","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436660+0000","last_change":"2024-04-27T03:30:13.378124+0000","last_active":"2024-04-27T03:31:01.436660+0000","last_peered":"2024-04-27T03:31:01.436660+0000","last_clean":"2024-04-27T03:31:01.436660+0000","last_became_active":"2024-04-27T03:30:13.377845+0000","last_became_peered":"2024-04-27T03:30:13.377845+0000","last_unstale":"2024-04-27T03:31:01.436660+0000","last_undegraded":"2024-04-27T03:31:01.436660+0000","last_fullsized":"2024-04-27T03:31:01.436660+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:54:37.770898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.639919+0000","last_change":"2024-04-27T03:30:11.362917+0000","last_active":"2024-04-27T03:33:20.639919+0000","last_peered":"2024-04-27T03:33:20.639919+0000","last_clean":"2024-04-27T03:33:20.639919+0000","last_became_active":"2024-04-27T03:30:11.362559+0000","last_became_peered":"2024-04-27T03:30:11.362559+0000","last_unstale":"2024-04-27T03:33:20.639919+0000","last_undegraded":"2024-04-27T03:33:20.639919+0000","last_fullsized":"2024-04-27T03:33:20.639919+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:12:55.587028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.436858+0000","last_change":"2024-04-27T03:30:11.363675+0000","last_active":"2024-04-27T03:31:01.436858+0000","last_peered":"2024-04-27T03:31:01.436858+0000","last_clean":"2024-04-27T03:31:01.436858+0000","last_became_active":"2024-04-27T03:30:11.363340+0000","last_became_peered":"2024-04-27T03:30:11.363340+0000","last_unstale":"2024-04-27T03:31:01.436858+0000","last_undegraded":"2024-04-27T03:31:01.436858+0000","last_fullsized":"2024-04-27T03:31:01.436858+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:26.857275+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"27'4","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:17.107465+0000","last_change":"2024-04-27T03:30:13.386029+0000","last_active":"2024-04-27T03:32:17.107465+0000","last_peered":"2024-04-27T03:32:17.107465+0000","last_clean":"2024-04-27T03:32:17.107465+0000","last_became_active":"2024-04-27T03:30:13.385705+0000","last_became_peered":"2024-04-27T03:30:13.385705+0000","last_unstale":"2024-04-27T03:32:17.107465+0000","last_undegraded":"2024-04-27T03:32:17.107465+0000","last_fullsized":"2024-04-27T03:32:17.107465+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:40:18.417469+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:31:01.430983+0000","last_change":"2024-04-27T03:30:11.360980+0000","last_active":"2024-04-27T03:31:01.430983+0000","last_peered":"2024-04-27T03:31:01.430983+0000","last_clean":"2024-04-27T03:31:01.430983+0000","last_became_active":"2024-04-27T03:30:11.360667+0000","last_became_peered":"2024-04-27T03:30:11.360667+0000","last_unstale":"2024-04-27T03:31:01.430983+0000","last_undegraded":"2024-04-27T03:31:01.430983+0000","last_fullsized":"2024-04-27T03:31:01.430983+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:56:51.203886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"27'14","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:20.305196+0000","last_change":"2024-04-27T03:30:13.380806+0000","last_active":"2024-04-27T03:32:20.305196+0000","last_peered":"2024-04-27T03:32:20.305196+0000","last_clean":"2024-04-27T03:32:20.305196+0000","last_became_active":"2024-04-27T03:30:13.380347+0000","last_became_peered":"2024-04-27T03:30:13.380347+0000","last_unstale":"2024-04-27T03:32:20.305196+0000","last_undegraded":"2024-04-27T03:32:20.305196+0000","last_fullsized":"2024-04-27T03:32:20.305196+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:26:48.533060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2280,"ondisk_log_size":2280,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":1888,"num_read_kb":1653,"num_write":789,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1569,"ondisk_log_size":1569,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":12,"ondisk_log_size":12,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":82,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705707,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35744,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94336096,"statfs":{"total":96636764160,"available":96600162304,"internally_reserved":0,"allocated":417792,"data_stored":120948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":36174279},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705708,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36256,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335584,"statfs":{"total":96636764160,"available":96599638016,"internally_reserved":0,"allocated":942080,"data_stored":708075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705707,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335556,"statfs":{"total":96636764160,"available":96599609344,"internally_reserved":0,"allocated":970752,"data_stored":708832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705709,"num_pgs":134,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39872,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331968,"statfs":{"total":96636764160,"available":96595935232,"internally_reserved":0,"allocated":380928,"data_stored":119531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T03:33:32.086 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.086 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.086 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.086 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.087 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.087 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.087 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.087 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.087 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.087 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.087 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.087 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.087 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.087 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.088 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.088 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.088 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.088 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.088 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.088 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.088 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.088 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.088 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.088 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.089 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.089 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.089 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.089 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.089 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.089 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.089 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.089 INFO:tasks.ceph:pgid 5.15 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.089 INFO:tasks.ceph:pgid 6.16 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.090 INFO:tasks.ceph:pgid 7.17 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.090 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.090 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.090 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.090 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.090 INFO:tasks.ceph:pgid 6.17 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.090 INFO:tasks.ceph:pgid 7.16 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.090 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.090 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.091 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.091 INFO:tasks.ceph:pgid 5.17 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.091 INFO:tasks.ceph:pgid 6.14 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.091 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.091 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.091 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.091 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.091 INFO:tasks.ceph:pgid 5.16 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.091 INFO:tasks.ceph:pgid 6.15 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.091 INFO:tasks.ceph:pgid 7.14 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.092 INFO:tasks.ceph:pgid 4.8 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.092 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.092 INFO:tasks.ceph:pgid 5.9 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.092 INFO:tasks.ceph:pgid 6.a last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.092 INFO:tasks.ceph:pgid 7.b last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.092 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.092 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.092 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.092 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.092 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.093 INFO:tasks.ceph:pgid 7.a last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.093 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.093 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.093 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.093 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.093 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.093 INFO:tasks.ceph:pgid 7.9 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.093 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.093 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.093 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.094 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.094 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.094 INFO:tasks.ceph:pgid 7.8 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.094 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.094 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.094 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.094 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.094 INFO:tasks.ceph:pgid 6.e last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.094 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.095 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.095 INFO:tasks.ceph:pgid 4.d last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.095 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.095 INFO:tasks.ceph:pgid 5.c last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.095 INFO:tasks.ceph:pgid 6.f last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.095 INFO:tasks.ceph:pgid 7.e last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.095 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.095 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.095 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.095 INFO:tasks.ceph:pgid 5.f last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.096 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.096 INFO:tasks.ceph:pgid 7.d last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.096 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.096 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.096 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.096 INFO:tasks.ceph:pgid 5.e last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.096 INFO:tasks.ceph:pgid 6.d last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.096 INFO:tasks.ceph:pgid 7.c last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.096 INFO:tasks.ceph:pgid 8.3 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.097 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.097 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.097 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.097 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.097 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.097 INFO:tasks.ceph:pgid 6.7 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.097 INFO:tasks.ceph:pgid 7.6 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.097 INFO:tasks.ceph:pgid 8.9 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.097 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.097 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.098 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.098 INFO:tasks.ceph:pgid 5.7 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.098 INFO:tasks.ceph:pgid 6.4 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.098 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.098 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.098 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.098 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.098 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.098 INFO:tasks.ceph:pgid 5.6 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.098 INFO:tasks.ceph:pgid 6.5 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.098 INFO:tasks.ceph:pgid 7.4 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.099 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.099 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.099 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.099 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.099 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.099 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.099 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.099 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.100 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.100 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.100 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.100 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.100 INFO:tasks.ceph:pgid 6.0 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.100 INFO:tasks.ceph:pgid 7.1 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.100 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.100 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.100 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.100 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.101 INFO:tasks.ceph:pgid 5.2 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.101 INFO:tasks.ceph:pgid 6.1 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.101 INFO:tasks.ceph:pgid 7.0 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.101 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.101 INFO:tasks.ceph:pgid 4.0 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.101 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.101 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.101 INFO:tasks.ceph:pgid 5.1 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.101 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.102 INFO:tasks.ceph:pgid 7.3 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.102 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.102 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.102 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.102 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.102 INFO:tasks.ceph:pgid 5.0 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.102 INFO:tasks.ceph:pgid 6.3 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.102 INFO:tasks.ceph:pgid 7.2 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.102 INFO:tasks.ceph:pgid 8.d last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.103 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.103 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.103 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.103 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.103 INFO:tasks.ceph:pgid 6.11 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.103 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.103 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.103 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.103 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.104 INFO:tasks.ceph:pgid 5.13 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.104 INFO:tasks.ceph:pgid 6.10 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.104 INFO:tasks.ceph:pgid 7.11 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.104 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.104 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.104 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.104 INFO:tasks.ceph:pgid 5.10 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.104 INFO:tasks.ceph:pgid 6.13 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.104 INFO:tasks.ceph:pgid 7.12 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.104 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.104 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.105 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.105 INFO:tasks.ceph:pgid 5.11 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.105 INFO:tasks.ceph:pgid 6.12 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.105 INFO:tasks.ceph:pgid 7.13 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.105 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.105 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.105 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.105 INFO:tasks.ceph:pgid 6.1d last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.105 INFO:tasks.ceph:pgid 7.1c last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.106 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.106 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.106 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.106 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.106 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.106 INFO:tasks.ceph:pgid 7.1d last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.107 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.107 INFO:tasks.ceph:pgid 4.1d last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.107 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.107 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.107 INFO:tasks.ceph:pgid 6.1f last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.107 INFO:tasks.ceph:pgid 7.1e last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.107 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.107 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.107 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.107 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.108 INFO:tasks.ceph:pgid 6.1e last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.108 INFO:tasks.ceph:pgid 7.1f last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.108 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.108 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.108 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.108 INFO:tasks.ceph:pgid 5.1a last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.108 INFO:tasks.ceph:pgid 6.19 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.108 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.108 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.109 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.109 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.109 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.109 INFO:tasks.ceph:pgid 6.18 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.109 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.109 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.109 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.109 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.109 INFO:tasks.ceph:pgid 5.18 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.109 INFO:tasks.ceph:pgid 6.1b last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.110 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.110 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.110 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.110 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.110 INFO:tasks.ceph:pgid 5.19 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.110 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.110 INFO:tasks.ceph:pgid 7.1b last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.110 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.111 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.111 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.111 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.112 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.112 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.112 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.112 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.112 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.112 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.112 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.112 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.112 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.113 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.113 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.113 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.113 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.113 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.113 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.113 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.114 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.114 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.114 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.114 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.114 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.114 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.114 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.114 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.114 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.115 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.115 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.115 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.115 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.115 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.115 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.115 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.115 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:32.115 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-27T03:33:52.116 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T03:33:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.217+0000 7f96c0cc8640 1 Processor -- start 2024-04-27T03:33:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.217+0000 7f96c0cc8640 1 -- start start 2024-04-27T03:33:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.218+0000 7f96c0cc8640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f96bc12dae0 0x7f96bc12fed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.218+0000 7f96c0cc8640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f96bc06f0f0 con 0x7f96bc06d690 2024-04-27T03:33:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.218+0000 7f96c0cc8640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f96bc071a60 con 0x7f96bc12dae0 2024-04-27T03:33:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.218+0000 7f96b9d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f96bc12dae0 0x7f96bc12fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.218+0000 7f96ba59c640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f96bc06d690 0x7f96bc06da70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:44308/0 (socket says 172.21.15.89:44308) 2024-04-27T03:33:52.224 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.218+0000 7f96b9d9b640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f96bc12dae0 0x7f96bc12fed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59788/0 (socket says 172.21.15.89:59788) 2024-04-27T03:33:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.218+0000 7f96ba59c640 1 -- 172.21.15.89:0/149625934 learned_addr learned my addr 172.21.15.89:0/149625934 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:33:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.218+0000 7f96b9d9b640 1 -- 172.21.15.89:0/149625934 >> v1:172.21.15.89:6789/0 conn(0x7f96bc06d690 legacy=0x7f96bc06da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:52.225 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.218+0000 7f96b9d9b640 1 -- 172.21.15.89:0/149625934 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc06f8b0 con 0x7f96bc12dae0 2024-04-27T03:33:52.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.219+0000 7f96b9d9b640 1 --2- 172.21.15.89:0/149625934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc12dae0 0x7f96bc12fed0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f96b000b9c0 tx=0x7f96b002e010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ddb618be46d936 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:52.227 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.219+0000 7f96b959a640 1 -- 172.21.15.89:0/149625934 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96b003b070 con 0x7f96bc12dae0 2024-04-27T03:33:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.219+0000 7f96b959a640 1 -- 172.21.15.89:0/149625934 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96b002ecf0 con 0x7f96bc12dae0 2024-04-27T03:33:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.219+0000 7f96b959a640 1 -- 172.21.15.89:0/149625934 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96b0036440 con 0x7f96bc12dae0 2024-04-27T03:33:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.220+0000 7f96c0cc8640 1 -- 172.21.15.89:0/149625934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc12dae0 msgr2=0x7f96bc12fed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.220+0000 7f96c0cc8640 1 --2- 172.21.15.89:0/149625934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc12dae0 0x7f96bc12fed0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f96b000b9c0 tx=0x7f96b002e010 comp rx=0 tx=0).stop 2024-04-27T03:33:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.220+0000 7f96c0cc8640 1 -- 172.21.15.89:0/149625934 shutdown_connections 2024-04-27T03:33:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.220+0000 7f96c0cc8640 1 --2- 172.21.15.89:0/149625934 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc12dae0 0x7f96bc12fed0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.220+0000 7f96c0cc8640 1 -- 172.21.15.89:0/149625934 >> 172.21.15.89:0/149625934 conn(0x7f96bc099680 msgr2=0x7f96bc099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.220+0000 7f96c0cc8640 1 -- 172.21.15.89:0/149625934 shutdown_connections 2024-04-27T03:33:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.220+0000 7f96c0cc8640 1 -- 172.21.15.89:0/149625934 wait complete. 2024-04-27T03:33:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.221+0000 7f96c0cc8640 1 Processor -- start 2024-04-27T03:33:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.221+0000 7f96c0cc8640 1 -- start start 2024-04-27T03:33:52.228 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.221+0000 7f96c0cc8640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc06d690 0x7f96bc1bee70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.221+0000 7f96c0cc8640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f96bc06e180 con 0x7f96bc06d690 2024-04-27T03:33:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.221+0000 7f96ba59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc06d690 0x7f96bc1bee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.221+0000 7f96ba59c640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc06d690 0x7f96bc1bee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:59794/0 (socket says 172.21.15.89:59794) 2024-04-27T03:33:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.221+0000 7f96ba59c640 1 -- 172.21.15.89:0/1800792561 learned_addr learned my addr 172.21.15.89:0/1800792561 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:33:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.222+0000 7f96ba59c640 1 -- 172.21.15.89:0/1800792561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f96bc08b200 con 0x7f96bc06d690 2024-04-27T03:33:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.222+0000 7f96ba59c640 1 --2- 172.21.15.89:0/1800792561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc06d690 0x7f96bc1bee70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00c480 tx=0x7f96ac00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.222+0000 7f96aaffd640 1 -- 172.21.15.89:0/1800792561 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96ac017070 con 0x7f96bc06d690 2024-04-27T03:33:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.222+0000 7f96aaffd640 1 -- 172.21.15.89:0/1800792561 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f96ac005c10 con 0x7f96bc06d690 2024-04-27T03:33:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.222+0000 7f96c0cc8640 1 -- 172.21.15.89:0/1800792561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f96bc08a7a0 con 0x7f96bc06d690 2024-04-27T03:33:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.222+0000 7f96aaffd640 1 -- 172.21.15.89:0/1800792561 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f96ac0070d0 con 0x7f96bc06d690 2024-04-27T03:33:52.229 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.222+0000 7f96c0cc8640 1 -- 172.21.15.89:0/1800792561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f96bc08ad50 con 0x7f96bc06d690 2024-04-27T03:33:52.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.223+0000 7f96aaffd640 1 -- 172.21.15.89:0/1800792561 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f96ac01a070 con 0x7f96bc06d690 2024-04-27T03:33:52.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.223+0000 7f96aaffd640 1 --2- 172.21.15.89:0/1800792561 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f969c038060 0x7f969c03a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:33:52.230 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.223+0000 7f96aaffd640 1 -- 172.21.15.89:0/1800792561 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f96ac04ab60 con 0x7f96bc06d690 2024-04-27T03:33:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.223+0000 7f96c0cc8640 1 -- 172.21.15.89:0/1800792561 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f96bc062da0 con 0x7f96bc06d690 2024-04-27T03:33:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.223+0000 7f96b9d9b640 1 --2- 172.21.15.89:0/1800792561 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f969c038060 0x7f969c03a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:33:52.231 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.224+0000 7f96b9d9b640 1 --2- 172.21.15.89:0/1800792561 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f969c038060 0x7f969c03a520 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f96b0038000 tx=0x7f96b0032000 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:33:52.233 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.228+0000 7f96aaffd640 1 -- 172.21.15.89:0/1800792561 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f96ac007270 con 0x7f96bc06d690 2024-04-27T03:33:52.401 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.395+0000 7f96c0cc8640 1 -- 172.21.15.89:0/1800792561 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f96bc071660 con 0x7f969c038060 2024-04-27T03:33:52.416 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.410+0000 7f96aaffd640 1 -- 172.21.15.89:0/1800792561 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+658366 (secure 0 0 0) 0x7f96bc071660 con 0x7f969c038060 2024-04-27T03:33:52.424 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:33:52.429 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-04-27T03:33:52.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.415+0000 7f96c0cc8640 1 -- 172.21.15.89:0/1800792561 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f969c038060 msgr2=0x7f969c03a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:52.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.415+0000 7f96c0cc8640 1 --2- 172.21.15.89:0/1800792561 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f969c038060 0x7f969c03a520 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f96b0038000 tx=0x7f96b0032000 comp rx=0 tx=0).stop 2024-04-27T03:33:52.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.415+0000 7f96c0cc8640 1 -- 172.21.15.89:0/1800792561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc06d690 msgr2=0x7f96bc1bee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:33:52.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.415+0000 7f96c0cc8640 1 --2- 172.21.15.89:0/1800792561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc06d690 0x7f96bc1bee70 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f96ac00c480 tx=0x7f96ac00c940 comp rx=0 tx=0).stop 2024-04-27T03:33:52.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.415+0000 7f96c0cc8640 1 -- 172.21.15.89:0/1800792561 shutdown_connections 2024-04-27T03:33:52.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.415+0000 7f96c0cc8640 1 --2- 172.21.15.89:0/1800792561 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f969c038060 0x7f969c03a520 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:52.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.415+0000 7f96c0cc8640 1 --2- 172.21.15.89:0/1800792561 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f96bc06d690 0x7f96bc1bee70 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:33:52.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.415+0000 7f96c0cc8640 1 -- 172.21.15.89:0/1800792561 >> 172.21.15.89:0/1800792561 conn(0x7f96bc099680 msgr2=0x7f96bc123a60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:33:52.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.415+0000 7f96c0cc8640 1 -- 172.21.15.89:0/1800792561 shutdown_connections 2024-04-27T03:33:52.432 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:33:52.415+0000 7f96c0cc8640 1 -- 172.21.15.89:0/1800792561 wait complete. 2024-04-27T03:33:52.432 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":138,"stamp":"2024-04-27T03:33:52.279176+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":252,"num_object_clones":0,"num_object_copies":504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":252,"num_whiteouts":0,"num_read":5378,"num_read_kb":4932,"num_write":2369,"num_write_kb":61373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1204,"num_omap_keys":6,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4938,"ondisk_log_size":4938,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":156472,"kb_used_data":2648,"kb_used_omap":6,"kb_used_meta":153785,"kb_avail":377330888,"statfs":{"total":386547056640,"available":386386829312,"internally_reserved":0,"allocated":2711552,"data_stored":1657386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":157476630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1204,"num_omap_keys":6,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004362"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.293766+0000","last_change":"2024-04-27T03:30:11.362820+0000","last_active":"2024-04-27T03:32:19.293766+0000","last_peered":"2024-04-27T03:32:19.293766+0000","last_clean":"2024-04-27T03:32:19.293766+0000","last_became_active":"2024-04-27T03:30:11.362574+0000","last_became_peered":"2024-04-27T03:30:11.362574+0000","last_unstale":"2024-04-27T03:32:19.293766+0000","last_undegraded":"2024-04-27T03:32:19.293766+0000","last_fullsized":"2024-04-27T03:32:19.293766+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"27'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.940769+0000","last_change":"2024-04-27T03:30:13.384819+0000","last_active":"2024-04-27T03:33:20.940769+0000","last_peered":"2024-04-27T03:33:20.940769+0000","last_clean":"2024-04-27T03:33:20.940769+0000","last_became_active":"2024-04-27T03:30:13.384580+0000","last_became_peered":"2024-04-27T03:30:13.384580+0000","last_unstale":"2024-04-27T03:33:20.940769+0000","last_undegraded":"2024-04-27T03:33:20.940769+0000","last_fullsized":"2024-04-27T03:33:20.940769+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"27'4","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.924082+0000","last_change":"2024-04-27T03:30:11.361366+0000","last_active":"2024-04-27T03:33:20.924082+0000","last_peered":"2024-04-27T03:33:20.924082+0000","last_clean":"2024-04-27T03:33:20.924082+0000","last_became_active":"2024-04-27T03:30:11.360857+0000","last_became_peered":"2024-04-27T03:30:11.360857+0000","last_unstale":"2024-04-27T03:33:20.924082+0000","last_undegraded":"2024-04-27T03:33:20.924082+0000","last_fullsized":"2024-04-27T03:33:20.924082+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"27'12","reported_seq":80,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.704039+0000","last_change":"2024-04-27T03:30:13.376545+0000","last_active":"2024-04-27T03:33:20.704039+0000","last_peered":"2024-04-27T03:33:20.704039+0000","last_clean":"2024-04-27T03:33:20.704039+0000","last_became_active":"2024-04-27T03:30:13.376221+0000","last_became_peered":"2024-04-27T03:30:13.376221+0000","last_unstale":"2024-04-27T03:33:20.704039+0000","last_undegraded":"2024-04-27T03:33:20.704039+0000","last_fullsized":"2024-04-27T03:33:20.704039+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.294417+0000","last_change":"2024-04-27T03:30:11.363460+0000","last_active":"2024-04-27T03:32:19.294417+0000","last_peered":"2024-04-27T03:32:19.294417+0000","last_clean":"2024-04-27T03:32:19.294417+0000","last_became_active":"2024-04-27T03:30:11.362739+0000","last_became_peered":"2024-04-27T03:30:11.362739+0000","last_unstale":"2024-04-27T03:32:19.294417+0000","last_undegraded":"2024-04-27T03:32:19.294417+0000","last_fullsized":"2024-04-27T03:32:19.294417+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"27'7","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.070147+0000","last_change":"2024-04-27T03:30:13.376819+0000","last_active":"2024-04-27T03:33:21.070147+0000","last_peered":"2024-04-27T03:33:21.070147+0000","last_clean":"2024-04-27T03:33:21.070147+0000","last_became_active":"2024-04-27T03:30:13.376524+0000","last_became_peered":"2024-04-27T03:30:13.376524+0000","last_unstale":"2024-04-27T03:33:21.070147+0000","last_undegraded":"2024-04-27T03:33:21.070147+0000","last_fullsized":"2024-04-27T03:33:21.070147+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.688799+0000","last_change":"2024-04-27T03:30:11.361039+0000","last_active":"2024-04-27T03:33:27.688799+0000","last_peered":"2024-04-27T03:33:27.688799+0000","last_clean":"2024-04-27T03:33:27.688799+0000","last_became_active":"2024-04-27T03:30:11.360188+0000","last_became_peered":"2024-04-27T03:30:11.360188+0000","last_unstale":"2024-04-27T03:33:27.688799+0000","last_undegraded":"2024-04-27T03:33:27.688799+0000","last_fullsized":"2024-04-27T03:33:27.688799+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"27'16","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.016128+0000","last_change":"2024-04-27T03:30:13.376574+0000","last_active":"2024-04-27T03:33:21.016128+0000","last_peered":"2024-04-27T03:33:21.016128+0000","last_clean":"2024-04-27T03:33:21.016128+0000","last_became_active":"2024-04-27T03:30:13.376288+0000","last_became_peered":"2024-04-27T03:30:13.376288+0000","last_unstale":"2024-04-27T03:33:21.016128+0000","last_undegraded":"2024-04-27T03:33:21.016128+0000","last_fullsized":"2024-04-27T03:33:21.016128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298576+0000","last_change":"2024-04-27T03:30:11.355274+0000","last_active":"2024-04-27T03:32:44.298576+0000","last_peered":"2024-04-27T03:32:44.298576+0000","last_clean":"2024-04-27T03:32:44.298576+0000","last_became_active":"2024-04-27T03:30:11.354989+0000","last_became_peered":"2024-04-27T03:30:11.354989+0000","last_unstale":"2024-04-27T03:32:44.298576+0000","last_undegraded":"2024-04-27T03:32:44.298576+0000","last_fullsized":"2024-04-27T03:32:44.298576+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"27'6","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.942784+0000","last_change":"2024-04-27T03:30:13.374182+0000","last_active":"2024-04-27T03:33:20.942784+0000","last_peered":"2024-04-27T03:33:20.942784+0000","last_clean":"2024-04-27T03:33:20.942784+0000","last_became_active":"2024-04-27T03:30:13.373424+0000","last_became_peered":"2024-04-27T03:30:13.373424+0000","last_unstale":"2024-04-27T03:33:20.942784+0000","last_undegraded":"2024-04-27T03:33:20.942784+0000","last_fullsized":"2024-04-27T03:33:20.942784+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.660727+0000","last_change":"2024-04-27T03:30:11.361019+0000","last_active":"2024-04-27T03:33:20.660727+0000","last_peered":"2024-04-27T03:33:20.660727+0000","last_clean":"2024-04-27T03:33:20.660727+0000","last_became_active":"2024-04-27T03:30:11.360219+0000","last_became_peered":"2024-04-27T03:30:11.360219+0000","last_unstale":"2024-04-27T03:33:20.660727+0000","last_undegraded":"2024-04-27T03:33:20.660727+0000","last_fullsized":"2024-04-27T03:33:20.660727+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"25'8","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.294382+0000","last_change":"2024-04-27T03:30:13.385912+0000","last_active":"2024-04-27T03:32:19.294382+0000","last_peered":"2024-04-27T03:32:19.294382+0000","last_clean":"2024-04-27T03:32:19.294382+0000","last_became_active":"2024-04-27T03:30:13.385665+0000","last_became_peered":"2024-04-27T03:30:13.385665+0000","last_unstale":"2024-04-27T03:32:19.294382+0000","last_undegraded":"2024-04-27T03:32:19.294382+0000","last_fullsized":"2024-04-27T03:32:19.294382+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298491+0000","last_change":"2024-04-27T03:30:11.373049+0000","last_active":"2024-04-27T03:32:44.298491+0000","last_peered":"2024-04-27T03:32:44.298491+0000","last_clean":"2024-04-27T03:32:44.298491+0000","last_became_active":"2024-04-27T03:30:11.372704+0000","last_became_peered":"2024-04-27T03:30:11.372704+0000","last_unstale":"2024-04-27T03:32:44.298491+0000","last_undegraded":"2024-04-27T03:32:44.298491+0000","last_fullsized":"2024-04-27T03:32:44.298491+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"27'4","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988488+0000","last_change":"2024-04-27T03:30:13.374520+0000","last_active":"2024-04-27T03:33:20.988488+0000","last_peered":"2024-04-27T03:33:20.988488+0000","last_clean":"2024-04-27T03:33:20.988488+0000","last_became_active":"2024-04-27T03:30:13.374199+0000","last_became_peered":"2024-04-27T03:30:13.374199+0000","last_unstale":"2024-04-27T03:33:20.988488+0000","last_undegraded":"2024-04-27T03:33:20.988488+0000","last_fullsized":"2024-04-27T03:33:20.988488+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.929186+0000","last_change":"2024-04-27T03:30:11.358470+0000","last_active":"2024-04-27T03:33:20.929186+0000","last_peered":"2024-04-27T03:33:20.929186+0000","last_clean":"2024-04-27T03:33:20.929186+0000","last_became_active":"2024-04-27T03:30:11.358181+0000","last_became_peered":"2024-04-27T03:30:11.358181+0000","last_unstale":"2024-04-27T03:33:20.929186+0000","last_undegraded":"2024-04-27T03:33:20.929186+0000","last_fullsized":"2024-04-27T03:33:20.929186+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"27'4","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.802571+0000","last_change":"2024-04-27T03:30:13.379341+0000","last_active":"2024-04-27T03:33:20.802571+0000","last_peered":"2024-04-27T03:33:20.802571+0000","last_clean":"2024-04-27T03:33:20.802571+0000","last_became_active":"2024-04-27T03:30:13.379059+0000","last_became_peered":"2024-04-27T03:30:13.379059+0000","last_unstale":"2024-04-27T03:33:20.802571+0000","last_undegraded":"2024-04-27T03:33:20.802571+0000","last_fullsized":"2024-04-27T03:33:20.802571+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.018889+0000","last_change":"2024-04-27T03:30:13.383971+0000","last_active":"2024-04-27T03:33:21.018889+0000","last_peered":"2024-04-27T03:33:21.018889+0000","last_clean":"2024-04-27T03:33:21.018889+0000","last_became_active":"2024-04-27T03:30:13.383694+0000","last_became_peered":"2024-04-27T03:30:13.383694+0000","last_unstale":"2024-04-27T03:33:21.018889+0000","last_undegraded":"2024-04-27T03:33:21.018889+0000","last_fullsized":"2024-04-27T03:33:21.018889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297382+0000","last_change":"2024-04-27T03:30:11.370455+0000","last_active":"2024-04-27T03:32:39.297382+0000","last_peered":"2024-04-27T03:32:39.297382+0000","last_clean":"2024-04-27T03:32:39.297382+0000","last_became_active":"2024-04-27T03:30:11.370157+0000","last_became_peered":"2024-04-27T03:30:11.370157+0000","last_unstale":"2024-04-27T03:32:39.297382+0000","last_undegraded":"2024-04-27T03:32:39.297382+0000","last_fullsized":"2024-04-27T03:32:39.297382+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"27'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.016615+0000","last_change":"2024-04-27T03:30:13.373269+0000","last_active":"2024-04-27T03:33:21.016615+0000","last_peered":"2024-04-27T03:33:21.016615+0000","last_clean":"2024-04-27T03:33:21.016615+0000","last_became_active":"2024-04-27T03:30:13.372973+0000","last_became_peered":"2024-04-27T03:30:13.372973+0000","last_unstale":"2024-04-27T03:33:21.016615+0000","last_undegraded":"2024-04-27T03:33:21.016615+0000","last_fullsized":"2024-04-27T03:33:21.016615+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.689033+0000","last_change":"2024-04-27T03:30:11.363764+0000","last_active":"2024-04-27T03:33:27.689033+0000","last_peered":"2024-04-27T03:33:27.689033+0000","last_clean":"2024-04-27T03:33:27.689033+0000","last_became_active":"2024-04-27T03:30:11.363109+0000","last_became_peered":"2024-04-27T03:30:11.363109+0000","last_unstale":"2024-04-27T03:33:27.689033+0000","last_undegraded":"2024-04-27T03:33:27.689033+0000","last_fullsized":"2024-04-27T03:33:27.689033+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"27'9","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.072505+0000","last_change":"2024-04-27T03:30:13.384110+0000","last_active":"2024-04-27T03:33:21.072505+0000","last_peered":"2024-04-27T03:33:21.072505+0000","last_clean":"2024-04-27T03:33:21.072505+0000","last_became_active":"2024-04-27T03:30:13.383742+0000","last_became_peered":"2024-04-27T03:30:13.383742+0000","last_unstale":"2024-04-27T03:33:21.072505+0000","last_undegraded":"2024-04-27T03:33:21.072505+0000","last_fullsized":"2024-04-27T03:33:21.072505+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"25'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297324+0000","last_change":"2024-04-27T03:30:11.370635+0000","last_active":"2024-04-27T03:32:39.297324+0000","last_peered":"2024-04-27T03:32:39.297324+0000","last_clean":"2024-04-27T03:32:39.297324+0000","last_became_active":"2024-04-27T03:30:11.370380+0000","last_became_peered":"2024-04-27T03:30:11.370380+0000","last_unstale":"2024-04-27T03:32:39.297324+0000","last_undegraded":"2024-04-27T03:32:39.297324+0000","last_fullsized":"2024-04-27T03:32:39.297324+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"27'6","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.016547+0000","last_change":"2024-04-27T03:31:01.446266+0000","last_active":"2024-04-27T03:33:21.016547+0000","last_peered":"2024-04-27T03:33:21.016547+0000","last_clean":"2024-04-27T03:33:21.016547+0000","last_became_active":"2024-04-27T03:31:01.440762+0000","last_became_peered":"2024-04-27T03:31:01.440762+0000","last_unstale":"2024-04-27T03:33:21.016547+0000","last_undegraded":"2024-04-27T03:33:21.016547+0000","last_fullsized":"2024-04-27T03:33:21.016547+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297309+0000","last_change":"2024-04-27T03:30:11.369834+0000","last_active":"2024-04-27T03:32:39.297309+0000","last_peered":"2024-04-27T03:32:39.297309+0000","last_clean":"2024-04-27T03:32:39.297309+0000","last_became_active":"2024-04-27T03:30:11.369551+0000","last_became_peered":"2024-04-27T03:30:11.369551+0000","last_unstale":"2024-04-27T03:32:39.297309+0000","last_undegraded":"2024-04-27T03:32:39.297309+0000","last_fullsized":"2024-04-27T03:32:39.297309+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"27'2","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988816+0000","last_change":"2024-04-27T03:30:13.382150+0000","last_active":"2024-04-27T03:33:20.988816+0000","last_peered":"2024-04-27T03:33:20.988816+0000","last_clean":"2024-04-27T03:33:20.988816+0000","last_became_active":"2024-04-27T03:30:13.381901+0000","last_became_peered":"2024-04-27T03:30:13.381901+0000","last_unstale":"2024-04-27T03:33:20.988816+0000","last_undegraded":"2024-04-27T03:33:20.988816+0000","last_fullsized":"2024-04-27T03:33:20.988816+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.291386+0000","last_change":"2024-04-27T03:30:11.361129+0000","last_active":"2024-04-27T03:32:39.291386+0000","last_peered":"2024-04-27T03:32:39.291386+0000","last_clean":"2024-04-27T03:32:39.291386+0000","last_became_active":"2024-04-27T03:30:11.360515+0000","last_became_peered":"2024-04-27T03:30:11.360515+0000","last_unstale":"2024-04-27T03:32:39.291386+0000","last_undegraded":"2024-04-27T03:32:39.291386+0000","last_fullsized":"2024-04-27T03:32:39.291386+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"27'4","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297334+0000","last_change":"2024-04-27T03:30:13.382249+0000","last_active":"2024-04-27T03:32:39.297334+0000","last_peered":"2024-04-27T03:32:39.297334+0000","last_clean":"2024-04-27T03:32:39.297334+0000","last_became_active":"2024-04-27T03:30:13.378932+0000","last_became_peered":"2024-04-27T03:30:13.378932+0000","last_unstale":"2024-04-27T03:32:39.297334+0000","last_undegraded":"2024-04-27T03:32:39.297334+0000","last_fullsized":"2024-04-27T03:32:39.297334+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296759+0000","last_change":"2024-04-27T03:30:11.372964+0000","last_active":"2024-04-27T03:32:34.296759+0000","last_peered":"2024-04-27T03:32:34.296759+0000","last_clean":"2024-04-27T03:32:34.296759+0000","last_became_active":"2024-04-27T03:30:11.372673+0000","last_became_peered":"2024-04-27T03:30:11.372673+0000","last_unstale":"2024-04-27T03:32:34.296759+0000","last_undegraded":"2024-04-27T03:32:34.296759+0000","last_fullsized":"2024-04-27T03:32:34.296759+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"27'35","reported_seq":100,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.028786+0000","last_change":"2024-04-27T03:30:21.413597+0000","last_active":"2024-04-27T03:33:21.028786+0000","last_peered":"2024-04-27T03:33:21.028786+0000","last_clean":"2024-04-27T03:33:21.028786+0000","last_became_active":"2024-04-27T03:30:21.413255+0000","last_became_peered":"2024-04-27T03:30:21.413255+0000","last_unstale":"2024-04-27T03:33:21.028786+0000","last_undegraded":"2024-04-27T03:33:21.028786+0000","last_fullsized":"2024-04-27T03:33:21.028786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"27'3","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.070022+0000","last_change":"2024-04-27T03:30:13.377382+0000","last_active":"2024-04-27T03:33:21.070022+0000","last_peered":"2024-04-27T03:33:21.070022+0000","last_clean":"2024-04-27T03:33:21.070022+0000","last_became_active":"2024-04-27T03:30:13.377110+0000","last_became_peered":"2024-04-27T03:30:13.377110+0000","last_unstale":"2024-04-27T03:33:21.070022+0000","last_undegraded":"2024-04-27T03:33:21.070022+0000","last_fullsized":"2024-04-27T03:33:21.070022+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296793+0000","last_change":"2024-04-27T03:30:11.360152+0000","last_active":"2024-04-27T03:32:34.296793+0000","last_peered":"2024-04-27T03:32:34.296793+0000","last_clean":"2024-04-27T03:32:34.296793+0000","last_became_active":"2024-04-27T03:30:11.359892+0000","last_became_peered":"2024-04-27T03:30:11.359892+0000","last_unstale":"2024-04-27T03:32:34.296793+0000","last_undegraded":"2024-04-27T03:32:34.296793+0000","last_fullsized":"2024-04-27T03:32:34.296793+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.148150+0000","last_change":"2024-04-27T03:33:38.148173+0000","last_active":"2024-04-27T03:33:38.148150+0000","last_peered":"2024-04-27T03:33:38.148150+0000","last_clean":"2024-04-27T03:33:38.148150+0000","last_became_active":"2024-04-27T03:30:15.380864+0000","last_became_peered":"2024-04-27T03:30:15.380864+0000","last_unstale":"2024-04-27T03:33:38.148150+0000","last_undegraded":"2024-04-27T03:33:38.148150+0000","last_fullsized":"2024-04-27T03:33:38.148150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:38.148061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:38.148061+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.148061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:53.865374+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"27'50","reported_seq":117,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.088201+0000","last_change":"2024-04-27T03:33:43.088242+0000","last_active":"2024-04-27T03:33:43.088201+0000","last_peered":"2024-04-27T03:33:43.088201+0000","last_clean":"2024-04-27T03:33:43.088201+0000","last_became_active":"2024-04-27T03:30:17.387391+0000","last_became_peered":"2024-04-27T03:30:17.387391+0000","last_unstale":"2024-04-27T03:33:43.088201+0000","last_undegraded":"2024-04-27T03:33:43.088201+0000","last_fullsized":"2024-04-27T03:33:43.088201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'50","last_scrub_stamp":"2024-04-27T03:33:43.088100+0000","last_deep_scrub":"27'50","last_deep_scrub_stamp":"2024-04-27T03:33:43.088100+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.088100+0000","objects_scrubbed":6,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:13:27.492179+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":45,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.971326+0000","last_change":"2024-04-27T03:33:44.971352+0000","last_active":"2024-04-27T03:33:44.971326+0000","last_peered":"2024-04-27T03:33:44.971326+0000","last_clean":"2024-04-27T03:33:44.971326+0000","last_became_active":"2024-04-27T03:30:19.407637+0000","last_became_peered":"2024-04-27T03:30:19.407637+0000","last_unstale":"2024-04-27T03:33:44.971326+0000","last_undegraded":"2024-04-27T03:33:44.971326+0000","last_fullsized":"2024-04-27T03:33:44.971326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:44.971239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:44.971239+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.971239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:22.448440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"27'9","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:49.299125+0000","last_change":"2024-04-27T03:30:21.414223+0000","last_active":"2024-04-27T03:32:49.299125+0000","last_peered":"2024-04-27T03:32:49.299125+0000","last_clean":"2024-04-27T03:32:49.299125+0000","last_became_active":"2024-04-27T03:30:21.413940+0000","last_became_peered":"2024-04-27T03:30:21.413940+0000","last_unstale":"2024-04-27T03:32:49.299125+0000","last_undegraded":"2024-04-27T03:32:49.299125+0000","last_fullsized":"2024-04-27T03:32:49.299125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"27'16","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988645+0000","last_change":"2024-04-27T03:30:13.381748+0000","last_active":"2024-04-27T03:33:20.988645+0000","last_peered":"2024-04-27T03:33:20.988645+0000","last_clean":"2024-04-27T03:33:20.988645+0000","last_became_active":"2024-04-27T03:30:13.381487+0000","last_became_peered":"2024-04-27T03:30:13.381487+0000","last_unstale":"2024-04-27T03:33:20.988645+0000","last_undegraded":"2024-04-27T03:33:20.988645+0000","last_fullsized":"2024-04-27T03:33:20.988645+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296267+0000","last_change":"2024-04-27T03:30:11.359690+0000","last_active":"2024-04-27T03:32:34.296267+0000","last_peered":"2024-04-27T03:32:34.296267+0000","last_clean":"2024-04-27T03:32:34.296267+0000","last_became_active":"2024-04-27T03:30:11.359446+0000","last_became_peered":"2024-04-27T03:30:11.359446+0000","last_unstale":"2024-04-27T03:32:34.296267+0000","last_undegraded":"2024-04-27T03:32:34.296267+0000","last_fullsized":"2024-04-27T03:32:34.296267+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.291420+0000","last_change":"2024-04-27T03:30:15.380566+0000","last_active":"2024-04-27T03:32:39.291420+0000","last_peered":"2024-04-27T03:32:39.291420+0000","last_clean":"2024-04-27T03:32:39.291420+0000","last_became_active":"2024-04-27T03:30:15.380313+0000","last_became_peered":"2024-04-27T03:30:15.380313+0000","last_unstale":"2024-04-27T03:32:39.291420+0000","last_undegraded":"2024-04-27T03:32:39.291420+0000","last_fullsized":"2024-04-27T03:32:39.291420+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"27'16","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.841774+0000","last_change":"2024-04-27T03:33:37.841840+0000","last_active":"2024-04-27T03:33:37.841774+0000","last_peered":"2024-04-27T03:33:37.841774+0000","last_clean":"2024-04-27T03:33:37.841774+0000","last_became_active":"2024-04-27T03:30:17.389126+0000","last_became_peered":"2024-04-27T03:30:17.389126+0000","last_unstale":"2024-04-27T03:33:37.841774+0000","last_undegraded":"2024-04-27T03:33:37.841774+0000","last_fullsized":"2024-04-27T03:33:37.841774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:33:37.841643+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:33:37.841643+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.841643+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:42.665962+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.837474+0000","last_change":"2024-04-27T03:33:36.837497+0000","last_active":"2024-04-27T03:33:36.837474+0000","last_peered":"2024-04-27T03:33:36.837474+0000","last_clean":"2024-04-27T03:33:36.837474+0000","last_became_active":"2024-04-27T03:30:19.405170+0000","last_became_peered":"2024-04-27T03:30:19.405170+0000","last_unstale":"2024-04-27T03:33:36.837474+0000","last_undegraded":"2024-04-27T03:33:36.837474+0000","last_fullsized":"2024-04-27T03:33:36.837474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.837388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.837388+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.837388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:14.962289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'6","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:54.293517+0000","last_change":"2024-04-27T03:30:21.408308+0000","last_active":"2024-04-27T03:32:54.293517+0000","last_peered":"2024-04-27T03:32:54.293517+0000","last_clean":"2024-04-27T03:32:54.293517+0000","last_became_active":"2024-04-27T03:30:21.407917+0000","last_became_peered":"2024-04-27T03:30:21.407917+0000","last_unstale":"2024-04-27T03:32:54.293517+0000","last_undegraded":"2024-04-27T03:32:54.293517+0000","last_fullsized":"2024-04-27T03:32:54.293517+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.804760+0000","last_change":"2024-04-27T03:30:13.378024+0000","last_active":"2024-04-27T03:33:20.804760+0000","last_peered":"2024-04-27T03:33:20.804760+0000","last_clean":"2024-04-27T03:33:20.804760+0000","last_became_active":"2024-04-27T03:30:13.377423+0000","last_became_peered":"2024-04-27T03:30:13.377423+0000","last_unstale":"2024-04-27T03:33:20.804760+0000","last_undegraded":"2024-04-27T03:33:20.804760+0000","last_fullsized":"2024-04-27T03:33:20.804760+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296713+0000","last_change":"2024-04-27T03:30:11.359542+0000","last_active":"2024-04-27T03:32:34.296713+0000","last_peered":"2024-04-27T03:32:34.296713+0000","last_clean":"2024-04-27T03:32:34.296713+0000","last_became_active":"2024-04-27T03:30:11.359261+0000","last_became_peered":"2024-04-27T03:30:11.359261+0000","last_unstale":"2024-04-27T03:32:34.296713+0000","last_undegraded":"2024-04-27T03:32:34.296713+0000","last_fullsized":"2024-04-27T03:32:34.296713+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.753269+0000","last_change":"2024-04-27T03:33:47.753294+0000","last_active":"2024-04-27T03:33:47.753269+0000","last_peered":"2024-04-27T03:33:47.753269+0000","last_clean":"2024-04-27T03:33:47.753269+0000","last_became_active":"2024-04-27T03:30:15.383618+0000","last_became_peered":"2024-04-27T03:30:15.383618+0000","last_unstale":"2024-04-27T03:33:47.753269+0000","last_undegraded":"2024-04-27T03:33:47.753269+0000","last_fullsized":"2024-04-27T03:33:47.753269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:47.753177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:47.753177+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.753177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:54.718147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"27'30","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.888348+0000","last_change":"2024-04-27T03:33:37.888412+0000","last_active":"2024-04-27T03:33:37.888348+0000","last_peered":"2024-04-27T03:33:37.888348+0000","last_clean":"2024-04-27T03:33:37.888348+0000","last_became_active":"2024-04-27T03:30:17.392223+0000","last_became_peered":"2024-04-27T03:30:17.392223+0000","last_unstale":"2024-04-27T03:33:37.888348+0000","last_undegraded":"2024-04-27T03:33:37.888348+0000","last_fullsized":"2024-04-27T03:33:37.888348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-04-27T03:33:37.888233+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-04-27T03:33:37.888233+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.888233+0000","objects_scrubbed":7,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:27:30.955280+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"27'285","reported_seq":385,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.712529+0000","last_change":"2024-04-27T03:30:19.406264+0000","last_active":"2024-04-27T03:33:27.712529+0000","last_peered":"2024-04-27T03:33:27.712529+0000","last_clean":"2024-04-27T03:33:27.712529+0000","last_became_active":"2024-04-27T03:30:19.405730+0000","last_became_peered":"2024-04-27T03:30:19.405730+0000","last_unstale":"2024-04-27T03:33:27.712529+0000","last_undegraded":"2024-04-27T03:33:27.712529+0000","last_fullsized":"2024-04-27T03:33:27.712529+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:54.299992+0000","last_change":"2024-04-27T03:30:21.409739+0000","last_active":"2024-04-27T03:32:54.299992+0000","last_peered":"2024-04-27T03:32:54.299992+0000","last_clean":"2024-04-27T03:32:54.299992+0000","last_became_active":"2024-04-27T03:30:21.409156+0000","last_became_peered":"2024-04-27T03:30:21.409156+0000","last_unstale":"2024-04-27T03:32:54.299992+0000","last_undegraded":"2024-04-27T03:32:54.299992+0000","last_fullsized":"2024-04-27T03:32:54.299992+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"27'5","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.070369+0000","last_change":"2024-04-27T03:30:13.385803+0000","last_active":"2024-04-27T03:33:21.070369+0000","last_peered":"2024-04-27T03:33:21.070369+0000","last_clean":"2024-04-27T03:33:21.070369+0000","last_became_active":"2024-04-27T03:30:13.385491+0000","last_became_peered":"2024-04-27T03:30:13.385491+0000","last_unstale":"2024-04-27T03:33:21.070369+0000","last_undegraded":"2024-04-27T03:33:21.070369+0000","last_fullsized":"2024-04-27T03:33:21.070369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296216+0000","last_change":"2024-04-27T03:30:11.370401+0000","last_active":"2024-04-27T03:32:34.296216+0000","last_peered":"2024-04-27T03:32:34.296216+0000","last_clean":"2024-04-27T03:32:34.296216+0000","last_became_active":"2024-04-27T03:30:11.369939+0000","last_became_peered":"2024-04-27T03:30:11.369939+0000","last_unstale":"2024-04-27T03:32:34.296216+0000","last_undegraded":"2024-04-27T03:32:34.296216+0000","last_fullsized":"2024-04-27T03:32:34.296216+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.134971+0000","last_change":"2024-04-27T03:33:33.134999+0000","last_active":"2024-04-27T03:33:33.134971+0000","last_peered":"2024-04-27T03:33:33.134971+0000","last_clean":"2024-04-27T03:33:33.134971+0000","last_became_active":"2024-04-27T03:30:15.384114+0000","last_became_peered":"2024-04-27T03:30:15.384114+0000","last_unstale":"2024-04-27T03:33:33.134971+0000","last_undegraded":"2024-04-27T03:33:33.134971+0000","last_fullsized":"2024-04-27T03:33:33.134971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:33.134840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:33.134840+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.134840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:57.479389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"27'43","reported_seq":104,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.859628+0000","last_change":"2024-04-27T03:33:38.859665+0000","last_active":"2024-04-27T03:33:38.859628+0000","last_peered":"2024-04-27T03:33:38.859628+0000","last_clean":"2024-04-27T03:33:38.859628+0000","last_became_active":"2024-04-27T03:30:17.391880+0000","last_became_peered":"2024-04-27T03:30:17.391880+0000","last_unstale":"2024-04-27T03:33:38.859628+0000","last_undegraded":"2024-04-27T03:33:38.859628+0000","last_fullsized":"2024-04-27T03:33:38.859628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'43","last_scrub_stamp":"2024-04-27T03:33:38.859525+0000","last_deep_scrub":"27'43","last_deep_scrub_stamp":"2024-04-27T03:33:38.859525+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.859525+0000","objects_scrubbed":5,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:06.442676+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"27'285","reported_seq":417,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.785523+0000","last_change":"2024-04-27T03:33:35.785553+0000","last_active":"2024-04-27T03:33:35.785523+0000","last_peered":"2024-04-27T03:33:35.785523+0000","last_clean":"2024-04-27T03:33:35.785523+0000","last_became_active":"2024-04-27T03:30:19.404744+0000","last_became_peered":"2024-04-27T03:30:19.404744+0000","last_unstale":"2024-04-27T03:33:35.785523+0000","last_undegraded":"2024-04-27T03:33:35.785523+0000","last_fullsized":"2024-04-27T03:33:35.785523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:35.785437+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:35.785437+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.785437+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:23.864435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'7","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.165585+0000","last_change":"2024-04-27T03:33:49.165611+0000","last_active":"2024-04-27T03:33:49.165585+0000","last_peered":"2024-04-27T03:33:49.165585+0000","last_clean":"2024-04-27T03:33:49.165585+0000","last_became_active":"2024-04-27T03:30:13.387335+0000","last_became_peered":"2024-04-27T03:30:13.387335+0000","last_unstale":"2024-04-27T03:33:49.165585+0000","last_undegraded":"2024-04-27T03:33:49.165585+0000","last_fullsized":"2024-04-27T03:33:49.165585+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:33:49.165500+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:33:49.165500+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.165500+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:22.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295452+0000","last_change":"2024-04-27T03:30:11.372061+0000","last_active":"2024-04-27T03:32:29.295452+0000","last_peered":"2024-04-27T03:32:29.295452+0000","last_clean":"2024-04-27T03:32:29.295452+0000","last_became_active":"2024-04-27T03:30:11.371600+0000","last_became_peered":"2024-04-27T03:30:11.371600+0000","last_unstale":"2024-04-27T03:32:29.295452+0000","last_undegraded":"2024-04-27T03:32:29.295452+0000","last_fullsized":"2024-04-27T03:32:29.295452+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.990712+0000","last_change":"2024-04-27T03:33:46.990760+0000","last_active":"2024-04-27T03:33:46.990712+0000","last_peered":"2024-04-27T03:33:46.990712+0000","last_clean":"2024-04-27T03:33:46.990712+0000","last_became_active":"2024-04-27T03:30:15.381359+0000","last_became_peered":"2024-04-27T03:30:15.381359+0000","last_unstale":"2024-04-27T03:33:46.990712+0000","last_undegraded":"2024-04-27T03:33:46.990712+0000","last_fullsized":"2024-04-27T03:33:46.990712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:46.990601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:46.990601+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.990601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:08.708219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"27'58","reported_seq":124,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.809607+0000","last_change":"2024-04-27T03:33:41.809671+0000","last_active":"2024-04-27T03:33:41.809607+0000","last_peered":"2024-04-27T03:33:41.809607+0000","last_clean":"2024-04-27T03:33:41.809607+0000","last_became_active":"2024-04-27T03:30:17.390021+0000","last_became_peered":"2024-04-27T03:30:17.390021+0000","last_unstale":"2024-04-27T03:33:41.809607+0000","last_undegraded":"2024-04-27T03:33:41.809607+0000","last_fullsized":"2024-04-27T03:33:41.809607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'58","last_scrub_stamp":"2024-04-27T03:33:41.809501+0000","last_deep_scrub":"27'58","last_deep_scrub_stamp":"2024-04-27T03:33:41.809501+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.809501+0000","objects_scrubbed":10,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:46.251050+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":30,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.738236+0000","last_change":"2024-04-27T03:33:32.738255+0000","last_active":"2024-04-27T03:33:32.738236+0000","last_peered":"2024-04-27T03:33:32.738236+0000","last_clean":"2024-04-27T03:33:32.738236+0000","last_became_active":"2024-04-27T03:30:19.404948+0000","last_became_peered":"2024-04-27T03:30:19.404948+0000","last_unstale":"2024-04-27T03:33:32.738236+0000","last_undegraded":"2024-04-27T03:33:32.738236+0000","last_fullsized":"2024-04-27T03:33:32.738236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:32.738165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:32.738165+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.738165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:19.681571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"27'3","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295334+0000","last_change":"2024-04-27T03:30:21.412598+0000","last_active":"2024-04-27T03:32:29.295334+0000","last_peered":"2024-04-27T03:32:29.295334+0000","last_clean":"2024-04-27T03:32:29.295334+0000","last_became_active":"2024-04-27T03:30:21.412331+0000","last_became_peered":"2024-04-27T03:30:21.412331+0000","last_unstale":"2024-04-27T03:32:29.295334+0000","last_undegraded":"2024-04-27T03:32:29.295334+0000","last_fullsized":"2024-04-27T03:32:29.295334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"25'6","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294523+0000","last_change":"2024-04-27T03:30:13.373532+0000","last_active":"2024-04-27T03:32:24.294523+0000","last_peered":"2024-04-27T03:32:24.294523+0000","last_clean":"2024-04-27T03:32:24.294523+0000","last_became_active":"2024-04-27T03:30:13.373226+0000","last_became_peered":"2024-04-27T03:30:13.373226+0000","last_unstale":"2024-04-27T03:32:24.294523+0000","last_undegraded":"2024-04-27T03:32:24.294523+0000","last_fullsized":"2024-04-27T03:32:24.294523+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295384+0000","last_change":"2024-04-27T03:30:11.373710+0000","last_active":"2024-04-27T03:32:29.295384+0000","last_peered":"2024-04-27T03:32:29.295384+0000","last_clean":"2024-04-27T03:32:29.295384+0000","last_became_active":"2024-04-27T03:30:11.373430+0000","last_became_peered":"2024-04-27T03:30:11.373430+0000","last_unstale":"2024-04-27T03:32:29.295384+0000","last_undegraded":"2024-04-27T03:32:29.295384+0000","last_fullsized":"2024-04-27T03:32:29.295384+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"18'1","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.552724+0000","last_change":"2024-04-27T03:31:01.447513+0000","last_active":"2024-04-27T03:33:27.552724+0000","last_peered":"2024-04-27T03:33:27.552724+0000","last_clean":"2024-04-27T03:33:27.552724+0000","last_became_active":"2024-04-27T03:31:01.443259+0000","last_became_peered":"2024-04-27T03:31:01.443259+0000","last_unstale":"2024-04-27T03:33:27.552724+0000","last_undegraded":"2024-04-27T03:33:27.552724+0000","last_fullsized":"2024-04-27T03:33:27.552724+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"27'42","reported_seq":95,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.694244+0000","last_change":"2024-04-27T03:30:17.389610+0000","last_active":"2024-04-27T03:33:27.694244+0000","last_peered":"2024-04-27T03:33:27.694244+0000","last_clean":"2024-04-27T03:33:27.694244+0000","last_became_active":"2024-04-27T03:30:17.388994+0000","last_became_peered":"2024-04-27T03:30:17.388994+0000","last_unstale":"2024-04-27T03:33:27.694244+0000","last_undegraded":"2024-04-27T03:33:27.694244+0000","last_fullsized":"2024-04-27T03:33:27.694244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:30.805151+0000","last_change":"2024-04-27T03:33:30.805201+0000","last_active":"2024-04-27T03:33:30.805151+0000","last_peered":"2024-04-27T03:33:30.805151+0000","last_clean":"2024-04-27T03:33:30.805151+0000","last_became_active":"2024-04-27T03:30:19.405334+0000","last_became_peered":"2024-04-27T03:30:19.405334+0000","last_unstale":"2024-04-27T03:33:30.805151+0000","last_undegraded":"2024-04-27T03:33:30.805151+0000","last_fullsized":"2024-04-27T03:33:30.805151+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:30.805019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:30.805019+0000","last_clean_scrub_stamp":"2024-04-27T03:33:30.805019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:46:37.451973+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"27'3","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295432+0000","last_change":"2024-04-27T03:30:21.410221+0000","last_active":"2024-04-27T03:32:29.295432+0000","last_peered":"2024-04-27T03:32:29.295432+0000","last_clean":"2024-04-27T03:32:29.295432+0000","last_became_active":"2024-04-27T03:30:21.409922+0000","last_became_peered":"2024-04-27T03:30:21.409922+0000","last_unstale":"2024-04-27T03:32:29.295432+0000","last_undegraded":"2024-04-27T03:32:29.295432+0000","last_fullsized":"2024-04-27T03:32:29.295432+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"27'6","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.019085+0000","last_change":"2024-04-27T03:30:13.381521+0000","last_active":"2024-04-27T03:33:21.019085+0000","last_peered":"2024-04-27T03:33:21.019085+0000","last_clean":"2024-04-27T03:33:21.019085+0000","last_became_active":"2024-04-27T03:30:13.381270+0000","last_became_peered":"2024-04-27T03:30:13.381270+0000","last_unstale":"2024-04-27T03:33:21.019085+0000","last_undegraded":"2024-04-27T03:33:21.019085+0000","last_fullsized":"2024-04-27T03:33:21.019085+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295920+0000","last_change":"2024-04-27T03:30:11.372966+0000","last_active":"2024-04-27T03:32:29.295920+0000","last_peered":"2024-04-27T03:32:29.295920+0000","last_clean":"2024-04-27T03:32:29.295920+0000","last_became_active":"2024-04-27T03:30:11.372583+0000","last_became_peered":"2024-04-27T03:30:11.372583+0000","last_unstale":"2024-04-27T03:32:29.295920+0000","last_undegraded":"2024-04-27T03:32:29.295920+0000","last_fullsized":"2024-04-27T03:32:29.295920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.290710+0000","last_change":"2024-04-27T03:30:15.382571+0000","last_active":"2024-04-27T03:32:34.290710+0000","last_peered":"2024-04-27T03:32:34.290710+0000","last_clean":"2024-04-27T03:32:34.290710+0000","last_became_active":"2024-04-27T03:30:15.382207+0000","last_became_peered":"2024-04-27T03:30:15.382207+0000","last_unstale":"2024-04-27T03:32:34.290710+0000","last_undegraded":"2024-04-27T03:32:34.290710+0000","last_fullsized":"2024-04-27T03:32:34.290710+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"27'52","reported_seq":114,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.655119+0000","last_change":"2024-04-27T03:30:17.390418+0000","last_active":"2024-04-27T03:33:27.655119+0000","last_peered":"2024-04-27T03:33:27.655119+0000","last_clean":"2024-04-27T03:33:27.655119+0000","last_became_active":"2024-04-27T03:30:17.389242+0000","last_became_peered":"2024-04-27T03:30:17.389242+0000","last_unstale":"2024-04-27T03:33:27.655119+0000","last_undegraded":"2024-04-27T03:33:27.655119+0000","last_fullsized":"2024-04-27T03:33:27.655119+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":62,"num_read_kb":51,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"27'285","reported_seq":360,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.734569+0000","last_change":"2024-04-27T03:33:38.734598+0000","last_active":"2024-04-27T03:33:38.734569+0000","last_peered":"2024-04-27T03:33:38.734569+0000","last_clean":"2024-04-27T03:33:38.734569+0000","last_became_active":"2024-04-27T03:30:19.406364+0000","last_became_peered":"2024-04-27T03:30:19.406364+0000","last_unstale":"2024-04-27T03:33:38.734569+0000","last_undegraded":"2024-04-27T03:33:38.734569+0000","last_fullsized":"2024-04-27T03:33:38.734569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:38.734486+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:38.734486+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.734486+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:11.608629+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.290755+0000","last_change":"2024-04-27T03:30:21.408481+0000","last_active":"2024-04-27T03:32:34.290755+0000","last_peered":"2024-04-27T03:32:34.290755+0000","last_clean":"2024-04-27T03:32:34.290755+0000","last_became_active":"2024-04-27T03:30:21.408168+0000","last_became_peered":"2024-04-27T03:30:21.408168+0000","last_unstale":"2024-04-27T03:32:34.290755+0000","last_undegraded":"2024-04-27T03:32:34.290755+0000","last_fullsized":"2024-04-27T03:32:34.290755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"27'14","reported_seq":102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.703859+0000","last_change":"2024-04-27T03:30:13.371773+0000","last_active":"2024-04-27T03:33:20.703859+0000","last_peered":"2024-04-27T03:33:20.703859+0000","last_clean":"2024-04-27T03:33:20.703859+0000","last_became_active":"2024-04-27T03:30:13.371503+0000","last_became_peered":"2024-04-27T03:30:13.371503+0000","last_unstale":"2024-04-27T03:33:20.703859+0000","last_undegraded":"2024-04-27T03:33:20.703859+0000","last_fullsized":"2024-04-27T03:33:20.703859+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295881+0000","last_change":"2024-04-27T03:30:11.357185+0000","last_active":"2024-04-27T03:32:29.295881+0000","last_peered":"2024-04-27T03:32:29.295881+0000","last_clean":"2024-04-27T03:32:29.295881+0000","last_became_active":"2024-04-27T03:30:11.356739+0000","last_became_peered":"2024-04-27T03:30:11.356739+0000","last_unstale":"2024-04-27T03:32:29.295881+0000","last_undegraded":"2024-04-27T03:32:29.295881+0000","last_fullsized":"2024-04-27T03:32:29.295881+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:14.303020+0000","last_change":"2024-04-27T03:31:01.443432+0000","last_active":"2024-04-27T03:33:14.303020+0000","last_peered":"2024-04-27T03:33:14.303020+0000","last_clean":"2024-04-27T03:33:14.303020+0000","last_became_active":"2024-04-27T03:31:01.443118+0000","last_became_peered":"2024-04-27T03:31:01.443118+0000","last_unstale":"2024-04-27T03:33:14.303020+0000","last_undegraded":"2024-04-27T03:33:14.303020+0000","last_fullsized":"2024-04-27T03:33:14.303020+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"27'74","reported_seq":151,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.703357+0000","last_change":"2024-04-27T03:30:17.392542+0000","last_active":"2024-04-27T03:33:27.703357+0000","last_peered":"2024-04-27T03:33:27.703357+0000","last_clean":"2024-04-27T03:33:27.703357+0000","last_became_active":"2024-04-27T03:30:17.387318+0000","last_became_peered":"2024-04-27T03:30:17.387318+0000","last_unstale":"2024-04-27T03:33:27.703357+0000","last_undegraded":"2024-04-27T03:33:27.703357+0000","last_fullsized":"2024-04-27T03:33:27.703357+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":74,"log_dups_size":0,"ondisk_log_size":74,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":79,"num_read_kb":68,"num_write":34,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.701834+0000","last_change":"2024-04-27T03:33:37.701877+0000","last_active":"2024-04-27T03:33:37.701834+0000","last_peered":"2024-04-27T03:33:37.701834+0000","last_clean":"2024-04-27T03:33:37.701834+0000","last_became_active":"2024-04-27T03:30:19.404265+0000","last_became_peered":"2024-04-27T03:30:19.404265+0000","last_unstale":"2024-04-27T03:33:37.701834+0000","last_undegraded":"2024-04-27T03:33:37.701834+0000","last_fullsized":"2024-04-27T03:33:37.701834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:37.701748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:37.701748+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.701748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:42.526067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"27'18","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.124420+0000","last_change":"2024-04-27T03:30:21.414109+0000","last_active":"2024-04-27T03:33:21.124420+0000","last_peered":"2024-04-27T03:33:21.124420+0000","last_clean":"2024-04-27T03:33:21.124420+0000","last_became_active":"2024-04-27T03:30:21.413757+0000","last_became_peered":"2024-04-27T03:30:21.413757+0000","last_unstale":"2024-04-27T03:33:21.124420+0000","last_undegraded":"2024-04-27T03:33:21.124420+0000","last_fullsized":"2024-04-27T03:33:21.124420+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988478+0000","last_change":"2024-04-27T03:30:13.380233+0000","last_active":"2024-04-27T03:33:20.988478+0000","last_peered":"2024-04-27T03:33:20.988478+0000","last_clean":"2024-04-27T03:33:20.988478+0000","last_became_active":"2024-04-27T03:30:13.379861+0000","last_became_peered":"2024-04-27T03:30:13.379861+0000","last_unstale":"2024-04-27T03:33:20.988478+0000","last_undegraded":"2024-04-27T03:33:20.988478+0000","last_fullsized":"2024-04-27T03:33:20.988478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.289805+0000","last_change":"2024-04-27T03:30:11.357503+0000","last_active":"2024-04-27T03:32:29.289805+0000","last_peered":"2024-04-27T03:32:29.289805+0000","last_clean":"2024-04-27T03:32:29.289805+0000","last_became_active":"2024-04-27T03:30:11.357176+0000","last_became_peered":"2024-04-27T03:30:11.357176+0000","last_unstale":"2024-04-27T03:32:29.289805+0000","last_undegraded":"2024-04-27T03:32:29.289805+0000","last_fullsized":"2024-04-27T03:32:29.289805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296361+0000","last_change":"2024-04-27T03:30:15.382614+0000","last_active":"2024-04-27T03:32:34.296361+0000","last_peered":"2024-04-27T03:32:34.296361+0000","last_clean":"2024-04-27T03:32:34.296361+0000","last_became_active":"2024-04-27T03:30:15.382290+0000","last_became_peered":"2024-04-27T03:30:15.382290+0000","last_unstale":"2024-04-27T03:32:34.296361+0000","last_undegraded":"2024-04-27T03:32:34.296361+0000","last_fullsized":"2024-04-27T03:32:34.296361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"27'46","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.971592+0000","last_change":"2024-04-27T03:33:41.971630+0000","last_active":"2024-04-27T03:33:41.971592+0000","last_peered":"2024-04-27T03:33:41.971592+0000","last_clean":"2024-04-27T03:33:41.971592+0000","last_became_active":"2024-04-27T03:30:17.385902+0000","last_became_peered":"2024-04-27T03:30:17.385902+0000","last_unstale":"2024-04-27T03:33:41.971592+0000","last_undegraded":"2024-04-27T03:33:41.971592+0000","last_fullsized":"2024-04-27T03:33:41.971592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:41.971499+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:41.971499+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.971499+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:48.621217+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296901+0000","last_change":"2024-04-27T03:30:19.406061+0000","last_active":"2024-04-27T03:32:34.296901+0000","last_peered":"2024-04-27T03:32:34.296901+0000","last_clean":"2024-04-27T03:32:34.296901+0000","last_became_active":"2024-04-27T03:30:19.405450+0000","last_became_peered":"2024-04-27T03:30:19.405450+0000","last_unstale":"2024-04-27T03:32:34.296901+0000","last_undegraded":"2024-04-27T03:32:34.296901+0000","last_fullsized":"2024-04-27T03:32:34.296901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"27'9","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.075726+0000","last_change":"2024-04-27T03:30:21.415884+0000","last_active":"2024-04-27T03:33:21.075726+0000","last_peered":"2024-04-27T03:33:21.075726+0000","last_clean":"2024-04-27T03:33:21.075726+0000","last_became_active":"2024-04-27T03:30:21.415494+0000","last_became_peered":"2024-04-27T03:30:21.415494+0000","last_unstale":"2024-04-27T03:33:21.075726+0000","last_undegraded":"2024-04-27T03:33:21.075726+0000","last_fullsized":"2024-04-27T03:33:21.075726+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'6","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.201209+0000","last_change":"2024-04-27T03:33:48.201236+0000","last_active":"2024-04-27T03:33:48.201209+0000","last_peered":"2024-04-27T03:33:48.201209+0000","last_clean":"2024-04-27T03:33:48.201209+0000","last_became_active":"2024-04-27T03:30:13.387533+0000","last_became_peered":"2024-04-27T03:30:13.387533+0000","last_unstale":"2024-04-27T03:33:48.201209+0000","last_undegraded":"2024-04-27T03:33:48.201209+0000","last_fullsized":"2024-04-27T03:33:48.201209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:48.201118+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:48.201118+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.201118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:48.546003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295161+0000","last_change":"2024-04-27T03:30:11.373706+0000","last_active":"2024-04-27T03:32:24.295161+0000","last_peered":"2024-04-27T03:32:24.295161+0000","last_clean":"2024-04-27T03:32:24.295161+0000","last_became_active":"2024-04-27T03:30:11.373440+0000","last_became_peered":"2024-04-27T03:30:11.373440+0000","last_unstale":"2024-04-27T03:32:24.295161+0000","last_undegraded":"2024-04-27T03:32:24.295161+0000","last_fullsized":"2024-04-27T03:32:24.295161+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.730998+0000","last_change":"2024-04-27T03:33:48.731024+0000","last_active":"2024-04-27T03:33:48.730998+0000","last_peered":"2024-04-27T03:33:48.730998+0000","last_clean":"2024-04-27T03:33:48.730998+0000","last_became_active":"2024-04-27T03:30:15.382913+0000","last_became_peered":"2024-04-27T03:30:15.382913+0000","last_unstale":"2024-04-27T03:33:48.730998+0000","last_undegraded":"2024-04-27T03:33:48.730998+0000","last_fullsized":"2024-04-27T03:33:48.730998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.730891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.730891+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.730891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:49.302463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"27'45","reported_seq":103,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.809157+0000","last_change":"2024-04-27T03:33:45.809220+0000","last_active":"2024-04-27T03:33:45.809157+0000","last_peered":"2024-04-27T03:33:45.809157+0000","last_clean":"2024-04-27T03:33:45.809157+0000","last_became_active":"2024-04-27T03:30:17.381106+0000","last_became_peered":"2024-04-27T03:30:17.381106+0000","last_unstale":"2024-04-27T03:33:45.809157+0000","last_undegraded":"2024-04-27T03:33:45.809157+0000","last_fullsized":"2024-04-27T03:33:45.809157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'45","last_scrub_stamp":"2024-04-27T03:33:45.809061+0000","last_deep_scrub":"27'45","last_deep_scrub_stamp":"2024-04-27T03:33:45.809061+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.809061+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:33.888060+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"27'285","reported_seq":415,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.864438+0000","last_change":"2024-04-27T03:33:31.864480+0000","last_active":"2024-04-27T03:33:31.864438+0000","last_peered":"2024-04-27T03:33:31.864438+0000","last_clean":"2024-04-27T03:33:31.864438+0000","last_became_active":"2024-04-27T03:30:19.403413+0000","last_became_peered":"2024-04-27T03:30:19.403413+0000","last_unstale":"2024-04-27T03:33:31.864438+0000","last_undegraded":"2024-04-27T03:33:31.864438+0000","last_fullsized":"2024-04-27T03:33:31.864438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:31.864348+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:31.864348+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.864348+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:27.874206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"27'14","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.077666+0000","last_change":"2024-04-27T03:30:21.403250+0000","last_active":"2024-04-27T03:33:21.077666+0000","last_peered":"2024-04-27T03:33:21.077666+0000","last_clean":"2024-04-27T03:33:21.077666+0000","last_became_active":"2024-04-27T03:30:21.402102+0000","last_became_peered":"2024-04-27T03:30:21.402102+0000","last_unstale":"2024-04-27T03:33:21.077666+0000","last_undegraded":"2024-04-27T03:33:21.077666+0000","last_fullsized":"2024-04-27T03:33:21.077666+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":23,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'12","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.802576+0000","last_change":"2024-04-27T03:30:13.383203+0000","last_active":"2024-04-27T03:33:20.802576+0000","last_peered":"2024-04-27T03:33:20.802576+0000","last_clean":"2024-04-27T03:33:20.802576+0000","last_became_active":"2024-04-27T03:30:13.382899+0000","last_became_peered":"2024-04-27T03:30:13.382899+0000","last_unstale":"2024-04-27T03:33:20.802576+0000","last_undegraded":"2024-04-27T03:33:20.802576+0000","last_fullsized":"2024-04-27T03:33:20.802576+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.926560+0000","last_change":"2024-04-27T03:31:01.444153+0000","last_active":"2024-04-27T03:33:20.926560+0000","last_peered":"2024-04-27T03:33:20.926560+0000","last_clean":"2024-04-27T03:33:20.926560+0000","last_became_active":"2024-04-27T03:31:01.439776+0000","last_became_peered":"2024-04-27T03:31:01.439776+0000","last_unstale":"2024-04-27T03:33:20.926560+0000","last_undegraded":"2024-04-27T03:33:20.926560+0000","last_fullsized":"2024-04-27T03:33:20.926560+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"27'5","reported_seq":428,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.950334+0000","last_change":"2024-04-27T03:33:43.950380+0000","last_active":"2024-04-27T03:33:43.950334+0000","last_peered":"2024-04-27T03:33:43.950334+0000","last_clean":"2024-04-27T03:33:43.950334+0000","last_became_active":"2024-04-27T03:30:15.382502+0000","last_became_peered":"2024-04-27T03:30:15.382502+0000","last_unstale":"2024-04-27T03:33:43.950334+0000","last_undegraded":"2024-04-27T03:33:43.950334+0000","last_fullsized":"2024-04-27T03:33:43.950334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:43.950258+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:43.950258+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.950258+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:48.355472+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"25'10","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296824+0000","last_change":"2024-04-27T03:30:17.389548+0000","last_active":"2024-04-27T03:32:34.296824+0000","last_peered":"2024-04-27T03:32:34.296824+0000","last_clean":"2024-04-27T03:32:34.296824+0000","last_became_active":"2024-04-27T03:30:17.387739+0000","last_became_peered":"2024-04-27T03:30:17.387739+0000","last_unstale":"2024-04-27T03:32:34.296824+0000","last_undegraded":"2024-04-27T03:32:34.296824+0000","last_fullsized":"2024-04-27T03:32:34.296824+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"27'285","reported_seq":522,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.933904+0000","last_change":"2024-04-27T03:33:32.933929+0000","last_active":"2024-04-27T03:33:32.933904+0000","last_peered":"2024-04-27T03:33:32.933904+0000","last_clean":"2024-04-27T03:33:32.933904+0000","last_became_active":"2024-04-27T03:30:19.402484+0000","last_became_peered":"2024-04-27T03:30:19.402484+0000","last_unstale":"2024-04-27T03:33:32.933904+0000","last_undegraded":"2024-04-27T03:33:32.933904+0000","last_fullsized":"2024-04-27T03:33:32.933904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:32.933800+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:32.933800+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.933800+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:19.924355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"27'4","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295248+0000","last_change":"2024-04-27T03:30:21.408801+0000","last_active":"2024-04-27T03:32:29.295248+0000","last_peered":"2024-04-27T03:32:29.295248+0000","last_clean":"2024-04-27T03:32:29.295248+0000","last_became_active":"2024-04-27T03:30:21.408512+0000","last_became_peered":"2024-04-27T03:30:21.408512+0000","last_unstale":"2024-04-27T03:32:29.295248+0000","last_undegraded":"2024-04-27T03:32:29.295248+0000","last_fullsized":"2024-04-27T03:32:29.295248+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'14","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.802467+0000","last_change":"2024-04-27T03:30:13.381639+0000","last_active":"2024-04-27T03:33:20.802467+0000","last_peered":"2024-04-27T03:33:20.802467+0000","last_clean":"2024-04-27T03:33:20.802467+0000","last_became_active":"2024-04-27T03:30:13.380540+0000","last_became_peered":"2024-04-27T03:30:13.380540+0000","last_unstale":"2024-04-27T03:33:20.802467+0000","last_undegraded":"2024-04-27T03:33:20.802467+0000","last_fullsized":"2024-04-27T03:33:20.802467+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295052+0000","last_change":"2024-04-27T03:30:11.372470+0000","last_active":"2024-04-27T03:32:24.295052+0000","last_peered":"2024-04-27T03:32:24.295052+0000","last_clean":"2024-04-27T03:32:24.295052+0000","last_became_active":"2024-04-27T03:30:11.372193+0000","last_became_peered":"2024-04-27T03:30:11.372193+0000","last_unstale":"2024-04-27T03:32:24.295052+0000","last_undegraded":"2024-04-27T03:32:24.295052+0000","last_fullsized":"2024-04-27T03:32:24.295052+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.115468+0000","last_change":"2024-04-27T03:33:35.115491+0000","last_active":"2024-04-27T03:33:35.115468+0000","last_peered":"2024-04-27T03:33:35.115468+0000","last_clean":"2024-04-27T03:33:35.115468+0000","last_became_active":"2024-04-27T03:30:15.383924+0000","last_became_peered":"2024-04-27T03:30:15.383924+0000","last_unstale":"2024-04-27T03:33:35.115468+0000","last_undegraded":"2024-04-27T03:33:35.115468+0000","last_fullsized":"2024-04-27T03:33:35.115468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:35.115383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:35.115383+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.115383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:37.564842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"27'48","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.883186+0000","last_change":"2024-04-27T03:33:38.883223+0000","last_active":"2024-04-27T03:33:38.883186+0000","last_peered":"2024-04-27T03:33:38.883186+0000","last_clean":"2024-04-27T03:33:38.883186+0000","last_became_active":"2024-04-27T03:30:17.387578+0000","last_became_peered":"2024-04-27T03:30:17.387578+0000","last_unstale":"2024-04-27T03:33:38.883186+0000","last_undegraded":"2024-04-27T03:33:38.883186+0000","last_fullsized":"2024-04-27T03:33:38.883186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-04-27T03:33:38.883110+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-04-27T03:33:38.883110+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.883110+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:11.757254+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.856113+0000","last_change":"2024-04-27T03:33:32.856137+0000","last_active":"2024-04-27T03:33:32.856113+0000","last_peered":"2024-04-27T03:33:32.856113+0000","last_clean":"2024-04-27T03:33:32.856113+0000","last_became_active":"2024-04-27T03:30:19.405512+0000","last_became_peered":"2024-04-27T03:30:19.405512+0000","last_unstale":"2024-04-27T03:33:32.856113+0000","last_undegraded":"2024-04-27T03:33:32.856113+0000","last_fullsized":"2024-04-27T03:33:32.856113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:32.856020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:32.856020+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.856020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:34.194673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"27'14","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.990549+0000","last_change":"2024-04-27T03:33:48.990573+0000","last_active":"2024-04-27T03:33:48.990549+0000","last_peered":"2024-04-27T03:33:48.990549+0000","last_clean":"2024-04-27T03:33:48.990549+0000","last_became_active":"2024-04-27T03:30:21.406883+0000","last_became_peered":"2024-04-27T03:30:21.406883+0000","last_unstale":"2024-04-27T03:33:48.990549+0000","last_undegraded":"2024-04-27T03:33:48.990549+0000","last_fullsized":"2024-04-27T03:33:48.990549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:33:48.990468+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:33:48.990468+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.990468+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:14:34.099716+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":59,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'20","reported_seq":80,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.941983+0000","last_change":"2024-04-27T03:30:13.387655+0000","last_active":"2024-04-27T03:33:20.941983+0000","last_peered":"2024-04-27T03:33:20.941983+0000","last_clean":"2024-04-27T03:33:20.941983+0000","last_became_active":"2024-04-27T03:30:13.387180+0000","last_became_peered":"2024-04-27T03:30:13.387180+0000","last_unstale":"2024-04-27T03:33:20.941983+0000","last_undegraded":"2024-04-27T03:33:20.941983+0000","last_fullsized":"2024-04-27T03:33:20.941983+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":194,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:09.292349+0000","last_change":"2024-04-27T03:30:06.104151+0000","last_active":"2024-04-27T03:32:09.292349+0000","last_peered":"2024-04-27T03:32:09.292349+0000","last_clean":"2024-04-27T03:32:09.292349+0000","last_became_active":"2024-04-27T03:30:06.102893+0000","last_became_peered":"2024-04-27T03:30:06.102893+0000","last_unstale":"2024-04-27T03:32:09.292349+0000","last_undegraded":"2024-04-27T03:32:09.292349+0000","last_fullsized":"2024-04-27T03:32:09.292349+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.293657+0000","last_change":"2024-04-27T03:30:06.106875+0000","last_active":"2024-04-27T03:32:14.293657+0000","last_peered":"2024-04-27T03:32:14.293657+0000","last_clean":"2024-04-27T03:32:14.293657+0000","last_became_active":"2024-04-27T03:30:06.106511+0000","last_became_peered":"2024-04-27T03:30:06.106511+0000","last_unstale":"2024-04-27T03:32:14.293657+0000","last_undegraded":"2024-04-27T03:32:14.293657+0000","last_fullsized":"2024-04-27T03:32:14.293657+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"27'4","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.007675+0000","last_change":"2024-04-27T03:30:11.362699+0000","last_active":"2024-04-27T03:33:21.007675+0000","last_peered":"2024-04-27T03:33:21.007675+0000","last_clean":"2024-04-27T03:33:21.007675+0000","last_became_active":"2024-04-27T03:30:11.362445+0000","last_became_peered":"2024-04-27T03:30:11.362445+0000","last_unstale":"2024-04-27T03:33:21.007675+0000","last_undegraded":"2024-04-27T03:33:21.007675+0000","last_fullsized":"2024-04-27T03:33:21.007675+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294623+0000","last_change":"2024-04-27T03:31:01.441027+0000","last_active":"2024-04-27T03:32:24.294623+0000","last_peered":"2024-04-27T03:32:24.294623+0000","last_clean":"2024-04-27T03:32:24.294623+0000","last_became_active":"2024-04-27T03:31:01.440678+0000","last_became_peered":"2024-04-27T03:31:01.440678+0000","last_unstale":"2024-04-27T03:32:24.294623+0000","last_undegraded":"2024-04-27T03:32:24.294623+0000","last_fullsized":"2024-04-27T03:32:24.294623+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"27'95","reported_seq":191,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.176299+0000","last_change":"2024-04-27T03:33:45.176369+0000","last_active":"2024-04-27T03:33:45.176299+0000","last_peered":"2024-04-27T03:33:45.176299+0000","last_clean":"2024-04-27T03:33:45.176299+0000","last_became_active":"2024-04-27T03:30:17.387707+0000","last_became_peered":"2024-04-27T03:30:17.387707+0000","last_unstale":"2024-04-27T03:33:45.176299+0000","last_undegraded":"2024-04-27T03:33:45.176299+0000","last_fullsized":"2024-04-27T03:33:45.176299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'95","last_scrub_stamp":"2024-04-27T03:33:45.176195+0000","last_deep_scrub":"27'95","last_deep_scrub_stamp":"2024-04-27T03:33:45.176195+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.176195+0000","objects_scrubbed":6,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:45.184076+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":93,"num_read_kb":81,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.779872+0000","last_change":"2024-04-27T03:33:31.779891+0000","last_active":"2024-04-27T03:33:31.779872+0000","last_peered":"2024-04-27T03:33:31.779872+0000","last_clean":"2024-04-27T03:33:31.779872+0000","last_became_active":"2024-04-27T03:30:19.405359+0000","last_became_peered":"2024-04-27T03:30:19.405359+0000","last_unstale":"2024-04-27T03:33:31.779872+0000","last_undegraded":"2024-04-27T03:33:31.779872+0000","last_fullsized":"2024-04-27T03:33:31.779872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:31.779786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:31.779786+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.779786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:45.852820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"27'81","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.838882+0000","last_change":"2024-04-27T03:33:44.838907+0000","last_active":"2024-04-27T03:33:44.838882+0000","last_peered":"2024-04-27T03:33:44.838882+0000","last_clean":"2024-04-27T03:33:44.838882+0000","last_became_active":"2024-04-27T03:30:21.404300+0000","last_became_peered":"2024-04-27T03:30:21.404300+0000","last_unstale":"2024-04-27T03:33:44.838882+0000","last_undegraded":"2024-04-27T03:33:44.838882+0000","last_fullsized":"2024-04-27T03:33:44.838882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'81","last_scrub_stamp":"2024-04-27T03:33:44.838760+0000","last_deep_scrub":"27'81","last_deep_scrub_stamp":"2024-04-27T03:33:44.838760+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.838760+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:18.504969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"25'2","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295248+0000","last_change":"2024-04-27T03:30:13.378248+0000","last_active":"2024-04-27T03:32:24.295248+0000","last_peered":"2024-04-27T03:32:24.295248+0000","last_clean":"2024-04-27T03:32:24.295248+0000","last_became_active":"2024-04-27T03:30:13.377713+0000","last_became_peered":"2024-04-27T03:30:13.377713+0000","last_unstale":"2024-04-27T03:32:24.295248+0000","last_undegraded":"2024-04-27T03:32:24.295248+0000","last_fullsized":"2024-04-27T03:32:24.295248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:09.292899+0000","last_change":"2024-04-27T03:30:06.106890+0000","last_active":"2024-04-27T03:32:09.292899+0000","last_peered":"2024-04-27T03:32:09.292899+0000","last_clean":"2024-04-27T03:32:09.292899+0000","last_became_active":"2024-04-27T03:30:06.106474+0000","last_became_peered":"2024-04-27T03:30:06.106474+0000","last_unstale":"2024-04-27T03:32:09.292899+0000","last_undegraded":"2024-04-27T03:32:09.292899+0000","last_fullsized":"2024-04-27T03:32:09.292899+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"27'4","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288125+0000","last_change":"2024-04-27T03:30:11.360679+0000","last_active":"2024-04-27T03:32:19.288125+0000","last_peered":"2024-04-27T03:32:19.288125+0000","last_clean":"2024-04-27T03:32:19.288125+0000","last_became_active":"2024-04-27T03:30:11.360244+0000","last_became_peered":"2024-04-27T03:30:11.360244+0000","last_unstale":"2024-04-27T03:32:19.288125+0000","last_undegraded":"2024-04-27T03:32:19.288125+0000","last_fullsized":"2024-04-27T03:32:19.288125+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.098611+0000","last_change":"2024-04-27T03:33:42.098651+0000","last_active":"2024-04-27T03:33:42.098611+0000","last_peered":"2024-04-27T03:33:42.098611+0000","last_clean":"2024-04-27T03:33:42.098611+0000","last_became_active":"2024-04-27T03:30:15.379448+0000","last_became_peered":"2024-04-27T03:30:15.379448+0000","last_unstale":"2024-04-27T03:33:42.098611+0000","last_undegraded":"2024-04-27T03:33:42.098611+0000","last_fullsized":"2024-04-27T03:33:42.098611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.098519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.098519+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.098519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:34:23.337627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"27'297","reported_seq":528,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.877296+0000","last_change":"2024-04-27T03:33:35.877345+0000","last_active":"2024-04-27T03:33:35.877296+0000","last_peered":"2024-04-27T03:33:35.877296+0000","last_clean":"2024-04-27T03:33:35.877296+0000","last_became_active":"2024-04-27T03:30:17.390884+0000","last_became_peered":"2024-04-27T03:30:17.390884+0000","last_unstale":"2024-04-27T03:33:35.877296+0000","last_undegraded":"2024-04-27T03:33:35.877296+0000","last_fullsized":"2024-04-27T03:33:35.877296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'297","last_scrub_stamp":"2024-04-27T03:33:35.877210+0000","last_deep_scrub":"27'297","last_deep_scrub_stamp":"2024-04-27T03:33:35.877210+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.877210+0000","objects_scrubbed":11,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:23.956209+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":387,"num_read_kb":371,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295201+0000","last_change":"2024-04-27T03:30:19.408904+0000","last_active":"2024-04-27T03:32:24.295201+0000","last_peered":"2024-04-27T03:32:24.295201+0000","last_clean":"2024-04-27T03:32:24.295201+0000","last_became_active":"2024-04-27T03:30:19.405777+0000","last_became_peered":"2024-04-27T03:30:19.405777+0000","last_unstale":"2024-04-27T03:32:24.295201+0000","last_undegraded":"2024-04-27T03:32:24.295201+0000","last_fullsized":"2024-04-27T03:32:24.295201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"27'7","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.013986+0000","last_change":"2024-04-27T03:30:21.409632+0000","last_active":"2024-04-27T03:33:21.013986+0000","last_peered":"2024-04-27T03:33:21.013986+0000","last_clean":"2024-04-27T03:33:21.013986+0000","last_became_active":"2024-04-27T03:30:21.409359+0000","last_became_peered":"2024-04-27T03:30:21.409359+0000","last_unstale":"2024-04-27T03:33:21.013986+0000","last_undegraded":"2024-04-27T03:33:21.013986+0000","last_fullsized":"2024-04-27T03:33:21.013986+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"25'10","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.289184+0000","last_change":"2024-04-27T03:30:13.385862+0000","last_active":"2024-04-27T03:32:24.289184+0000","last_peered":"2024-04-27T03:32:24.289184+0000","last_clean":"2024-04-27T03:32:24.289184+0000","last_became_active":"2024-04-27T03:30:13.385521+0000","last_became_peered":"2024-04-27T03:30:13.385521+0000","last_unstale":"2024-04-27T03:32:24.289184+0000","last_undegraded":"2024-04-27T03:32:24.289184+0000","last_fullsized":"2024-04-27T03:32:24.289184+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:09.292273+0000","last_change":"2024-04-27T03:30:06.103722+0000","last_active":"2024-04-27T03:32:09.292273+0000","last_peered":"2024-04-27T03:32:09.292273+0000","last_clean":"2024-04-27T03:32:09.292273+0000","last_became_active":"2024-04-27T03:30:06.103441+0000","last_became_peered":"2024-04-27T03:30:06.103441+0000","last_unstale":"2024-04-27T03:32:09.292273+0000","last_undegraded":"2024-04-27T03:32:09.292273+0000","last_fullsized":"2024-04-27T03:32:09.292273+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.287482+0000","last_change":"2024-04-27T03:30:11.360789+0000","last_active":"2024-04-27T03:32:14.287482+0000","last_peered":"2024-04-27T03:32:14.287482+0000","last_clean":"2024-04-27T03:32:14.287482+0000","last_became_active":"2024-04-27T03:30:11.360443+0000","last_became_peered":"2024-04-27T03:30:11.360443+0000","last_unstale":"2024-04-27T03:32:14.287482+0000","last_undegraded":"2024-04-27T03:32:14.287482+0000","last_fullsized":"2024-04-27T03:32:14.287482+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.772880+0000","last_change":"2024-04-27T03:33:46.772935+0000","last_active":"2024-04-27T03:33:46.772880+0000","last_peered":"2024-04-27T03:33:46.772880+0000","last_clean":"2024-04-27T03:33:46.772880+0000","last_became_active":"2024-04-27T03:30:15.383213+0000","last_became_peered":"2024-04-27T03:30:15.383213+0000","last_unstale":"2024-04-27T03:33:46.772880+0000","last_undegraded":"2024-04-27T03:33:46.772880+0000","last_fullsized":"2024-04-27T03:33:46.772880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:46.772765+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:46.772765+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.772765+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:24.897665+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"27'23","reported_seq":100,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.859478+0000","last_change":"2024-04-27T03:33:42.859533+0000","last_active":"2024-04-27T03:33:42.859478+0000","last_peered":"2024-04-27T03:33:42.859478+0000","last_clean":"2024-04-27T03:33:42.859478+0000","last_became_active":"2024-04-27T03:30:17.388588+0000","last_became_peered":"2024-04-27T03:30:17.388588+0000","last_unstale":"2024-04-27T03:33:42.859478+0000","last_undegraded":"2024-04-27T03:33:42.859478+0000","last_fullsized":"2024-04-27T03:33:42.859478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-04-27T03:33:42.859384+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-04-27T03:33:42.859384+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.859384+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:29.849938+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.729956+0000","last_change":"2024-04-27T03:33:46.729982+0000","last_active":"2024-04-27T03:33:46.729956+0000","last_peered":"2024-04-27T03:33:46.729956+0000","last_clean":"2024-04-27T03:33:46.729956+0000","last_became_active":"2024-04-27T03:30:19.407004+0000","last_became_peered":"2024-04-27T03:30:19.407004+0000","last_unstale":"2024-04-27T03:33:46.729956+0000","last_undegraded":"2024-04-27T03:33:46.729956+0000","last_fullsized":"2024-04-27T03:33:46.729956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:46.729851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:46.729851+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.729851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:36.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"25'2","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297632+0000","last_change":"2024-04-27T03:30:21.417022+0000","last_active":"2024-04-27T03:32:39.297632+0000","last_peered":"2024-04-27T03:32:39.297632+0000","last_clean":"2024-04-27T03:32:39.297632+0000","last_became_active":"2024-04-27T03:30:21.416389+0000","last_became_peered":"2024-04-27T03:30:21.416389+0000","last_unstale":"2024-04-27T03:32:39.297632+0000","last_undegraded":"2024-04-27T03:32:39.297632+0000","last_fullsized":"2024-04-27T03:32:39.297632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'8","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.704007+0000","last_change":"2024-04-27T03:30:13.369982+0000","last_active":"2024-04-27T03:33:20.704007+0000","last_peered":"2024-04-27T03:33:20.704007+0000","last_clean":"2024-04-27T03:33:20.704007+0000","last_became_active":"2024-04-27T03:30:13.369638+0000","last_became_peered":"2024-04-27T03:30:13.369638+0000","last_unstale":"2024-04-27T03:33:20.704007+0000","last_undegraded":"2024-04-27T03:33:20.704007+0000","last_fullsized":"2024-04-27T03:33:20.704007+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:09.292988+0000","last_change":"2024-04-27T03:30:06.107772+0000","last_active":"2024-04-27T03:32:09.292988+0000","last_peered":"2024-04-27T03:32:09.292988+0000","last_clean":"2024-04-27T03:32:09.292988+0000","last_became_active":"2024-04-27T03:30:06.107356+0000","last_became_peered":"2024-04-27T03:30:06.107356+0000","last_unstale":"2024-04-27T03:32:09.292988+0000","last_undegraded":"2024-04-27T03:32:09.292988+0000","last_fullsized":"2024-04-27T03:32:09.292988+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.293662+0000","last_change":"2024-04-27T03:30:11.357072+0000","last_active":"2024-04-27T03:32:19.293662+0000","last_peered":"2024-04-27T03:32:19.293662+0000","last_clean":"2024-04-27T03:32:19.293662+0000","last_became_active":"2024-04-27T03:30:11.356685+0000","last_became_peered":"2024-04-27T03:30:11.356685+0000","last_unstale":"2024-04-27T03:32:19.293662+0000","last_undegraded":"2024-04-27T03:32:19.293662+0000","last_fullsized":"2024-04-27T03:32:19.293662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.289129+0000","last_change":"2024-04-27T03:30:15.382673+0000","last_active":"2024-04-27T03:32:24.289129+0000","last_peered":"2024-04-27T03:32:24.289129+0000","last_clean":"2024-04-27T03:32:24.289129+0000","last_became_active":"2024-04-27T03:30:15.382326+0000","last_became_peered":"2024-04-27T03:30:15.382326+0000","last_unstale":"2024-04-27T03:32:24.289129+0000","last_undegraded":"2024-04-27T03:32:24.289129+0000","last_fullsized":"2024-04-27T03:32:24.289129+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"27'13","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.336717+0000","last_change":"2024-04-27T03:30:17.390316+0000","last_active":"2024-04-27T03:33:20.336717+0000","last_peered":"2024-04-27T03:33:20.336717+0000","last_clean":"2024-04-27T03:33:20.336717+0000","last_became_active":"2024-04-27T03:30:17.389373+0000","last_became_peered":"2024-04-27T03:30:17.389373+0000","last_unstale":"2024-04-27T03:33:20.336717+0000","last_undegraded":"2024-04-27T03:33:20.336717+0000","last_fullsized":"2024-04-27T03:33:20.336717+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295992+0000","last_change":"2024-04-27T03:30:19.406634+0000","last_active":"2024-04-27T03:32:29.295992+0000","last_peered":"2024-04-27T03:32:29.295992+0000","last_clean":"2024-04-27T03:32:29.295992+0000","last_became_active":"2024-04-27T03:30:19.405953+0000","last_became_peered":"2024-04-27T03:30:19.405953+0000","last_unstale":"2024-04-27T03:32:29.295992+0000","last_undegraded":"2024-04-27T03:32:29.295992+0000","last_fullsized":"2024-04-27T03:32:29.295992+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"27'6","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.137122+0000","last_change":"2024-04-27T03:30:21.413040+0000","last_active":"2024-04-27T03:33:21.137122+0000","last_peered":"2024-04-27T03:33:21.137122+0000","last_clean":"2024-04-27T03:33:21.137122+0000","last_became_active":"2024-04-27T03:30:21.412790+0000","last_became_peered":"2024-04-27T03:30:21.412790+0000","last_unstale":"2024-04-27T03:33:21.137122+0000","last_undegraded":"2024-04-27T03:33:21.137122+0000","last_fullsized":"2024-04-27T03:33:21.137122+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'18","reported_seq":87,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.940786+0000","last_change":"2024-04-27T03:30:13.381418+0000","last_active":"2024-04-27T03:33:20.940786+0000","last_peered":"2024-04-27T03:33:20.940786+0000","last_clean":"2024-04-27T03:33:20.940786+0000","last_became_active":"2024-04-27T03:30:13.380888+0000","last_became_peered":"2024-04-27T03:30:13.380888+0000","last_unstale":"2024-04-27T03:33:20.940786+0000","last_undegraded":"2024-04-27T03:33:20.940786+0000","last_fullsized":"2024-04-27T03:33:20.940786+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":73,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.293716+0000","last_change":"2024-04-27T03:30:11.362550+0000","last_active":"2024-04-27T03:32:19.293716+0000","last_peered":"2024-04-27T03:32:19.293716+0000","last_clean":"2024-04-27T03:32:19.293716+0000","last_became_active":"2024-04-27T03:30:11.361786+0000","last_became_peered":"2024-04-27T03:30:11.361786+0000","last_unstale":"2024-04-27T03:32:19.293716+0000","last_undegraded":"2024-04-27T03:32:19.293716+0000","last_fullsized":"2024-04-27T03:32:19.293716+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.287557+0000","last_change":"2024-04-27T03:30:06.107861+0000","last_active":"2024-04-27T03:32:14.287557+0000","last_peered":"2024-04-27T03:32:14.287557+0000","last_clean":"2024-04-27T03:32:14.287557+0000","last_became_active":"2024-04-27T03:30:06.107594+0000","last_became_peered":"2024-04-27T03:30:06.107594+0000","last_unstale":"2024-04-27T03:32:14.287557+0000","last_undegraded":"2024-04-27T03:32:14.287557+0000","last_fullsized":"2024-04-27T03:32:14.287557+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"27'1","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.553388+0000","last_change":"2024-04-27T03:30:15.381770+0000","last_active":"2024-04-27T03:33:27.553388+0000","last_peered":"2024-04-27T03:33:27.553388+0000","last_clean":"2024-04-27T03:33:27.553388+0000","last_became_active":"2024-04-27T03:30:15.381392+0000","last_became_peered":"2024-04-27T03:30:15.381392+0000","last_unstale":"2024-04-27T03:33:27.553388+0000","last_undegraded":"2024-04-27T03:33:27.553388+0000","last_fullsized":"2024-04-27T03:33:27.553388+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"27'87","reported_seq":175,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.739920+0000","last_change":"2024-04-27T03:33:39.739991+0000","last_active":"2024-04-27T03:33:39.739920+0000","last_peered":"2024-04-27T03:33:39.739920+0000","last_clean":"2024-04-27T03:33:39.739920+0000","last_became_active":"2024-04-27T03:30:17.390409+0000","last_became_peered":"2024-04-27T03:30:17.390409+0000","last_unstale":"2024-04-27T03:33:39.739920+0000","last_undegraded":"2024-04-27T03:33:39.739920+0000","last_fullsized":"2024-04-27T03:33:39.739920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'87","last_scrub_stamp":"2024-04-27T03:33:39.739797+0000","last_deep_scrub":"27'87","last_deep_scrub_stamp":"2024-04-27T03:33:39.739797+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.739797+0000","objects_scrubbed":11,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:27:02.606163+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":89,"num_read_kb":75,"num_write":42,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":292,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.814593+0000","last_change":"2024-04-27T03:33:33.814624+0000","last_active":"2024-04-27T03:33:33.814593+0000","last_peered":"2024-04-27T03:33:33.814593+0000","last_clean":"2024-04-27T03:33:33.814593+0000","last_became_active":"2024-04-27T03:30:19.402977+0000","last_became_peered":"2024-04-27T03:30:19.402977+0000","last_unstale":"2024-04-27T03:33:33.814593+0000","last_undegraded":"2024-04-27T03:33:33.814593+0000","last_fullsized":"2024-04-27T03:33:33.814593+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:33.814493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:33.814493+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.814493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:14:40.755714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"27'11","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.990016+0000","last_change":"2024-04-27T03:30:21.410763+0000","last_active":"2024-04-27T03:33:20.990016+0000","last_peered":"2024-04-27T03:33:20.990016+0000","last_clean":"2024-04-27T03:33:20.990016+0000","last_became_active":"2024-04-27T03:30:21.410511+0000","last_became_peered":"2024-04-27T03:30:21.410511+0000","last_unstale":"2024-04-27T03:33:20.990016+0000","last_undegraded":"2024-04-27T03:33:20.990016+0000","last_fullsized":"2024-04-27T03:33:20.990016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"27'3","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.072466+0000","last_change":"2024-04-27T03:30:13.380442+0000","last_active":"2024-04-27T03:33:21.072466+0000","last_peered":"2024-04-27T03:33:21.072466+0000","last_clean":"2024-04-27T03:33:21.072466+0000","last_became_active":"2024-04-27T03:30:13.379985+0000","last_became_peered":"2024-04-27T03:30:13.379985+0000","last_unstale":"2024-04-27T03:33:21.072466+0000","last_undegraded":"2024-04-27T03:33:21.072466+0000","last_fullsized":"2024-04-27T03:33:21.072466+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.688977+0000","last_change":"2024-04-27T03:30:11.361129+0000","last_active":"2024-04-27T03:33:27.688977+0000","last_peered":"2024-04-27T03:33:27.688977+0000","last_clean":"2024-04-27T03:33:27.688977+0000","last_became_active":"2024-04-27T03:30:11.360352+0000","last_became_peered":"2024-04-27T03:30:11.360352+0000","last_unstale":"2024-04-27T03:33:27.688977+0000","last_undegraded":"2024-04-27T03:33:27.688977+0000","last_fullsized":"2024-04-27T03:33:27.688977+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.293704+0000","last_change":"2024-04-27T03:30:06.106816+0000","last_active":"2024-04-27T03:32:14.293704+0000","last_peered":"2024-04-27T03:32:14.293704+0000","last_clean":"2024-04-27T03:32:14.293704+0000","last_became_active":"2024-04-27T03:30:06.106475+0000","last_became_peered":"2024-04-27T03:30:06.106475+0000","last_unstale":"2024-04-27T03:32:14.293704+0000","last_undegraded":"2024-04-27T03:32:14.293704+0000","last_fullsized":"2024-04-27T03:32:14.293704+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.089676+0000","last_change":"2024-04-27T03:33:41.089706+0000","last_active":"2024-04-27T03:33:41.089676+0000","last_peered":"2024-04-27T03:33:41.089676+0000","last_clean":"2024-04-27T03:33:41.089676+0000","last_became_active":"2024-04-27T03:30:15.380030+0000","last_became_peered":"2024-04-27T03:30:15.380030+0000","last_unstale":"2024-04-27T03:33:41.089676+0000","last_undegraded":"2024-04-27T03:33:41.089676+0000","last_fullsized":"2024-04-27T03:33:41.089676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:41.089578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:41.089578+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.089578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:36.584947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"27'28","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.880878+0000","last_change":"2024-04-27T03:33:34.880920+0000","last_active":"2024-04-27T03:33:34.880878+0000","last_peered":"2024-04-27T03:33:34.880878+0000","last_clean":"2024-04-27T03:33:34.880878+0000","last_became_active":"2024-04-27T03:30:17.393253+0000","last_became_peered":"2024-04-27T03:30:17.393253+0000","last_unstale":"2024-04-27T03:33:34.880878+0000","last_undegraded":"2024-04-27T03:33:34.880878+0000","last_fullsized":"2024-04-27T03:33:34.880878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-04-27T03:33:34.880792+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-04-27T03:33:34.880792+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.880792+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:08.547001+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"27'285","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.921391+0000","last_change":"2024-04-27T03:33:33.921440+0000","last_active":"2024-04-27T03:33:33.921391+0000","last_peered":"2024-04-27T03:33:33.921391+0000","last_clean":"2024-04-27T03:33:33.921391+0000","last_became_active":"2024-04-27T03:30:19.402319+0000","last_became_peered":"2024-04-27T03:30:19.402319+0000","last_unstale":"2024-04-27T03:33:33.921391+0000","last_undegraded":"2024-04-27T03:33:33.921391+0000","last_fullsized":"2024-04-27T03:33:33.921391+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:33.921301+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:33.921301+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.921301+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:42.843568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297804+0000","last_change":"2024-04-27T03:30:21.414388+0000","last_active":"2024-04-27T03:32:39.297804+0000","last_peered":"2024-04-27T03:32:39.297804+0000","last_clean":"2024-04-27T03:32:39.297804+0000","last_became_active":"2024-04-27T03:30:21.414109+0000","last_became_peered":"2024-04-27T03:30:21.414109+0000","last_unstale":"2024-04-27T03:32:39.297804+0000","last_undegraded":"2024-04-27T03:32:39.297804+0000","last_fullsized":"2024-04-27T03:32:39.297804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.192285+0000","last_change":"2024-04-27T03:33:46.192314+0000","last_active":"2024-04-27T03:33:46.192285+0000","last_peered":"2024-04-27T03:33:46.192285+0000","last_clean":"2024-04-27T03:33:46.192285+0000","last_became_active":"2024-04-27T03:30:13.387764+0000","last_became_peered":"2024-04-27T03:30:13.387764+0000","last_unstale":"2024-04-27T03:33:46.192285+0000","last_undegraded":"2024-04-27T03:33:46.192285+0000","last_fullsized":"2024-04-27T03:33:46.192285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:33:46.192183+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:33:46.192183+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.192183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:21:54.976582+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.921434+0000","last_change":"2024-04-27T03:30:11.356555+0000","last_active":"2024-04-27T03:33:20.921434+0000","last_peered":"2024-04-27T03:33:20.921434+0000","last_clean":"2024-04-27T03:33:20.921434+0000","last_became_active":"2024-04-27T03:30:11.355704+0000","last_became_peered":"2024-04-27T03:30:11.355704+0000","last_unstale":"2024-04-27T03:33:20.921434+0000","last_undegraded":"2024-04-27T03:33:20.921434+0000","last_fullsized":"2024-04-27T03:33:20.921434+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.287594+0000","last_change":"2024-04-27T03:30:06.104067+0000","last_active":"2024-04-27T03:32:14.287594+0000","last_peered":"2024-04-27T03:32:14.287594+0000","last_clean":"2024-04-27T03:32:14.287594+0000","last_became_active":"2024-04-27T03:30:06.103759+0000","last_became_peered":"2024-04-27T03:30:06.103759+0000","last_unstale":"2024-04-27T03:32:14.287594+0000","last_undegraded":"2024-04-27T03:32:14.287594+0000","last_fullsized":"2024-04-27T03:32:14.287594+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"27'5","reported_seq":238,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.180866+0000","last_change":"2024-04-27T03:33:32.180895+0000","last_active":"2024-04-27T03:33:32.180866+0000","last_peered":"2024-04-27T03:33:32.180866+0000","last_clean":"2024-04-27T03:33:32.180866+0000","last_became_active":"2024-04-27T03:30:15.384372+0000","last_became_peered":"2024-04-27T03:30:15.384372+0000","last_unstale":"2024-04-27T03:33:32.180866+0000","last_undegraded":"2024-04-27T03:33:32.180866+0000","last_fullsized":"2024-04-27T03:33:32.180866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:32.180778+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:32.180778+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.180778+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:01:22.337645+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'20","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.021661+0000","last_change":"2024-04-27T03:30:17.389511+0000","last_active":"2024-04-27T03:33:21.021661+0000","last_peered":"2024-04-27T03:33:21.021661+0000","last_clean":"2024-04-27T03:33:21.021661+0000","last_became_active":"2024-04-27T03:30:17.389154+0000","last_became_peered":"2024-04-27T03:30:17.389154+0000","last_unstale":"2024-04-27T03:33:21.021661+0000","last_undegraded":"2024-04-27T03:33:21.021661+0000","last_fullsized":"2024-04-27T03:33:21.021661+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"27'285","reported_seq":410,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.780298+0000","last_change":"2024-04-27T03:33:33.780327+0000","last_active":"2024-04-27T03:33:33.780298+0000","last_peered":"2024-04-27T03:33:33.780298+0000","last_clean":"2024-04-27T03:33:33.780298+0000","last_became_active":"2024-04-27T03:30:19.404867+0000","last_became_peered":"2024-04-27T03:30:19.404867+0000","last_unstale":"2024-04-27T03:33:33.780298+0000","last_undegraded":"2024-04-27T03:33:33.780298+0000","last_fullsized":"2024-04-27T03:33:33.780298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:33.780200+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:33.780200+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.780200+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:42.702468+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"25'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297292+0000","last_change":"2024-04-27T03:30:21.413577+0000","last_active":"2024-04-27T03:32:39.297292+0000","last_peered":"2024-04-27T03:32:39.297292+0000","last_clean":"2024-04-27T03:32:39.297292+0000","last_became_active":"2024-04-27T03:30:21.413264+0000","last_became_peered":"2024-04-27T03:30:21.413264+0000","last_unstale":"2024-04-27T03:32:39.297292+0000","last_undegraded":"2024-04-27T03:32:39.297292+0000","last_fullsized":"2024-04-27T03:32:39.297292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"27'4","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.703816+0000","last_change":"2024-04-27T03:30:13.383496+0000","last_active":"2024-04-27T03:33:20.703816+0000","last_peered":"2024-04-27T03:33:20.703816+0000","last_clean":"2024-04-27T03:33:20.703816+0000","last_became_active":"2024-04-27T03:30:13.383208+0000","last_became_peered":"2024-04-27T03:30:13.383208+0000","last_unstale":"2024-04-27T03:33:20.703816+0000","last_undegraded":"2024-04-27T03:33:20.703816+0000","last_fullsized":"2024-04-27T03:33:20.703816+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294682+0000","last_change":"2024-04-27T03:30:11.370648+0000","last_active":"2024-04-27T03:32:24.294682+0000","last_peered":"2024-04-27T03:32:24.294682+0000","last_clean":"2024-04-27T03:32:24.294682+0000","last_became_active":"2024-04-27T03:30:11.369568+0000","last_became_peered":"2024-04-27T03:30:11.369568+0000","last_unstale":"2024-04-27T03:32:24.294682+0000","last_undegraded":"2024-04-27T03:32:24.294682+0000","last_fullsized":"2024-04-27T03:32:24.294682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.293751+0000","last_change":"2024-04-27T03:30:06.107954+0000","last_active":"2024-04-27T03:32:14.293751+0000","last_peered":"2024-04-27T03:32:14.293751+0000","last_clean":"2024-04-27T03:32:14.293751+0000","last_became_active":"2024-04-27T03:30:06.107600+0000","last_became_peered":"2024-04-27T03:30:06.107600+0000","last_unstale":"2024-04-27T03:32:14.293751+0000","last_undegraded":"2024-04-27T03:32:14.293751+0000","last_fullsized":"2024-04-27T03:32:14.293751+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.124770+0000","last_change":"2024-04-27T03:33:40.124794+0000","last_active":"2024-04-27T03:33:40.124770+0000","last_peered":"2024-04-27T03:33:40.124770+0000","last_clean":"2024-04-27T03:33:40.124770+0000","last_became_active":"2024-04-27T03:30:15.380598+0000","last_became_peered":"2024-04-27T03:30:15.380598+0000","last_unstale":"2024-04-27T03:33:40.124770+0000","last_undegraded":"2024-04-27T03:33:40.124770+0000","last_fullsized":"2024-04-27T03:33:40.124770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:40.124684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:40.124684+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.124684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:12.385924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"27'104","reported_seq":203,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.834918+0000","last_change":"2024-04-27T03:33:39.834973+0000","last_active":"2024-04-27T03:33:39.834918+0000","last_peered":"2024-04-27T03:33:39.834918+0000","last_clean":"2024-04-27T03:33:39.834918+0000","last_became_active":"2024-04-27T03:30:17.391191+0000","last_became_peered":"2024-04-27T03:30:17.391191+0000","last_unstale":"2024-04-27T03:33:39.834918+0000","last_undegraded":"2024-04-27T03:33:39.834918+0000","last_fullsized":"2024-04-27T03:33:39.834918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'104","last_scrub_stamp":"2024-04-27T03:33:39.834804+0000","last_deep_scrub":"27'104","last_deep_scrub_stamp":"2024-04-27T03:33:39.834804+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.834804+0000","objects_scrubbed":9,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:46.431245+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":102,"num_read_kb":87,"num_write":40,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.744748+0000","last_change":"2024-04-27T03:33:36.744784+0000","last_active":"2024-04-27T03:33:36.744748+0000","last_peered":"2024-04-27T03:33:36.744748+0000","last_clean":"2024-04-27T03:33:36.744748+0000","last_became_active":"2024-04-27T03:30:19.404187+0000","last_became_peered":"2024-04-27T03:30:19.404187+0000","last_unstale":"2024-04-27T03:33:36.744748+0000","last_undegraded":"2024-04-27T03:33:36.744748+0000","last_fullsized":"2024-04-27T03:33:36.744748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.744626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.744626+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.744626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:14.869527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.769625+0000","last_change":"2024-04-27T03:33:48.769648+0000","last_active":"2024-04-27T03:33:48.769625+0000","last_peered":"2024-04-27T03:33:48.769625+0000","last_clean":"2024-04-27T03:33:48.769625+0000","last_became_active":"2024-04-27T03:30:21.407857+0000","last_became_peered":"2024-04-27T03:30:21.407857+0000","last_unstale":"2024-04-27T03:33:48.769625+0000","last_undegraded":"2024-04-27T03:33:48.769625+0000","last_fullsized":"2024-04-27T03:33:48.769625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.769538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.769538+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.769538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:21.643681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"25'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295427+0000","last_change":"2024-04-27T03:30:21.411358+0000","last_active":"2024-04-27T03:32:29.295427+0000","last_peered":"2024-04-27T03:32:29.295427+0000","last_clean":"2024-04-27T03:32:29.295427+0000","last_became_active":"2024-04-27T03:30:21.411054+0000","last_became_peered":"2024-04-27T03:30:21.411054+0000","last_unstale":"2024-04-27T03:32:29.295427+0000","last_undegraded":"2024-04-27T03:32:29.295427+0000","last_fullsized":"2024-04-27T03:32:29.295427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"27'6","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296298+0000","last_change":"2024-04-27T03:30:13.383003+0000","last_active":"2024-04-27T03:32:34.296298+0000","last_peered":"2024-04-27T03:32:34.296298+0000","last_clean":"2024-04-27T03:32:34.296298+0000","last_became_active":"2024-04-27T03:30:13.382675+0000","last_became_peered":"2024-04-27T03:30:13.382675+0000","last_unstale":"2024-04-27T03:32:34.296298+0000","last_undegraded":"2024-04-27T03:32:34.296298+0000","last_fullsized":"2024-04-27T03:32:34.296298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296814+0000","last_change":"2024-04-27T03:30:11.359531+0000","last_active":"2024-04-27T03:32:34.296814+0000","last_peered":"2024-04-27T03:32:34.296814+0000","last_clean":"2024-04-27T03:32:34.296814+0000","last_became_active":"2024-04-27T03:30:11.359195+0000","last_became_peered":"2024-04-27T03:30:11.359195+0000","last_unstale":"2024-04-27T03:32:34.296814+0000","last_undegraded":"2024-04-27T03:32:34.296814+0000","last_fullsized":"2024-04-27T03:32:34.296814+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297265+0000","last_change":"2024-04-27T03:30:15.379854+0000","last_active":"2024-04-27T03:32:39.297265+0000","last_peered":"2024-04-27T03:32:39.297265+0000","last_clean":"2024-04-27T03:32:39.297265+0000","last_became_active":"2024-04-27T03:30:15.379541+0000","last_became_peered":"2024-04-27T03:30:15.379541+0000","last_unstale":"2024-04-27T03:32:39.297265+0000","last_undegraded":"2024-04-27T03:32:39.297265+0000","last_fullsized":"2024-04-27T03:32:39.297265+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"27'46","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.177061+0000","last_change":"2024-04-27T03:33:37.177105+0000","last_active":"2024-04-27T03:33:37.177061+0000","last_peered":"2024-04-27T03:33:37.177061+0000","last_clean":"2024-04-27T03:33:37.177061+0000","last_became_active":"2024-04-27T03:30:17.386496+0000","last_became_peered":"2024-04-27T03:30:17.386496+0000","last_unstale":"2024-04-27T03:33:37.177061+0000","last_undegraded":"2024-04-27T03:33:37.177061+0000","last_fullsized":"2024-04-27T03:33:37.177061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:37.176970+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:37.176970+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.176970+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:15.021583+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297552+0000","last_change":"2024-04-27T03:30:19.405864+0000","last_active":"2024-04-27T03:32:39.297552+0000","last_peered":"2024-04-27T03:32:39.297552+0000","last_clean":"2024-04-27T03:32:39.297552+0000","last_became_active":"2024-04-27T03:30:19.405189+0000","last_became_peered":"2024-04-27T03:30:19.405189+0000","last_unstale":"2024-04-27T03:32:39.297552+0000","last_undegraded":"2024-04-27T03:32:39.297552+0000","last_fullsized":"2024-04-27T03:32:39.297552+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"27'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.289139+0000","last_change":"2024-04-27T03:30:21.414203+0000","last_active":"2024-04-27T03:32:24.289139+0000","last_peered":"2024-04-27T03:32:24.289139+0000","last_clean":"2024-04-27T03:32:24.289139+0000","last_became_active":"2024-04-27T03:30:21.413920+0000","last_became_peered":"2024-04-27T03:30:21.413920+0000","last_unstale":"2024-04-27T03:32:24.289139+0000","last_undegraded":"2024-04-27T03:32:24.289139+0000","last_fullsized":"2024-04-27T03:32:24.289139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"27'16","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.989051+0000","last_change":"2024-04-27T03:30:13.381187+0000","last_active":"2024-04-27T03:33:20.989051+0000","last_peered":"2024-04-27T03:33:20.989051+0000","last_clean":"2024-04-27T03:33:20.989051+0000","last_became_active":"2024-04-27T03:30:13.380240+0000","last_became_peered":"2024-04-27T03:30:13.380240+0000","last_unstale":"2024-04-27T03:33:20.989051+0000","last_undegraded":"2024-04-27T03:33:20.989051+0000","last_fullsized":"2024-04-27T03:33:20.989051+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":31,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297331+0000","last_change":"2024-04-27T03:30:11.361106+0000","last_active":"2024-04-27T03:32:39.297331+0000","last_peered":"2024-04-27T03:32:39.297331+0000","last_clean":"2024-04-27T03:32:39.297331+0000","last_became_active":"2024-04-27T03:30:11.360217+0000","last_became_peered":"2024-04-27T03:30:11.360217+0000","last_unstale":"2024-04-27T03:32:39.297331+0000","last_undegraded":"2024-04-27T03:32:39.297331+0000","last_fullsized":"2024-04-27T03:32:39.297331+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.125251+0000","last_change":"2024-04-27T03:33:36.125275+0000","last_active":"2024-04-27T03:33:36.125251+0000","last_peered":"2024-04-27T03:33:36.125251+0000","last_clean":"2024-04-27T03:33:36.125251+0000","last_became_active":"2024-04-27T03:30:15.381507+0000","last_became_peered":"2024-04-27T03:30:15.381507+0000","last_unstale":"2024-04-27T03:33:36.125251+0000","last_undegraded":"2024-04-27T03:33:36.125251+0000","last_fullsized":"2024-04-27T03:33:36.125251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.125165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.125165+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.125165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:43.471223+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"27'9","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.702947+0000","last_change":"2024-04-27T03:33:44.703001+0000","last_active":"2024-04-27T03:33:44.702947+0000","last_peered":"2024-04-27T03:33:44.702947+0000","last_clean":"2024-04-27T03:33:44.702947+0000","last_became_active":"2024-04-27T03:30:17.385318+0000","last_became_peered":"2024-04-27T03:30:17.385318+0000","last_unstale":"2024-04-27T03:33:44.702947+0000","last_undegraded":"2024-04-27T03:33:44.702947+0000","last_fullsized":"2024-04-27T03:33:44.702947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-04-27T03:33:44.702827+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-04-27T03:33:44.702827+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.702827+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:55.754975+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.873119+0000","last_change":"2024-04-27T03:33:36.873142+0000","last_active":"2024-04-27T03:33:36.873119+0000","last_peered":"2024-04-27T03:33:36.873119+0000","last_clean":"2024-04-27T03:33:36.873119+0000","last_became_active":"2024-04-27T03:30:19.406736+0000","last_became_peered":"2024-04-27T03:30:19.406736+0000","last_unstale":"2024-04-27T03:33:36.873119+0000","last_undegraded":"2024-04-27T03:33:36.873119+0000","last_fullsized":"2024-04-27T03:33:36.873119+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.873031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.873031+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.873031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:57.751682+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"27'4","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.805997+0000","last_change":"2024-04-27T03:30:21.408419+0000","last_active":"2024-04-27T03:33:20.805997+0000","last_peered":"2024-04-27T03:33:20.805997+0000","last_clean":"2024-04-27T03:33:20.805997+0000","last_became_active":"2024-04-27T03:30:21.408079+0000","last_became_peered":"2024-04-27T03:30:21.408079+0000","last_unstale":"2024-04-27T03:33:20.805997+0000","last_undegraded":"2024-04-27T03:33:20.805997+0000","last_fullsized":"2024-04-27T03:33:20.805997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"27'4","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.019125+0000","last_change":"2024-04-27T03:30:13.375656+0000","last_active":"2024-04-27T03:33:21.019125+0000","last_peered":"2024-04-27T03:33:21.019125+0000","last_clean":"2024-04-27T03:33:21.019125+0000","last_became_active":"2024-04-27T03:30:13.375350+0000","last_became_peered":"2024-04-27T03:30:13.375350+0000","last_unstale":"2024-04-27T03:33:21.019125+0000","last_undegraded":"2024-04-27T03:33:21.019125+0000","last_fullsized":"2024-04-27T03:33:21.019125+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297355+0000","last_change":"2024-04-27T03:30:11.361187+0000","last_active":"2024-04-27T03:32:39.297355+0000","last_peered":"2024-04-27T03:32:39.297355+0000","last_clean":"2024-04-27T03:32:39.297355+0000","last_became_active":"2024-04-27T03:30:11.360938+0000","last_became_peered":"2024-04-27T03:30:11.360938+0000","last_unstale":"2024-04-27T03:32:39.297355+0000","last_undegraded":"2024-04-27T03:32:39.297355+0000","last_fullsized":"2024-04-27T03:32:39.297355+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.767320+0000","last_change":"2024-04-27T03:33:47.767359+0000","last_active":"2024-04-27T03:33:47.767320+0000","last_peered":"2024-04-27T03:33:47.767320+0000","last_clean":"2024-04-27T03:33:47.767320+0000","last_became_active":"2024-04-27T03:30:15.382287+0000","last_became_peered":"2024-04-27T03:30:15.382287+0000","last_unstale":"2024-04-27T03:33:47.767320+0000","last_undegraded":"2024-04-27T03:33:47.767320+0000","last_fullsized":"2024-04-27T03:33:47.767320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:47.767216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:47.767216+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.767216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:52.591536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"27'47","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.768186+0000","last_change":"2024-04-27T03:33:41.768233+0000","last_active":"2024-04-27T03:33:41.768186+0000","last_peered":"2024-04-27T03:33:41.768186+0000","last_clean":"2024-04-27T03:33:41.768186+0000","last_became_active":"2024-04-27T03:30:17.388260+0000","last_became_peered":"2024-04-27T03:30:17.388260+0000","last_unstale":"2024-04-27T03:33:41.768186+0000","last_undegraded":"2024-04-27T03:33:41.768186+0000","last_fullsized":"2024-04-27T03:33:41.768186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-04-27T03:33:41.768105+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-04-27T03:33:41.768105+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.768105+0000","objects_scrubbed":6,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:48.417823+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":662,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.927890+0000","last_change":"2024-04-27T03:33:31.927916+0000","last_active":"2024-04-27T03:33:31.927890+0000","last_peered":"2024-04-27T03:33:31.927890+0000","last_clean":"2024-04-27T03:33:31.927890+0000","last_became_active":"2024-04-27T03:30:19.402430+0000","last_became_peered":"2024-04-27T03:30:19.402430+0000","last_unstale":"2024-04-27T03:33:31.927890+0000","last_undegraded":"2024-04-27T03:33:31.927890+0000","last_fullsized":"2024-04-27T03:33:31.927890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:31.927798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:31.927798+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.927798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:36.369347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:54.300287+0000","last_change":"2024-04-27T03:30:21.416316+0000","last_active":"2024-04-27T03:32:54.300287+0000","last_peered":"2024-04-27T03:32:54.300287+0000","last_clean":"2024-04-27T03:32:54.300287+0000","last_became_active":"2024-04-27T03:30:21.415984+0000","last_became_peered":"2024-04-27T03:30:21.415984+0000","last_unstale":"2024-04-27T03:32:54.300287+0000","last_undegraded":"2024-04-27T03:32:54.300287+0000","last_fullsized":"2024-04-27T03:32:54.300287+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"27'5","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.072429+0000","last_change":"2024-04-27T03:30:13.387737+0000","last_active":"2024-04-27T03:33:21.072429+0000","last_peered":"2024-04-27T03:33:21.072429+0000","last_clean":"2024-04-27T03:33:21.072429+0000","last_became_active":"2024-04-27T03:30:13.386985+0000","last_became_peered":"2024-04-27T03:30:13.386985+0000","last_unstale":"2024-04-27T03:33:21.072429+0000","last_undegraded":"2024-04-27T03:33:21.072429+0000","last_fullsized":"2024-04-27T03:33:21.072429+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297371+0000","last_change":"2024-04-27T03:30:11.370182+0000","last_active":"2024-04-27T03:32:39.297371+0000","last_peered":"2024-04-27T03:32:39.297371+0000","last_clean":"2024-04-27T03:32:39.297371+0000","last_became_active":"2024-04-27T03:30:11.369824+0000","last_became_peered":"2024-04-27T03:30:11.369824+0000","last_unstale":"2024-04-27T03:32:39.297371+0000","last_undegraded":"2024-04-27T03:32:39.297371+0000","last_fullsized":"2024-04-27T03:32:39.297371+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.154995+0000","last_change":"2024-04-27T03:33:34.155037+0000","last_active":"2024-04-27T03:33:34.154995+0000","last_peered":"2024-04-27T03:33:34.154995+0000","last_clean":"2024-04-27T03:33:34.154995+0000","last_became_active":"2024-04-27T03:30:15.383771+0000","last_became_peered":"2024-04-27T03:30:15.383771+0000","last_unstale":"2024-04-27T03:33:34.154995+0000","last_undegraded":"2024-04-27T03:33:34.154995+0000","last_fullsized":"2024-04-27T03:33:34.154995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.154879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.154879+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.154879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:59.127137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"27'36","reported_seq":91,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.840074+0000","last_change":"2024-04-27T03:33:43.840113+0000","last_active":"2024-04-27T03:33:43.840074+0000","last_peered":"2024-04-27T03:33:43.840074+0000","last_clean":"2024-04-27T03:33:43.840074+0000","last_became_active":"2024-04-27T03:30:17.385488+0000","last_became_peered":"2024-04-27T03:30:17.385488+0000","last_unstale":"2024-04-27T03:33:43.840074+0000","last_undegraded":"2024-04-27T03:33:43.840074+0000","last_fullsized":"2024-04-27T03:33:43.840074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-04-27T03:33:43.839983+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-04-27T03:33:43.839983+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.839983+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:52.762250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.844652+0000","last_change":"2024-04-27T03:33:35.844680+0000","last_active":"2024-04-27T03:33:35.844652+0000","last_peered":"2024-04-27T03:33:35.844652+0000","last_clean":"2024-04-27T03:33:35.844652+0000","last_became_active":"2024-04-27T03:30:19.398484+0000","last_became_peered":"2024-04-27T03:30:19.398484+0000","last_unstale":"2024-04-27T03:33:35.844652+0000","last_undegraded":"2024-04-27T03:33:35.844652+0000","last_fullsized":"2024-04-27T03:33:35.844652+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:35.844545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:35.844545+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.844545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:17.570301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"25'2","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.294470+0000","last_change":"2024-04-27T03:30:13.377494+0000","last_active":"2024-04-27T03:32:19.294470+0000","last_peered":"2024-04-27T03:32:19.294470+0000","last_clean":"2024-04-27T03:32:19.294470+0000","last_became_active":"2024-04-27T03:30:13.377217+0000","last_became_peered":"2024-04-27T03:30:13.377217+0000","last_unstale":"2024-04-27T03:32:19.294470+0000","last_undegraded":"2024-04-27T03:32:19.294470+0000","last_fullsized":"2024-04-27T03:32:19.294470+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297519+0000","last_change":"2024-04-27T03:30:11.370820+0000","last_active":"2024-04-27T03:32:39.297519+0000","last_peered":"2024-04-27T03:32:39.297519+0000","last_clean":"2024-04-27T03:32:39.297519+0000","last_became_active":"2024-04-27T03:30:11.370547+0000","last_became_peered":"2024-04-27T03:30:11.370547+0000","last_unstale":"2024-04-27T03:32:39.297519+0000","last_undegraded":"2024-04-27T03:32:39.297519+0000","last_fullsized":"2024-04-27T03:32:39.297519+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288238+0000","last_change":"2024-04-27T03:30:15.382292+0000","last_active":"2024-04-27T03:32:19.288238+0000","last_peered":"2024-04-27T03:32:19.288238+0000","last_clean":"2024-04-27T03:32:19.288238+0000","last_became_active":"2024-04-27T03:30:15.381956+0000","last_became_peered":"2024-04-27T03:30:15.381956+0000","last_unstale":"2024-04-27T03:32:19.288238+0000","last_undegraded":"2024-04-27T03:32:19.288238+0000","last_fullsized":"2024-04-27T03:32:19.288238+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"27'70","reported_seq":149,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.743749+0000","last_change":"2024-04-27T03:33:45.743793+0000","last_active":"2024-04-27T03:33:45.743749+0000","last_peered":"2024-04-27T03:33:45.743749+0000","last_clean":"2024-04-27T03:33:45.743749+0000","last_became_active":"2024-04-27T03:30:17.383178+0000","last_became_peered":"2024-04-27T03:30:17.383178+0000","last_unstale":"2024-04-27T03:33:45.743749+0000","last_undegraded":"2024-04-27T03:33:45.743749+0000","last_fullsized":"2024-04-27T03:33:45.743749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'70","last_scrub_stamp":"2024-04-27T03:33:45.743668+0000","last_deep_scrub":"27'70","last_deep_scrub_stamp":"2024-04-27T03:33:45.743668+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.743668+0000","objects_scrubbed":6,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:37.293903+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":28,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.812908+0000","last_change":"2024-04-27T03:33:34.812933+0000","last_active":"2024-04-27T03:33:34.812908+0000","last_peered":"2024-04-27T03:33:34.812908+0000","last_clean":"2024-04-27T03:33:34.812908+0000","last_became_active":"2024-04-27T03:30:19.402781+0000","last_became_peered":"2024-04-27T03:30:19.402781+0000","last_unstale":"2024-04-27T03:33:34.812908+0000","last_undegraded":"2024-04-27T03:33:34.812908+0000","last_fullsized":"2024-04-27T03:33:34.812908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.812814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.812814+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.812814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:12.666738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"27'16","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.292116+0000","last_change":"2024-04-27T03:30:21.408415+0000","last_active":"2024-04-27T03:32:44.292116+0000","last_peered":"2024-04-27T03:32:44.292116+0000","last_clean":"2024-04-27T03:32:44.292116+0000","last_became_active":"2024-04-27T03:30:21.408049+0000","last_became_peered":"2024-04-27T03:30:21.408049+0000","last_unstale":"2024-04-27T03:32:44.292116+0000","last_undegraded":"2024-04-27T03:32:44.292116+0000","last_fullsized":"2024-04-27T03:32:44.292116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":21,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.287607+0000","last_change":"2024-04-27T03:30:13.373552+0000","last_active":"2024-04-27T03:32:14.287607+0000","last_peered":"2024-04-27T03:32:14.287607+0000","last_clean":"2024-04-27T03:32:14.287607+0000","last_became_active":"2024-04-27T03:30:13.373257+0000","last_became_peered":"2024-04-27T03:30:13.373257+0000","last_unstale":"2024-04-27T03:32:14.287607+0000","last_undegraded":"2024-04-27T03:32:14.287607+0000","last_fullsized":"2024-04-27T03:32:14.287607+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297190+0000","last_change":"2024-04-27T03:30:11.371431+0000","last_active":"2024-04-27T03:32:39.297190+0000","last_peered":"2024-04-27T03:32:39.297190+0000","last_clean":"2024-04-27T03:32:39.297190+0000","last_became_active":"2024-04-27T03:30:11.370091+0000","last_became_peered":"2024-04-27T03:30:11.370091+0000","last_unstale":"2024-04-27T03:32:39.297190+0000","last_undegraded":"2024-04-27T03:32:39.297190+0000","last_fullsized":"2024-04-27T03:32:39.297190+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294700+0000","last_change":"2024-04-27T03:30:15.380494+0000","last_active":"2024-04-27T03:32:24.294700+0000","last_peered":"2024-04-27T03:32:24.294700+0000","last_clean":"2024-04-27T03:32:24.294700+0000","last_became_active":"2024-04-27T03:30:15.379800+0000","last_became_peered":"2024-04-27T03:30:15.379800+0000","last_unstale":"2024-04-27T03:32:24.294700+0000","last_undegraded":"2024-04-27T03:32:24.294700+0000","last_fullsized":"2024-04-27T03:32:24.294700+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"27'39","reported_seq":89,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.612373+0000","last_change":"2024-04-27T03:30:17.389821+0000","last_active":"2024-04-27T03:33:27.612373+0000","last_peered":"2024-04-27T03:33:27.612373+0000","last_clean":"2024-04-27T03:33:27.612373+0000","last_became_active":"2024-04-27T03:30:17.388246+0000","last_became_peered":"2024-04-27T03:30:17.388246+0000","last_unstale":"2024-04-27T03:33:27.612373+0000","last_undegraded":"2024-04-27T03:33:27.612373+0000","last_fullsized":"2024-04-27T03:33:27.612373+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":39,"log_dups_size":0,"ondisk_log_size":39,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.948322+0000","last_change":"2024-04-27T03:33:40.948348+0000","last_active":"2024-04-27T03:33:40.948322+0000","last_peered":"2024-04-27T03:33:40.948322+0000","last_clean":"2024-04-27T03:33:40.948322+0000","last_became_active":"2024-04-27T03:30:19.406707+0000","last_became_peered":"2024-04-27T03:30:19.406707+0000","last_unstale":"2024-04-27T03:33:40.948322+0000","last_undegraded":"2024-04-27T03:33:40.948322+0000","last_fullsized":"2024-04-27T03:33:40.948322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:40.948229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:40.948229+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.948229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:11.173852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298513+0000","last_change":"2024-04-27T03:30:21.415821+0000","last_active":"2024-04-27T03:32:44.298513+0000","last_peered":"2024-04-27T03:32:44.298513+0000","last_clean":"2024-04-27T03:32:44.298513+0000","last_became_active":"2024-04-27T03:30:21.415441+0000","last_became_peered":"2024-04-27T03:30:21.415441+0000","last_unstale":"2024-04-27T03:32:44.298513+0000","last_undegraded":"2024-04-27T03:32:44.298513+0000","last_fullsized":"2024-04-27T03:32:44.298513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.127154+0000","last_change":"2024-04-27T03:33:44.127210+0000","last_active":"2024-04-27T03:33:44.127154+0000","last_peered":"2024-04-27T03:33:44.127154+0000","last_clean":"2024-04-27T03:33:44.127154+0000","last_became_active":"2024-04-27T03:30:13.387397+0000","last_became_peered":"2024-04-27T03:30:13.387397+0000","last_unstale":"2024-04-27T03:33:44.127154+0000","last_undegraded":"2024-04-27T03:33:44.127154+0000","last_fullsized":"2024-04-27T03:33:44.127154+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:44.127050+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:44.127050+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.127050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:24.841845+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298192+0000","last_change":"2024-04-27T03:30:11.370706+0000","last_active":"2024-04-27T03:32:44.298192+0000","last_peered":"2024-04-27T03:32:44.298192+0000","last_clean":"2024-04-27T03:32:44.298192+0000","last_became_active":"2024-04-27T03:30:11.369799+0000","last_became_peered":"2024-04-27T03:30:11.369799+0000","last_unstale":"2024-04-27T03:32:44.298192+0000","last_undegraded":"2024-04-27T03:32:44.298192+0000","last_fullsized":"2024-04-27T03:32:44.298192+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:49.299233+0000","last_change":"2024-04-27T03:30:15.380214+0000","last_active":"2024-04-27T03:32:49.299233+0000","last_peered":"2024-04-27T03:32:49.299233+0000","last_clean":"2024-04-27T03:32:49.299233+0000","last_became_active":"2024-04-27T03:30:15.379873+0000","last_became_peered":"2024-04-27T03:30:15.379873+0000","last_unstale":"2024-04-27T03:32:49.299233+0000","last_undegraded":"2024-04-27T03:32:49.299233+0000","last_fullsized":"2024-04-27T03:32:49.299233+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"27'48","reported_seq":110,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.721960+0000","last_change":"2024-04-27T03:33:43.722022+0000","last_active":"2024-04-27T03:33:43.721960+0000","last_peered":"2024-04-27T03:33:43.721960+0000","last_clean":"2024-04-27T03:33:43.721960+0000","last_became_active":"2024-04-27T03:30:17.387205+0000","last_became_peered":"2024-04-27T03:30:17.387205+0000","last_unstale":"2024-04-27T03:33:43.721960+0000","last_undegraded":"2024-04-27T03:33:43.721960+0000","last_fullsized":"2024-04-27T03:33:43.721960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-04-27T03:33:43.721859+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-04-27T03:33:43.721859+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.721859+0000","objects_scrubbed":6,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:42.507737+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":63,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.822242+0000","last_change":"2024-04-27T03:33:34.822276+0000","last_active":"2024-04-27T03:33:34.822242+0000","last_peered":"2024-04-27T03:33:34.822242+0000","last_clean":"2024-04-27T03:33:34.822242+0000","last_became_active":"2024-04-27T03:30:19.405462+0000","last_became_peered":"2024-04-27T03:30:19.405462+0000","last_unstale":"2024-04-27T03:33:34.822242+0000","last_undegraded":"2024-04-27T03:33:34.822242+0000","last_fullsized":"2024-04-27T03:33:34.822242+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.822124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.822124+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.822124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:04:28.428726+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"27'7","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298466+0000","last_change":"2024-04-27T03:30:21.416894+0000","last_active":"2024-04-27T03:32:44.298466+0000","last_peered":"2024-04-27T03:32:44.298466+0000","last_clean":"2024-04-27T03:32:44.298466+0000","last_became_active":"2024-04-27T03:30:21.416592+0000","last_became_peered":"2024-04-27T03:30:21.416592+0000","last_unstale":"2024-04-27T03:32:44.298466+0000","last_undegraded":"2024-04-27T03:32:44.298466+0000","last_fullsized":"2024-04-27T03:32:44.298466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"27'6","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.292145+0000","last_change":"2024-04-27T03:30:13.370253+0000","last_active":"2024-04-27T03:32:44.292145+0000","last_peered":"2024-04-27T03:32:44.292145+0000","last_clean":"2024-04-27T03:32:44.292145+0000","last_became_active":"2024-04-27T03:30:13.369643+0000","last_became_peered":"2024-04-27T03:30:13.369643+0000","last_unstale":"2024-04-27T03:32:44.292145+0000","last_undegraded":"2024-04-27T03:32:44.292145+0000","last_fullsized":"2024-04-27T03:32:44.292145+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.688098+0000","last_change":"2024-04-27T03:30:11.372867+0000","last_active":"2024-04-27T03:33:27.688098+0000","last_peered":"2024-04-27T03:33:27.688098+0000","last_clean":"2024-04-27T03:33:27.688098+0000","last_became_active":"2024-04-27T03:30:11.372220+0000","last_became_peered":"2024-04-27T03:30:11.372220+0000","last_unstale":"2024-04-27T03:33:27.688098+0000","last_undegraded":"2024-04-27T03:33:27.688098+0000","last_fullsized":"2024-04-27T03:33:27.688098+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:49.299231+0000","last_change":"2024-04-27T03:30:15.380858+0000","last_active":"2024-04-27T03:32:49.299231+0000","last_peered":"2024-04-27T03:32:49.299231+0000","last_clean":"2024-04-27T03:32:49.299231+0000","last_became_active":"2024-04-27T03:30:15.380573+0000","last_became_peered":"2024-04-27T03:30:15.380573+0000","last_unstale":"2024-04-27T03:32:49.299231+0000","last_undegraded":"2024-04-27T03:32:49.299231+0000","last_fullsized":"2024-04-27T03:32:49.299231+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"27'22","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.170858+0000","last_change":"2024-04-27T03:33:47.170926+0000","last_active":"2024-04-27T03:33:47.170858+0000","last_peered":"2024-04-27T03:33:47.170858+0000","last_clean":"2024-04-27T03:33:47.170858+0000","last_became_active":"2024-04-27T03:30:17.388289+0000","last_became_peered":"2024-04-27T03:30:17.388289+0000","last_unstale":"2024-04-27T03:33:47.170858+0000","last_undegraded":"2024-04-27T03:33:47.170858+0000","last_fullsized":"2024-04-27T03:33:47.170858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-04-27T03:33:47.170743+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-04-27T03:33:47.170743+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.170743+0000","objects_scrubbed":6,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:01.287895+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"27'285","reported_seq":384,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.829060+0000","last_change":"2024-04-27T03:33:40.829103+0000","last_active":"2024-04-27T03:33:40.829060+0000","last_peered":"2024-04-27T03:33:40.829060+0000","last_clean":"2024-04-27T03:33:40.829060+0000","last_became_active":"2024-04-27T03:30:19.406854+0000","last_became_peered":"2024-04-27T03:30:19.406854+0000","last_unstale":"2024-04-27T03:33:40.829060+0000","last_undegraded":"2024-04-27T03:33:40.829060+0000","last_fullsized":"2024-04-27T03:33:40.829060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:40.828943+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:40.828943+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.828943+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:59.341953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298415+0000","last_change":"2024-04-27T03:30:21.415992+0000","last_active":"2024-04-27T03:32:44.298415+0000","last_peered":"2024-04-27T03:32:44.298415+0000","last_clean":"2024-04-27T03:32:44.298415+0000","last_became_active":"2024-04-27T03:30:21.415534+0000","last_became_peered":"2024-04-27T03:30:21.415534+0000","last_unstale":"2024-04-27T03:32:44.298415+0000","last_undegraded":"2024-04-27T03:32:44.298415+0000","last_fullsized":"2024-04-27T03:32:44.298415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"27'24","reported_seq":103,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988774+0000","last_change":"2024-04-27T03:30:13.371535+0000","last_active":"2024-04-27T03:33:20.988774+0000","last_peered":"2024-04-27T03:33:20.988774+0000","last_clean":"2024-04-27T03:33:20.988774+0000","last_became_active":"2024-04-27T03:30:13.371158+0000","last_became_peered":"2024-04-27T03:30:13.371158+0000","last_unstale":"2024-04-27T03:33:20.988774+0000","last_undegraded":"2024-04-27T03:33:20.988774+0000","last_fullsized":"2024-04-27T03:33:20.988774+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":88,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298246+0000","last_change":"2024-04-27T03:30:11.359205+0000","last_active":"2024-04-27T03:32:44.298246+0000","last_peered":"2024-04-27T03:32:44.298246+0000","last_clean":"2024-04-27T03:32:44.298246+0000","last_became_active":"2024-04-27T03:30:11.358893+0000","last_became_peered":"2024-04-27T03:30:11.358893+0000","last_unstale":"2024-04-27T03:32:44.298246+0000","last_undegraded":"2024-04-27T03:32:44.298246+0000","last_fullsized":"2024-04-27T03:32:44.298246+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.016872+0000","last_change":"2024-04-27T03:33:48.016893+0000","last_active":"2024-04-27T03:33:48.016872+0000","last_peered":"2024-04-27T03:33:48.016872+0000","last_clean":"2024-04-27T03:33:48.016872+0000","last_became_active":"2024-04-27T03:30:15.381175+0000","last_became_peered":"2024-04-27T03:30:15.381175+0000","last_unstale":"2024-04-27T03:33:48.016872+0000","last_undegraded":"2024-04-27T03:33:48.016872+0000","last_fullsized":"2024-04-27T03:33:48.016872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.016786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.016786+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.016786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:39:37.651908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"27'73","reported_seq":155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.916431+0000","last_change":"2024-04-27T03:33:39.916480+0000","last_active":"2024-04-27T03:33:39.916431+0000","last_peered":"2024-04-27T03:33:39.916431+0000","last_clean":"2024-04-27T03:33:39.916431+0000","last_became_active":"2024-04-27T03:30:17.386673+0000","last_became_peered":"2024-04-27T03:30:17.386673+0000","last_unstale":"2024-04-27T03:33:39.916431+0000","last_undegraded":"2024-04-27T03:33:39.916431+0000","last_fullsized":"2024-04-27T03:33:39.916431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'73","last_scrub_stamp":"2024-04-27T03:33:39.916339+0000","last_deep_scrub":"27'73","last_deep_scrub_stamp":"2024-04-27T03:33:39.916339+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.916339+0000","objects_scrubbed":11,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:47.317240+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":80,"num_read_kb":67,"num_write":44,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.292035+0000","last_change":"2024-04-27T03:30:19.407289+0000","last_active":"2024-04-27T03:32:44.292035+0000","last_peered":"2024-04-27T03:32:44.292035+0000","last_clean":"2024-04-27T03:32:44.292035+0000","last_became_active":"2024-04-27T03:30:19.406990+0000","last_became_peered":"2024-04-27T03:30:19.406990+0000","last_unstale":"2024-04-27T03:32:44.292035+0000","last_undegraded":"2024-04-27T03:32:44.292035+0000","last_fullsized":"2024-04-27T03:32:44.292035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:49.299260+0000","last_change":"2024-04-27T03:30:21.410357+0000","last_active":"2024-04-27T03:32:49.299260+0000","last_peered":"2024-04-27T03:32:49.299260+0000","last_clean":"2024-04-27T03:32:49.299260+0000","last_became_active":"2024-04-27T03:30:21.410060+0000","last_became_peered":"2024-04-27T03:30:21.410060+0000","last_unstale":"2024-04-27T03:32:49.299260+0000","last_undegraded":"2024-04-27T03:32:49.299260+0000","last_fullsized":"2024-04-27T03:32:49.299260+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"27'10","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.015929+0000","last_change":"2024-04-27T03:30:13.372186+0000","last_active":"2024-04-27T03:33:21.015929+0000","last_peered":"2024-04-27T03:33:21.015929+0000","last_clean":"2024-04-27T03:33:21.015929+0000","last_became_active":"2024-04-27T03:30:13.371866+0000","last_became_peered":"2024-04-27T03:30:13.371866+0000","last_unstale":"2024-04-27T03:33:21.015929+0000","last_undegraded":"2024-04-27T03:33:21.015929+0000","last_fullsized":"2024-04-27T03:33:21.015929+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"27'4","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.931430+0000","last_change":"2024-04-27T03:30:11.362991+0000","last_active":"2024-04-27T03:33:20.931430+0000","last_peered":"2024-04-27T03:33:20.931430+0000","last_clean":"2024-04-27T03:33:20.931430+0000","last_became_active":"2024-04-27T03:30:11.362420+0000","last_became_peered":"2024-04-27T03:30:11.362420+0000","last_unstale":"2024-04-27T03:33:20.931430+0000","last_undegraded":"2024-04-27T03:33:20.931430+0000","last_fullsized":"2024-04-27T03:33:20.931430+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:49.299192+0000","last_change":"2024-04-27T03:30:15.382620+0000","last_active":"2024-04-27T03:32:49.299192+0000","last_peered":"2024-04-27T03:32:49.299192+0000","last_clean":"2024-04-27T03:32:49.299192+0000","last_became_active":"2024-04-27T03:30:15.382305+0000","last_became_peered":"2024-04-27T03:30:15.382305+0000","last_unstale":"2024-04-27T03:32:49.299192+0000","last_undegraded":"2024-04-27T03:32:49.299192+0000","last_fullsized":"2024-04-27T03:32:49.299192+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"27'46","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.785014+0000","last_change":"2024-04-27T03:33:40.785057+0000","last_active":"2024-04-27T03:33:40.785014+0000","last_peered":"2024-04-27T03:33:40.785014+0000","last_clean":"2024-04-27T03:33:40.785014+0000","last_became_active":"2024-04-27T03:30:17.390281+0000","last_became_peered":"2024-04-27T03:30:17.390281+0000","last_unstale":"2024-04-27T03:33:40.785014+0000","last_undegraded":"2024-04-27T03:33:40.785014+0000","last_fullsized":"2024-04-27T03:33:40.785014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:40.784918+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:40.784918+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.784918+0000","objects_scrubbed":5,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:11.010541+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":50,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298477+0000","last_change":"2024-04-27T03:30:19.406218+0000","last_active":"2024-04-27T03:32:44.298477+0000","last_peered":"2024-04-27T03:32:44.298477+0000","last_clean":"2024-04-27T03:32:44.298477+0000","last_became_active":"2024-04-27T03:30:19.405353+0000","last_became_peered":"2024-04-27T03:30:19.405353+0000","last_unstale":"2024-04-27T03:32:44.298477+0000","last_undegraded":"2024-04-27T03:32:44.298477+0000","last_fullsized":"2024-04-27T03:32:44.298477+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:49.299534+0000","last_change":"2024-04-27T03:30:21.416876+0000","last_active":"2024-04-27T03:32:49.299534+0000","last_peered":"2024-04-27T03:32:49.299534+0000","last_clean":"2024-04-27T03:32:49.299534+0000","last_became_active":"2024-04-27T03:30:21.416276+0000","last_became_peered":"2024-04-27T03:30:21.416276+0000","last_unstale":"2024-04-27T03:32:49.299534+0000","last_undegraded":"2024-04-27T03:32:49.299534+0000","last_fullsized":"2024-04-27T03:32:49.299534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'24","reported_seq":102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.706403+0000","last_change":"2024-04-27T03:30:13.377820+0000","last_active":"2024-04-27T03:33:20.706403+0000","last_peered":"2024-04-27T03:33:20.706403+0000","last_clean":"2024-04-27T03:33:20.706403+0000","last_became_active":"2024-04-27T03:30:13.376415+0000","last_became_peered":"2024-04-27T03:30:13.376415+0000","last_unstale":"2024-04-27T03:33:20.706403+0000","last_undegraded":"2024-04-27T03:33:20.706403+0000","last_fullsized":"2024-04-27T03:33:20.706403+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.688922+0000","last_change":"2024-04-27T03:30:11.361140+0000","last_active":"2024-04-27T03:33:27.688922+0000","last_peered":"2024-04-27T03:33:27.688922+0000","last_clean":"2024-04-27T03:33:27.688922+0000","last_became_active":"2024-04-27T03:30:11.360590+0000","last_became_peered":"2024-04-27T03:30:11.360590+0000","last_unstale":"2024-04-27T03:33:27.688922+0000","last_undegraded":"2024-04-27T03:33:27.688922+0000","last_fullsized":"2024-04-27T03:33:27.688922+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.996324+0000","last_change":"2024-04-27T03:33:42.996361+0000","last_active":"2024-04-27T03:33:42.996324+0000","last_peered":"2024-04-27T03:33:42.996324+0000","last_clean":"2024-04-27T03:33:42.996324+0000","last_became_active":"2024-04-27T03:30:15.385614+0000","last_became_peered":"2024-04-27T03:30:15.385614+0000","last_unstale":"2024-04-27T03:33:42.996324+0000","last_undegraded":"2024-04-27T03:33:42.996324+0000","last_fullsized":"2024-04-27T03:33:42.996324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.996219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.996219+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.996219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:58.172124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"27'37","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.164208+0000","last_change":"2024-04-27T03:33:39.164242+0000","last_active":"2024-04-27T03:33:39.164208+0000","last_peered":"2024-04-27T03:33:39.164208+0000","last_clean":"2024-04-27T03:33:39.164208+0000","last_became_active":"2024-04-27T03:30:17.386983+0000","last_became_peered":"2024-04-27T03:30:17.386983+0000","last_unstale":"2024-04-27T03:33:39.164208+0000","last_undegraded":"2024-04-27T03:33:39.164208+0000","last_fullsized":"2024-04-27T03:33:39.164208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-04-27T03:33:39.164121+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-04-27T03:33:39.164121+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.164121+0000","objects_scrubbed":3,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:26.316134+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":33,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:49.299428+0000","last_change":"2024-04-27T03:30:19.402931+0000","last_active":"2024-04-27T03:32:49.299428+0000","last_peered":"2024-04-27T03:32:49.299428+0000","last_clean":"2024-04-27T03:32:49.299428+0000","last_became_active":"2024-04-27T03:30:19.402652+0000","last_became_peered":"2024-04-27T03:30:19.402652+0000","last_unstale":"2024-04-27T03:32:49.299428+0000","last_undegraded":"2024-04-27T03:32:49.299428+0000","last_fullsized":"2024-04-27T03:32:49.299428+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"27'6","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.937826+0000","last_change":"2024-04-27T03:30:21.420988+0000","last_active":"2024-04-27T03:33:20.937826+0000","last_peered":"2024-04-27T03:33:20.937826+0000","last_clean":"2024-04-27T03:33:20.937826+0000","last_became_active":"2024-04-27T03:30:21.420725+0000","last_became_peered":"2024-04-27T03:30:21.420725+0000","last_unstale":"2024-04-27T03:33:20.937826+0000","last_undegraded":"2024-04-27T03:33:20.937826+0000","last_fullsized":"2024-04-27T03:33:20.937826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297155+0000","last_change":"2024-04-27T03:30:13.375384+0000","last_active":"2024-04-27T03:32:39.297155+0000","last_peered":"2024-04-27T03:32:39.297155+0000","last_clean":"2024-04-27T03:32:39.297155+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:32:39.297155+0000","last_undegraded":"2024-04-27T03:32:39.297155+0000","last_fullsized":"2024-04-27T03:32:39.297155+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.293781+0000","last_change":"2024-04-27T03:30:11.370518+0000","last_active":"2024-04-27T03:32:19.293781+0000","last_peered":"2024-04-27T03:32:19.293781+0000","last_clean":"2024-04-27T03:32:19.293781+0000","last_became_active":"2024-04-27T03:30:11.369877+0000","last_became_peered":"2024-04-27T03:30:11.369877+0000","last_unstale":"2024-04-27T03:32:19.293781+0000","last_undegraded":"2024-04-27T03:32:19.293781+0000","last_fullsized":"2024-04-27T03:32:19.293781+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"27'3","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.974605+0000","last_change":"2024-04-27T03:33:45.974632+0000","last_active":"2024-04-27T03:33:45.974605+0000","last_peered":"2024-04-27T03:33:45.974605+0000","last_clean":"2024-04-27T03:33:45.974605+0000","last_became_active":"2024-04-27T03:30:15.381574+0000","last_became_peered":"2024-04-27T03:30:15.381574+0000","last_unstale":"2024-04-27T03:33:45.974605+0000","last_undegraded":"2024-04-27T03:33:45.974605+0000","last_fullsized":"2024-04-27T03:33:45.974605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-27T03:33:45.974523+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-27T03:33:45.974523+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.974523+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:37.524758+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"27'47","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.667001+0000","last_change":"2024-04-27T03:30:17.391410+0000","last_active":"2024-04-27T03:33:27.667001+0000","last_peered":"2024-04-27T03:33:27.667001+0000","last_clean":"2024-04-27T03:33:27.667001+0000","last_became_active":"2024-04-27T03:30:17.388277+0000","last_became_peered":"2024-04-27T03:30:17.388277+0000","last_unstale":"2024-04-27T03:33:27.667001+0000","last_undegraded":"2024-04-27T03:33:27.667001+0000","last_fullsized":"2024-04-27T03:33:27.667001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:16.368058+0000","last_clean_scrub_stamp":"2024-04-27T03:30:16.368058+0000","objects_scrubbed":0,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":52,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.728511+0000","last_change":"2024-04-27T03:33:42.728536+0000","last_active":"2024-04-27T03:33:42.728511+0000","last_peered":"2024-04-27T03:33:42.728511+0000","last_clean":"2024-04-27T03:33:42.728511+0000","last_became_active":"2024-04-27T03:30:19.406565+0000","last_became_peered":"2024-04-27T03:30:19.406565+0000","last_unstale":"2024-04-27T03:33:42.728511+0000","last_undegraded":"2024-04-27T03:33:42.728511+0000","last_fullsized":"2024-04-27T03:33:42.728511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.728420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.728420+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.728420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:57.904325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"27'4","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298588+0000","last_change":"2024-04-27T03:30:21.414808+0000","last_active":"2024-04-27T03:32:44.298588+0000","last_peered":"2024-04-27T03:32:44.298588+0000","last_clean":"2024-04-27T03:32:44.298588+0000","last_became_active":"2024-04-27T03:30:21.414510+0000","last_became_peered":"2024-04-27T03:30:21.414510+0000","last_unstale":"2024-04-27T03:32:44.298588+0000","last_undegraded":"2024-04-27T03:32:44.298588+0000","last_fullsized":"2024-04-27T03:32:44.298588+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"25'8","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294712+0000","last_change":"2024-04-27T03:30:13.387657+0000","last_active":"2024-04-27T03:32:24.294712+0000","last_peered":"2024-04-27T03:32:24.294712+0000","last_clean":"2024-04-27T03:32:24.294712+0000","last_became_active":"2024-04-27T03:30:13.387149+0000","last_became_peered":"2024-04-27T03:30:13.387149+0000","last_unstale":"2024-04-27T03:32:24.294712+0000","last_undegraded":"2024-04-27T03:32:24.294712+0000","last_fullsized":"2024-04-27T03:32:24.294712+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"25'6","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.293591+0000","last_change":"2024-04-27T03:30:11.358174+0000","last_active":"2024-04-27T03:32:19.293591+0000","last_peered":"2024-04-27T03:32:19.293591+0000","last_clean":"2024-04-27T03:32:19.293591+0000","last_became_active":"2024-04-27T03:30:11.357878+0000","last_became_peered":"2024-04-27T03:30:11.357878+0000","last_unstale":"2024-04-27T03:32:19.293591+0000","last_undegraded":"2024-04-27T03:32:19.293591+0000","last_fullsized":"2024-04-27T03:32:19.293591+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"27'4","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.289160+0000","last_change":"2024-04-27T03:30:13.369979+0000","last_active":"2024-04-27T03:32:24.289160+0000","last_peered":"2024-04-27T03:32:24.289160+0000","last_clean":"2024-04-27T03:32:24.289160+0000","last_became_active":"2024-04-27T03:30:13.369681+0000","last_became_peered":"2024-04-27T03:30:13.369681+0000","last_unstale":"2024-04-27T03:32:24.289160+0000","last_undegraded":"2024-04-27T03:32:24.289160+0000","last_fullsized":"2024-04-27T03:32:24.289160+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288101+0000","last_change":"2024-04-27T03:30:11.373422+0000","last_active":"2024-04-27T03:32:19.288101+0000","last_peered":"2024-04-27T03:32:19.288101+0000","last_clean":"2024-04-27T03:32:19.288101+0000","last_became_active":"2024-04-27T03:30:11.373186+0000","last_became_peered":"2024-04-27T03:30:11.373186+0000","last_unstale":"2024-04-27T03:32:19.288101+0000","last_undegraded":"2024-04-27T03:32:19.288101+0000","last_fullsized":"2024-04-27T03:32:19.288101+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"27'2","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.940562+0000","last_change":"2024-04-27T03:30:13.382330+0000","last_active":"2024-04-27T03:33:20.940562+0000","last_peered":"2024-04-27T03:33:20.940562+0000","last_clean":"2024-04-27T03:33:20.940562+0000","last_became_active":"2024-04-27T03:30:13.382069+0000","last_became_peered":"2024-04-27T03:30:13.382069+0000","last_unstale":"2024-04-27T03:33:20.940562+0000","last_undegraded":"2024-04-27T03:33:20.940562+0000","last_fullsized":"2024-04-27T03:33:20.940562+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"27'3","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.294333+0000","last_change":"2024-04-27T03:30:11.350547+0000","last_active":"2024-04-27T03:32:19.294333+0000","last_peered":"2024-04-27T03:32:19.294333+0000","last_clean":"2024-04-27T03:32:19.294333+0000","last_became_active":"2024-04-27T03:30:11.350295+0000","last_became_peered":"2024-04-27T03:30:11.350295+0000","last_unstale":"2024-04-27T03:32:19.294333+0000","last_undegraded":"2024-04-27T03:32:19.294333+0000","last_fullsized":"2024-04-27T03:32:19.294333+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.940114+0000","last_change":"2024-04-27T03:30:13.370393+0000","last_active":"2024-04-27T03:33:20.940114+0000","last_peered":"2024-04-27T03:33:20.940114+0000","last_clean":"2024-04-27T03:33:20.940114+0000","last_became_active":"2024-04-27T03:30:13.370066+0000","last_became_peered":"2024-04-27T03:30:13.370066+0000","last_unstale":"2024-04-27T03:33:20.940114+0000","last_undegraded":"2024-04-27T03:33:20.940114+0000","last_fullsized":"2024-04-27T03:33:20.940114+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"27'4","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.689177+0000","last_change":"2024-04-27T03:30:11.360775+0000","last_active":"2024-04-27T03:33:27.689177+0000","last_peered":"2024-04-27T03:33:27.689177+0000","last_clean":"2024-04-27T03:33:27.689177+0000","last_became_active":"2024-04-27T03:30:11.360376+0000","last_became_peered":"2024-04-27T03:30:11.360376+0000","last_unstale":"2024-04-27T03:33:27.689177+0000","last_undegraded":"2024-04-27T03:33:27.689177+0000","last_fullsized":"2024-04-27T03:33:27.689177+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"27'4","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.804947+0000","last_change":"2024-04-27T03:30:13.378662+0000","last_active":"2024-04-27T03:33:20.804947+0000","last_peered":"2024-04-27T03:33:20.804947+0000","last_clean":"2024-04-27T03:33:20.804947+0000","last_became_active":"2024-04-27T03:30:13.378368+0000","last_became_peered":"2024-04-27T03:30:13.378368+0000","last_unstale":"2024-04-27T03:33:20.804947+0000","last_undegraded":"2024-04-27T03:33:20.804947+0000","last_fullsized":"2024-04-27T03:33:20.804947+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"25'2","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.289111+0000","last_change":"2024-04-27T03:30:11.371839+0000","last_active":"2024-04-27T03:32:24.289111+0000","last_peered":"2024-04-27T03:32:24.289111+0000","last_clean":"2024-04-27T03:32:24.289111+0000","last_became_active":"2024-04-27T03:30:11.371553+0000","last_became_peered":"2024-04-27T03:30:11.371553+0000","last_unstale":"2024-04-27T03:32:24.289111+0000","last_undegraded":"2024-04-27T03:32:24.289111+0000","last_fullsized":"2024-04-27T03:32:24.289111+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"27'10","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.016605+0000","last_change":"2024-04-27T03:31:01.440910+0000","last_active":"2024-04-27T03:33:21.016605+0000","last_peered":"2024-04-27T03:33:21.016605+0000","last_clean":"2024-04-27T03:33:21.016605+0000","last_became_active":"2024-04-27T03:31:01.440576+0000","last_became_peered":"2024-04-27T03:31:01.440576+0000","last_unstale":"2024-04-27T03:33:21.016605+0000","last_undegraded":"2024-04-27T03:33:21.016605+0000","last_fullsized":"2024-04-27T03:33:21.016605+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294653+0000","last_change":"2024-04-27T03:30:11.371034+0000","last_active":"2024-04-27T03:32:24.294653+0000","last_peered":"2024-04-27T03:32:24.294653+0000","last_clean":"2024-04-27T03:32:24.294653+0000","last_became_active":"2024-04-27T03:30:11.369517+0000","last_became_peered":"2024-04-27T03:30:11.369517+0000","last_unstale":"2024-04-27T03:32:24.294653+0000","last_undegraded":"2024-04-27T03:32:24.294653+0000","last_fullsized":"2024-04-27T03:32:24.294653+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288115+0000","last_change":"2024-04-27T03:30:13.374070+0000","last_active":"2024-04-27T03:32:19.288115+0000","last_peered":"2024-04-27T03:32:19.288115+0000","last_clean":"2024-04-27T03:32:19.288115+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:32:19.288115+0000","last_undegraded":"2024-04-27T03:32:19.288115+0000","last_fullsized":"2024-04-27T03:32:19.288115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294652+0000","last_change":"2024-04-27T03:31:01.441506+0000","last_active":"2024-04-27T03:32:24.294652+0000","last_peered":"2024-04-27T03:32:24.294652+0000","last_clean":"2024-04-27T03:32:24.294652+0000","last_became_active":"2024-04-27T03:31:01.441159+0000","last_became_peered":"2024-04-27T03:31:01.441159+0000","last_unstale":"2024-04-27T03:32:24.294652+0000","last_undegraded":"2024-04-27T03:32:24.294652+0000","last_fullsized":"2024-04-27T03:32:24.294652+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"27'3","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.069897+0000","last_change":"2024-04-27T03:30:13.378083+0000","last_active":"2024-04-27T03:33:21.069897+0000","last_peered":"2024-04-27T03:33:21.069897+0000","last_clean":"2024-04-27T03:33:21.069897+0000","last_became_active":"2024-04-27T03:30:13.377821+0000","last_became_peered":"2024-04-27T03:30:13.377821+0000","last_unstale":"2024-04-27T03:33:21.069897+0000","last_undegraded":"2024-04-27T03:33:21.069897+0000","last_fullsized":"2024-04-27T03:33:21.069897+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.289175+0000","last_change":"2024-04-27T03:30:11.371906+0000","last_active":"2024-04-27T03:32:24.289175+0000","last_peered":"2024-04-27T03:32:24.289175+0000","last_clean":"2024-04-27T03:32:24.289175+0000","last_became_active":"2024-04-27T03:30:11.371604+0000","last_became_peered":"2024-04-27T03:30:11.371604+0000","last_unstale":"2024-04-27T03:32:24.289175+0000","last_undegraded":"2024-04-27T03:32:24.289175+0000","last_fullsized":"2024-04-27T03:32:24.289175+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"27'8","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988787+0000","last_change":"2024-04-27T03:30:13.388711+0000","last_active":"2024-04-27T03:33:20.988787+0000","last_peered":"2024-04-27T03:33:20.988787+0000","last_clean":"2024-04-27T03:33:20.988787+0000","last_became_active":"2024-04-27T03:30:13.388421+0000","last_became_peered":"2024-04-27T03:30:13.388421+0000","last_unstale":"2024-04-27T03:33:20.988787+0000","last_undegraded":"2024-04-27T03:33:20.988787+0000","last_fullsized":"2024-04-27T03:33:20.988787+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.680852+0000","last_change":"2024-04-27T03:30:11.355453+0000","last_active":"2024-04-27T03:33:20.680852+0000","last_peered":"2024-04-27T03:33:20.680852+0000","last_clean":"2024-04-27T03:33:20.680852+0000","last_became_active":"2024-04-27T03:30:11.355183+0000","last_became_peered":"2024-04-27T03:30:11.355183+0000","last_unstale":"2024-04-27T03:33:20.680852+0000","last_undegraded":"2024-04-27T03:33:20.680852+0000","last_fullsized":"2024-04-27T03:33:20.680852+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"27'4","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296887+0000","last_change":"2024-04-27T03:30:13.377155+0000","last_active":"2024-04-27T03:32:34.296887+0000","last_peered":"2024-04-27T03:32:34.296887+0000","last_clean":"2024-04-27T03:32:34.296887+0000","last_became_active":"2024-04-27T03:30:13.376875+0000","last_became_peered":"2024-04-27T03:30:13.376875+0000","last_unstale":"2024-04-27T03:32:34.296887+0000","last_undegraded":"2024-04-27T03:32:34.296887+0000","last_fullsized":"2024-04-27T03:32:34.296887+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.289753+0000","last_change":"2024-04-27T03:30:11.361373+0000","last_active":"2024-04-27T03:32:29.289753+0000","last_peered":"2024-04-27T03:32:29.289753+0000","last_clean":"2024-04-27T03:32:29.289753+0000","last_became_active":"2024-04-27T03:30:11.360299+0000","last_became_peered":"2024-04-27T03:30:11.360299+0000","last_unstale":"2024-04-27T03:32:29.289753+0000","last_undegraded":"2024-04-27T03:32:29.289753+0000","last_fullsized":"2024-04-27T03:32:29.289753+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"27'10","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988466+0000","last_change":"2024-04-27T03:30:13.376269+0000","last_active":"2024-04-27T03:33:20.988466+0000","last_peered":"2024-04-27T03:33:20.988466+0000","last_clean":"2024-04-27T03:33:20.988466+0000","last_became_active":"2024-04-27T03:30:13.376087+0000","last_became_peered":"2024-04-27T03:30:13.376087+0000","last_unstale":"2024-04-27T03:33:20.988466+0000","last_undegraded":"2024-04-27T03:33:20.988466+0000","last_fullsized":"2024-04-27T03:33:20.988466+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295291+0000","last_change":"2024-04-27T03:30:11.371237+0000","last_active":"2024-04-27T03:32:29.295291+0000","last_peered":"2024-04-27T03:32:29.295291+0000","last_clean":"2024-04-27T03:32:29.295291+0000","last_became_active":"2024-04-27T03:30:11.370837+0000","last_became_peered":"2024-04-27T03:30:11.370837+0000","last_unstale":"2024-04-27T03:32:29.295291+0000","last_undegraded":"2024-04-27T03:32:29.295291+0000","last_fullsized":"2024-04-27T03:32:29.295291+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"27'6","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.704051+0000","last_change":"2024-04-27T03:30:13.376130+0000","last_active":"2024-04-27T03:33:20.704051+0000","last_peered":"2024-04-27T03:33:20.704051+0000","last_clean":"2024-04-27T03:33:20.704051+0000","last_became_active":"2024-04-27T03:30:13.373511+0000","last_became_peered":"2024-04-27T03:30:13.373511+0000","last_unstale":"2024-04-27T03:33:20.704051+0000","last_undegraded":"2024-04-27T03:33:20.704051+0000","last_fullsized":"2024-04-27T03:33:20.704051+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295373+0000","last_change":"2024-04-27T03:30:11.370821+0000","last_active":"2024-04-27T03:32:29.295373+0000","last_peered":"2024-04-27T03:32:29.295373+0000","last_clean":"2024-04-27T03:32:29.295373+0000","last_became_active":"2024-04-27T03:30:11.370283+0000","last_became_peered":"2024-04-27T03:30:11.370283+0000","last_unstale":"2024-04-27T03:32:29.295373+0000","last_undegraded":"2024-04-27T03:32:29.295373+0000","last_fullsized":"2024-04-27T03:32:29.295373+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"27'11","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.072440+0000","last_change":"2024-04-27T03:30:13.380862+0000","last_active":"2024-04-27T03:33:21.072440+0000","last_peered":"2024-04-27T03:33:21.072440+0000","last_clean":"2024-04-27T03:33:21.072440+0000","last_became_active":"2024-04-27T03:30:13.380404+0000","last_became_peered":"2024-04-27T03:30:13.380404+0000","last_unstale":"2024-04-27T03:33:21.072440+0000","last_undegraded":"2024-04-27T03:33:21.072440+0000","last_fullsized":"2024-04-27T03:33:21.072440+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295898+0000","last_change":"2024-04-27T03:30:11.373282+0000","last_active":"2024-04-27T03:32:29.295898+0000","last_peered":"2024-04-27T03:32:29.295898+0000","last_clean":"2024-04-27T03:32:29.295898+0000","last_became_active":"2024-04-27T03:30:11.373026+0000","last_became_peered":"2024-04-27T03:30:11.373026+0000","last_unstale":"2024-04-27T03:32:29.295898+0000","last_undegraded":"2024-04-27T03:32:29.295898+0000","last_fullsized":"2024-04-27T03:32:29.295898+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"27'1","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.072393+0000","last_change":"2024-04-27T03:30:13.386073+0000","last_active":"2024-04-27T03:33:21.072393+0000","last_peered":"2024-04-27T03:33:21.072393+0000","last_clean":"2024-04-27T03:33:21.072393+0000","last_became_active":"2024-04-27T03:30:13.385791+0000","last_became_peered":"2024-04-27T03:30:13.385791+0000","last_unstale":"2024-04-27T03:33:21.072393+0000","last_undegraded":"2024-04-27T03:33:21.072393+0000","last_fullsized":"2024-04-27T03:33:21.072393+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295932+0000","last_change":"2024-04-27T03:30:11.358239+0000","last_active":"2024-04-27T03:32:29.295932+0000","last_peered":"2024-04-27T03:32:29.295932+0000","last_clean":"2024-04-27T03:32:29.295932+0000","last_became_active":"2024-04-27T03:30:11.357898+0000","last_became_peered":"2024-04-27T03:30:11.357898+0000","last_unstale":"2024-04-27T03:32:29.295932+0000","last_undegraded":"2024-04-27T03:32:29.295932+0000","last_fullsized":"2024-04-27T03:32:29.295932+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"27'5","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.070313+0000","last_change":"2024-04-27T03:31:01.444873+0000","last_active":"2024-04-27T03:33:21.070313+0000","last_peered":"2024-04-27T03:33:21.070313+0000","last_clean":"2024-04-27T03:33:21.070313+0000","last_became_active":"2024-04-27T03:31:01.440820+0000","last_became_peered":"2024-04-27T03:31:01.440820+0000","last_unstale":"2024-04-27T03:33:21.070313+0000","last_undegraded":"2024-04-27T03:33:21.070313+0000","last_fullsized":"2024-04-27T03:33:21.070313+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296405+0000","last_change":"2024-04-27T03:30:11.363676+0000","last_active":"2024-04-27T03:32:34.296405+0000","last_peered":"2024-04-27T03:32:34.296405+0000","last_clean":"2024-04-27T03:32:34.296405+0000","last_became_active":"2024-04-27T03:30:11.362788+0000","last_became_peered":"2024-04-27T03:30:11.362788+0000","last_unstale":"2024-04-27T03:32:34.296405+0000","last_undegraded":"2024-04-27T03:32:34.296405+0000","last_fullsized":"2024-04-27T03:32:34.296405+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"25'2","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294481+0000","last_change":"2024-04-27T03:30:13.378124+0000","last_active":"2024-04-27T03:32:24.294481+0000","last_peered":"2024-04-27T03:32:24.294481+0000","last_clean":"2024-04-27T03:32:24.294481+0000","last_became_active":"2024-04-27T03:30:13.377845+0000","last_became_peered":"2024-04-27T03:30:13.377845+0000","last_unstale":"2024-04-27T03:32:24.294481+0000","last_undegraded":"2024-04-27T03:32:24.294481+0000","last_fullsized":"2024-04-27T03:32:24.294481+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.688620+0000","last_change":"2024-04-27T03:30:11.362917+0000","last_active":"2024-04-27T03:33:27.688620+0000","last_peered":"2024-04-27T03:33:27.688620+0000","last_clean":"2024-04-27T03:33:27.688620+0000","last_became_active":"2024-04-27T03:30:11.362559+0000","last_became_peered":"2024-04-27T03:30:11.362559+0000","last_unstale":"2024-04-27T03:33:27.688620+0000","last_undegraded":"2024-04-27T03:33:27.688620+0000","last_fullsized":"2024-04-27T03:33:27.688620+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296630+0000","last_change":"2024-04-27T03:30:11.363675+0000","last_active":"2024-04-27T03:32:34.296630+0000","last_peered":"2024-04-27T03:32:34.296630+0000","last_clean":"2024-04-27T03:32:34.296630+0000","last_became_active":"2024-04-27T03:30:11.363340+0000","last_became_peered":"2024-04-27T03:30:11.363340+0000","last_unstale":"2024-04-27T03:32:34.296630+0000","last_undegraded":"2024-04-27T03:32:34.296630+0000","last_fullsized":"2024-04-27T03:32:34.296630+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.291504+0000","last_change":"2024-04-27T03:30:13.386029+0000","last_active":"2024-04-27T03:32:39.291504+0000","last_peered":"2024-04-27T03:32:39.291504+0000","last_clean":"2024-04-27T03:32:39.291504+0000","last_became_active":"2024-04-27T03:30:13.385705+0000","last_became_peered":"2024-04-27T03:30:13.385705+0000","last_unstale":"2024-04-27T03:32:39.291504+0000","last_undegraded":"2024-04-27T03:32:39.291504+0000","last_fullsized":"2024-04-27T03:32:39.291504+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.290659+0000","last_change":"2024-04-27T03:30:11.360980+0000","last_active":"2024-04-27T03:32:34.290659+0000","last_peered":"2024-04-27T03:32:34.290659+0000","last_clean":"2024-04-27T03:32:34.290659+0000","last_became_active":"2024-04-27T03:30:11.360667+0000","last_became_peered":"2024-04-27T03:30:11.360667+0000","last_unstale":"2024-04-27T03:32:34.290659+0000","last_undegraded":"2024-04-27T03:32:34.290659+0000","last_fullsized":"2024-04-27T03:32:34.290659+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"27'14","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297355+0000","last_change":"2024-04-27T03:30:13.380806+0000","last_active":"2024-04-27T03:32:39.297355+0000","last_peered":"2024-04-27T03:32:39.297355+0000","last_clean":"2024-04-27T03:32:39.297355+0000","last_became_active":"2024-04-27T03:30:13.380347+0000","last_became_peered":"2024-04-27T03:30:13.380347+0000","last_unstale":"2024-04-27T03:32:39.297355+0000","last_undegraded":"2024-04-27T03:32:39.297355+0000","last_fullsized":"2024-04-27T03:32:39.297355+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":267,"ondisk_log_size":267,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2280,"ondisk_log_size":2280,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":1888,"num_read_kb":1653,"num_write":789,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1204,"num_omap_keys":6,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1701,"ondisk_log_size":1701,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":473,"ondisk_log_size":473,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":82,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705711,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39904,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331936,"statfs":{"total":96636764160,"available":96595902464,"internally_reserved":0,"allocated":417792,"data_stored":120948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705712,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40416,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331424,"statfs":{"total":96636764160,"available":96595378176,"internally_reserved":0,"allocated":942080,"data_stored":708075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705711,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335556,"statfs":{"total":96636764160,"available":96599609344,"internally_reserved":0,"allocated":970752,"data_stored":708832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705713,"num_pgs":134,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39868,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331972,"statfs":{"total":96636764160,"available":96595939328,"internally_reserved":0,"allocated":380928,"data_stored":119531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T03:33:52.481 INFO:tasks.ceph:pgid 3.3f last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.481 INFO:tasks.ceph:pgid 4.38 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.481 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.481 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.481 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.481 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.481 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.481 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.481 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.482 INFO:tasks.ceph:pgid 4.3c last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.482 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.482 INFO:tasks.ceph:pgid 4.3d last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.482 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.482 INFO:tasks.ceph:pgid 4.3e last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.482 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.482 INFO:tasks.ceph:pgid 4.3f last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.482 INFO:tasks.ceph:pgid 4.30 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.482 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.483 INFO:tasks.ceph:pgid 4.31 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.483 INFO:tasks.ceph:pgid 3.36 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.483 INFO:tasks.ceph:pgid 4.32 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.483 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.483 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.483 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.483 INFO:tasks.ceph:pgid 4.34 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.483 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.483 INFO:tasks.ceph:pgid 4.35 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.483 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.484 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.484 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.484 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.484 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.484 INFO:tasks.ceph:pgid 4.15 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.484 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.484 INFO:tasks.ceph:pgid 5.14 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.484 INFO:tasks.ceph:pgid 8.1a last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.484 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.485 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.485 INFO:tasks.ceph:pgid 7.15 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.485 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.485 INFO:tasks.ceph:pgid 4.17 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.485 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.485 INFO:tasks.ceph:pgid 3.f last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.485 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.485 INFO:tasks.ceph:pgid 4.9 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.485 INFO:tasks.ceph:pgid 3.e last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.486 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.486 INFO:tasks.ceph:pgid 6.b last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.486 INFO:tasks.ceph:pgid 8.5 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.486 INFO:tasks.ceph:pgid 4.a last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.486 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.486 INFO:tasks.ceph:pgid 5.b last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.486 INFO:tasks.ceph:pgid 6.8 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.486 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.486 INFO:tasks.ceph:pgid 4.b last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.487 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.487 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.487 INFO:tasks.ceph:pgid 6.9 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.487 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.487 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.487 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.487 INFO:tasks.ceph:pgid 5.d last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.487 INFO:tasks.ceph:pgid 7.f last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.487 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.488 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.488 INFO:tasks.ceph:pgid 8.1 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.488 INFO:tasks.ceph:pgid 4.e last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.488 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.488 INFO:tasks.ceph:pgid 6.c last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.488 INFO:tasks.ceph:pgid 8.2 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.488 INFO:tasks.ceph:pgid 4.f last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.488 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.489 INFO:tasks.ceph:pgid 4.5 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.489 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.489 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.489 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.489 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.489 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.489 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.489 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.489 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.489 INFO:tasks.ceph:pgid 8.a last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.490 INFO:tasks.ceph:pgid 4.7 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.490 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.490 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.490 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.490 INFO:tasks.ceph:pgid 4.4 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.490 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.490 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.490 INFO:tasks.ceph:pgid 5.5 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.491 INFO:tasks.ceph:pgid 6.6 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.491 INFO:tasks.ceph:pgid 7.7 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.491 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.491 INFO:tasks.ceph:pgid 4.2 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.491 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.491 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.491 INFO:tasks.ceph:pgid 5.3 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.491 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.491 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.492 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.492 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.492 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.492 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.492 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.492 INFO:tasks.ceph:pgid 6.2 last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.492 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.492 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.492 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.493 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.493 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.493 INFO:tasks.ceph:pgid 4.13 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.493 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.493 INFO:tasks.ceph:pgid 5.12 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.493 INFO:tasks.ceph:pgid 7.10 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.493 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.493 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.493 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.493 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.494 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.494 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.494 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.494 INFO:tasks.ceph:pgid 4.10 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.494 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.494 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.494 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.494 INFO:tasks.ceph:pgid 5.1e last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.495 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.495 INFO:tasks.ceph:pgid 4.1e last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.495 INFO:tasks.ceph:pgid 3.19 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.495 INFO:tasks.ceph:pgid 5.1f last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.495 INFO:tasks.ceph:pgid 6.1c last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.495 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.495 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.495 INFO:tasks.ceph:pgid 5.1c last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.495 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.496 INFO:tasks.ceph:pgid 4.1c last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.496 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.496 INFO:tasks.ceph:pgid 5.1d last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.496 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.496 INFO:tasks.ceph:pgid 4.1b last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.496 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.496 INFO:tasks.ceph:pgid 7.18 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.496 INFO:tasks.ceph:pgid 8.17 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.497 INFO:tasks.ceph:pgid 4.1a last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.497 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.497 INFO:tasks.ceph:pgid 5.1b last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.497 INFO:tasks.ceph:pgid 7.19 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.497 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.497 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.497 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.497 INFO:tasks.ceph:pgid 7.1a last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.497 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.498 INFO:tasks.ceph:pgid 4.18 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.498 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.498 INFO:tasks.ceph:pgid 6.1a last_scrub_stamp 2024-04-27T03:30:16.368058+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=16, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.498 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.498 INFO:tasks.ceph:pgid 4.27 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.498 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.498 INFO:tasks.ceph:pgid 4.26 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.498 INFO:tasks.ceph:pgid 3.21 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.498 INFO:tasks.ceph:pgid 4.25 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.499 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.499 INFO:tasks.ceph:pgid 4.24 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.499 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.499 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.499 INFO:tasks.ceph:pgid 3.24 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.499 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.499 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.499 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.499 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.499 INFO:tasks.ceph:pgid 4.20 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.499 INFO:tasks.ceph:pgid 3.27 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.500 INFO:tasks.ceph:pgid 4.2f last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.500 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.500 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.500 INFO:tasks.ceph:pgid 3.29 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.500 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.500 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.500 INFO:tasks.ceph:pgid 4.2c last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.500 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.500 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.500 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.501 INFO:tasks.ceph:pgid 4.2a last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.501 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.501 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.501 INFO:tasks.ceph:pgid 3.2e last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.501 INFO:tasks.ceph:pgid 4.28 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.501 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.501 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.501 INFO:tasks.ceph:pgid 4.37 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.501 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.501 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:33:52.502 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-27T03:34:12.503 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T03:34:12.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.606+0000 7f22c59bd640 1 Processor -- start 2024-04-27T03:34:12.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.606+0000 7f22c59bd640 1 -- start start 2024-04-27T03:34:12.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.606+0000 7f22c59bd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f22c006f260 0x7f22c006f640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:34:12.612 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.606+0000 7f22c59bd640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f22c006fff0 con 0x7f22c006fb80 2024-04-27T03:34:12.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.606+0000 7f22c59bd640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f22c0070bc0 con 0x7f22c006f260 2024-04-27T03:34:12.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.606+0000 7f22beffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f22c006f260 0x7f22c006f640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:34:12.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.606+0000 7f22be7fc640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f22c006fb80 0x7f22c012e540 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:45178/0 (socket says 172.21.15.89:45178) 2024-04-27T03:34:12.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.606+0000 7f22be7fc640 1 -- 172.21.15.89:0/2613022501 learned_addr learned my addr 172.21.15.89:0/2613022501 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:34:12.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.607+0000 7f22beffd640 1 -- 172.21.15.89:0/2613022501 >> v1:172.21.15.89:6789/0 conn(0x7f22c006fb80 legacy=0x7f22c012e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:34:12.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.607+0000 7f22beffd640 1 -- 172.21.15.89:0/2613022501 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c012ec50 con 0x7f22c006f260 2024-04-27T03:34:12.613 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.607+0000 7f22beffd640 1 --2- 172.21.15.89:0/2613022501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22c006f260 0x7f22c006f640 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009720 tx=0x7f22b002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181eb31dae887b11 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:34:12.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.608+0000 7f22bdffb640 1 -- 172.21.15.89:0/2613022501 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22b00316d0 con 0x7f22c006f260 2024-04-27T03:34:12.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.608+0000 7f22bdffb640 1 -- 172.21.15.89:0/2613022501 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b0037070 con 0x7f22c006f260 2024-04-27T03:34:12.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.608+0000 7f22bdffb640 1 -- 172.21.15.89:0/2613022501 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22b00319c0 con 0x7f22c006f260 2024-04-27T03:34:12.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.609+0000 7f22c59bd640 1 -- 172.21.15.89:0/2613022501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22c006f260 msgr2=0x7f22c006f640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:34:12.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.609+0000 7f22c59bd640 1 --2- 172.21.15.89:0/2613022501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22c006f260 0x7f22c006f640 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009720 tx=0x7f22b002b840 comp rx=0 tx=0).stop 2024-04-27T03:34:12.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.609+0000 7f22c59bd640 1 -- 172.21.15.89:0/2613022501 shutdown_connections 2024-04-27T03:34:12.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.609+0000 7f22c59bd640 1 --2- 172.21.15.89:0/2613022501 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22c006f260 0x7f22c006f640 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:34:12.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.609+0000 7f22c59bd640 1 -- 172.21.15.89:0/2613022501 >> 172.21.15.89:0/2613022501 conn(0x7f22c009e170 msgr2=0x7f22c006d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:34:12.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.609+0000 7f22c59bd640 1 -- 172.21.15.89:0/2613022501 shutdown_connections 2024-04-27T03:34:12.615 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.609+0000 7f22c59bd640 1 -- 172.21.15.89:0/2613022501 wait complete. 2024-04-27T03:34:12.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.610+0000 7f22c59bd640 1 Processor -- start 2024-04-27T03:34:12.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.610+0000 7f22c59bd640 1 -- start start 2024-04-27T03:34:12.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.611+0000 7f22c59bd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22c006f260 0x7f22c0117480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:34:12.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.611+0000 7f22c59bd640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f22c012f400 con 0x7f22c006f260 2024-04-27T03:34:12.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.611+0000 7f22beffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22c006f260 0x7f22c0117480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:34:12.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.611+0000 7f22beffd640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22c006f260 0x7f22c0117480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:51952/0 (socket says 172.21.15.89:51952) 2024-04-27T03:34:12.617 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.611+0000 7f22beffd640 1 -- 172.21.15.89:0/1082676585 learned_addr learned my addr 172.21.15.89:0/1082676585 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:34:12.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.611+0000 7f22beffd640 1 -- 172.21.15.89:0/1082676585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22c011a8a0 con 0x7f22c006f260 2024-04-27T03:34:12.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.611+0000 7f22beffd640 1 --2- 172.21.15.89:0/1082676585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22c006f260 0x7f22c0117480 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009840 tx=0x7f22b00046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:34:12.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.611+0000 7f22a7fff640 1 -- 172.21.15.89:0/1082676585 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22b0032640 con 0x7f22c006f260 2024-04-27T03:34:12.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.611+0000 7f22a7fff640 1 -- 172.21.15.89:0/1082676585 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f22b00327e0 con 0x7f22c006f260 2024-04-27T03:34:12.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.611+0000 7f22a7fff640 1 -- 172.21.15.89:0/1082676585 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f22b0032b00 con 0x7f22c006f260 2024-04-27T03:34:12.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.611+0000 7f22c59bd640 1 -- 172.21.15.89:0/1082676585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22c0119c70 con 0x7f22c006f260 2024-04-27T03:34:12.618 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.611+0000 7f22c59bd640 1 -- 172.21.15.89:0/1082676585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22c011a3f0 con 0x7f22c006f260 2024-04-27T03:34:12.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.612+0000 7f22a7fff640 1 -- 172.21.15.89:0/1082676585 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f22b004c060 con 0x7f22c006f260 2024-04-27T03:34:12.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.612+0000 7f22a7fff640 1 --2- 172.21.15.89:0/1082676585 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f22ac038010 0x7f22ac03a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:34:12.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.612+0000 7f22c59bd640 1 -- 172.21.15.89:0/1082676585 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22c00a18a0 con 0x7f22c006f260 2024-04-27T03:34:12.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.612+0000 7f22a7fff640 1 -- 172.21.15.89:0/1082676585 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f22b002f050 con 0x7f22c006f260 2024-04-27T03:34:12.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.613+0000 7f22be7fc640 1 --2- 172.21.15.89:0/1082676585 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f22ac038010 0x7f22ac03a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:34:12.619 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.613+0000 7f22be7fc640 1 --2- 172.21.15.89:0/1082676585 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f22ac038010 0x7f22ac03a4d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f22b4007fd0 tx=0x7f22b4007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:34:12.622 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.617+0000 7f22a7fff640 1 -- 172.21.15.89:0/1082676585 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f22c00a18a0 con 0x7f22c006f260 2024-04-27T03:34:12.795 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.789+0000 7f22c59bd640 1 -- 172.21.15.89:0/1082676585 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f22c01347f0 con 0x7f22ac038010 2024-04-27T03:34:12.810 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.805+0000 7f22a7fff640 1 -- 172.21.15.89:0/1082676585 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+661495 (secure 0 0 0) 0x7f22c01347f0 con 0x7f22ac038010 2024-04-27T03:34:12.818 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:34:12.823 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-04-27T03:34:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.809+0000 7f22c59bd640 1 -- 172.21.15.89:0/1082676585 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f22ac038010 msgr2=0x7f22ac03a4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:34:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.809+0000 7f22c59bd640 1 --2- 172.21.15.89:0/1082676585 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f22ac038010 0x7f22ac03a4d0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f22b4007fd0 tx=0x7f22b4007ac0 comp rx=0 tx=0).stop 2024-04-27T03:34:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.809+0000 7f22c59bd640 1 -- 172.21.15.89:0/1082676585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22c006f260 msgr2=0x7f22c0117480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:34:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.809+0000 7f22c59bd640 1 --2- 172.21.15.89:0/1082676585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22c006f260 0x7f22c0117480 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7f22b0009840 tx=0x7f22b00046c0 comp rx=0 tx=0).stop 2024-04-27T03:34:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.809+0000 7f22c59bd640 1 -- 172.21.15.89:0/1082676585 shutdown_connections 2024-04-27T03:34:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.809+0000 7f22c59bd640 1 --2- 172.21.15.89:0/1082676585 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f22ac038010 0x7f22ac03a4d0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:34:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.809+0000 7f22c59bd640 1 --2- 172.21.15.89:0/1082676585 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f22c006f260 0x7f22c0117480 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:34:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.809+0000 7f22c59bd640 1 -- 172.21.15.89:0/1082676585 >> 172.21.15.89:0/1082676585 conn(0x7f22c009e170 msgr2=0x7f22c012b5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:34:12.826 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.809+0000 7f22c59bd640 1 -- 172.21.15.89:0/1082676585 shutdown_connections 2024-04-27T03:34:12.827 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:12.809+0000 7f22c59bd640 1 -- 172.21.15.89:0/1082676585 wait complete. 2024-04-27T03:34:12.827 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":148,"stamp":"2024-04-27T03:34:12.285397+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":252,"num_object_clones":0,"num_object_copies":504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":252,"num_whiteouts":0,"num_read":5378,"num_read_kb":4932,"num_write":2369,"num_write_kb":61373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3041,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":4996,"ondisk_log_size":4996,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":156472,"kb_used_data":2648,"kb_used_omap":6,"kb_used_meta":153785,"kb_avail":377330888,"statfs":{"total":386547056640,"available":386386829312,"internally_reserved":0,"allocated":2711552,"data_stored":1657386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":157476630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":350,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004135"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.834218+0000","last_change":"2024-04-27T03:34:07.834248+0000","last_active":"2024-04-27T03:34:07.834218+0000","last_peered":"2024-04-27T03:34:07.834218+0000","last_clean":"2024-04-27T03:34:07.834218+0000","last_became_active":"2024-04-27T03:30:11.362574+0000","last_became_peered":"2024-04-27T03:30:11.362574+0000","last_unstale":"2024-04-27T03:34:07.834218+0000","last_undegraded":"2024-04-27T03:34:07.834218+0000","last_fullsized":"2024-04-27T03:34:07.834218+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:07.834117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:07.834117+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.834117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:15.180175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.077803+0000","last_change":"2024-04-27T03:33:56.077828+0000","last_active":"2024-04-27T03:33:56.077803+0000","last_peered":"2024-04-27T03:33:56.077803+0000","last_clean":"2024-04-27T03:33:56.077803+0000","last_became_active":"2024-04-27T03:30:13.384580+0000","last_became_peered":"2024-04-27T03:30:13.384580+0000","last_unstale":"2024-04-27T03:33:56.077803+0000","last_undegraded":"2024-04-27T03:33:56.077803+0000","last_fullsized":"2024-04-27T03:33:56.077803+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:56.077710+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:56.077710+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.077710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:21.049969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"27'4","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.924082+0000","last_change":"2024-04-27T03:30:11.361366+0000","last_active":"2024-04-27T03:33:20.924082+0000","last_peered":"2024-04-27T03:33:20.924082+0000","last_clean":"2024-04-27T03:33:20.924082+0000","last_became_active":"2024-04-27T03:30:11.360857+0000","last_became_peered":"2024-04-27T03:30:11.360857+0000","last_unstale":"2024-04-27T03:33:20.924082+0000","last_undegraded":"2024-04-27T03:33:20.924082+0000","last_fullsized":"2024-04-27T03:33:20.924082+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"27'12","reported_seq":80,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.704039+0000","last_change":"2024-04-27T03:30:13.376545+0000","last_active":"2024-04-27T03:33:20.704039+0000","last_peered":"2024-04-27T03:33:20.704039+0000","last_clean":"2024-04-27T03:33:20.704039+0000","last_became_active":"2024-04-27T03:30:13.376221+0000","last_became_peered":"2024-04-27T03:30:13.376221+0000","last_unstale":"2024-04-27T03:33:20.704039+0000","last_undegraded":"2024-04-27T03:33:20.704039+0000","last_fullsized":"2024-04-27T03:33:20.704039+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.294417+0000","last_change":"2024-04-27T03:30:11.363460+0000","last_active":"2024-04-27T03:32:19.294417+0000","last_peered":"2024-04-27T03:32:19.294417+0000","last_clean":"2024-04-27T03:32:19.294417+0000","last_became_active":"2024-04-27T03:30:11.362739+0000","last_became_peered":"2024-04-27T03:30:11.362739+0000","last_unstale":"2024-04-27T03:32:19.294417+0000","last_undegraded":"2024-04-27T03:32:19.294417+0000","last_fullsized":"2024-04-27T03:32:19.294417+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"27'7","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.070147+0000","last_change":"2024-04-27T03:30:13.376819+0000","last_active":"2024-04-27T03:33:21.070147+0000","last_peered":"2024-04-27T03:33:21.070147+0000","last_clean":"2024-04-27T03:33:21.070147+0000","last_became_active":"2024-04-27T03:30:13.376524+0000","last_became_peered":"2024-04-27T03:30:13.376524+0000","last_unstale":"2024-04-27T03:33:21.070147+0000","last_undegraded":"2024-04-27T03:33:21.070147+0000","last_fullsized":"2024-04-27T03:33:21.070147+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.688799+0000","last_change":"2024-04-27T03:30:11.361039+0000","last_active":"2024-04-27T03:33:27.688799+0000","last_peered":"2024-04-27T03:33:27.688799+0000","last_clean":"2024-04-27T03:33:27.688799+0000","last_became_active":"2024-04-27T03:30:11.360188+0000","last_became_peered":"2024-04-27T03:30:11.360188+0000","last_unstale":"2024-04-27T03:33:27.688799+0000","last_undegraded":"2024-04-27T03:33:27.688799+0000","last_fullsized":"2024-04-27T03:33:27.688799+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"27'16","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.016128+0000","last_change":"2024-04-27T03:30:13.376574+0000","last_active":"2024-04-27T03:33:21.016128+0000","last_peered":"2024-04-27T03:33:21.016128+0000","last_clean":"2024-04-27T03:33:21.016128+0000","last_became_active":"2024-04-27T03:30:13.376288+0000","last_became_peered":"2024-04-27T03:30:13.376288+0000","last_unstale":"2024-04-27T03:33:21.016128+0000","last_undegraded":"2024-04-27T03:33:21.016128+0000","last_fullsized":"2024-04-27T03:33:21.016128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298576+0000","last_change":"2024-04-27T03:30:11.355274+0000","last_active":"2024-04-27T03:32:44.298576+0000","last_peered":"2024-04-27T03:32:44.298576+0000","last_clean":"2024-04-27T03:32:44.298576+0000","last_became_active":"2024-04-27T03:30:11.354989+0000","last_became_peered":"2024-04-27T03:30:11.354989+0000","last_unstale":"2024-04-27T03:32:44.298576+0000","last_undegraded":"2024-04-27T03:32:44.298576+0000","last_fullsized":"2024-04-27T03:32:44.298576+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"27'6","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.200055+0000","last_change":"2024-04-27T03:34:04.200104+0000","last_active":"2024-04-27T03:34:04.200055+0000","last_peered":"2024-04-27T03:34:04.200055+0000","last_clean":"2024-04-27T03:34:04.200055+0000","last_became_active":"2024-04-27T03:30:13.373424+0000","last_became_peered":"2024-04-27T03:30:13.373424+0000","last_unstale":"2024-04-27T03:34:04.200055+0000","last_undegraded":"2024-04-27T03:34:04.200055+0000","last_fullsized":"2024-04-27T03:34:04.200055+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:04.199924+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:04.199924+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.199924+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:59.695293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.660727+0000","last_change":"2024-04-27T03:30:11.361019+0000","last_active":"2024-04-27T03:33:20.660727+0000","last_peered":"2024-04-27T03:33:20.660727+0000","last_clean":"2024-04-27T03:33:20.660727+0000","last_became_active":"2024-04-27T03:30:11.360219+0000","last_became_peered":"2024-04-27T03:30:11.360219+0000","last_unstale":"2024-04-27T03:33:20.660727+0000","last_undegraded":"2024-04-27T03:33:20.660727+0000","last_fullsized":"2024-04-27T03:33:20.660727+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"25'8","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.275540+0000","last_change":"2024-04-27T03:34:05.275565+0000","last_active":"2024-04-27T03:34:05.275540+0000","last_peered":"2024-04-27T03:34:05.275540+0000","last_clean":"2024-04-27T03:34:05.275540+0000","last_became_active":"2024-04-27T03:30:13.385665+0000","last_became_peered":"2024-04-27T03:30:13.385665+0000","last_unstale":"2024-04-27T03:34:05.275540+0000","last_undegraded":"2024-04-27T03:34:05.275540+0000","last_fullsized":"2024-04-27T03:34:05.275540+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'8","last_scrub_stamp":"2024-04-27T03:34:05.275413+0000","last_deep_scrub":"25'8","last_deep_scrub_stamp":"2024-04-27T03:34:05.275413+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.275413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:20:59.991475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298491+0000","last_change":"2024-04-27T03:30:11.373049+0000","last_active":"2024-04-27T03:32:44.298491+0000","last_peered":"2024-04-27T03:32:44.298491+0000","last_clean":"2024-04-27T03:32:44.298491+0000","last_became_active":"2024-04-27T03:30:11.372704+0000","last_became_peered":"2024-04-27T03:30:11.372704+0000","last_unstale":"2024-04-27T03:32:44.298491+0000","last_undegraded":"2024-04-27T03:32:44.298491+0000","last_fullsized":"2024-04-27T03:32:44.298491+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"27'4","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.771755+0000","last_change":"2024-04-27T03:33:56.771780+0000","last_active":"2024-04-27T03:33:56.771755+0000","last_peered":"2024-04-27T03:33:56.771755+0000","last_clean":"2024-04-27T03:33:56.771755+0000","last_became_active":"2024-04-27T03:30:13.374199+0000","last_became_peered":"2024-04-27T03:30:13.374199+0000","last_unstale":"2024-04-27T03:33:56.771755+0000","last_undegraded":"2024-04-27T03:33:56.771755+0000","last_fullsized":"2024-04-27T03:33:56.771755+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:56.771668+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:56.771668+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.771668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:18.489287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.929186+0000","last_change":"2024-04-27T03:30:11.358470+0000","last_active":"2024-04-27T03:33:20.929186+0000","last_peered":"2024-04-27T03:33:20.929186+0000","last_clean":"2024-04-27T03:33:20.929186+0000","last_became_active":"2024-04-27T03:30:11.358181+0000","last_became_peered":"2024-04-27T03:30:11.358181+0000","last_unstale":"2024-04-27T03:33:20.929186+0000","last_undegraded":"2024-04-27T03:33:20.929186+0000","last_fullsized":"2024-04-27T03:33:20.929186+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"27'4","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.810866+0000","last_change":"2024-04-27T03:34:00.810889+0000","last_active":"2024-04-27T03:34:00.810866+0000","last_peered":"2024-04-27T03:34:00.810866+0000","last_clean":"2024-04-27T03:34:00.810866+0000","last_became_active":"2024-04-27T03:30:13.379059+0000","last_became_peered":"2024-04-27T03:30:13.379059+0000","last_unstale":"2024-04-27T03:34:00.810866+0000","last_undegraded":"2024-04-27T03:34:00.810866+0000","last_fullsized":"2024-04-27T03:34:00.810866+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:00.810774+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:00.810774+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.810774+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:01.151920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"27'6","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.269064+0000","last_change":"2024-04-27T03:34:06.269111+0000","last_active":"2024-04-27T03:34:06.269064+0000","last_peered":"2024-04-27T03:34:06.269064+0000","last_clean":"2024-04-27T03:34:06.269064+0000","last_became_active":"2024-04-27T03:30:13.383694+0000","last_became_peered":"2024-04-27T03:30:13.383694+0000","last_unstale":"2024-04-27T03:34:06.269064+0000","last_undegraded":"2024-04-27T03:34:06.269064+0000","last_fullsized":"2024-04-27T03:34:06.269064+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:06.268965+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:06.268965+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.268965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:49.525413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297382+0000","last_change":"2024-04-27T03:30:11.370455+0000","last_active":"2024-04-27T03:32:39.297382+0000","last_peered":"2024-04-27T03:32:39.297382+0000","last_clean":"2024-04-27T03:32:39.297382+0000","last_became_active":"2024-04-27T03:30:11.370157+0000","last_became_peered":"2024-04-27T03:30:11.370157+0000","last_unstale":"2024-04-27T03:32:39.297382+0000","last_undegraded":"2024-04-27T03:32:39.297382+0000","last_fullsized":"2024-04-27T03:32:39.297382+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"27'6","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.201544+0000","last_change":"2024-04-27T03:34:03.201590+0000","last_active":"2024-04-27T03:34:03.201544+0000","last_peered":"2024-04-27T03:34:03.201544+0000","last_clean":"2024-04-27T03:34:03.201544+0000","last_became_active":"2024-04-27T03:30:13.372973+0000","last_became_peered":"2024-04-27T03:30:13.372973+0000","last_unstale":"2024-04-27T03:34:03.201544+0000","last_undegraded":"2024-04-27T03:34:03.201544+0000","last_fullsized":"2024-04-27T03:34:03.201544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:03.201445+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:03.201445+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.201445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:57.765391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"27'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.817465+0000","last_change":"2024-04-27T03:34:03.817515+0000","last_active":"2024-04-27T03:34:03.817465+0000","last_peered":"2024-04-27T03:34:03.817465+0000","last_clean":"2024-04-27T03:34:03.817465+0000","last_became_active":"2024-04-27T03:30:11.363109+0000","last_became_peered":"2024-04-27T03:30:11.363109+0000","last_unstale":"2024-04-27T03:34:03.817465+0000","last_undegraded":"2024-04-27T03:34:03.817465+0000","last_fullsized":"2024-04-27T03:34:03.817465+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:03.817361+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:03.817361+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.817361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:30.353138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"27'10","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.121833+0000","last_change":"2024-04-27T03:33:58.121867+0000","last_active":"2024-04-27T03:33:58.121833+0000","last_peered":"2024-04-27T03:33:58.121833+0000","last_clean":"2024-04-27T03:33:58.121833+0000","last_became_active":"2024-04-27T03:30:13.383742+0000","last_became_peered":"2024-04-27T03:30:13.383742+0000","last_unstale":"2024-04-27T03:33:58.121833+0000","last_undegraded":"2024-04-27T03:33:58.121833+0000","last_fullsized":"2024-04-27T03:33:58.121833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:33:58.121739+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:33:58.121739+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.121739+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:37.923650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"25'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297324+0000","last_change":"2024-04-27T03:30:11.370635+0000","last_active":"2024-04-27T03:32:39.297324+0000","last_peered":"2024-04-27T03:32:39.297324+0000","last_clean":"2024-04-27T03:32:39.297324+0000","last_became_active":"2024-04-27T03:30:11.370380+0000","last_became_peered":"2024-04-27T03:30:11.370380+0000","last_unstale":"2024-04-27T03:32:39.297324+0000","last_undegraded":"2024-04-27T03:32:39.297324+0000","last_fullsized":"2024-04-27T03:32:39.297324+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"27'6","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.016547+0000","last_change":"2024-04-27T03:31:01.446266+0000","last_active":"2024-04-27T03:33:21.016547+0000","last_peered":"2024-04-27T03:33:21.016547+0000","last_clean":"2024-04-27T03:33:21.016547+0000","last_became_active":"2024-04-27T03:31:01.440762+0000","last_became_peered":"2024-04-27T03:31:01.440762+0000","last_unstale":"2024-04-27T03:33:21.016547+0000","last_undegraded":"2024-04-27T03:33:21.016547+0000","last_fullsized":"2024-04-27T03:33:21.016547+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297309+0000","last_change":"2024-04-27T03:30:11.369834+0000","last_active":"2024-04-27T03:32:39.297309+0000","last_peered":"2024-04-27T03:32:39.297309+0000","last_clean":"2024-04-27T03:32:39.297309+0000","last_became_active":"2024-04-27T03:30:11.369551+0000","last_became_peered":"2024-04-27T03:30:11.369551+0000","last_unstale":"2024-04-27T03:32:39.297309+0000","last_undegraded":"2024-04-27T03:32:39.297309+0000","last_fullsized":"2024-04-27T03:32:39.297309+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"27'2","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.850385+0000","last_change":"2024-04-27T03:34:04.850412+0000","last_active":"2024-04-27T03:34:04.850385+0000","last_peered":"2024-04-27T03:34:04.850385+0000","last_clean":"2024-04-27T03:34:04.850385+0000","last_became_active":"2024-04-27T03:30:13.381901+0000","last_became_peered":"2024-04-27T03:30:13.381901+0000","last_unstale":"2024-04-27T03:34:04.850385+0000","last_undegraded":"2024-04-27T03:34:04.850385+0000","last_fullsized":"2024-04-27T03:34:04.850385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:04.850293+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:04.850293+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.850293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:29.194843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.291386+0000","last_change":"2024-04-27T03:30:11.361129+0000","last_active":"2024-04-27T03:32:39.291386+0000","last_peered":"2024-04-27T03:32:39.291386+0000","last_clean":"2024-04-27T03:32:39.291386+0000","last_became_active":"2024-04-27T03:30:11.360515+0000","last_became_peered":"2024-04-27T03:30:11.360515+0000","last_unstale":"2024-04-27T03:32:39.291386+0000","last_undegraded":"2024-04-27T03:32:39.291386+0000","last_fullsized":"2024-04-27T03:32:39.291386+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"27'4","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.155138+0000","last_change":"2024-04-27T03:33:59.155166+0000","last_active":"2024-04-27T03:33:59.155138+0000","last_peered":"2024-04-27T03:33:59.155138+0000","last_clean":"2024-04-27T03:33:59.155138+0000","last_became_active":"2024-04-27T03:30:13.378932+0000","last_became_peered":"2024-04-27T03:30:13.378932+0000","last_unstale":"2024-04-27T03:33:59.155138+0000","last_undegraded":"2024-04-27T03:33:59.155138+0000","last_fullsized":"2024-04-27T03:33:59.155138+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:59.155047+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:59.155047+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.155047+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:36.999661+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296759+0000","last_change":"2024-04-27T03:30:11.372964+0000","last_active":"2024-04-27T03:32:34.296759+0000","last_peered":"2024-04-27T03:32:34.296759+0000","last_clean":"2024-04-27T03:32:34.296759+0000","last_became_active":"2024-04-27T03:30:11.372673+0000","last_became_peered":"2024-04-27T03:30:11.372673+0000","last_unstale":"2024-04-27T03:32:34.296759+0000","last_undegraded":"2024-04-27T03:32:34.296759+0000","last_fullsized":"2024-04-27T03:32:34.296759+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"27'35","reported_seq":100,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.028786+0000","last_change":"2024-04-27T03:30:21.413597+0000","last_active":"2024-04-27T03:33:21.028786+0000","last_peered":"2024-04-27T03:33:21.028786+0000","last_clean":"2024-04-27T03:33:21.028786+0000","last_became_active":"2024-04-27T03:30:21.413255+0000","last_became_peered":"2024-04-27T03:30:21.413255+0000","last_unstale":"2024-04-27T03:33:21.028786+0000","last_undegraded":"2024-04-27T03:33:21.028786+0000","last_fullsized":"2024-04-27T03:33:21.028786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"27'3","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.070022+0000","last_change":"2024-04-27T03:30:13.377382+0000","last_active":"2024-04-27T03:33:21.070022+0000","last_peered":"2024-04-27T03:33:21.070022+0000","last_clean":"2024-04-27T03:33:21.070022+0000","last_became_active":"2024-04-27T03:30:13.377110+0000","last_became_peered":"2024-04-27T03:30:13.377110+0000","last_unstale":"2024-04-27T03:33:21.070022+0000","last_undegraded":"2024-04-27T03:33:21.070022+0000","last_fullsized":"2024-04-27T03:33:21.070022+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296793+0000","last_change":"2024-04-27T03:30:11.360152+0000","last_active":"2024-04-27T03:32:34.296793+0000","last_peered":"2024-04-27T03:32:34.296793+0000","last_clean":"2024-04-27T03:32:34.296793+0000","last_became_active":"2024-04-27T03:30:11.359892+0000","last_became_peered":"2024-04-27T03:30:11.359892+0000","last_unstale":"2024-04-27T03:32:34.296793+0000","last_undegraded":"2024-04-27T03:32:34.296793+0000","last_fullsized":"2024-04-27T03:32:34.296793+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.148150+0000","last_change":"2024-04-27T03:33:38.148173+0000","last_active":"2024-04-27T03:33:38.148150+0000","last_peered":"2024-04-27T03:33:38.148150+0000","last_clean":"2024-04-27T03:33:38.148150+0000","last_became_active":"2024-04-27T03:30:15.380864+0000","last_became_peered":"2024-04-27T03:30:15.380864+0000","last_unstale":"2024-04-27T03:33:38.148150+0000","last_undegraded":"2024-04-27T03:33:38.148150+0000","last_fullsized":"2024-04-27T03:33:38.148150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:38.148061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:38.148061+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.148061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:53.865374+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"27'50","reported_seq":117,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.088201+0000","last_change":"2024-04-27T03:33:43.088242+0000","last_active":"2024-04-27T03:33:43.088201+0000","last_peered":"2024-04-27T03:33:43.088201+0000","last_clean":"2024-04-27T03:33:43.088201+0000","last_became_active":"2024-04-27T03:30:17.387391+0000","last_became_peered":"2024-04-27T03:30:17.387391+0000","last_unstale":"2024-04-27T03:33:43.088201+0000","last_undegraded":"2024-04-27T03:33:43.088201+0000","last_fullsized":"2024-04-27T03:33:43.088201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'50","last_scrub_stamp":"2024-04-27T03:33:43.088100+0000","last_deep_scrub":"27'50","last_deep_scrub_stamp":"2024-04-27T03:33:43.088100+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.088100+0000","objects_scrubbed":6,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:13:27.492179+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":45,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.971326+0000","last_change":"2024-04-27T03:33:44.971352+0000","last_active":"2024-04-27T03:33:44.971326+0000","last_peered":"2024-04-27T03:33:44.971326+0000","last_clean":"2024-04-27T03:33:44.971326+0000","last_became_active":"2024-04-27T03:30:19.407637+0000","last_became_peered":"2024-04-27T03:30:19.407637+0000","last_unstale":"2024-04-27T03:33:44.971326+0000","last_undegraded":"2024-04-27T03:33:44.971326+0000","last_fullsized":"2024-04-27T03:33:44.971326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:44.971239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:44.971239+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.971239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:22.448440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"27'9","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:49.299125+0000","last_change":"2024-04-27T03:30:21.414223+0000","last_active":"2024-04-27T03:32:49.299125+0000","last_peered":"2024-04-27T03:32:49.299125+0000","last_clean":"2024-04-27T03:32:49.299125+0000","last_became_active":"2024-04-27T03:30:21.413940+0000","last_became_peered":"2024-04-27T03:30:21.413940+0000","last_unstale":"2024-04-27T03:32:49.299125+0000","last_undegraded":"2024-04-27T03:32:49.299125+0000","last_fullsized":"2024-04-27T03:32:49.299125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"27'16","reported_seq":76,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.265928+0000","last_change":"2024-04-27T03:34:09.266003+0000","last_active":"2024-04-27T03:34:09.265928+0000","last_peered":"2024-04-27T03:34:09.265928+0000","last_clean":"2024-04-27T03:34:09.265928+0000","last_became_active":"2024-04-27T03:30:13.381487+0000","last_became_peered":"2024-04-27T03:30:13.381487+0000","last_unstale":"2024-04-27T03:34:09.265928+0000","last_undegraded":"2024-04-27T03:34:09.265928+0000","last_fullsized":"2024-04-27T03:34:09.265928+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:09.265817+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:09.265817+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.265817+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:46.218428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296267+0000","last_change":"2024-04-27T03:30:11.359690+0000","last_active":"2024-04-27T03:32:34.296267+0000","last_peered":"2024-04-27T03:32:34.296267+0000","last_clean":"2024-04-27T03:32:34.296267+0000","last_became_active":"2024-04-27T03:30:11.359446+0000","last_became_peered":"2024-04-27T03:30:11.359446+0000","last_unstale":"2024-04-27T03:32:34.296267+0000","last_undegraded":"2024-04-27T03:32:34.296267+0000","last_fullsized":"2024-04-27T03:32:34.296267+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.671578+0000","last_change":"2024-04-27T03:33:55.671603+0000","last_active":"2024-04-27T03:33:55.671578+0000","last_peered":"2024-04-27T03:33:55.671578+0000","last_clean":"2024-04-27T03:33:55.671578+0000","last_became_active":"2024-04-27T03:30:15.380313+0000","last_became_peered":"2024-04-27T03:30:15.380313+0000","last_unstale":"2024-04-27T03:33:55.671578+0000","last_undegraded":"2024-04-27T03:33:55.671578+0000","last_fullsized":"2024-04-27T03:33:55.671578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:55.671471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:55.671471+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.671471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:20.643729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"27'16","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.841774+0000","last_change":"2024-04-27T03:33:37.841840+0000","last_active":"2024-04-27T03:33:37.841774+0000","last_peered":"2024-04-27T03:33:37.841774+0000","last_clean":"2024-04-27T03:33:37.841774+0000","last_became_active":"2024-04-27T03:30:17.389126+0000","last_became_peered":"2024-04-27T03:30:17.389126+0000","last_unstale":"2024-04-27T03:33:37.841774+0000","last_undegraded":"2024-04-27T03:33:37.841774+0000","last_fullsized":"2024-04-27T03:33:37.841774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:33:37.841643+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:33:37.841643+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.841643+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:42.665962+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.837474+0000","last_change":"2024-04-27T03:33:36.837497+0000","last_active":"2024-04-27T03:33:36.837474+0000","last_peered":"2024-04-27T03:33:36.837474+0000","last_clean":"2024-04-27T03:33:36.837474+0000","last_became_active":"2024-04-27T03:30:19.405170+0000","last_became_peered":"2024-04-27T03:30:19.405170+0000","last_unstale":"2024-04-27T03:33:36.837474+0000","last_undegraded":"2024-04-27T03:33:36.837474+0000","last_fullsized":"2024-04-27T03:33:36.837474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.837388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.837388+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.837388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:14.962289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'6","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.661566+0000","last_change":"2024-04-27T03:33:58.661623+0000","last_active":"2024-04-27T03:33:58.661566+0000","last_peered":"2024-04-27T03:33:58.661566+0000","last_clean":"2024-04-27T03:33:58.661566+0000","last_became_active":"2024-04-27T03:30:21.407917+0000","last_became_peered":"2024-04-27T03:30:21.407917+0000","last_unstale":"2024-04-27T03:33:58.661566+0000","last_undegraded":"2024-04-27T03:33:58.661566+0000","last_fullsized":"2024-04-27T03:33:58.661566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:58.661469+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:58.661469+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.661469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:52.661163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.804760+0000","last_change":"2024-04-27T03:30:13.378024+0000","last_active":"2024-04-27T03:33:20.804760+0000","last_peered":"2024-04-27T03:33:20.804760+0000","last_clean":"2024-04-27T03:33:20.804760+0000","last_became_active":"2024-04-27T03:30:13.377423+0000","last_became_peered":"2024-04-27T03:30:13.377423+0000","last_unstale":"2024-04-27T03:33:20.804760+0000","last_undegraded":"2024-04-27T03:33:20.804760+0000","last_fullsized":"2024-04-27T03:33:20.804760+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296713+0000","last_change":"2024-04-27T03:30:11.359542+0000","last_active":"2024-04-27T03:32:34.296713+0000","last_peered":"2024-04-27T03:32:34.296713+0000","last_clean":"2024-04-27T03:32:34.296713+0000","last_became_active":"2024-04-27T03:30:11.359261+0000","last_became_peered":"2024-04-27T03:30:11.359261+0000","last_unstale":"2024-04-27T03:32:34.296713+0000","last_undegraded":"2024-04-27T03:32:34.296713+0000","last_fullsized":"2024-04-27T03:32:34.296713+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.753269+0000","last_change":"2024-04-27T03:33:47.753294+0000","last_active":"2024-04-27T03:33:47.753269+0000","last_peered":"2024-04-27T03:33:47.753269+0000","last_clean":"2024-04-27T03:33:47.753269+0000","last_became_active":"2024-04-27T03:30:15.383618+0000","last_became_peered":"2024-04-27T03:30:15.383618+0000","last_unstale":"2024-04-27T03:33:47.753269+0000","last_undegraded":"2024-04-27T03:33:47.753269+0000","last_fullsized":"2024-04-27T03:33:47.753269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:47.753177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:47.753177+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.753177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:54.718147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"27'30","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.888348+0000","last_change":"2024-04-27T03:33:37.888412+0000","last_active":"2024-04-27T03:33:37.888348+0000","last_peered":"2024-04-27T03:33:37.888348+0000","last_clean":"2024-04-27T03:33:37.888348+0000","last_became_active":"2024-04-27T03:30:17.392223+0000","last_became_peered":"2024-04-27T03:30:17.392223+0000","last_unstale":"2024-04-27T03:33:37.888348+0000","last_undegraded":"2024-04-27T03:33:37.888348+0000","last_fullsized":"2024-04-27T03:33:37.888348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-04-27T03:33:37.888233+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-04-27T03:33:37.888233+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.888233+0000","objects_scrubbed":7,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:27:30.955280+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"27'286","reported_seq":394,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.206383+0000","last_change":"2024-04-27T03:34:03.206432+0000","last_active":"2024-04-27T03:34:03.206383+0000","last_peered":"2024-04-27T03:34:03.206383+0000","last_clean":"2024-04-27T03:34:03.206383+0000","last_became_active":"2024-04-27T03:30:19.405730+0000","last_became_peered":"2024-04-27T03:30:19.405730+0000","last_unstale":"2024-04-27T03:34:03.206383+0000","last_undegraded":"2024-04-27T03:34:03.206383+0000","last_fullsized":"2024-04-27T03:34:03.206383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'286","last_scrub_stamp":"2024-04-27T03:34:03.206276+0000","last_deep_scrub":"27'286","last_deep_scrub_stamp":"2024-04-27T03:34:03.206276+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.206276+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:25.670340+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:54.299992+0000","last_change":"2024-04-27T03:30:21.409739+0000","last_active":"2024-04-27T03:32:54.299992+0000","last_peered":"2024-04-27T03:32:54.299992+0000","last_clean":"2024-04-27T03:32:54.299992+0000","last_became_active":"2024-04-27T03:30:21.409156+0000","last_became_peered":"2024-04-27T03:30:21.409156+0000","last_unstale":"2024-04-27T03:32:54.299992+0000","last_undegraded":"2024-04-27T03:32:54.299992+0000","last_fullsized":"2024-04-27T03:32:54.299992+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"27'6","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.704469+0000","last_change":"2024-04-27T03:33:59.704499+0000","last_active":"2024-04-27T03:33:59.704469+0000","last_peered":"2024-04-27T03:33:59.704469+0000","last_clean":"2024-04-27T03:33:59.704469+0000","last_became_active":"2024-04-27T03:30:13.385491+0000","last_became_peered":"2024-04-27T03:30:13.385491+0000","last_unstale":"2024-04-27T03:33:59.704469+0000","last_undegraded":"2024-04-27T03:33:59.704469+0000","last_fullsized":"2024-04-27T03:33:59.704469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:59.704383+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:59.704383+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.704383+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:13:15.421696+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296216+0000","last_change":"2024-04-27T03:30:11.370401+0000","last_active":"2024-04-27T03:32:34.296216+0000","last_peered":"2024-04-27T03:32:34.296216+0000","last_clean":"2024-04-27T03:32:34.296216+0000","last_became_active":"2024-04-27T03:30:11.369939+0000","last_became_peered":"2024-04-27T03:30:11.369939+0000","last_unstale":"2024-04-27T03:32:34.296216+0000","last_undegraded":"2024-04-27T03:32:34.296216+0000","last_fullsized":"2024-04-27T03:32:34.296216+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.134971+0000","last_change":"2024-04-27T03:33:33.134999+0000","last_active":"2024-04-27T03:33:33.134971+0000","last_peered":"2024-04-27T03:33:33.134971+0000","last_clean":"2024-04-27T03:33:33.134971+0000","last_became_active":"2024-04-27T03:30:15.384114+0000","last_became_peered":"2024-04-27T03:30:15.384114+0000","last_unstale":"2024-04-27T03:33:33.134971+0000","last_undegraded":"2024-04-27T03:33:33.134971+0000","last_fullsized":"2024-04-27T03:33:33.134971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:33.134840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:33.134840+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.134840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:57.479389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"27'43","reported_seq":104,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.859628+0000","last_change":"2024-04-27T03:33:38.859665+0000","last_active":"2024-04-27T03:33:38.859628+0000","last_peered":"2024-04-27T03:33:38.859628+0000","last_clean":"2024-04-27T03:33:38.859628+0000","last_became_active":"2024-04-27T03:30:17.391880+0000","last_became_peered":"2024-04-27T03:30:17.391880+0000","last_unstale":"2024-04-27T03:33:38.859628+0000","last_undegraded":"2024-04-27T03:33:38.859628+0000","last_fullsized":"2024-04-27T03:33:38.859628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'43","last_scrub_stamp":"2024-04-27T03:33:38.859525+0000","last_deep_scrub":"27'43","last_deep_scrub_stamp":"2024-04-27T03:33:38.859525+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.859525+0000","objects_scrubbed":5,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:06.442676+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"27'285","reported_seq":417,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.785523+0000","last_change":"2024-04-27T03:33:35.785553+0000","last_active":"2024-04-27T03:33:35.785523+0000","last_peered":"2024-04-27T03:33:35.785523+0000","last_clean":"2024-04-27T03:33:35.785523+0000","last_became_active":"2024-04-27T03:30:19.404744+0000","last_became_peered":"2024-04-27T03:30:19.404744+0000","last_unstale":"2024-04-27T03:33:35.785523+0000","last_undegraded":"2024-04-27T03:33:35.785523+0000","last_fullsized":"2024-04-27T03:33:35.785523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:35.785437+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:35.785437+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.785437+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:23.864435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'7","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.165585+0000","last_change":"2024-04-27T03:33:49.165611+0000","last_active":"2024-04-27T03:33:49.165585+0000","last_peered":"2024-04-27T03:33:49.165585+0000","last_clean":"2024-04-27T03:33:49.165585+0000","last_became_active":"2024-04-27T03:30:13.387335+0000","last_became_peered":"2024-04-27T03:30:13.387335+0000","last_unstale":"2024-04-27T03:33:49.165585+0000","last_undegraded":"2024-04-27T03:33:49.165585+0000","last_fullsized":"2024-04-27T03:33:49.165585+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:33:49.165500+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:33:49.165500+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.165500+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:22.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.841302+0000","last_change":"2024-04-27T03:34:01.841348+0000","last_active":"2024-04-27T03:34:01.841302+0000","last_peered":"2024-04-27T03:34:01.841302+0000","last_clean":"2024-04-27T03:34:01.841302+0000","last_became_active":"2024-04-27T03:30:11.371600+0000","last_became_peered":"2024-04-27T03:30:11.371600+0000","last_unstale":"2024-04-27T03:34:01.841302+0000","last_undegraded":"2024-04-27T03:34:01.841302+0000","last_fullsized":"2024-04-27T03:34:01.841302+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:01.841193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:01.841193+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.841193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:21:13.198341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.990712+0000","last_change":"2024-04-27T03:33:46.990760+0000","last_active":"2024-04-27T03:33:46.990712+0000","last_peered":"2024-04-27T03:33:46.990712+0000","last_clean":"2024-04-27T03:33:46.990712+0000","last_became_active":"2024-04-27T03:30:15.381359+0000","last_became_peered":"2024-04-27T03:30:15.381359+0000","last_unstale":"2024-04-27T03:33:46.990712+0000","last_undegraded":"2024-04-27T03:33:46.990712+0000","last_fullsized":"2024-04-27T03:33:46.990712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:46.990601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:46.990601+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.990601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:08.708219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"27'58","reported_seq":124,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.809607+0000","last_change":"2024-04-27T03:33:41.809671+0000","last_active":"2024-04-27T03:33:41.809607+0000","last_peered":"2024-04-27T03:33:41.809607+0000","last_clean":"2024-04-27T03:33:41.809607+0000","last_became_active":"2024-04-27T03:30:17.390021+0000","last_became_peered":"2024-04-27T03:30:17.390021+0000","last_unstale":"2024-04-27T03:33:41.809607+0000","last_undegraded":"2024-04-27T03:33:41.809607+0000","last_fullsized":"2024-04-27T03:33:41.809607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'58","last_scrub_stamp":"2024-04-27T03:33:41.809501+0000","last_deep_scrub":"27'58","last_deep_scrub_stamp":"2024-04-27T03:33:41.809501+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.809501+0000","objects_scrubbed":10,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:46.251050+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":30,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.738236+0000","last_change":"2024-04-27T03:33:32.738255+0000","last_active":"2024-04-27T03:33:32.738236+0000","last_peered":"2024-04-27T03:33:32.738236+0000","last_clean":"2024-04-27T03:33:32.738236+0000","last_became_active":"2024-04-27T03:30:19.404948+0000","last_became_peered":"2024-04-27T03:30:19.404948+0000","last_unstale":"2024-04-27T03:33:32.738236+0000","last_undegraded":"2024-04-27T03:33:32.738236+0000","last_fullsized":"2024-04-27T03:33:32.738236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:32.738165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:32.738165+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.738165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:19.681571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"27'3","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295334+0000","last_change":"2024-04-27T03:30:21.412598+0000","last_active":"2024-04-27T03:32:29.295334+0000","last_peered":"2024-04-27T03:32:29.295334+0000","last_clean":"2024-04-27T03:32:29.295334+0000","last_became_active":"2024-04-27T03:30:21.412331+0000","last_became_peered":"2024-04-27T03:30:21.412331+0000","last_unstale":"2024-04-27T03:32:29.295334+0000","last_undegraded":"2024-04-27T03:32:29.295334+0000","last_fullsized":"2024-04-27T03:32:29.295334+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"25'6","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.184692+0000","last_change":"2024-04-27T03:34:06.184718+0000","last_active":"2024-04-27T03:34:06.184692+0000","last_peered":"2024-04-27T03:34:06.184692+0000","last_clean":"2024-04-27T03:34:06.184692+0000","last_became_active":"2024-04-27T03:30:13.373226+0000","last_became_peered":"2024-04-27T03:30:13.373226+0000","last_unstale":"2024-04-27T03:34:06.184692+0000","last_undegraded":"2024-04-27T03:34:06.184692+0000","last_fullsized":"2024-04-27T03:34:06.184692+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'6","last_scrub_stamp":"2024-04-27T03:34:06.184601+0000","last_deep_scrub":"25'6","last_deep_scrub_stamp":"2024-04-27T03:34:06.184601+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.184601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:06.834111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.791842+0000","last_change":"2024-04-27T03:33:59.791865+0000","last_active":"2024-04-27T03:33:59.791842+0000","last_peered":"2024-04-27T03:33:59.791842+0000","last_clean":"2024-04-27T03:33:59.791842+0000","last_became_active":"2024-04-27T03:30:11.373430+0000","last_became_peered":"2024-04-27T03:30:11.373430+0000","last_unstale":"2024-04-27T03:33:59.791842+0000","last_undegraded":"2024-04-27T03:33:59.791842+0000","last_fullsized":"2024-04-27T03:33:59.791842+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:59.791751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:59.791751+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.791751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:23.025879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"18'1","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.552724+0000","last_change":"2024-04-27T03:31:01.447513+0000","last_active":"2024-04-27T03:33:27.552724+0000","last_peered":"2024-04-27T03:33:27.552724+0000","last_clean":"2024-04-27T03:33:27.552724+0000","last_became_active":"2024-04-27T03:31:01.443259+0000","last_became_peered":"2024-04-27T03:31:01.443259+0000","last_unstale":"2024-04-27T03:33:27.552724+0000","last_undegraded":"2024-04-27T03:33:27.552724+0000","last_fullsized":"2024-04-27T03:33:27.552724+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"27'47","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.230606+0000","last_change":"2024-04-27T03:34:04.230642+0000","last_active":"2024-04-27T03:34:04.230606+0000","last_peered":"2024-04-27T03:34:04.230606+0000","last_clean":"2024-04-27T03:34:04.230606+0000","last_became_active":"2024-04-27T03:30:17.388994+0000","last_became_peered":"2024-04-27T03:30:17.388994+0000","last_unstale":"2024-04-27T03:34:04.230606+0000","last_undegraded":"2024-04-27T03:34:04.230606+0000","last_fullsized":"2024-04-27T03:34:04.230606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-04-27T03:34:04.230519+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-04-27T03:34:04.230519+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.230519+0000","objects_scrubbed":5,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:43:53.086291+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:30.805151+0000","last_change":"2024-04-27T03:33:30.805201+0000","last_active":"2024-04-27T03:33:30.805151+0000","last_peered":"2024-04-27T03:33:30.805151+0000","last_clean":"2024-04-27T03:33:30.805151+0000","last_became_active":"2024-04-27T03:30:19.405334+0000","last_became_peered":"2024-04-27T03:30:19.405334+0000","last_unstale":"2024-04-27T03:33:30.805151+0000","last_undegraded":"2024-04-27T03:33:30.805151+0000","last_fullsized":"2024-04-27T03:33:30.805151+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:30.805019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:30.805019+0000","last_clean_scrub_stamp":"2024-04-27T03:33:30.805019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:46:37.451973+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"27'4","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.099583+0000","last_change":"2024-04-27T03:33:57.099611+0000","last_active":"2024-04-27T03:33:57.099583+0000","last_peered":"2024-04-27T03:33:57.099583+0000","last_clean":"2024-04-27T03:33:57.099583+0000","last_became_active":"2024-04-27T03:30:21.409922+0000","last_became_peered":"2024-04-27T03:30:21.409922+0000","last_unstale":"2024-04-27T03:33:57.099583+0000","last_undegraded":"2024-04-27T03:33:57.099583+0000","last_fullsized":"2024-04-27T03:33:57.099583+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:57.099495+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:57.099495+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.099495+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:01.218125+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"27'6","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.819531+0000","last_change":"2024-04-27T03:34:02.819584+0000","last_active":"2024-04-27T03:34:02.819531+0000","last_peered":"2024-04-27T03:34:02.819531+0000","last_clean":"2024-04-27T03:34:02.819531+0000","last_became_active":"2024-04-27T03:30:13.381270+0000","last_became_peered":"2024-04-27T03:30:13.381270+0000","last_unstale":"2024-04-27T03:34:02.819531+0000","last_undegraded":"2024-04-27T03:34:02.819531+0000","last_fullsized":"2024-04-27T03:34:02.819531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:02.819411+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:02.819411+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.819411+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:59.129585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295920+0000","last_change":"2024-04-27T03:30:11.372966+0000","last_active":"2024-04-27T03:32:29.295920+0000","last_peered":"2024-04-27T03:32:29.295920+0000","last_clean":"2024-04-27T03:32:29.295920+0000","last_became_active":"2024-04-27T03:30:11.372583+0000","last_became_peered":"2024-04-27T03:30:11.372583+0000","last_unstale":"2024-04-27T03:32:29.295920+0000","last_undegraded":"2024-04-27T03:32:29.295920+0000","last_fullsized":"2024-04-27T03:32:29.295920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.709284+0000","last_change":"2024-04-27T03:33:51.709320+0000","last_active":"2024-04-27T03:33:51.709284+0000","last_peered":"2024-04-27T03:33:51.709284+0000","last_clean":"2024-04-27T03:33:51.709284+0000","last_became_active":"2024-04-27T03:30:15.382207+0000","last_became_peered":"2024-04-27T03:30:15.382207+0000","last_unstale":"2024-04-27T03:33:51.709284+0000","last_undegraded":"2024-04-27T03:33:51.709284+0000","last_fullsized":"2024-04-27T03:33:51.709284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:51.709192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:51.709192+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.709192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:40.618465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"27'61","reported_seq":131,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.183997+0000","last_change":"2024-04-27T03:34:00.184072+0000","last_active":"2024-04-27T03:34:00.183997+0000","last_peered":"2024-04-27T03:34:00.183997+0000","last_clean":"2024-04-27T03:34:00.183997+0000","last_became_active":"2024-04-27T03:30:17.389242+0000","last_became_peered":"2024-04-27T03:30:17.389242+0000","last_unstale":"2024-04-27T03:34:00.183997+0000","last_undegraded":"2024-04-27T03:34:00.183997+0000","last_fullsized":"2024-04-27T03:34:00.183997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'61","last_scrub_stamp":"2024-04-27T03:34:00.183892+0000","last_deep_scrub":"27'61","last_deep_scrub_stamp":"2024-04-27T03:34:00.183892+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.183892+0000","objects_scrubbed":9,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:30:17.176389+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":62,"num_read_kb":51,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"27'285","reported_seq":360,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.734569+0000","last_change":"2024-04-27T03:33:38.734598+0000","last_active":"2024-04-27T03:33:38.734569+0000","last_peered":"2024-04-27T03:33:38.734569+0000","last_clean":"2024-04-27T03:33:38.734569+0000","last_became_active":"2024-04-27T03:30:19.406364+0000","last_became_peered":"2024-04-27T03:30:19.406364+0000","last_unstale":"2024-04-27T03:33:38.734569+0000","last_undegraded":"2024-04-27T03:33:38.734569+0000","last_fullsized":"2024-04-27T03:33:38.734569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:38.734486+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:38.734486+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.734486+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:11.608629+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.290755+0000","last_change":"2024-04-27T03:30:21.408481+0000","last_active":"2024-04-27T03:32:34.290755+0000","last_peered":"2024-04-27T03:32:34.290755+0000","last_clean":"2024-04-27T03:32:34.290755+0000","last_became_active":"2024-04-27T03:30:21.408168+0000","last_became_peered":"2024-04-27T03:30:21.408168+0000","last_unstale":"2024-04-27T03:32:34.290755+0000","last_undegraded":"2024-04-27T03:32:34.290755+0000","last_fullsized":"2024-04-27T03:32:34.290755+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"27'14","reported_seq":110,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.819117+0000","last_change":"2024-04-27T03:33:55.819144+0000","last_active":"2024-04-27T03:33:55.819117+0000","last_peered":"2024-04-27T03:33:55.819117+0000","last_clean":"2024-04-27T03:33:55.819117+0000","last_became_active":"2024-04-27T03:30:13.371503+0000","last_became_peered":"2024-04-27T03:30:13.371503+0000","last_unstale":"2024-04-27T03:33:55.819117+0000","last_undegraded":"2024-04-27T03:33:55.819117+0000","last_fullsized":"2024-04-27T03:33:55.819117+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:33:55.819026+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:33:55.819026+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.819026+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:53.861728+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295881+0000","last_change":"2024-04-27T03:30:11.357185+0000","last_active":"2024-04-27T03:32:29.295881+0000","last_peered":"2024-04-27T03:32:29.295881+0000","last_clean":"2024-04-27T03:32:29.295881+0000","last_became_active":"2024-04-27T03:30:11.356739+0000","last_became_peered":"2024-04-27T03:30:11.356739+0000","last_unstale":"2024-04-27T03:32:29.295881+0000","last_undegraded":"2024-04-27T03:32:29.295881+0000","last_fullsized":"2024-04-27T03:32:29.295881+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":12,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:14.303020+0000","last_change":"2024-04-27T03:31:01.443432+0000","last_active":"2024-04-27T03:33:14.303020+0000","last_peered":"2024-04-27T03:33:14.303020+0000","last_clean":"2024-04-27T03:33:14.303020+0000","last_became_active":"2024-04-27T03:31:01.443118+0000","last_became_peered":"2024-04-27T03:31:01.443118+0000","last_unstale":"2024-04-27T03:33:14.303020+0000","last_undegraded":"2024-04-27T03:33:14.303020+0000","last_fullsized":"2024-04-27T03:33:14.303020+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"27'79","reported_seq":164,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.115580+0000","last_change":"2024-04-27T03:33:56.115644+0000","last_active":"2024-04-27T03:33:56.115580+0000","last_peered":"2024-04-27T03:33:56.115580+0000","last_clean":"2024-04-27T03:33:56.115580+0000","last_became_active":"2024-04-27T03:30:17.387318+0000","last_became_peered":"2024-04-27T03:30:17.387318+0000","last_unstale":"2024-04-27T03:33:56.115580+0000","last_undegraded":"2024-04-27T03:33:56.115580+0000","last_fullsized":"2024-04-27T03:33:56.115580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'79","last_scrub_stamp":"2024-04-27T03:33:56.115468+0000","last_deep_scrub":"27'79","last_deep_scrub_stamp":"2024-04-27T03:33:56.115468+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.115468+0000","objects_scrubbed":7,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:06.434627+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":79,"num_read_kb":68,"num_write":34,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.701834+0000","last_change":"2024-04-27T03:33:37.701877+0000","last_active":"2024-04-27T03:33:37.701834+0000","last_peered":"2024-04-27T03:33:37.701834+0000","last_clean":"2024-04-27T03:33:37.701834+0000","last_became_active":"2024-04-27T03:30:19.404265+0000","last_became_peered":"2024-04-27T03:30:19.404265+0000","last_unstale":"2024-04-27T03:33:37.701834+0000","last_undegraded":"2024-04-27T03:33:37.701834+0000","last_fullsized":"2024-04-27T03:33:37.701834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:37.701748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:37.701748+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.701748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:42.526067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"27'18","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.124420+0000","last_change":"2024-04-27T03:30:21.414109+0000","last_active":"2024-04-27T03:33:21.124420+0000","last_peered":"2024-04-27T03:33:21.124420+0000","last_clean":"2024-04-27T03:33:21.124420+0000","last_became_active":"2024-04-27T03:30:21.413757+0000","last_became_peered":"2024-04-27T03:30:21.413757+0000","last_unstale":"2024-04-27T03:33:21.124420+0000","last_undegraded":"2024-04-27T03:33:21.124420+0000","last_fullsized":"2024-04-27T03:33:21.124420+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988478+0000","last_change":"2024-04-27T03:30:13.380233+0000","last_active":"2024-04-27T03:33:20.988478+0000","last_peered":"2024-04-27T03:33:20.988478+0000","last_clean":"2024-04-27T03:33:20.988478+0000","last_became_active":"2024-04-27T03:30:13.379861+0000","last_became_peered":"2024-04-27T03:30:13.379861+0000","last_unstale":"2024-04-27T03:33:20.988478+0000","last_undegraded":"2024-04-27T03:33:20.988478+0000","last_fullsized":"2024-04-27T03:33:20.988478+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.289805+0000","last_change":"2024-04-27T03:30:11.357503+0000","last_active":"2024-04-27T03:32:29.289805+0000","last_peered":"2024-04-27T03:32:29.289805+0000","last_clean":"2024-04-27T03:32:29.289805+0000","last_became_active":"2024-04-27T03:30:11.357176+0000","last_became_peered":"2024-04-27T03:30:11.357176+0000","last_unstale":"2024-04-27T03:32:29.289805+0000","last_undegraded":"2024-04-27T03:32:29.289805+0000","last_fullsized":"2024-04-27T03:32:29.289805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.754096+0000","last_change":"2024-04-27T03:33:49.754119+0000","last_active":"2024-04-27T03:33:49.754096+0000","last_peered":"2024-04-27T03:33:49.754096+0000","last_clean":"2024-04-27T03:33:49.754096+0000","last_became_active":"2024-04-27T03:30:15.382290+0000","last_became_peered":"2024-04-27T03:30:15.382290+0000","last_unstale":"2024-04-27T03:33:49.754096+0000","last_undegraded":"2024-04-27T03:33:49.754096+0000","last_fullsized":"2024-04-27T03:33:49.754096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:49.754004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:49.754004+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.754004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:57.154906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"27'46","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.971592+0000","last_change":"2024-04-27T03:33:41.971630+0000","last_active":"2024-04-27T03:33:41.971592+0000","last_peered":"2024-04-27T03:33:41.971592+0000","last_clean":"2024-04-27T03:33:41.971592+0000","last_became_active":"2024-04-27T03:30:17.385902+0000","last_became_peered":"2024-04-27T03:30:17.385902+0000","last_unstale":"2024-04-27T03:33:41.971592+0000","last_undegraded":"2024-04-27T03:33:41.971592+0000","last_fullsized":"2024-04-27T03:33:41.971592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:41.971499+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:41.971499+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.971499+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:48.621217+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.094297+0000","last_change":"2024-04-27T03:33:55.094322+0000","last_active":"2024-04-27T03:33:55.094297+0000","last_peered":"2024-04-27T03:33:55.094297+0000","last_clean":"2024-04-27T03:33:55.094297+0000","last_became_active":"2024-04-27T03:30:19.405450+0000","last_became_peered":"2024-04-27T03:30:19.405450+0000","last_unstale":"2024-04-27T03:33:55.094297+0000","last_undegraded":"2024-04-27T03:33:55.094297+0000","last_fullsized":"2024-04-27T03:33:55.094297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:55.094207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:55.094207+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.094207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:33.101519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"27'9","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.075726+0000","last_change":"2024-04-27T03:30:21.415884+0000","last_active":"2024-04-27T03:33:21.075726+0000","last_peered":"2024-04-27T03:33:21.075726+0000","last_clean":"2024-04-27T03:33:21.075726+0000","last_became_active":"2024-04-27T03:30:21.415494+0000","last_became_peered":"2024-04-27T03:30:21.415494+0000","last_unstale":"2024-04-27T03:33:21.075726+0000","last_undegraded":"2024-04-27T03:33:21.075726+0000","last_fullsized":"2024-04-27T03:33:21.075726+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'6","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.201209+0000","last_change":"2024-04-27T03:33:48.201236+0000","last_active":"2024-04-27T03:33:48.201209+0000","last_peered":"2024-04-27T03:33:48.201209+0000","last_clean":"2024-04-27T03:33:48.201209+0000","last_became_active":"2024-04-27T03:30:13.387533+0000","last_became_peered":"2024-04-27T03:30:13.387533+0000","last_unstale":"2024-04-27T03:33:48.201209+0000","last_undegraded":"2024-04-27T03:33:48.201209+0000","last_fullsized":"2024-04-27T03:33:48.201209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:48.201118+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:48.201118+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.201118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:48.546003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295161+0000","last_change":"2024-04-27T03:30:11.373706+0000","last_active":"2024-04-27T03:32:24.295161+0000","last_peered":"2024-04-27T03:32:24.295161+0000","last_clean":"2024-04-27T03:32:24.295161+0000","last_became_active":"2024-04-27T03:30:11.373440+0000","last_became_peered":"2024-04-27T03:30:11.373440+0000","last_unstale":"2024-04-27T03:32:24.295161+0000","last_undegraded":"2024-04-27T03:32:24.295161+0000","last_fullsized":"2024-04-27T03:32:24.295161+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.730998+0000","last_change":"2024-04-27T03:33:48.731024+0000","last_active":"2024-04-27T03:33:48.730998+0000","last_peered":"2024-04-27T03:33:48.730998+0000","last_clean":"2024-04-27T03:33:48.730998+0000","last_became_active":"2024-04-27T03:30:15.382913+0000","last_became_peered":"2024-04-27T03:30:15.382913+0000","last_unstale":"2024-04-27T03:33:48.730998+0000","last_undegraded":"2024-04-27T03:33:48.730998+0000","last_fullsized":"2024-04-27T03:33:48.730998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.730891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.730891+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.730891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:49.302463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"27'45","reported_seq":103,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.809157+0000","last_change":"2024-04-27T03:33:45.809220+0000","last_active":"2024-04-27T03:33:45.809157+0000","last_peered":"2024-04-27T03:33:45.809157+0000","last_clean":"2024-04-27T03:33:45.809157+0000","last_became_active":"2024-04-27T03:30:17.381106+0000","last_became_peered":"2024-04-27T03:30:17.381106+0000","last_unstale":"2024-04-27T03:33:45.809157+0000","last_undegraded":"2024-04-27T03:33:45.809157+0000","last_fullsized":"2024-04-27T03:33:45.809157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'45","last_scrub_stamp":"2024-04-27T03:33:45.809061+0000","last_deep_scrub":"27'45","last_deep_scrub_stamp":"2024-04-27T03:33:45.809061+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.809061+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:33.888060+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"27'285","reported_seq":415,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.864438+0000","last_change":"2024-04-27T03:33:31.864480+0000","last_active":"2024-04-27T03:33:31.864438+0000","last_peered":"2024-04-27T03:33:31.864438+0000","last_clean":"2024-04-27T03:33:31.864438+0000","last_became_active":"2024-04-27T03:30:19.403413+0000","last_became_peered":"2024-04-27T03:30:19.403413+0000","last_unstale":"2024-04-27T03:33:31.864438+0000","last_undegraded":"2024-04-27T03:33:31.864438+0000","last_fullsized":"2024-04-27T03:33:31.864438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:31.864348+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:31.864348+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.864348+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:27.874206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"27'15","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.705541+0000","last_change":"2024-04-27T03:33:54.705571+0000","last_active":"2024-04-27T03:33:54.705541+0000","last_peered":"2024-04-27T03:33:54.705541+0000","last_clean":"2024-04-27T03:33:54.705541+0000","last_became_active":"2024-04-27T03:30:21.402102+0000","last_became_peered":"2024-04-27T03:30:21.402102+0000","last_unstale":"2024-04-27T03:33:54.705541+0000","last_undegraded":"2024-04-27T03:33:54.705541+0000","last_fullsized":"2024-04-27T03:33:54.705541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-04-27T03:33:54.705410+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-04-27T03:33:54.705410+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.705410+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:00.713477+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":23,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'12","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.884557+0000","last_change":"2024-04-27T03:34:08.884586+0000","last_active":"2024-04-27T03:34:08.884557+0000","last_peered":"2024-04-27T03:34:08.884557+0000","last_clean":"2024-04-27T03:34:08.884557+0000","last_became_active":"2024-04-27T03:30:13.382899+0000","last_became_peered":"2024-04-27T03:30:13.382899+0000","last_unstale":"2024-04-27T03:34:08.884557+0000","last_undegraded":"2024-04-27T03:34:08.884557+0000","last_fullsized":"2024-04-27T03:34:08.884557+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-04-27T03:34:08.884470+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-04-27T03:34:08.884470+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.884470+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:46.729084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.926560+0000","last_change":"2024-04-27T03:31:01.444153+0000","last_active":"2024-04-27T03:33:20.926560+0000","last_peered":"2024-04-27T03:33:20.926560+0000","last_clean":"2024-04-27T03:33:20.926560+0000","last_became_active":"2024-04-27T03:31:01.439776+0000","last_became_peered":"2024-04-27T03:31:01.439776+0000","last_unstale":"2024-04-27T03:33:20.926560+0000","last_undegraded":"2024-04-27T03:33:20.926560+0000","last_fullsized":"2024-04-27T03:33:20.926560+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"27'5","reported_seq":428,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.950334+0000","last_change":"2024-04-27T03:33:43.950380+0000","last_active":"2024-04-27T03:33:43.950334+0000","last_peered":"2024-04-27T03:33:43.950334+0000","last_clean":"2024-04-27T03:33:43.950334+0000","last_became_active":"2024-04-27T03:30:15.382502+0000","last_became_peered":"2024-04-27T03:30:15.382502+0000","last_unstale":"2024-04-27T03:33:43.950334+0000","last_undegraded":"2024-04-27T03:33:43.950334+0000","last_fullsized":"2024-04-27T03:33:43.950334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:43.950258+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:43.950258+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.950258+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:48.355472+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"27'15","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.197694+0000","last_change":"2024-04-27T03:33:58.197757+0000","last_active":"2024-04-27T03:33:58.197694+0000","last_peered":"2024-04-27T03:33:58.197694+0000","last_clean":"2024-04-27T03:33:58.197694+0000","last_became_active":"2024-04-27T03:30:17.387739+0000","last_became_peered":"2024-04-27T03:30:17.387739+0000","last_unstale":"2024-04-27T03:33:58.197694+0000","last_undegraded":"2024-04-27T03:33:58.197694+0000","last_fullsized":"2024-04-27T03:33:58.197694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-04-27T03:33:58.197595+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-04-27T03:33:58.197595+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.197595+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:01.324323+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"27'285","reported_seq":522,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.933904+0000","last_change":"2024-04-27T03:33:32.933929+0000","last_active":"2024-04-27T03:33:32.933904+0000","last_peered":"2024-04-27T03:33:32.933904+0000","last_clean":"2024-04-27T03:33:32.933904+0000","last_became_active":"2024-04-27T03:30:19.402484+0000","last_became_peered":"2024-04-27T03:30:19.402484+0000","last_unstale":"2024-04-27T03:33:32.933904+0000","last_undegraded":"2024-04-27T03:33:32.933904+0000","last_fullsized":"2024-04-27T03:33:32.933904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:32.933800+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:32.933800+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.933800+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:19.924355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.778511+0000","last_change":"2024-04-27T03:33:52.778552+0000","last_active":"2024-04-27T03:33:52.778511+0000","last_peered":"2024-04-27T03:33:52.778511+0000","last_clean":"2024-04-27T03:33:52.778511+0000","last_became_active":"2024-04-27T03:30:21.408512+0000","last_became_peered":"2024-04-27T03:30:21.408512+0000","last_unstale":"2024-04-27T03:33:52.778511+0000","last_undegraded":"2024-04-27T03:33:52.778511+0000","last_fullsized":"2024-04-27T03:33:52.778511+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:52.778420+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:52.778420+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.778420+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:57.183634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'14","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.153948+0000","last_change":"2024-04-27T03:34:02.153985+0000","last_active":"2024-04-27T03:34:02.153948+0000","last_peered":"2024-04-27T03:34:02.153948+0000","last_clean":"2024-04-27T03:34:02.153948+0000","last_became_active":"2024-04-27T03:30:13.380540+0000","last_became_peered":"2024-04-27T03:30:13.380540+0000","last_unstale":"2024-04-27T03:34:02.153948+0000","last_undegraded":"2024-04-27T03:34:02.153948+0000","last_fullsized":"2024-04-27T03:34:02.153948+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:34:02.153863+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:34:02.153863+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.153863+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:47.534998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295052+0000","last_change":"2024-04-27T03:30:11.372470+0000","last_active":"2024-04-27T03:32:24.295052+0000","last_peered":"2024-04-27T03:32:24.295052+0000","last_clean":"2024-04-27T03:32:24.295052+0000","last_became_active":"2024-04-27T03:30:11.372193+0000","last_became_peered":"2024-04-27T03:30:11.372193+0000","last_unstale":"2024-04-27T03:32:24.295052+0000","last_undegraded":"2024-04-27T03:32:24.295052+0000","last_fullsized":"2024-04-27T03:32:24.295052+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.115468+0000","last_change":"2024-04-27T03:33:35.115491+0000","last_active":"2024-04-27T03:33:35.115468+0000","last_peered":"2024-04-27T03:33:35.115468+0000","last_clean":"2024-04-27T03:33:35.115468+0000","last_became_active":"2024-04-27T03:30:15.383924+0000","last_became_peered":"2024-04-27T03:30:15.383924+0000","last_unstale":"2024-04-27T03:33:35.115468+0000","last_undegraded":"2024-04-27T03:33:35.115468+0000","last_fullsized":"2024-04-27T03:33:35.115468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:35.115383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:35.115383+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.115383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:37.564842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"27'48","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.883186+0000","last_change":"2024-04-27T03:33:38.883223+0000","last_active":"2024-04-27T03:33:38.883186+0000","last_peered":"2024-04-27T03:33:38.883186+0000","last_clean":"2024-04-27T03:33:38.883186+0000","last_became_active":"2024-04-27T03:30:17.387578+0000","last_became_peered":"2024-04-27T03:30:17.387578+0000","last_unstale":"2024-04-27T03:33:38.883186+0000","last_undegraded":"2024-04-27T03:33:38.883186+0000","last_fullsized":"2024-04-27T03:33:38.883186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-04-27T03:33:38.883110+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-04-27T03:33:38.883110+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.883110+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:11.757254+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.856113+0000","last_change":"2024-04-27T03:33:32.856137+0000","last_active":"2024-04-27T03:33:32.856113+0000","last_peered":"2024-04-27T03:33:32.856113+0000","last_clean":"2024-04-27T03:33:32.856113+0000","last_became_active":"2024-04-27T03:30:19.405512+0000","last_became_peered":"2024-04-27T03:30:19.405512+0000","last_unstale":"2024-04-27T03:33:32.856113+0000","last_undegraded":"2024-04-27T03:33:32.856113+0000","last_fullsized":"2024-04-27T03:33:32.856113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:32.856020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:32.856020+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.856020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:34.194673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"27'14","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.990549+0000","last_change":"2024-04-27T03:33:48.990573+0000","last_active":"2024-04-27T03:33:48.990549+0000","last_peered":"2024-04-27T03:33:48.990549+0000","last_clean":"2024-04-27T03:33:48.990549+0000","last_became_active":"2024-04-27T03:30:21.406883+0000","last_became_peered":"2024-04-27T03:30:21.406883+0000","last_unstale":"2024-04-27T03:33:48.990549+0000","last_undegraded":"2024-04-27T03:33:48.990549+0000","last_fullsized":"2024-04-27T03:33:48.990549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:33:48.990468+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:33:48.990468+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.990468+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:14:34.099716+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":59,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'20","reported_seq":88,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.148586+0000","last_change":"2024-04-27T03:33:52.148613+0000","last_active":"2024-04-27T03:33:52.148586+0000","last_peered":"2024-04-27T03:33:52.148586+0000","last_clean":"2024-04-27T03:33:52.148586+0000","last_became_active":"2024-04-27T03:30:13.387180+0000","last_became_peered":"2024-04-27T03:30:13.387180+0000","last_unstale":"2024-04-27T03:33:52.148586+0000","last_undegraded":"2024-04-27T03:33:52.148586+0000","last_fullsized":"2024-04-27T03:33:52.148586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-04-27T03:33:52.148491+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-04-27T03:33:52.148491+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.148491+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:24:54.949485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"11'117","reported_seq":195,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:30.911015+0000","last_change":"2024-04-27T03:30:06.104151+0000","last_active":"2024-04-27T03:33:30.911015+0000","last_peered":"2024-04-27T03:33:30.911015+0000","last_clean":"2024-04-27T03:33:30.911015+0000","last_became_active":"2024-04-27T03:30:06.102893+0000","last_became_peered":"2024-04-27T03:30:06.102893+0000","last_unstale":"2024-04-27T03:33:30.911015+0000","last_undegraded":"2024-04-27T03:33:30.911015+0000","last_fullsized":"2024-04-27T03:33:30.911015+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.293657+0000","last_change":"2024-04-27T03:30:06.106875+0000","last_active":"2024-04-27T03:32:14.293657+0000","last_peered":"2024-04-27T03:32:14.293657+0000","last_clean":"2024-04-27T03:32:14.293657+0000","last_became_active":"2024-04-27T03:30:06.106511+0000","last_became_peered":"2024-04-27T03:30:06.106511+0000","last_unstale":"2024-04-27T03:32:14.293657+0000","last_undegraded":"2024-04-27T03:32:14.293657+0000","last_fullsized":"2024-04-27T03:32:14.293657+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"27'4","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.007675+0000","last_change":"2024-04-27T03:30:11.362699+0000","last_active":"2024-04-27T03:33:21.007675+0000","last_peered":"2024-04-27T03:33:21.007675+0000","last_clean":"2024-04-27T03:33:21.007675+0000","last_became_active":"2024-04-27T03:30:11.362445+0000","last_became_peered":"2024-04-27T03:30:11.362445+0000","last_unstale":"2024-04-27T03:33:21.007675+0000","last_undegraded":"2024-04-27T03:33:21.007675+0000","last_fullsized":"2024-04-27T03:33:21.007675+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294623+0000","last_change":"2024-04-27T03:31:01.441027+0000","last_active":"2024-04-27T03:32:24.294623+0000","last_peered":"2024-04-27T03:32:24.294623+0000","last_clean":"2024-04-27T03:32:24.294623+0000","last_became_active":"2024-04-27T03:31:01.440678+0000","last_became_peered":"2024-04-27T03:31:01.440678+0000","last_unstale":"2024-04-27T03:32:24.294623+0000","last_undegraded":"2024-04-27T03:32:24.294623+0000","last_fullsized":"2024-04-27T03:32:24.294623+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"27'95","reported_seq":191,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.176299+0000","last_change":"2024-04-27T03:33:45.176369+0000","last_active":"2024-04-27T03:33:45.176299+0000","last_peered":"2024-04-27T03:33:45.176299+0000","last_clean":"2024-04-27T03:33:45.176299+0000","last_became_active":"2024-04-27T03:30:17.387707+0000","last_became_peered":"2024-04-27T03:30:17.387707+0000","last_unstale":"2024-04-27T03:33:45.176299+0000","last_undegraded":"2024-04-27T03:33:45.176299+0000","last_fullsized":"2024-04-27T03:33:45.176299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'95","last_scrub_stamp":"2024-04-27T03:33:45.176195+0000","last_deep_scrub":"27'95","last_deep_scrub_stamp":"2024-04-27T03:33:45.176195+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.176195+0000","objects_scrubbed":6,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:45.184076+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":93,"num_read_kb":81,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.779872+0000","last_change":"2024-04-27T03:33:31.779891+0000","last_active":"2024-04-27T03:33:31.779872+0000","last_peered":"2024-04-27T03:33:31.779872+0000","last_clean":"2024-04-27T03:33:31.779872+0000","last_became_active":"2024-04-27T03:30:19.405359+0000","last_became_peered":"2024-04-27T03:30:19.405359+0000","last_unstale":"2024-04-27T03:33:31.779872+0000","last_undegraded":"2024-04-27T03:33:31.779872+0000","last_fullsized":"2024-04-27T03:33:31.779872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:31.779786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:31.779786+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.779786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:45.852820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"27'81","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.838882+0000","last_change":"2024-04-27T03:33:44.838907+0000","last_active":"2024-04-27T03:33:44.838882+0000","last_peered":"2024-04-27T03:33:44.838882+0000","last_clean":"2024-04-27T03:33:44.838882+0000","last_became_active":"2024-04-27T03:30:21.404300+0000","last_became_peered":"2024-04-27T03:30:21.404300+0000","last_unstale":"2024-04-27T03:33:44.838882+0000","last_undegraded":"2024-04-27T03:33:44.838882+0000","last_fullsized":"2024-04-27T03:33:44.838882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'81","last_scrub_stamp":"2024-04-27T03:33:44.838760+0000","last_deep_scrub":"27'81","last_deep_scrub_stamp":"2024-04-27T03:33:44.838760+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.838760+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:18.504969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"25'2","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295248+0000","last_change":"2024-04-27T03:30:13.378248+0000","last_active":"2024-04-27T03:32:24.295248+0000","last_peered":"2024-04-27T03:32:24.295248+0000","last_clean":"2024-04-27T03:32:24.295248+0000","last_became_active":"2024-04-27T03:30:13.377713+0000","last_became_peered":"2024-04-27T03:30:13.377713+0000","last_unstale":"2024-04-27T03:32:24.295248+0000","last_undegraded":"2024-04-27T03:32:24.295248+0000","last_fullsized":"2024-04-27T03:32:24.295248+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.755713+0000","last_change":"2024-04-27T03:30:06.106890+0000","last_active":"2024-04-27T03:33:31.755713+0000","last_peered":"2024-04-27T03:33:31.755713+0000","last_clean":"2024-04-27T03:33:31.755713+0000","last_became_active":"2024-04-27T03:30:06.106474+0000","last_became_peered":"2024-04-27T03:30:06.106474+0000","last_unstale":"2024-04-27T03:33:31.755713+0000","last_undegraded":"2024-04-27T03:33:31.755713+0000","last_fullsized":"2024-04-27T03:33:31.755713+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"27'4","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288125+0000","last_change":"2024-04-27T03:30:11.360679+0000","last_active":"2024-04-27T03:32:19.288125+0000","last_peered":"2024-04-27T03:32:19.288125+0000","last_clean":"2024-04-27T03:32:19.288125+0000","last_became_active":"2024-04-27T03:30:11.360244+0000","last_became_peered":"2024-04-27T03:30:11.360244+0000","last_unstale":"2024-04-27T03:32:19.288125+0000","last_undegraded":"2024-04-27T03:32:19.288125+0000","last_fullsized":"2024-04-27T03:32:19.288125+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.098611+0000","last_change":"2024-04-27T03:33:42.098651+0000","last_active":"2024-04-27T03:33:42.098611+0000","last_peered":"2024-04-27T03:33:42.098611+0000","last_clean":"2024-04-27T03:33:42.098611+0000","last_became_active":"2024-04-27T03:30:15.379448+0000","last_became_peered":"2024-04-27T03:30:15.379448+0000","last_unstale":"2024-04-27T03:33:42.098611+0000","last_undegraded":"2024-04-27T03:33:42.098611+0000","last_fullsized":"2024-04-27T03:33:42.098611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.098519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.098519+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.098519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:34:23.337627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"27'297","reported_seq":528,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.877296+0000","last_change":"2024-04-27T03:33:35.877345+0000","last_active":"2024-04-27T03:33:35.877296+0000","last_peered":"2024-04-27T03:33:35.877296+0000","last_clean":"2024-04-27T03:33:35.877296+0000","last_became_active":"2024-04-27T03:30:17.390884+0000","last_became_peered":"2024-04-27T03:30:17.390884+0000","last_unstale":"2024-04-27T03:33:35.877296+0000","last_undegraded":"2024-04-27T03:33:35.877296+0000","last_fullsized":"2024-04-27T03:33:35.877296+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'297","last_scrub_stamp":"2024-04-27T03:33:35.877210+0000","last_deep_scrub":"27'297","last_deep_scrub_stamp":"2024-04-27T03:33:35.877210+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.877210+0000","objects_scrubbed":11,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:23.956209+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":387,"num_read_kb":371,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295201+0000","last_change":"2024-04-27T03:30:19.408904+0000","last_active":"2024-04-27T03:32:24.295201+0000","last_peered":"2024-04-27T03:32:24.295201+0000","last_clean":"2024-04-27T03:32:24.295201+0000","last_became_active":"2024-04-27T03:30:19.405777+0000","last_became_peered":"2024-04-27T03:30:19.405777+0000","last_unstale":"2024-04-27T03:32:24.295201+0000","last_undegraded":"2024-04-27T03:32:24.295201+0000","last_fullsized":"2024-04-27T03:32:24.295201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:18.376286+0000","last_clean_scrub_stamp":"2024-04-27T03:30:18.376286+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.973284+0000","last_change":"2024-04-27T03:33:52.973335+0000","last_active":"2024-04-27T03:33:52.973284+0000","last_peered":"2024-04-27T03:33:52.973284+0000","last_clean":"2024-04-27T03:33:52.973284+0000","last_became_active":"2024-04-27T03:30:21.409359+0000","last_became_peered":"2024-04-27T03:30:21.409359+0000","last_unstale":"2024-04-27T03:33:52.973284+0000","last_undegraded":"2024-04-27T03:33:52.973284+0000","last_fullsized":"2024-04-27T03:33:52.973284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:33:52.973192+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:33:52.973192+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.973192+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:49.283366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"25'10","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.632263+0000","last_change":"2024-04-27T03:33:57.632293+0000","last_active":"2024-04-27T03:33:57.632263+0000","last_peered":"2024-04-27T03:33:57.632263+0000","last_clean":"2024-04-27T03:33:57.632263+0000","last_became_active":"2024-04-27T03:30:13.385521+0000","last_became_peered":"2024-04-27T03:30:13.385521+0000","last_unstale":"2024-04-27T03:33:57.632263+0000","last_undegraded":"2024-04-27T03:33:57.632263+0000","last_fullsized":"2024-04-27T03:33:57.632263+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'10","last_scrub_stamp":"2024-04-27T03:33:57.632133+0000","last_deep_scrub":"25'10","last_deep_scrub_stamp":"2024-04-27T03:33:57.632133+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.632133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:37.434045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:09.292273+0000","last_change":"2024-04-27T03:30:06.103722+0000","last_active":"2024-04-27T03:32:09.292273+0000","last_peered":"2024-04-27T03:32:09.292273+0000","last_clean":"2024-04-27T03:32:09.292273+0000","last_became_active":"2024-04-27T03:30:06.103441+0000","last_became_peered":"2024-04-27T03:30:06.103441+0000","last_unstale":"2024-04-27T03:32:09.292273+0000","last_undegraded":"2024-04-27T03:32:09.292273+0000","last_fullsized":"2024-04-27T03:32:09.292273+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.287482+0000","last_change":"2024-04-27T03:30:11.360789+0000","last_active":"2024-04-27T03:32:14.287482+0000","last_peered":"2024-04-27T03:32:14.287482+0000","last_clean":"2024-04-27T03:32:14.287482+0000","last_became_active":"2024-04-27T03:30:11.360443+0000","last_became_peered":"2024-04-27T03:30:11.360443+0000","last_unstale":"2024-04-27T03:32:14.287482+0000","last_undegraded":"2024-04-27T03:32:14.287482+0000","last_fullsized":"2024-04-27T03:32:14.287482+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.772880+0000","last_change":"2024-04-27T03:33:46.772935+0000","last_active":"2024-04-27T03:33:46.772880+0000","last_peered":"2024-04-27T03:33:46.772880+0000","last_clean":"2024-04-27T03:33:46.772880+0000","last_became_active":"2024-04-27T03:30:15.383213+0000","last_became_peered":"2024-04-27T03:30:15.383213+0000","last_unstale":"2024-04-27T03:33:46.772880+0000","last_undegraded":"2024-04-27T03:33:46.772880+0000","last_fullsized":"2024-04-27T03:33:46.772880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:46.772765+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:46.772765+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.772765+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:24.897665+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"27'23","reported_seq":100,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.859478+0000","last_change":"2024-04-27T03:33:42.859533+0000","last_active":"2024-04-27T03:33:42.859478+0000","last_peered":"2024-04-27T03:33:42.859478+0000","last_clean":"2024-04-27T03:33:42.859478+0000","last_became_active":"2024-04-27T03:30:17.388588+0000","last_became_peered":"2024-04-27T03:30:17.388588+0000","last_unstale":"2024-04-27T03:33:42.859478+0000","last_undegraded":"2024-04-27T03:33:42.859478+0000","last_fullsized":"2024-04-27T03:33:42.859478+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-04-27T03:33:42.859384+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-04-27T03:33:42.859384+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.859384+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:29.849938+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.729956+0000","last_change":"2024-04-27T03:33:46.729982+0000","last_active":"2024-04-27T03:33:46.729956+0000","last_peered":"2024-04-27T03:33:46.729956+0000","last_clean":"2024-04-27T03:33:46.729956+0000","last_became_active":"2024-04-27T03:30:19.407004+0000","last_became_peered":"2024-04-27T03:30:19.407004+0000","last_unstale":"2024-04-27T03:33:46.729956+0000","last_undegraded":"2024-04-27T03:33:46.729956+0000","last_fullsized":"2024-04-27T03:33:46.729956+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:46.729851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:46.729851+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.729851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:36.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"25'2","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297632+0000","last_change":"2024-04-27T03:30:21.417022+0000","last_active":"2024-04-27T03:32:39.297632+0000","last_peered":"2024-04-27T03:32:39.297632+0000","last_clean":"2024-04-27T03:32:39.297632+0000","last_became_active":"2024-04-27T03:30:21.416389+0000","last_became_peered":"2024-04-27T03:30:21.416389+0000","last_unstale":"2024-04-27T03:32:39.297632+0000","last_undegraded":"2024-04-27T03:32:39.297632+0000","last_fullsized":"2024-04-27T03:32:39.297632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'8","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.675543+0000","last_change":"2024-04-27T03:34:00.675570+0000","last_active":"2024-04-27T03:34:00.675543+0000","last_peered":"2024-04-27T03:34:00.675543+0000","last_clean":"2024-04-27T03:34:00.675543+0000","last_became_active":"2024-04-27T03:30:13.369638+0000","last_became_peered":"2024-04-27T03:30:13.369638+0000","last_unstale":"2024-04-27T03:34:00.675543+0000","last_undegraded":"2024-04-27T03:34:00.675543+0000","last_fullsized":"2024-04-27T03:34:00.675543+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-04-27T03:34:00.675392+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-04-27T03:34:00.675392+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.675392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:47.827405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:09.292988+0000","last_change":"2024-04-27T03:30:06.107772+0000","last_active":"2024-04-27T03:32:09.292988+0000","last_peered":"2024-04-27T03:32:09.292988+0000","last_clean":"2024-04-27T03:32:09.292988+0000","last_became_active":"2024-04-27T03:30:06.107356+0000","last_became_peered":"2024-04-27T03:30:06.107356+0000","last_unstale":"2024-04-27T03:32:09.292988+0000","last_undegraded":"2024-04-27T03:32:09.292988+0000","last_fullsized":"2024-04-27T03:32:09.292988+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.293662+0000","last_change":"2024-04-27T03:30:11.357072+0000","last_active":"2024-04-27T03:32:19.293662+0000","last_peered":"2024-04-27T03:32:19.293662+0000","last_clean":"2024-04-27T03:32:19.293662+0000","last_became_active":"2024-04-27T03:30:11.356685+0000","last_became_peered":"2024-04-27T03:30:11.356685+0000","last_unstale":"2024-04-27T03:32:19.293662+0000","last_undegraded":"2024-04-27T03:32:19.293662+0000","last_fullsized":"2024-04-27T03:32:19.293662+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.706667+0000","last_change":"2024-04-27T03:33:49.706693+0000","last_active":"2024-04-27T03:33:49.706667+0000","last_peered":"2024-04-27T03:33:49.706667+0000","last_clean":"2024-04-27T03:33:49.706667+0000","last_became_active":"2024-04-27T03:30:15.382326+0000","last_became_peered":"2024-04-27T03:30:15.382326+0000","last_unstale":"2024-04-27T03:33:49.706667+0000","last_undegraded":"2024-04-27T03:33:49.706667+0000","last_fullsized":"2024-04-27T03:33:49.706667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:49.706574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:49.706574+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.706574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:45.223380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"27'20","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.220468+0000","last_change":"2024-04-27T03:34:01.220511+0000","last_active":"2024-04-27T03:34:01.220468+0000","last_peered":"2024-04-27T03:34:01.220468+0000","last_clean":"2024-04-27T03:34:01.220468+0000","last_became_active":"2024-04-27T03:30:17.389373+0000","last_became_peered":"2024-04-27T03:30:17.389373+0000","last_unstale":"2024-04-27T03:34:01.220468+0000","last_undegraded":"2024-04-27T03:34:01.220468+0000","last_fullsized":"2024-04-27T03:34:01.220468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-04-27T03:34:01.220376+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-04-27T03:34:01.220376+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.220376+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:49.020148+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":350,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.297507+0000","last_change":"2024-04-27T03:34:07.297531+0000","last_active":"2024-04-27T03:34:07.297507+0000","last_peered":"2024-04-27T03:34:07.297507+0000","last_clean":"2024-04-27T03:34:07.297507+0000","last_became_active":"2024-04-27T03:30:19.405953+0000","last_became_peered":"2024-04-27T03:30:19.405953+0000","last_unstale":"2024-04-27T03:34:07.297507+0000","last_undegraded":"2024-04-27T03:34:07.297507+0000","last_fullsized":"2024-04-27T03:34:07.297507+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:07.297411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:07.297411+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.297411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:38.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"27'6","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.137122+0000","last_change":"2024-04-27T03:30:21.413040+0000","last_active":"2024-04-27T03:33:21.137122+0000","last_peered":"2024-04-27T03:33:21.137122+0000","last_clean":"2024-04-27T03:33:21.137122+0000","last_became_active":"2024-04-27T03:30:21.412790+0000","last_became_peered":"2024-04-27T03:30:21.412790+0000","last_unstale":"2024-04-27T03:33:21.137122+0000","last_undegraded":"2024-04-27T03:33:21.137122+0000","last_fullsized":"2024-04-27T03:33:21.137122+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'18","reported_seq":95,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.201594+0000","last_change":"2024-04-27T03:34:00.201621+0000","last_active":"2024-04-27T03:34:00.201594+0000","last_peered":"2024-04-27T03:34:00.201594+0000","last_clean":"2024-04-27T03:34:00.201594+0000","last_became_active":"2024-04-27T03:30:13.380888+0000","last_became_peered":"2024-04-27T03:30:13.380888+0000","last_unstale":"2024-04-27T03:34:00.201594+0000","last_undegraded":"2024-04-27T03:34:00.201594+0000","last_fullsized":"2024-04-27T03:34:00.201594+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-04-27T03:34:00.201503+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-04-27T03:34:00.201503+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.201503+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:13:15.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":73,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.293716+0000","last_change":"2024-04-27T03:30:11.362550+0000","last_active":"2024-04-27T03:32:19.293716+0000","last_peered":"2024-04-27T03:32:19.293716+0000","last_clean":"2024-04-27T03:32:19.293716+0000","last_became_active":"2024-04-27T03:30:11.361786+0000","last_became_peered":"2024-04-27T03:30:11.361786+0000","last_unstale":"2024-04-27T03:32:19.293716+0000","last_undegraded":"2024-04-27T03:32:19.293716+0000","last_fullsized":"2024-04-27T03:32:19.293716+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.287557+0000","last_change":"2024-04-27T03:30:06.107861+0000","last_active":"2024-04-27T03:32:14.287557+0000","last_peered":"2024-04-27T03:32:14.287557+0000","last_clean":"2024-04-27T03:32:14.287557+0000","last_became_active":"2024-04-27T03:30:06.107594+0000","last_became_peered":"2024-04-27T03:30:06.107594+0000","last_unstale":"2024-04-27T03:32:14.287557+0000","last_undegraded":"2024-04-27T03:32:14.287557+0000","last_fullsized":"2024-04-27T03:32:14.287557+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:53.688641+0000","last_change":"2024-04-27T03:33:53.688679+0000","last_active":"2024-04-27T03:33:53.688641+0000","last_peered":"2024-04-27T03:33:53.688641+0000","last_clean":"2024-04-27T03:33:53.688641+0000","last_became_active":"2024-04-27T03:30:15.381392+0000","last_became_peered":"2024-04-27T03:30:15.381392+0000","last_unstale":"2024-04-27T03:33:53.688641+0000","last_undegraded":"2024-04-27T03:33:53.688641+0000","last_fullsized":"2024-04-27T03:33:53.688641+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:53.688558+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:53.688558+0000","last_clean_scrub_stamp":"2024-04-27T03:33:53.688558+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:08:53.046929+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"27'87","reported_seq":175,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.739920+0000","last_change":"2024-04-27T03:33:39.739991+0000","last_active":"2024-04-27T03:33:39.739920+0000","last_peered":"2024-04-27T03:33:39.739920+0000","last_clean":"2024-04-27T03:33:39.739920+0000","last_became_active":"2024-04-27T03:30:17.390409+0000","last_became_peered":"2024-04-27T03:30:17.390409+0000","last_unstale":"2024-04-27T03:33:39.739920+0000","last_undegraded":"2024-04-27T03:33:39.739920+0000","last_fullsized":"2024-04-27T03:33:39.739920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'87","last_scrub_stamp":"2024-04-27T03:33:39.739797+0000","last_deep_scrub":"27'87","last_deep_scrub_stamp":"2024-04-27T03:33:39.739797+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.739797+0000","objects_scrubbed":11,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:27:02.606163+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":89,"num_read_kb":75,"num_write":42,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":292,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.814593+0000","last_change":"2024-04-27T03:33:33.814624+0000","last_active":"2024-04-27T03:33:33.814593+0000","last_peered":"2024-04-27T03:33:33.814593+0000","last_clean":"2024-04-27T03:33:33.814593+0000","last_became_active":"2024-04-27T03:30:19.402977+0000","last_became_peered":"2024-04-27T03:30:19.402977+0000","last_unstale":"2024-04-27T03:33:33.814593+0000","last_undegraded":"2024-04-27T03:33:33.814593+0000","last_fullsized":"2024-04-27T03:33:33.814593+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:33.814493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:33.814493+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.814493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:14:40.755714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"27'11","reported_seq":35,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.990016+0000","last_change":"2024-04-27T03:30:21.410763+0000","last_active":"2024-04-27T03:33:20.990016+0000","last_peered":"2024-04-27T03:33:20.990016+0000","last_clean":"2024-04-27T03:33:20.990016+0000","last_became_active":"2024-04-27T03:30:21.410511+0000","last_became_peered":"2024-04-27T03:30:21.410511+0000","last_unstale":"2024-04-27T03:33:20.990016+0000","last_undegraded":"2024-04-27T03:33:20.990016+0000","last_fullsized":"2024-04-27T03:33:20.990016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"27'3","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.072466+0000","last_change":"2024-04-27T03:30:13.380442+0000","last_active":"2024-04-27T03:33:21.072466+0000","last_peered":"2024-04-27T03:33:21.072466+0000","last_clean":"2024-04-27T03:33:21.072466+0000","last_became_active":"2024-04-27T03:30:13.379985+0000","last_became_peered":"2024-04-27T03:30:13.379985+0000","last_unstale":"2024-04-27T03:33:21.072466+0000","last_undegraded":"2024-04-27T03:33:21.072466+0000","last_fullsized":"2024-04-27T03:33:21.072466+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.688977+0000","last_change":"2024-04-27T03:30:11.361129+0000","last_active":"2024-04-27T03:33:27.688977+0000","last_peered":"2024-04-27T03:33:27.688977+0000","last_clean":"2024-04-27T03:33:27.688977+0000","last_became_active":"2024-04-27T03:30:11.360352+0000","last_became_peered":"2024-04-27T03:30:11.360352+0000","last_unstale":"2024-04-27T03:33:27.688977+0000","last_undegraded":"2024-04-27T03:33:27.688977+0000","last_fullsized":"2024-04-27T03:33:27.688977+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.293704+0000","last_change":"2024-04-27T03:30:06.106816+0000","last_active":"2024-04-27T03:32:14.293704+0000","last_peered":"2024-04-27T03:32:14.293704+0000","last_clean":"2024-04-27T03:32:14.293704+0000","last_became_active":"2024-04-27T03:30:06.106475+0000","last_became_peered":"2024-04-27T03:30:06.106475+0000","last_unstale":"2024-04-27T03:32:14.293704+0000","last_undegraded":"2024-04-27T03:32:14.293704+0000","last_fullsized":"2024-04-27T03:32:14.293704+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.089676+0000","last_change":"2024-04-27T03:33:41.089706+0000","last_active":"2024-04-27T03:33:41.089676+0000","last_peered":"2024-04-27T03:33:41.089676+0000","last_clean":"2024-04-27T03:33:41.089676+0000","last_became_active":"2024-04-27T03:30:15.380030+0000","last_became_peered":"2024-04-27T03:30:15.380030+0000","last_unstale":"2024-04-27T03:33:41.089676+0000","last_undegraded":"2024-04-27T03:33:41.089676+0000","last_fullsized":"2024-04-27T03:33:41.089676+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:41.089578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:41.089578+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.089578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:36.584947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"27'28","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.880878+0000","last_change":"2024-04-27T03:33:34.880920+0000","last_active":"2024-04-27T03:33:34.880878+0000","last_peered":"2024-04-27T03:33:34.880878+0000","last_clean":"2024-04-27T03:33:34.880878+0000","last_became_active":"2024-04-27T03:30:17.393253+0000","last_became_peered":"2024-04-27T03:30:17.393253+0000","last_unstale":"2024-04-27T03:33:34.880878+0000","last_undegraded":"2024-04-27T03:33:34.880878+0000","last_fullsized":"2024-04-27T03:33:34.880878+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-04-27T03:33:34.880792+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-04-27T03:33:34.880792+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.880792+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:08.547001+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"27'285","reported_seq":398,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.921391+0000","last_change":"2024-04-27T03:33:33.921440+0000","last_active":"2024-04-27T03:33:33.921391+0000","last_peered":"2024-04-27T03:33:33.921391+0000","last_clean":"2024-04-27T03:33:33.921391+0000","last_became_active":"2024-04-27T03:30:19.402319+0000","last_became_peered":"2024-04-27T03:30:19.402319+0000","last_unstale":"2024-04-27T03:33:33.921391+0000","last_undegraded":"2024-04-27T03:33:33.921391+0000","last_fullsized":"2024-04-27T03:33:33.921391+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:33.921301+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:33.921301+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.921301+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:42.843568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297804+0000","last_change":"2024-04-27T03:30:21.414388+0000","last_active":"2024-04-27T03:32:39.297804+0000","last_peered":"2024-04-27T03:32:39.297804+0000","last_clean":"2024-04-27T03:32:39.297804+0000","last_became_active":"2024-04-27T03:30:21.414109+0000","last_became_peered":"2024-04-27T03:30:21.414109+0000","last_unstale":"2024-04-27T03:32:39.297804+0000","last_undegraded":"2024-04-27T03:32:39.297804+0000","last_fullsized":"2024-04-27T03:32:39.297804+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.192285+0000","last_change":"2024-04-27T03:33:46.192314+0000","last_active":"2024-04-27T03:33:46.192285+0000","last_peered":"2024-04-27T03:33:46.192285+0000","last_clean":"2024-04-27T03:33:46.192285+0000","last_became_active":"2024-04-27T03:30:13.387764+0000","last_became_peered":"2024-04-27T03:30:13.387764+0000","last_unstale":"2024-04-27T03:33:46.192285+0000","last_undegraded":"2024-04-27T03:33:46.192285+0000","last_fullsized":"2024-04-27T03:33:46.192285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:33:46.192183+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:33:46.192183+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.192183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:21:54.976582+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.921434+0000","last_change":"2024-04-27T03:30:11.356555+0000","last_active":"2024-04-27T03:33:20.921434+0000","last_peered":"2024-04-27T03:33:20.921434+0000","last_clean":"2024-04-27T03:33:20.921434+0000","last_became_active":"2024-04-27T03:30:11.355704+0000","last_became_peered":"2024-04-27T03:30:11.355704+0000","last_unstale":"2024-04-27T03:33:20.921434+0000","last_undegraded":"2024-04-27T03:33:20.921434+0000","last_fullsized":"2024-04-27T03:33:20.921434+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.287594+0000","last_change":"2024-04-27T03:30:06.104067+0000","last_active":"2024-04-27T03:32:14.287594+0000","last_peered":"2024-04-27T03:32:14.287594+0000","last_clean":"2024-04-27T03:32:14.287594+0000","last_became_active":"2024-04-27T03:30:06.103759+0000","last_became_peered":"2024-04-27T03:30:06.103759+0000","last_unstale":"2024-04-27T03:32:14.287594+0000","last_undegraded":"2024-04-27T03:32:14.287594+0000","last_fullsized":"2024-04-27T03:32:14.287594+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"27'5","reported_seq":238,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.180866+0000","last_change":"2024-04-27T03:33:32.180895+0000","last_active":"2024-04-27T03:33:32.180866+0000","last_peered":"2024-04-27T03:33:32.180866+0000","last_clean":"2024-04-27T03:33:32.180866+0000","last_became_active":"2024-04-27T03:30:15.384372+0000","last_became_peered":"2024-04-27T03:30:15.384372+0000","last_unstale":"2024-04-27T03:33:32.180866+0000","last_undegraded":"2024-04-27T03:33:32.180866+0000","last_fullsized":"2024-04-27T03:33:32.180866+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:32.180778+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:32.180778+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.180778+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:01:22.337645+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'26","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.104765+0000","last_change":"2024-04-27T03:33:51.104813+0000","last_active":"2024-04-27T03:33:51.104765+0000","last_peered":"2024-04-27T03:33:51.104765+0000","last_clean":"2024-04-27T03:33:51.104765+0000","last_became_active":"2024-04-27T03:30:17.389154+0000","last_became_peered":"2024-04-27T03:30:17.389154+0000","last_unstale":"2024-04-27T03:33:51.104765+0000","last_undegraded":"2024-04-27T03:33:51.104765+0000","last_fullsized":"2024-04-27T03:33:51.104765+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-04-27T03:33:51.104672+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-04-27T03:33:51.104672+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.104672+0000","objects_scrubbed":6,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:09.096082+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"27'285","reported_seq":410,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.780298+0000","last_change":"2024-04-27T03:33:33.780327+0000","last_active":"2024-04-27T03:33:33.780298+0000","last_peered":"2024-04-27T03:33:33.780298+0000","last_clean":"2024-04-27T03:33:33.780298+0000","last_became_active":"2024-04-27T03:30:19.404867+0000","last_became_peered":"2024-04-27T03:30:19.404867+0000","last_unstale":"2024-04-27T03:33:33.780298+0000","last_undegraded":"2024-04-27T03:33:33.780298+0000","last_fullsized":"2024-04-27T03:33:33.780298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:33.780200+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:33.780200+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.780200+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:42.702468+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"25'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297292+0000","last_change":"2024-04-27T03:30:21.413577+0000","last_active":"2024-04-27T03:32:39.297292+0000","last_peered":"2024-04-27T03:32:39.297292+0000","last_clean":"2024-04-27T03:32:39.297292+0000","last_became_active":"2024-04-27T03:30:21.413264+0000","last_became_peered":"2024-04-27T03:30:21.413264+0000","last_unstale":"2024-04-27T03:32:39.297292+0000","last_undegraded":"2024-04-27T03:32:39.297292+0000","last_fullsized":"2024-04-27T03:32:39.297292+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"27'4","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.703816+0000","last_change":"2024-04-27T03:30:13.383496+0000","last_active":"2024-04-27T03:33:20.703816+0000","last_peered":"2024-04-27T03:33:20.703816+0000","last_clean":"2024-04-27T03:33:20.703816+0000","last_became_active":"2024-04-27T03:30:13.383208+0000","last_became_peered":"2024-04-27T03:30:13.383208+0000","last_unstale":"2024-04-27T03:33:20.703816+0000","last_undegraded":"2024-04-27T03:33:20.703816+0000","last_fullsized":"2024-04-27T03:33:20.703816+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294682+0000","last_change":"2024-04-27T03:30:11.370648+0000","last_active":"2024-04-27T03:32:24.294682+0000","last_peered":"2024-04-27T03:32:24.294682+0000","last_clean":"2024-04-27T03:32:24.294682+0000","last_became_active":"2024-04-27T03:30:11.369568+0000","last_became_peered":"2024-04-27T03:30:11.369568+0000","last_unstale":"2024-04-27T03:32:24.294682+0000","last_undegraded":"2024-04-27T03:32:24.294682+0000","last_fullsized":"2024-04-27T03:32:24.294682+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:14.293751+0000","last_change":"2024-04-27T03:30:06.107954+0000","last_active":"2024-04-27T03:32:14.293751+0000","last_peered":"2024-04-27T03:32:14.293751+0000","last_clean":"2024-04-27T03:32:14.293751+0000","last_became_active":"2024-04-27T03:30:06.107600+0000","last_became_peered":"2024-04-27T03:30:06.107600+0000","last_unstale":"2024-04-27T03:32:14.293751+0000","last_undegraded":"2024-04-27T03:32:14.293751+0000","last_fullsized":"2024-04-27T03:32:14.293751+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.124770+0000","last_change":"2024-04-27T03:33:40.124794+0000","last_active":"2024-04-27T03:33:40.124770+0000","last_peered":"2024-04-27T03:33:40.124770+0000","last_clean":"2024-04-27T03:33:40.124770+0000","last_became_active":"2024-04-27T03:30:15.380598+0000","last_became_peered":"2024-04-27T03:30:15.380598+0000","last_unstale":"2024-04-27T03:33:40.124770+0000","last_undegraded":"2024-04-27T03:33:40.124770+0000","last_fullsized":"2024-04-27T03:33:40.124770+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:40.124684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:40.124684+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.124684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:12.385924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"27'104","reported_seq":203,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.834918+0000","last_change":"2024-04-27T03:33:39.834973+0000","last_active":"2024-04-27T03:33:39.834918+0000","last_peered":"2024-04-27T03:33:39.834918+0000","last_clean":"2024-04-27T03:33:39.834918+0000","last_became_active":"2024-04-27T03:30:17.391191+0000","last_became_peered":"2024-04-27T03:30:17.391191+0000","last_unstale":"2024-04-27T03:33:39.834918+0000","last_undegraded":"2024-04-27T03:33:39.834918+0000","last_fullsized":"2024-04-27T03:33:39.834918+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'104","last_scrub_stamp":"2024-04-27T03:33:39.834804+0000","last_deep_scrub":"27'104","last_deep_scrub_stamp":"2024-04-27T03:33:39.834804+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.834804+0000","objects_scrubbed":9,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:46.431245+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":102,"num_read_kb":87,"num_write":40,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.744748+0000","last_change":"2024-04-27T03:33:36.744784+0000","last_active":"2024-04-27T03:33:36.744748+0000","last_peered":"2024-04-27T03:33:36.744748+0000","last_clean":"2024-04-27T03:33:36.744748+0000","last_became_active":"2024-04-27T03:30:19.404187+0000","last_became_peered":"2024-04-27T03:30:19.404187+0000","last_unstale":"2024-04-27T03:33:36.744748+0000","last_undegraded":"2024-04-27T03:33:36.744748+0000","last_fullsized":"2024-04-27T03:33:36.744748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.744626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.744626+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.744626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:14.869527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.769625+0000","last_change":"2024-04-27T03:33:48.769648+0000","last_active":"2024-04-27T03:33:48.769625+0000","last_peered":"2024-04-27T03:33:48.769625+0000","last_clean":"2024-04-27T03:33:48.769625+0000","last_became_active":"2024-04-27T03:30:21.407857+0000","last_became_peered":"2024-04-27T03:30:21.407857+0000","last_unstale":"2024-04-27T03:33:48.769625+0000","last_undegraded":"2024-04-27T03:33:48.769625+0000","last_fullsized":"2024-04-27T03:33:48.769625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.769538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.769538+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.769538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:21.643681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"25'2","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295427+0000","last_change":"2024-04-27T03:30:21.411358+0000","last_active":"2024-04-27T03:32:29.295427+0000","last_peered":"2024-04-27T03:32:29.295427+0000","last_clean":"2024-04-27T03:32:29.295427+0000","last_became_active":"2024-04-27T03:30:21.411054+0000","last_became_peered":"2024-04-27T03:30:21.411054+0000","last_unstale":"2024-04-27T03:32:29.295427+0000","last_undegraded":"2024-04-27T03:32:29.295427+0000","last_fullsized":"2024-04-27T03:32:29.295427+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"27'6","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.813031+0000","last_change":"2024-04-27T03:34:06.813056+0000","last_active":"2024-04-27T03:34:06.813031+0000","last_peered":"2024-04-27T03:34:06.813031+0000","last_clean":"2024-04-27T03:34:06.813031+0000","last_became_active":"2024-04-27T03:30:13.382675+0000","last_became_peered":"2024-04-27T03:30:13.382675+0000","last_unstale":"2024-04-27T03:34:06.813031+0000","last_undegraded":"2024-04-27T03:34:06.813031+0000","last_fullsized":"2024-04-27T03:34:06.813031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:06.812933+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:06.812933+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.812933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:10.931563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296814+0000","last_change":"2024-04-27T03:30:11.359531+0000","last_active":"2024-04-27T03:32:34.296814+0000","last_peered":"2024-04-27T03:32:34.296814+0000","last_clean":"2024-04-27T03:32:34.296814+0000","last_became_active":"2024-04-27T03:30:11.359195+0000","last_became_peered":"2024-04-27T03:30:11.359195+0000","last_unstale":"2024-04-27T03:32:34.296814+0000","last_undegraded":"2024-04-27T03:32:34.296814+0000","last_fullsized":"2024-04-27T03:32:34.296814+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.994229+0000","last_change":"2024-04-27T03:33:50.994275+0000","last_active":"2024-04-27T03:33:50.994229+0000","last_peered":"2024-04-27T03:33:50.994229+0000","last_clean":"2024-04-27T03:33:50.994229+0000","last_became_active":"2024-04-27T03:30:15.379541+0000","last_became_peered":"2024-04-27T03:30:15.379541+0000","last_unstale":"2024-04-27T03:33:50.994229+0000","last_undegraded":"2024-04-27T03:33:50.994229+0000","last_fullsized":"2024-04-27T03:33:50.994229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:50.994123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:50.994123+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.994123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:51.335268+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"27'46","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.177061+0000","last_change":"2024-04-27T03:33:37.177105+0000","last_active":"2024-04-27T03:33:37.177061+0000","last_peered":"2024-04-27T03:33:37.177061+0000","last_clean":"2024-04-27T03:33:37.177061+0000","last_became_active":"2024-04-27T03:30:17.386496+0000","last_became_peered":"2024-04-27T03:30:17.386496+0000","last_unstale":"2024-04-27T03:33:37.177061+0000","last_undegraded":"2024-04-27T03:33:37.177061+0000","last_fullsized":"2024-04-27T03:33:37.177061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:37.176970+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:37.176970+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.176970+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:15.021583+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:53.149637+0000","last_change":"2024-04-27T03:33:53.149659+0000","last_active":"2024-04-27T03:33:53.149637+0000","last_peered":"2024-04-27T03:33:53.149637+0000","last_clean":"2024-04-27T03:33:53.149637+0000","last_became_active":"2024-04-27T03:30:19.405189+0000","last_became_peered":"2024-04-27T03:30:19.405189+0000","last_unstale":"2024-04-27T03:33:53.149637+0000","last_undegraded":"2024-04-27T03:33:53.149637+0000","last_fullsized":"2024-04-27T03:33:53.149637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:53.149540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:53.149540+0000","last_clean_scrub_stamp":"2024-04-27T03:33:53.149540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:17:47.405355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"27'3","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.289139+0000","last_change":"2024-04-27T03:30:21.414203+0000","last_active":"2024-04-27T03:32:24.289139+0000","last_peered":"2024-04-27T03:32:24.289139+0000","last_clean":"2024-04-27T03:32:24.289139+0000","last_became_active":"2024-04-27T03:30:21.413920+0000","last_became_peered":"2024-04-27T03:30:21.413920+0000","last_unstale":"2024-04-27T03:32:24.289139+0000","last_undegraded":"2024-04-27T03:32:24.289139+0000","last_fullsized":"2024-04-27T03:32:24.289139+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"27'16","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.989051+0000","last_change":"2024-04-27T03:30:13.381187+0000","last_active":"2024-04-27T03:33:20.989051+0000","last_peered":"2024-04-27T03:33:20.989051+0000","last_clean":"2024-04-27T03:33:20.989051+0000","last_became_active":"2024-04-27T03:30:13.380240+0000","last_became_peered":"2024-04-27T03:30:13.380240+0000","last_unstale":"2024-04-27T03:33:20.989051+0000","last_undegraded":"2024-04-27T03:33:20.989051+0000","last_fullsized":"2024-04-27T03:33:20.989051+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":31,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297331+0000","last_change":"2024-04-27T03:30:11.361106+0000","last_active":"2024-04-27T03:32:39.297331+0000","last_peered":"2024-04-27T03:32:39.297331+0000","last_clean":"2024-04-27T03:32:39.297331+0000","last_became_active":"2024-04-27T03:30:11.360217+0000","last_became_peered":"2024-04-27T03:30:11.360217+0000","last_unstale":"2024-04-27T03:32:39.297331+0000","last_undegraded":"2024-04-27T03:32:39.297331+0000","last_fullsized":"2024-04-27T03:32:39.297331+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.125251+0000","last_change":"2024-04-27T03:33:36.125275+0000","last_active":"2024-04-27T03:33:36.125251+0000","last_peered":"2024-04-27T03:33:36.125251+0000","last_clean":"2024-04-27T03:33:36.125251+0000","last_became_active":"2024-04-27T03:30:15.381507+0000","last_became_peered":"2024-04-27T03:30:15.381507+0000","last_unstale":"2024-04-27T03:33:36.125251+0000","last_undegraded":"2024-04-27T03:33:36.125251+0000","last_fullsized":"2024-04-27T03:33:36.125251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.125165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.125165+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.125165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:43.471223+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"27'9","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.702947+0000","last_change":"2024-04-27T03:33:44.703001+0000","last_active":"2024-04-27T03:33:44.702947+0000","last_peered":"2024-04-27T03:33:44.702947+0000","last_clean":"2024-04-27T03:33:44.702947+0000","last_became_active":"2024-04-27T03:30:17.385318+0000","last_became_peered":"2024-04-27T03:30:17.385318+0000","last_unstale":"2024-04-27T03:33:44.702947+0000","last_undegraded":"2024-04-27T03:33:44.702947+0000","last_fullsized":"2024-04-27T03:33:44.702947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-04-27T03:33:44.702827+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-04-27T03:33:44.702827+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.702827+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:55.754975+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.873119+0000","last_change":"2024-04-27T03:33:36.873142+0000","last_active":"2024-04-27T03:33:36.873119+0000","last_peered":"2024-04-27T03:33:36.873119+0000","last_clean":"2024-04-27T03:33:36.873119+0000","last_became_active":"2024-04-27T03:30:19.406736+0000","last_became_peered":"2024-04-27T03:30:19.406736+0000","last_unstale":"2024-04-27T03:33:36.873119+0000","last_undegraded":"2024-04-27T03:33:36.873119+0000","last_fullsized":"2024-04-27T03:33:36.873119+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.873031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.873031+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.873031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:57.751682+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"27'4","reported_seq":22,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.805997+0000","last_change":"2024-04-27T03:30:21.408419+0000","last_active":"2024-04-27T03:33:20.805997+0000","last_peered":"2024-04-27T03:33:20.805997+0000","last_clean":"2024-04-27T03:33:20.805997+0000","last_became_active":"2024-04-27T03:30:21.408079+0000","last_became_peered":"2024-04-27T03:30:21.408079+0000","last_unstale":"2024-04-27T03:33:20.805997+0000","last_undegraded":"2024-04-27T03:33:20.805997+0000","last_fullsized":"2024-04-27T03:33:20.805997+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"27'4","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.019125+0000","last_change":"2024-04-27T03:30:13.375656+0000","last_active":"2024-04-27T03:33:21.019125+0000","last_peered":"2024-04-27T03:33:21.019125+0000","last_clean":"2024-04-27T03:33:21.019125+0000","last_became_active":"2024-04-27T03:30:13.375350+0000","last_became_peered":"2024-04-27T03:30:13.375350+0000","last_unstale":"2024-04-27T03:33:21.019125+0000","last_undegraded":"2024-04-27T03:33:21.019125+0000","last_fullsized":"2024-04-27T03:33:21.019125+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297355+0000","last_change":"2024-04-27T03:30:11.361187+0000","last_active":"2024-04-27T03:32:39.297355+0000","last_peered":"2024-04-27T03:32:39.297355+0000","last_clean":"2024-04-27T03:32:39.297355+0000","last_became_active":"2024-04-27T03:30:11.360938+0000","last_became_peered":"2024-04-27T03:30:11.360938+0000","last_unstale":"2024-04-27T03:32:39.297355+0000","last_undegraded":"2024-04-27T03:32:39.297355+0000","last_fullsized":"2024-04-27T03:32:39.297355+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.767320+0000","last_change":"2024-04-27T03:33:47.767359+0000","last_active":"2024-04-27T03:33:47.767320+0000","last_peered":"2024-04-27T03:33:47.767320+0000","last_clean":"2024-04-27T03:33:47.767320+0000","last_became_active":"2024-04-27T03:30:15.382287+0000","last_became_peered":"2024-04-27T03:30:15.382287+0000","last_unstale":"2024-04-27T03:33:47.767320+0000","last_undegraded":"2024-04-27T03:33:47.767320+0000","last_fullsized":"2024-04-27T03:33:47.767320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:47.767216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:47.767216+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.767216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:52.591536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"27'47","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.768186+0000","last_change":"2024-04-27T03:33:41.768233+0000","last_active":"2024-04-27T03:33:41.768186+0000","last_peered":"2024-04-27T03:33:41.768186+0000","last_clean":"2024-04-27T03:33:41.768186+0000","last_became_active":"2024-04-27T03:30:17.388260+0000","last_became_peered":"2024-04-27T03:30:17.388260+0000","last_unstale":"2024-04-27T03:33:41.768186+0000","last_undegraded":"2024-04-27T03:33:41.768186+0000","last_fullsized":"2024-04-27T03:33:41.768186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-04-27T03:33:41.768105+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-04-27T03:33:41.768105+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.768105+0000","objects_scrubbed":6,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:48.417823+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":662,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.927890+0000","last_change":"2024-04-27T03:33:31.927916+0000","last_active":"2024-04-27T03:33:31.927890+0000","last_peered":"2024-04-27T03:33:31.927890+0000","last_clean":"2024-04-27T03:33:31.927890+0000","last_became_active":"2024-04-27T03:30:19.402430+0000","last_became_peered":"2024-04-27T03:30:19.402430+0000","last_unstale":"2024-04-27T03:33:31.927890+0000","last_undegraded":"2024-04-27T03:33:31.927890+0000","last_fullsized":"2024-04-27T03:33:31.927890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:31.927798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:31.927798+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.927798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:36.369347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:54.300287+0000","last_change":"2024-04-27T03:30:21.416316+0000","last_active":"2024-04-27T03:32:54.300287+0000","last_peered":"2024-04-27T03:32:54.300287+0000","last_clean":"2024-04-27T03:32:54.300287+0000","last_became_active":"2024-04-27T03:30:21.415984+0000","last_became_peered":"2024-04-27T03:30:21.415984+0000","last_unstale":"2024-04-27T03:32:54.300287+0000","last_undegraded":"2024-04-27T03:32:54.300287+0000","last_fullsized":"2024-04-27T03:32:54.300287+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"27'5","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.144432+0000","last_change":"2024-04-27T03:33:50.144460+0000","last_active":"2024-04-27T03:33:50.144432+0000","last_peered":"2024-04-27T03:33:50.144432+0000","last_clean":"2024-04-27T03:33:50.144432+0000","last_became_active":"2024-04-27T03:30:13.386985+0000","last_became_peered":"2024-04-27T03:30:13.386985+0000","last_unstale":"2024-04-27T03:33:50.144432+0000","last_undegraded":"2024-04-27T03:33:50.144432+0000","last_fullsized":"2024-04-27T03:33:50.144432+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:50.144340+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:50.144340+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.144340+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:15.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297371+0000","last_change":"2024-04-27T03:30:11.370182+0000","last_active":"2024-04-27T03:32:39.297371+0000","last_peered":"2024-04-27T03:32:39.297371+0000","last_clean":"2024-04-27T03:32:39.297371+0000","last_became_active":"2024-04-27T03:30:11.369824+0000","last_became_peered":"2024-04-27T03:30:11.369824+0000","last_unstale":"2024-04-27T03:32:39.297371+0000","last_undegraded":"2024-04-27T03:32:39.297371+0000","last_fullsized":"2024-04-27T03:32:39.297371+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.154995+0000","last_change":"2024-04-27T03:33:34.155037+0000","last_active":"2024-04-27T03:33:34.154995+0000","last_peered":"2024-04-27T03:33:34.154995+0000","last_clean":"2024-04-27T03:33:34.154995+0000","last_became_active":"2024-04-27T03:30:15.383771+0000","last_became_peered":"2024-04-27T03:30:15.383771+0000","last_unstale":"2024-04-27T03:33:34.154995+0000","last_undegraded":"2024-04-27T03:33:34.154995+0000","last_fullsized":"2024-04-27T03:33:34.154995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.154879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.154879+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.154879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:59.127137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"27'36","reported_seq":91,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.840074+0000","last_change":"2024-04-27T03:33:43.840113+0000","last_active":"2024-04-27T03:33:43.840074+0000","last_peered":"2024-04-27T03:33:43.840074+0000","last_clean":"2024-04-27T03:33:43.840074+0000","last_became_active":"2024-04-27T03:30:17.385488+0000","last_became_peered":"2024-04-27T03:30:17.385488+0000","last_unstale":"2024-04-27T03:33:43.840074+0000","last_undegraded":"2024-04-27T03:33:43.840074+0000","last_fullsized":"2024-04-27T03:33:43.840074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-04-27T03:33:43.839983+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-04-27T03:33:43.839983+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.839983+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:52.762250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.844652+0000","last_change":"2024-04-27T03:33:35.844680+0000","last_active":"2024-04-27T03:33:35.844652+0000","last_peered":"2024-04-27T03:33:35.844652+0000","last_clean":"2024-04-27T03:33:35.844652+0000","last_became_active":"2024-04-27T03:30:19.398484+0000","last_became_peered":"2024-04-27T03:30:19.398484+0000","last_unstale":"2024-04-27T03:33:35.844652+0000","last_undegraded":"2024-04-27T03:33:35.844652+0000","last_fullsized":"2024-04-27T03:33:35.844652+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:35.844545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:35.844545+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.844545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:17.570301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"25'2","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.294470+0000","last_change":"2024-04-27T03:30:13.377494+0000","last_active":"2024-04-27T03:32:19.294470+0000","last_peered":"2024-04-27T03:32:19.294470+0000","last_clean":"2024-04-27T03:32:19.294470+0000","last_became_active":"2024-04-27T03:30:13.377217+0000","last_became_peered":"2024-04-27T03:30:13.377217+0000","last_unstale":"2024-04-27T03:32:19.294470+0000","last_undegraded":"2024-04-27T03:32:19.294470+0000","last_fullsized":"2024-04-27T03:32:19.294470+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297519+0000","last_change":"2024-04-27T03:30:11.370820+0000","last_active":"2024-04-27T03:32:39.297519+0000","last_peered":"2024-04-27T03:32:39.297519+0000","last_clean":"2024-04-27T03:32:39.297519+0000","last_became_active":"2024-04-27T03:30:11.370547+0000","last_became_peered":"2024-04-27T03:30:11.370547+0000","last_unstale":"2024-04-27T03:32:39.297519+0000","last_undegraded":"2024-04-27T03:32:39.297519+0000","last_fullsized":"2024-04-27T03:32:39.297519+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.700561+0000","last_change":"2024-04-27T03:33:52.700581+0000","last_active":"2024-04-27T03:33:52.700561+0000","last_peered":"2024-04-27T03:33:52.700561+0000","last_clean":"2024-04-27T03:33:52.700561+0000","last_became_active":"2024-04-27T03:30:15.381956+0000","last_became_peered":"2024-04-27T03:30:15.381956+0000","last_unstale":"2024-04-27T03:33:52.700561+0000","last_undegraded":"2024-04-27T03:33:52.700561+0000","last_fullsized":"2024-04-27T03:33:52.700561+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:52.700486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:52.700486+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.700486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:49.010661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"27'70","reported_seq":149,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.743749+0000","last_change":"2024-04-27T03:33:45.743793+0000","last_active":"2024-04-27T03:33:45.743749+0000","last_peered":"2024-04-27T03:33:45.743749+0000","last_clean":"2024-04-27T03:33:45.743749+0000","last_became_active":"2024-04-27T03:30:17.383178+0000","last_became_peered":"2024-04-27T03:30:17.383178+0000","last_unstale":"2024-04-27T03:33:45.743749+0000","last_undegraded":"2024-04-27T03:33:45.743749+0000","last_fullsized":"2024-04-27T03:33:45.743749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'70","last_scrub_stamp":"2024-04-27T03:33:45.743668+0000","last_deep_scrub":"27'70","last_deep_scrub_stamp":"2024-04-27T03:33:45.743668+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.743668+0000","objects_scrubbed":6,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:37.293903+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":28,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.812908+0000","last_change":"2024-04-27T03:33:34.812933+0000","last_active":"2024-04-27T03:33:34.812908+0000","last_peered":"2024-04-27T03:33:34.812908+0000","last_clean":"2024-04-27T03:33:34.812908+0000","last_became_active":"2024-04-27T03:30:19.402781+0000","last_became_peered":"2024-04-27T03:30:19.402781+0000","last_unstale":"2024-04-27T03:33:34.812908+0000","last_undegraded":"2024-04-27T03:33:34.812908+0000","last_fullsized":"2024-04-27T03:33:34.812908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.812814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.812814+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.812814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:12.666738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"27'16","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.292116+0000","last_change":"2024-04-27T03:30:21.408415+0000","last_active":"2024-04-27T03:32:44.292116+0000","last_peered":"2024-04-27T03:32:44.292116+0000","last_clean":"2024-04-27T03:32:44.292116+0000","last_became_active":"2024-04-27T03:30:21.408049+0000","last_became_peered":"2024-04-27T03:30:21.408049+0000","last_unstale":"2024-04-27T03:32:44.292116+0000","last_undegraded":"2024-04-27T03:32:44.292116+0000","last_fullsized":"2024-04-27T03:32:44.292116+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":21,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.693500+0000","last_change":"2024-04-27T03:34:08.693523+0000","last_active":"2024-04-27T03:34:08.693500+0000","last_peered":"2024-04-27T03:34:08.693500+0000","last_clean":"2024-04-27T03:34:08.693500+0000","last_became_active":"2024-04-27T03:30:13.373257+0000","last_became_peered":"2024-04-27T03:30:13.373257+0000","last_unstale":"2024-04-27T03:34:08.693500+0000","last_undegraded":"2024-04-27T03:34:08.693500+0000","last_fullsized":"2024-04-27T03:34:08.693500+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:08.693374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:08.693374+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.693374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:22.810525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.222541+0000","last_change":"2024-04-27T03:34:08.222565+0000","last_active":"2024-04-27T03:34:08.222541+0000","last_peered":"2024-04-27T03:34:08.222541+0000","last_clean":"2024-04-27T03:34:08.222541+0000","last_became_active":"2024-04-27T03:30:11.370091+0000","last_became_peered":"2024-04-27T03:30:11.370091+0000","last_unstale":"2024-04-27T03:34:08.222541+0000","last_undegraded":"2024-04-27T03:34:08.222541+0000","last_fullsized":"2024-04-27T03:34:08.222541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:08.222454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:08.222454+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.222454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:22:17.006853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.756533+0000","last_change":"2024-04-27T03:33:51.756557+0000","last_active":"2024-04-27T03:33:51.756533+0000","last_peered":"2024-04-27T03:33:51.756533+0000","last_clean":"2024-04-27T03:33:51.756533+0000","last_became_active":"2024-04-27T03:30:15.379800+0000","last_became_peered":"2024-04-27T03:30:15.379800+0000","last_unstale":"2024-04-27T03:33:51.756533+0000","last_undegraded":"2024-04-27T03:33:51.756533+0000","last_fullsized":"2024-04-27T03:33:51.756533+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:51.756438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:51.756438+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.756438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:14:49.565231+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"27'46","reported_seq":104,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.217186+0000","last_change":"2024-04-27T03:34:02.217229+0000","last_active":"2024-04-27T03:34:02.217186+0000","last_peered":"2024-04-27T03:34:02.217186+0000","last_clean":"2024-04-27T03:34:02.217186+0000","last_became_active":"2024-04-27T03:30:17.388246+0000","last_became_peered":"2024-04-27T03:30:17.388246+0000","last_unstale":"2024-04-27T03:34:02.217186+0000","last_undegraded":"2024-04-27T03:34:02.217186+0000","last_fullsized":"2024-04-27T03:34:02.217186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:34:02.217099+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:34:02.217099+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.217099+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:17:44.534386+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.948322+0000","last_change":"2024-04-27T03:33:40.948348+0000","last_active":"2024-04-27T03:33:40.948322+0000","last_peered":"2024-04-27T03:33:40.948322+0000","last_clean":"2024-04-27T03:33:40.948322+0000","last_became_active":"2024-04-27T03:30:19.406707+0000","last_became_peered":"2024-04-27T03:30:19.406707+0000","last_unstale":"2024-04-27T03:33:40.948322+0000","last_undegraded":"2024-04-27T03:33:40.948322+0000","last_fullsized":"2024-04-27T03:33:40.948322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:40.948229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:40.948229+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.948229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:11.173852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298513+0000","last_change":"2024-04-27T03:30:21.415821+0000","last_active":"2024-04-27T03:32:44.298513+0000","last_peered":"2024-04-27T03:32:44.298513+0000","last_clean":"2024-04-27T03:32:44.298513+0000","last_became_active":"2024-04-27T03:30:21.415441+0000","last_became_peered":"2024-04-27T03:30:21.415441+0000","last_unstale":"2024-04-27T03:32:44.298513+0000","last_undegraded":"2024-04-27T03:32:44.298513+0000","last_fullsized":"2024-04-27T03:32:44.298513+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.127154+0000","last_change":"2024-04-27T03:33:44.127210+0000","last_active":"2024-04-27T03:33:44.127154+0000","last_peered":"2024-04-27T03:33:44.127154+0000","last_clean":"2024-04-27T03:33:44.127154+0000","last_became_active":"2024-04-27T03:30:13.387397+0000","last_became_peered":"2024-04-27T03:30:13.387397+0000","last_unstale":"2024-04-27T03:33:44.127154+0000","last_undegraded":"2024-04-27T03:33:44.127154+0000","last_fullsized":"2024-04-27T03:33:44.127154+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:44.127050+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:44.127050+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.127050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:24.841845+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298192+0000","last_change":"2024-04-27T03:30:11.370706+0000","last_active":"2024-04-27T03:32:44.298192+0000","last_peered":"2024-04-27T03:32:44.298192+0000","last_clean":"2024-04-27T03:32:44.298192+0000","last_became_active":"2024-04-27T03:30:11.369799+0000","last_became_peered":"2024-04-27T03:30:11.369799+0000","last_unstale":"2024-04-27T03:32:44.298192+0000","last_undegraded":"2024-04-27T03:32:44.298192+0000","last_fullsized":"2024-04-27T03:32:44.298192+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:53.786343+0000","last_change":"2024-04-27T03:33:53.786369+0000","last_active":"2024-04-27T03:33:53.786343+0000","last_peered":"2024-04-27T03:33:53.786343+0000","last_clean":"2024-04-27T03:33:53.786343+0000","last_became_active":"2024-04-27T03:30:15.379873+0000","last_became_peered":"2024-04-27T03:30:15.379873+0000","last_unstale":"2024-04-27T03:33:53.786343+0000","last_undegraded":"2024-04-27T03:33:53.786343+0000","last_fullsized":"2024-04-27T03:33:53.786343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:53.786220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:53.786220+0000","last_clean_scrub_stamp":"2024-04-27T03:33:53.786220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:31.263421+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"27'48","reported_seq":110,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.721960+0000","last_change":"2024-04-27T03:33:43.722022+0000","last_active":"2024-04-27T03:33:43.721960+0000","last_peered":"2024-04-27T03:33:43.721960+0000","last_clean":"2024-04-27T03:33:43.721960+0000","last_became_active":"2024-04-27T03:30:17.387205+0000","last_became_peered":"2024-04-27T03:30:17.387205+0000","last_unstale":"2024-04-27T03:33:43.721960+0000","last_undegraded":"2024-04-27T03:33:43.721960+0000","last_fullsized":"2024-04-27T03:33:43.721960+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-04-27T03:33:43.721859+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-04-27T03:33:43.721859+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.721859+0000","objects_scrubbed":6,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:42.507737+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":63,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.822242+0000","last_change":"2024-04-27T03:33:34.822276+0000","last_active":"2024-04-27T03:33:34.822242+0000","last_peered":"2024-04-27T03:33:34.822242+0000","last_clean":"2024-04-27T03:33:34.822242+0000","last_became_active":"2024-04-27T03:30:19.405462+0000","last_became_peered":"2024-04-27T03:30:19.405462+0000","last_unstale":"2024-04-27T03:33:34.822242+0000","last_undegraded":"2024-04-27T03:33:34.822242+0000","last_fullsized":"2024-04-27T03:33:34.822242+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.822124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.822124+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.822124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:04:28.428726+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"27'7","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298466+0000","last_change":"2024-04-27T03:30:21.416894+0000","last_active":"2024-04-27T03:32:44.298466+0000","last_peered":"2024-04-27T03:32:44.298466+0000","last_clean":"2024-04-27T03:32:44.298466+0000","last_became_active":"2024-04-27T03:30:21.416592+0000","last_became_peered":"2024-04-27T03:30:21.416592+0000","last_unstale":"2024-04-27T03:32:44.298466+0000","last_undegraded":"2024-04-27T03:32:44.298466+0000","last_fullsized":"2024-04-27T03:32:44.298466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"27'6","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.718193+0000","last_change":"2024-04-27T03:34:04.718246+0000","last_active":"2024-04-27T03:34:04.718193+0000","last_peered":"2024-04-27T03:34:04.718193+0000","last_clean":"2024-04-27T03:34:04.718193+0000","last_became_active":"2024-04-27T03:30:13.369643+0000","last_became_peered":"2024-04-27T03:30:13.369643+0000","last_unstale":"2024-04-27T03:34:04.718193+0000","last_undegraded":"2024-04-27T03:34:04.718193+0000","last_fullsized":"2024-04-27T03:34:04.718193+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:04.718068+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:04.718068+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.718068+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:13:49.122146+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.688098+0000","last_change":"2024-04-27T03:30:11.372867+0000","last_active":"2024-04-27T03:33:27.688098+0000","last_peered":"2024-04-27T03:33:27.688098+0000","last_clean":"2024-04-27T03:33:27.688098+0000","last_became_active":"2024-04-27T03:30:11.372220+0000","last_became_peered":"2024-04-27T03:30:11.372220+0000","last_unstale":"2024-04-27T03:33:27.688098+0000","last_undegraded":"2024-04-27T03:33:27.688098+0000","last_fullsized":"2024-04-27T03:33:27.688098+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.972519+0000","last_change":"2024-04-27T03:33:49.972545+0000","last_active":"2024-04-27T03:33:49.972519+0000","last_peered":"2024-04-27T03:33:49.972519+0000","last_clean":"2024-04-27T03:33:49.972519+0000","last_became_active":"2024-04-27T03:30:15.380573+0000","last_became_peered":"2024-04-27T03:30:15.380573+0000","last_unstale":"2024-04-27T03:33:49.972519+0000","last_undegraded":"2024-04-27T03:33:49.972519+0000","last_fullsized":"2024-04-27T03:33:49.972519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:49.972421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:49.972421+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.972421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:13.206549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"27'22","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.170858+0000","last_change":"2024-04-27T03:33:47.170926+0000","last_active":"2024-04-27T03:33:47.170858+0000","last_peered":"2024-04-27T03:33:47.170858+0000","last_clean":"2024-04-27T03:33:47.170858+0000","last_became_active":"2024-04-27T03:30:17.388289+0000","last_became_peered":"2024-04-27T03:30:17.388289+0000","last_unstale":"2024-04-27T03:33:47.170858+0000","last_undegraded":"2024-04-27T03:33:47.170858+0000","last_fullsized":"2024-04-27T03:33:47.170858+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-04-27T03:33:47.170743+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-04-27T03:33:47.170743+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.170743+0000","objects_scrubbed":6,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:01.287895+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"27'285","reported_seq":384,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.829060+0000","last_change":"2024-04-27T03:33:40.829103+0000","last_active":"2024-04-27T03:33:40.829060+0000","last_peered":"2024-04-27T03:33:40.829060+0000","last_clean":"2024-04-27T03:33:40.829060+0000","last_became_active":"2024-04-27T03:30:19.406854+0000","last_became_peered":"2024-04-27T03:30:19.406854+0000","last_unstale":"2024-04-27T03:33:40.829060+0000","last_undegraded":"2024-04-27T03:33:40.829060+0000","last_fullsized":"2024-04-27T03:33:40.829060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:40.828943+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:40.828943+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.828943+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:59.341953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298415+0000","last_change":"2024-04-27T03:30:21.415992+0000","last_active":"2024-04-27T03:32:44.298415+0000","last_peered":"2024-04-27T03:32:44.298415+0000","last_clean":"2024-04-27T03:32:44.298415+0000","last_became_active":"2024-04-27T03:30:21.415534+0000","last_became_peered":"2024-04-27T03:30:21.415534+0000","last_unstale":"2024-04-27T03:32:44.298415+0000","last_undegraded":"2024-04-27T03:32:44.298415+0000","last_fullsized":"2024-04-27T03:32:44.298415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"27'24","reported_seq":111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.819891+0000","last_change":"2024-04-27T03:33:57.819934+0000","last_active":"2024-04-27T03:33:57.819891+0000","last_peered":"2024-04-27T03:33:57.819891+0000","last_clean":"2024-04-27T03:33:57.819891+0000","last_became_active":"2024-04-27T03:30:13.371158+0000","last_became_peered":"2024-04-27T03:30:13.371158+0000","last_unstale":"2024-04-27T03:33:57.819891+0000","last_undegraded":"2024-04-27T03:33:57.819891+0000","last_fullsized":"2024-04-27T03:33:57.819891+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-04-27T03:33:57.819758+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-04-27T03:33:57.819758+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.819758+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:10:53.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":88,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298246+0000","last_change":"2024-04-27T03:30:11.359205+0000","last_active":"2024-04-27T03:32:44.298246+0000","last_peered":"2024-04-27T03:32:44.298246+0000","last_clean":"2024-04-27T03:32:44.298246+0000","last_became_active":"2024-04-27T03:30:11.358893+0000","last_became_peered":"2024-04-27T03:30:11.358893+0000","last_unstale":"2024-04-27T03:32:44.298246+0000","last_undegraded":"2024-04-27T03:32:44.298246+0000","last_fullsized":"2024-04-27T03:32:44.298246+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.016872+0000","last_change":"2024-04-27T03:33:48.016893+0000","last_active":"2024-04-27T03:33:48.016872+0000","last_peered":"2024-04-27T03:33:48.016872+0000","last_clean":"2024-04-27T03:33:48.016872+0000","last_became_active":"2024-04-27T03:30:15.381175+0000","last_became_peered":"2024-04-27T03:30:15.381175+0000","last_unstale":"2024-04-27T03:33:48.016872+0000","last_undegraded":"2024-04-27T03:33:48.016872+0000","last_fullsized":"2024-04-27T03:33:48.016872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.016786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.016786+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.016786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:39:37.651908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"27'73","reported_seq":155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.916431+0000","last_change":"2024-04-27T03:33:39.916480+0000","last_active":"2024-04-27T03:33:39.916431+0000","last_peered":"2024-04-27T03:33:39.916431+0000","last_clean":"2024-04-27T03:33:39.916431+0000","last_became_active":"2024-04-27T03:30:17.386673+0000","last_became_peered":"2024-04-27T03:30:17.386673+0000","last_unstale":"2024-04-27T03:33:39.916431+0000","last_undegraded":"2024-04-27T03:33:39.916431+0000","last_fullsized":"2024-04-27T03:33:39.916431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'73","last_scrub_stamp":"2024-04-27T03:33:39.916339+0000","last_deep_scrub":"27'73","last_deep_scrub_stamp":"2024-04-27T03:33:39.916339+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.916339+0000","objects_scrubbed":11,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:47.317240+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":80,"num_read_kb":67,"num_write":44,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.711840+0000","last_change":"2024-04-27T03:33:50.711878+0000","last_active":"2024-04-27T03:33:50.711840+0000","last_peered":"2024-04-27T03:33:50.711840+0000","last_clean":"2024-04-27T03:33:50.711840+0000","last_became_active":"2024-04-27T03:30:19.406990+0000","last_became_peered":"2024-04-27T03:30:19.406990+0000","last_unstale":"2024-04-27T03:33:50.711840+0000","last_undegraded":"2024-04-27T03:33:50.711840+0000","last_fullsized":"2024-04-27T03:33:50.711840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:50.711713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:50.711713+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.711713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:29.194718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.179312+0000","last_change":"2024-04-27T03:34:01.179339+0000","last_active":"2024-04-27T03:34:01.179312+0000","last_peered":"2024-04-27T03:34:01.179312+0000","last_clean":"2024-04-27T03:34:01.179312+0000","last_became_active":"2024-04-27T03:30:21.410060+0000","last_became_peered":"2024-04-27T03:30:21.410060+0000","last_unstale":"2024-04-27T03:34:01.179312+0000","last_undegraded":"2024-04-27T03:34:01.179312+0000","last_fullsized":"2024-04-27T03:34:01.179312+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:01.179224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:01.179224+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.179224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:48.331237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"27'10","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.805577+0000","last_change":"2024-04-27T03:33:54.805600+0000","last_active":"2024-04-27T03:33:54.805577+0000","last_peered":"2024-04-27T03:33:54.805577+0000","last_clean":"2024-04-27T03:33:54.805577+0000","last_became_active":"2024-04-27T03:30:13.371866+0000","last_became_peered":"2024-04-27T03:30:13.371866+0000","last_unstale":"2024-04-27T03:33:54.805577+0000","last_undegraded":"2024-04-27T03:33:54.805577+0000","last_fullsized":"2024-04-27T03:33:54.805577+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:33:54.805478+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:33:54.805478+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.805478+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:21:58.032925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"27'4","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.931430+0000","last_change":"2024-04-27T03:30:11.362991+0000","last_active":"2024-04-27T03:33:20.931430+0000","last_peered":"2024-04-27T03:33:20.931430+0000","last_clean":"2024-04-27T03:33:20.931430+0000","last_became_active":"2024-04-27T03:30:11.362420+0000","last_became_peered":"2024-04-27T03:30:11.362420+0000","last_unstale":"2024-04-27T03:33:20.931430+0000","last_undegraded":"2024-04-27T03:33:20.931430+0000","last_fullsized":"2024-04-27T03:33:20.931430+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.769662+0000","last_change":"2024-04-27T03:33:50.769691+0000","last_active":"2024-04-27T03:33:50.769662+0000","last_peered":"2024-04-27T03:33:50.769662+0000","last_clean":"2024-04-27T03:33:50.769662+0000","last_became_active":"2024-04-27T03:30:15.382305+0000","last_became_peered":"2024-04-27T03:30:15.382305+0000","last_unstale":"2024-04-27T03:33:50.769662+0000","last_undegraded":"2024-04-27T03:33:50.769662+0000","last_fullsized":"2024-04-27T03:33:50.769662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:50.769565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:50.769565+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.769565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:30.717513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"27'46","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.785014+0000","last_change":"2024-04-27T03:33:40.785057+0000","last_active":"2024-04-27T03:33:40.785014+0000","last_peered":"2024-04-27T03:33:40.785014+0000","last_clean":"2024-04-27T03:33:40.785014+0000","last_became_active":"2024-04-27T03:30:17.390281+0000","last_became_peered":"2024-04-27T03:30:17.390281+0000","last_unstale":"2024-04-27T03:33:40.785014+0000","last_undegraded":"2024-04-27T03:33:40.785014+0000","last_fullsized":"2024-04-27T03:33:40.785014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:40.784918+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:40.784918+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.784918+0000","objects_scrubbed":5,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:11.010541+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":50,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.154146+0000","last_change":"2024-04-27T03:33:59.154184+0000","last_active":"2024-04-27T03:33:59.154146+0000","last_peered":"2024-04-27T03:33:59.154146+0000","last_clean":"2024-04-27T03:33:59.154146+0000","last_became_active":"2024-04-27T03:30:19.405353+0000","last_became_peered":"2024-04-27T03:30:19.405353+0000","last_unstale":"2024-04-27T03:33:59.154146+0000","last_undegraded":"2024-04-27T03:33:59.154146+0000","last_fullsized":"2024-04-27T03:33:59.154146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:59.154022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:59.154022+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.154022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:53:09.531303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:49.299534+0000","last_change":"2024-04-27T03:30:21.416876+0000","last_active":"2024-04-27T03:32:49.299534+0000","last_peered":"2024-04-27T03:32:49.299534+0000","last_clean":"2024-04-27T03:32:49.299534+0000","last_became_active":"2024-04-27T03:30:21.416276+0000","last_became_peered":"2024-04-27T03:30:21.416276+0000","last_unstale":"2024-04-27T03:32:49.299534+0000","last_undegraded":"2024-04-27T03:32:49.299534+0000","last_fullsized":"2024-04-27T03:32:49.299534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'24","reported_seq":102,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.706403+0000","last_change":"2024-04-27T03:30:13.377820+0000","last_active":"2024-04-27T03:33:20.706403+0000","last_peered":"2024-04-27T03:33:20.706403+0000","last_clean":"2024-04-27T03:33:20.706403+0000","last_became_active":"2024-04-27T03:30:13.376415+0000","last_became_peered":"2024-04-27T03:30:13.376415+0000","last_unstale":"2024-04-27T03:33:20.706403+0000","last_undegraded":"2024-04-27T03:33:20.706403+0000","last_fullsized":"2024-04-27T03:33:20.706403+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.688922+0000","last_change":"2024-04-27T03:30:11.361140+0000","last_active":"2024-04-27T03:33:27.688922+0000","last_peered":"2024-04-27T03:33:27.688922+0000","last_clean":"2024-04-27T03:33:27.688922+0000","last_became_active":"2024-04-27T03:30:11.360590+0000","last_became_peered":"2024-04-27T03:30:11.360590+0000","last_unstale":"2024-04-27T03:33:27.688922+0000","last_undegraded":"2024-04-27T03:33:27.688922+0000","last_fullsized":"2024-04-27T03:33:27.688922+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.996324+0000","last_change":"2024-04-27T03:33:42.996361+0000","last_active":"2024-04-27T03:33:42.996324+0000","last_peered":"2024-04-27T03:33:42.996324+0000","last_clean":"2024-04-27T03:33:42.996324+0000","last_became_active":"2024-04-27T03:30:15.385614+0000","last_became_peered":"2024-04-27T03:30:15.385614+0000","last_unstale":"2024-04-27T03:33:42.996324+0000","last_undegraded":"2024-04-27T03:33:42.996324+0000","last_fullsized":"2024-04-27T03:33:42.996324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.996219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.996219+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.996219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:58.172124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"27'37","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.164208+0000","last_change":"2024-04-27T03:33:39.164242+0000","last_active":"2024-04-27T03:33:39.164208+0000","last_peered":"2024-04-27T03:33:39.164208+0000","last_clean":"2024-04-27T03:33:39.164208+0000","last_became_active":"2024-04-27T03:30:17.386983+0000","last_became_peered":"2024-04-27T03:30:17.386983+0000","last_unstale":"2024-04-27T03:33:39.164208+0000","last_undegraded":"2024-04-27T03:33:39.164208+0000","last_fullsized":"2024-04-27T03:33:39.164208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-04-27T03:33:39.164121+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-04-27T03:33:39.164121+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.164121+0000","objects_scrubbed":3,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:26.316134+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":33,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.113482+0000","last_change":"2024-04-27T03:33:54.113533+0000","last_active":"2024-04-27T03:33:54.113482+0000","last_peered":"2024-04-27T03:33:54.113482+0000","last_clean":"2024-04-27T03:33:54.113482+0000","last_became_active":"2024-04-27T03:30:19.402652+0000","last_became_peered":"2024-04-27T03:30:19.402652+0000","last_unstale":"2024-04-27T03:33:54.113482+0000","last_undegraded":"2024-04-27T03:33:54.113482+0000","last_fullsized":"2024-04-27T03:33:54.113482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:54.113385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:54.113385+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.113385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:04:42.933126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"27'6","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.937826+0000","last_change":"2024-04-27T03:30:21.420988+0000","last_active":"2024-04-27T03:33:20.937826+0000","last_peered":"2024-04-27T03:33:20.937826+0000","last_clean":"2024-04-27T03:33:20.937826+0000","last_became_active":"2024-04-27T03:30:21.420725+0000","last_became_peered":"2024-04-27T03:30:21.420725+0000","last_unstale":"2024-04-27T03:33:20.937826+0000","last_undegraded":"2024-04-27T03:33:20.937826+0000","last_fullsized":"2024-04-27T03:33:20.937826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.788128+0000","last_change":"2024-04-27T03:33:58.788151+0000","last_active":"2024-04-27T03:33:58.788128+0000","last_peered":"2024-04-27T03:33:58.788128+0000","last_clean":"2024-04-27T03:33:58.788128+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:33:58.788128+0000","last_undegraded":"2024-04-27T03:33:58.788128+0000","last_fullsized":"2024-04-27T03:33:58.788128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:58.788034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:58.788034+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.788034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:14:43.897282+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.293781+0000","last_change":"2024-04-27T03:30:11.370518+0000","last_active":"2024-04-27T03:32:19.293781+0000","last_peered":"2024-04-27T03:32:19.293781+0000","last_clean":"2024-04-27T03:32:19.293781+0000","last_became_active":"2024-04-27T03:30:11.369877+0000","last_became_peered":"2024-04-27T03:30:11.369877+0000","last_unstale":"2024-04-27T03:32:19.293781+0000","last_undegraded":"2024-04-27T03:32:19.293781+0000","last_fullsized":"2024-04-27T03:32:19.293781+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"27'3","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.974605+0000","last_change":"2024-04-27T03:33:45.974632+0000","last_active":"2024-04-27T03:33:45.974605+0000","last_peered":"2024-04-27T03:33:45.974605+0000","last_clean":"2024-04-27T03:33:45.974605+0000","last_became_active":"2024-04-27T03:30:15.381574+0000","last_became_peered":"2024-04-27T03:30:15.381574+0000","last_unstale":"2024-04-27T03:33:45.974605+0000","last_undegraded":"2024-04-27T03:33:45.974605+0000","last_fullsized":"2024-04-27T03:33:45.974605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-27T03:33:45.974523+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-27T03:33:45.974523+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.974523+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:37.524758+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"27'53","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.159896+0000","last_change":"2024-04-27T03:33:57.159964+0000","last_active":"2024-04-27T03:33:57.159896+0000","last_peered":"2024-04-27T03:33:57.159896+0000","last_clean":"2024-04-27T03:33:57.159896+0000","last_became_active":"2024-04-27T03:30:17.388277+0000","last_became_peered":"2024-04-27T03:30:17.388277+0000","last_unstale":"2024-04-27T03:33:57.159896+0000","last_undegraded":"2024-04-27T03:33:57.159896+0000","last_fullsized":"2024-04-27T03:33:57.159896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'53","last_scrub_stamp":"2024-04-27T03:33:57.159788+0000","last_deep_scrub":"27'53","last_deep_scrub_stamp":"2024-04-27T03:33:57.159788+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.159788+0000","objects_scrubbed":7,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:03.430517+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":52,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1138,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.728511+0000","last_change":"2024-04-27T03:33:42.728536+0000","last_active":"2024-04-27T03:33:42.728511+0000","last_peered":"2024-04-27T03:33:42.728511+0000","last_clean":"2024-04-27T03:33:42.728511+0000","last_became_active":"2024-04-27T03:30:19.406565+0000","last_became_peered":"2024-04-27T03:30:19.406565+0000","last_unstale":"2024-04-27T03:33:42.728511+0000","last_undegraded":"2024-04-27T03:33:42.728511+0000","last_fullsized":"2024-04-27T03:33:42.728511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.728420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.728420+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.728420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:57.904325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"27'4","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298588+0000","last_change":"2024-04-27T03:30:21.414808+0000","last_active":"2024-04-27T03:32:44.298588+0000","last_peered":"2024-04-27T03:32:44.298588+0000","last_clean":"2024-04-27T03:32:44.298588+0000","last_became_active":"2024-04-27T03:30:21.414510+0000","last_became_peered":"2024-04-27T03:30:21.414510+0000","last_unstale":"2024-04-27T03:32:44.298588+0000","last_undegraded":"2024-04-27T03:32:44.298588+0000","last_fullsized":"2024-04-27T03:32:44.298588+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"25'8","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.006545+0000","last_change":"2024-04-27T03:33:54.006570+0000","last_active":"2024-04-27T03:33:54.006545+0000","last_peered":"2024-04-27T03:33:54.006545+0000","last_clean":"2024-04-27T03:33:54.006545+0000","last_became_active":"2024-04-27T03:30:13.387149+0000","last_became_peered":"2024-04-27T03:30:13.387149+0000","last_unstale":"2024-04-27T03:33:54.006545+0000","last_undegraded":"2024-04-27T03:33:54.006545+0000","last_fullsized":"2024-04-27T03:33:54.006545+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'8","last_scrub_stamp":"2024-04-27T03:33:54.006451+0000","last_deep_scrub":"25'8","last_deep_scrub_stamp":"2024-04-27T03:33:54.006451+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.006451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:20.542228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"25'6","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.293591+0000","last_change":"2024-04-27T03:30:11.358174+0000","last_active":"2024-04-27T03:32:19.293591+0000","last_peered":"2024-04-27T03:32:19.293591+0000","last_clean":"2024-04-27T03:32:19.293591+0000","last_became_active":"2024-04-27T03:30:11.357878+0000","last_became_peered":"2024-04-27T03:30:11.357878+0000","last_unstale":"2024-04-27T03:32:19.293591+0000","last_undegraded":"2024-04-27T03:32:19.293591+0000","last_fullsized":"2024-04-27T03:32:19.293591+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"27'4","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.707627+0000","last_change":"2024-04-27T03:34:02.707653+0000","last_active":"2024-04-27T03:34:02.707627+0000","last_peered":"2024-04-27T03:34:02.707627+0000","last_clean":"2024-04-27T03:34:02.707627+0000","last_became_active":"2024-04-27T03:30:13.369681+0000","last_became_peered":"2024-04-27T03:30:13.369681+0000","last_unstale":"2024-04-27T03:34:02.707627+0000","last_undegraded":"2024-04-27T03:34:02.707627+0000","last_fullsized":"2024-04-27T03:34:02.707627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:02.707538+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:02.707538+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.707538+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:58.202906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"25'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.696765+0000","last_change":"2024-04-27T03:34:01.696795+0000","last_active":"2024-04-27T03:34:01.696765+0000","last_peered":"2024-04-27T03:34:01.696765+0000","last_clean":"2024-04-27T03:34:01.696765+0000","last_became_active":"2024-04-27T03:30:11.373186+0000","last_became_peered":"2024-04-27T03:30:11.373186+0000","last_unstale":"2024-04-27T03:34:01.696765+0000","last_undegraded":"2024-04-27T03:34:01.696765+0000","last_fullsized":"2024-04-27T03:34:01.696765+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:01.696661+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:01.696661+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.696661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:33.957902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.249733+0000","last_change":"2024-04-27T03:34:08.249759+0000","last_active":"2024-04-27T03:34:08.249733+0000","last_peered":"2024-04-27T03:34:08.249733+0000","last_clean":"2024-04-27T03:34:08.249733+0000","last_became_active":"2024-04-27T03:30:13.382069+0000","last_became_peered":"2024-04-27T03:30:13.382069+0000","last_unstale":"2024-04-27T03:34:08.249733+0000","last_undegraded":"2024-04-27T03:34:08.249733+0000","last_fullsized":"2024-04-27T03:34:08.249733+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:08.249643+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:08.249643+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.249643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:17:08.768291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"27'3","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.294333+0000","last_change":"2024-04-27T03:30:11.350547+0000","last_active":"2024-04-27T03:32:19.294333+0000","last_peered":"2024-04-27T03:32:19.294333+0000","last_clean":"2024-04-27T03:32:19.294333+0000","last_became_active":"2024-04-27T03:30:11.350295+0000","last_became_peered":"2024-04-27T03:30:11.350295+0000","last_unstale":"2024-04-27T03:32:19.294333+0000","last_undegraded":"2024-04-27T03:32:19.294333+0000","last_fullsized":"2024-04-27T03:32:19.294333+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"27'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.676368+0000","last_change":"2024-04-27T03:34:06.676435+0000","last_active":"2024-04-27T03:34:06.676368+0000","last_peered":"2024-04-27T03:34:06.676368+0000","last_clean":"2024-04-27T03:34:06.676368+0000","last_became_active":"2024-04-27T03:30:13.370066+0000","last_became_peered":"2024-04-27T03:30:13.370066+0000","last_unstale":"2024-04-27T03:34:06.676368+0000","last_undegraded":"2024-04-27T03:34:06.676368+0000","last_fullsized":"2024-04-27T03:34:06.676368+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:06.676262+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:06.676262+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.676262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:06.684143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"27'4","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.689177+0000","last_change":"2024-04-27T03:30:11.360775+0000","last_active":"2024-04-27T03:33:27.689177+0000","last_peered":"2024-04-27T03:33:27.689177+0000","last_clean":"2024-04-27T03:33:27.689177+0000","last_became_active":"2024-04-27T03:30:11.360376+0000","last_became_peered":"2024-04-27T03:30:11.360376+0000","last_unstale":"2024-04-27T03:33:27.689177+0000","last_undegraded":"2024-04-27T03:33:27.689177+0000","last_fullsized":"2024-04-27T03:33:27.689177+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"27'4","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.804947+0000","last_change":"2024-04-27T03:30:13.378662+0000","last_active":"2024-04-27T03:33:20.804947+0000","last_peered":"2024-04-27T03:33:20.804947+0000","last_clean":"2024-04-27T03:33:20.804947+0000","last_became_active":"2024-04-27T03:30:13.378368+0000","last_became_peered":"2024-04-27T03:30:13.378368+0000","last_unstale":"2024-04-27T03:33:20.804947+0000","last_undegraded":"2024-04-27T03:33:20.804947+0000","last_fullsized":"2024-04-27T03:33:20.804947+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.669173+0000","last_change":"2024-04-27T03:34:05.669200+0000","last_active":"2024-04-27T03:34:05.669173+0000","last_peered":"2024-04-27T03:34:05.669173+0000","last_clean":"2024-04-27T03:34:05.669173+0000","last_became_active":"2024-04-27T03:30:11.371553+0000","last_became_peered":"2024-04-27T03:30:11.371553+0000","last_unstale":"2024-04-27T03:34:05.669173+0000","last_undegraded":"2024-04-27T03:34:05.669173+0000","last_fullsized":"2024-04-27T03:34:05.669173+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:05.669082+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:05.669082+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.669082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:46.383877+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"27'10","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.016605+0000","last_change":"2024-04-27T03:31:01.440910+0000","last_active":"2024-04-27T03:33:21.016605+0000","last_peered":"2024-04-27T03:33:21.016605+0000","last_clean":"2024-04-27T03:33:21.016605+0000","last_became_active":"2024-04-27T03:31:01.440576+0000","last_became_peered":"2024-04-27T03:31:01.440576+0000","last_unstale":"2024-04-27T03:33:21.016605+0000","last_undegraded":"2024-04-27T03:33:21.016605+0000","last_fullsized":"2024-04-27T03:33:21.016605+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294653+0000","last_change":"2024-04-27T03:30:11.371034+0000","last_active":"2024-04-27T03:32:24.294653+0000","last_peered":"2024-04-27T03:32:24.294653+0000","last_clean":"2024-04-27T03:32:24.294653+0000","last_became_active":"2024-04-27T03:30:11.369517+0000","last_became_peered":"2024-04-27T03:30:11.369517+0000","last_unstale":"2024-04-27T03:32:24.294653+0000","last_undegraded":"2024-04-27T03:32:24.294653+0000","last_fullsized":"2024-04-27T03:32:24.294653+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"27'4","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:19.288115+0000","last_change":"2024-04-27T03:30:13.374070+0000","last_active":"2024-04-27T03:32:19.288115+0000","last_peered":"2024-04-27T03:32:19.288115+0000","last_clean":"2024-04-27T03:32:19.288115+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:32:19.288115+0000","last_undegraded":"2024-04-27T03:32:19.288115+0000","last_fullsized":"2024-04-27T03:32:19.288115+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.294652+0000","last_change":"2024-04-27T03:31:01.441506+0000","last_active":"2024-04-27T03:32:24.294652+0000","last_peered":"2024-04-27T03:32:24.294652+0000","last_clean":"2024-04-27T03:32:24.294652+0000","last_became_active":"2024-04-27T03:31:01.441159+0000","last_became_peered":"2024-04-27T03:31:01.441159+0000","last_unstale":"2024-04-27T03:32:24.294652+0000","last_undegraded":"2024-04-27T03:32:24.294652+0000","last_fullsized":"2024-04-27T03:32:24.294652+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"27'4","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.181915+0000","last_change":"2024-04-27T03:34:07.181941+0000","last_active":"2024-04-27T03:34:07.181915+0000","last_peered":"2024-04-27T03:34:07.181915+0000","last_clean":"2024-04-27T03:34:07.181915+0000","last_became_active":"2024-04-27T03:30:13.377821+0000","last_became_peered":"2024-04-27T03:30:13.377821+0000","last_unstale":"2024-04-27T03:34:07.181915+0000","last_undegraded":"2024-04-27T03:34:07.181915+0000","last_fullsized":"2024-04-27T03:34:07.181915+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:07.181834+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:07.181834+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.181834+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:07.189714+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.697093+0000","last_change":"2024-04-27T03:34:03.697117+0000","last_active":"2024-04-27T03:34:03.697093+0000","last_peered":"2024-04-27T03:34:03.697093+0000","last_clean":"2024-04-27T03:34:03.697093+0000","last_became_active":"2024-04-27T03:30:11.371604+0000","last_became_peered":"2024-04-27T03:30:11.371604+0000","last_unstale":"2024-04-27T03:34:03.697093+0000","last_undegraded":"2024-04-27T03:34:03.697093+0000","last_fullsized":"2024-04-27T03:34:03.697093+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:03.697000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:03.697000+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.697000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:34:44.936107+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"27'8","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.956162+0000","last_change":"2024-04-27T03:33:51.956191+0000","last_active":"2024-04-27T03:33:51.956162+0000","last_peered":"2024-04-27T03:33:51.956162+0000","last_clean":"2024-04-27T03:33:51.956162+0000","last_became_active":"2024-04-27T03:30:13.388421+0000","last_became_peered":"2024-04-27T03:30:13.388421+0000","last_unstale":"2024-04-27T03:33:51.956162+0000","last_undegraded":"2024-04-27T03:33:51.956162+0000","last_fullsized":"2024-04-27T03:33:51.956162+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-04-27T03:33:51.956075+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-04-27T03:33:51.956075+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.956075+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:21:03.313222+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.680852+0000","last_change":"2024-04-27T03:30:11.355453+0000","last_active":"2024-04-27T03:33:20.680852+0000","last_peered":"2024-04-27T03:33:20.680852+0000","last_clean":"2024-04-27T03:33:20.680852+0000","last_became_active":"2024-04-27T03:30:11.355183+0000","last_became_peered":"2024-04-27T03:30:11.355183+0000","last_unstale":"2024-04-27T03:33:20.680852+0000","last_undegraded":"2024-04-27T03:33:20.680852+0000","last_fullsized":"2024-04-27T03:33:20.680852+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"27'4","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296887+0000","last_change":"2024-04-27T03:30:13.377155+0000","last_active":"2024-04-27T03:32:34.296887+0000","last_peered":"2024-04-27T03:32:34.296887+0000","last_clean":"2024-04-27T03:32:34.296887+0000","last_became_active":"2024-04-27T03:30:13.376875+0000","last_became_peered":"2024-04-27T03:30:13.376875+0000","last_unstale":"2024-04-27T03:32:34.296887+0000","last_undegraded":"2024-04-27T03:32:34.296887+0000","last_fullsized":"2024-04-27T03:32:34.296887+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"25'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.648387+0000","last_change":"2024-04-27T03:34:07.648432+0000","last_active":"2024-04-27T03:34:07.648387+0000","last_peered":"2024-04-27T03:34:07.648387+0000","last_clean":"2024-04-27T03:34:07.648387+0000","last_became_active":"2024-04-27T03:30:11.360299+0000","last_became_peered":"2024-04-27T03:30:11.360299+0000","last_unstale":"2024-04-27T03:34:07.648387+0000","last_undegraded":"2024-04-27T03:34:07.648387+0000","last_fullsized":"2024-04-27T03:34:07.648387+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:07.648243+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:07.648243+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.648243+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:22:16.432642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"27'10","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.988466+0000","last_change":"2024-04-27T03:30:13.376269+0000","last_active":"2024-04-27T03:33:20.988466+0000","last_peered":"2024-04-27T03:33:20.988466+0000","last_clean":"2024-04-27T03:33:20.988466+0000","last_became_active":"2024-04-27T03:30:13.376087+0000","last_became_peered":"2024-04-27T03:30:13.376087+0000","last_unstale":"2024-04-27T03:33:20.988466+0000","last_undegraded":"2024-04-27T03:33:20.988466+0000","last_fullsized":"2024-04-27T03:33:20.988466+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295291+0000","last_change":"2024-04-27T03:30:11.371237+0000","last_active":"2024-04-27T03:32:29.295291+0000","last_peered":"2024-04-27T03:32:29.295291+0000","last_clean":"2024-04-27T03:32:29.295291+0000","last_became_active":"2024-04-27T03:30:11.370837+0000","last_became_peered":"2024-04-27T03:30:11.370837+0000","last_unstale":"2024-04-27T03:32:29.295291+0000","last_undegraded":"2024-04-27T03:32:29.295291+0000","last_fullsized":"2024-04-27T03:32:29.295291+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"27'6","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.179298+0000","last_change":"2024-04-27T03:34:05.179323+0000","last_active":"2024-04-27T03:34:05.179298+0000","last_peered":"2024-04-27T03:34:05.179298+0000","last_clean":"2024-04-27T03:34:05.179298+0000","last_became_active":"2024-04-27T03:30:13.373511+0000","last_became_peered":"2024-04-27T03:30:13.373511+0000","last_unstale":"2024-04-27T03:34:05.179298+0000","last_undegraded":"2024-04-27T03:34:05.179298+0000","last_fullsized":"2024-04-27T03:34:05.179298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:05.179213+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:05.179213+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.179213+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:49.583854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295373+0000","last_change":"2024-04-27T03:30:11.370821+0000","last_active":"2024-04-27T03:32:29.295373+0000","last_peered":"2024-04-27T03:32:29.295373+0000","last_clean":"2024-04-27T03:32:29.295373+0000","last_became_active":"2024-04-27T03:30:11.370283+0000","last_became_peered":"2024-04-27T03:30:11.370283+0000","last_unstale":"2024-04-27T03:32:29.295373+0000","last_undegraded":"2024-04-27T03:32:29.295373+0000","last_fullsized":"2024-04-27T03:32:29.295373+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"27'11","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.072440+0000","last_change":"2024-04-27T03:30:13.380862+0000","last_active":"2024-04-27T03:33:21.072440+0000","last_peered":"2024-04-27T03:33:21.072440+0000","last_clean":"2024-04-27T03:33:21.072440+0000","last_became_active":"2024-04-27T03:30:13.380404+0000","last_became_peered":"2024-04-27T03:30:13.380404+0000","last_unstale":"2024-04-27T03:33:21.072440+0000","last_undegraded":"2024-04-27T03:33:21.072440+0000","last_fullsized":"2024-04-27T03:33:21.072440+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295898+0000","last_change":"2024-04-27T03:30:11.373282+0000","last_active":"2024-04-27T03:32:29.295898+0000","last_peered":"2024-04-27T03:32:29.295898+0000","last_clean":"2024-04-27T03:32:29.295898+0000","last_became_active":"2024-04-27T03:30:11.373026+0000","last_became_peered":"2024-04-27T03:30:11.373026+0000","last_unstale":"2024-04-27T03:32:29.295898+0000","last_undegraded":"2024-04-27T03:32:29.295898+0000","last_fullsized":"2024-04-27T03:32:29.295898+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"27'2","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.031529+0000","last_change":"2024-04-27T03:33:55.031559+0000","last_active":"2024-04-27T03:33:55.031529+0000","last_peered":"2024-04-27T03:33:55.031529+0000","last_clean":"2024-04-27T03:33:55.031529+0000","last_became_active":"2024-04-27T03:30:13.385791+0000","last_became_peered":"2024-04-27T03:30:13.385791+0000","last_unstale":"2024-04-27T03:33:55.031529+0000","last_undegraded":"2024-04-27T03:33:55.031529+0000","last_fullsized":"2024-04-27T03:33:55.031529+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:55.031445+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:55.031445+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.031445+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:01.039513+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295932+0000","last_change":"2024-04-27T03:30:11.358239+0000","last_active":"2024-04-27T03:32:29.295932+0000","last_peered":"2024-04-27T03:32:29.295932+0000","last_clean":"2024-04-27T03:32:29.295932+0000","last_became_active":"2024-04-27T03:30:11.357898+0000","last_became_peered":"2024-04-27T03:30:11.357898+0000","last_unstale":"2024-04-27T03:32:29.295932+0000","last_undegraded":"2024-04-27T03:32:29.295932+0000","last_fullsized":"2024-04-27T03:32:29.295932+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"27'5","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.070313+0000","last_change":"2024-04-27T03:31:01.444873+0000","last_active":"2024-04-27T03:33:21.070313+0000","last_peered":"2024-04-27T03:33:21.070313+0000","last_clean":"2024-04-27T03:33:21.070313+0000","last_became_active":"2024-04-27T03:31:01.440820+0000","last_became_peered":"2024-04-27T03:31:01.440820+0000","last_unstale":"2024-04-27T03:33:21.070313+0000","last_undegraded":"2024-04-27T03:33:21.070313+0000","last_fullsized":"2024-04-27T03:33:21.070313+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.846496+0000","last_change":"2024-04-27T03:34:05.846517+0000","last_active":"2024-04-27T03:34:05.846496+0000","last_peered":"2024-04-27T03:34:05.846496+0000","last_clean":"2024-04-27T03:34:05.846496+0000","last_became_active":"2024-04-27T03:30:11.362788+0000","last_became_peered":"2024-04-27T03:30:11.362788+0000","last_unstale":"2024-04-27T03:34:05.846496+0000","last_undegraded":"2024-04-27T03:34:05.846496+0000","last_fullsized":"2024-04-27T03:34:05.846496+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:05.846403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:05.846403+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.846403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:30.818661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"25'2","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.209177+0000","last_change":"2024-04-27T03:34:09.209205+0000","last_active":"2024-04-27T03:34:09.209177+0000","last_peered":"2024-04-27T03:34:09.209177+0000","last_clean":"2024-04-27T03:34:09.209177+0000","last_became_active":"2024-04-27T03:30:13.377845+0000","last_became_peered":"2024-04-27T03:30:13.377845+0000","last_unstale":"2024-04-27T03:34:09.209177+0000","last_undegraded":"2024-04-27T03:34:09.209177+0000","last_fullsized":"2024-04-27T03:34:09.209177+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:09.209073+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:09.209073+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.209073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:40.804195+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.688620+0000","last_change":"2024-04-27T03:30:11.362917+0000","last_active":"2024-04-27T03:33:27.688620+0000","last_peered":"2024-04-27T03:33:27.688620+0000","last_clean":"2024-04-27T03:33:27.688620+0000","last_became_active":"2024-04-27T03:30:11.362559+0000","last_became_peered":"2024-04-27T03:30:11.362559+0000","last_unstale":"2024-04-27T03:33:27.688620+0000","last_undegraded":"2024-04-27T03:33:27.688620+0000","last_fullsized":"2024-04-27T03:33:27.688620+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296630+0000","last_change":"2024-04-27T03:30:11.363675+0000","last_active":"2024-04-27T03:32:34.296630+0000","last_peered":"2024-04-27T03:32:34.296630+0000","last_clean":"2024-04-27T03:32:34.296630+0000","last_became_active":"2024-04-27T03:30:11.363340+0000","last_became_peered":"2024-04-27T03:30:11.363340+0000","last_unstale":"2024-04-27T03:32:34.296630+0000","last_undegraded":"2024-04-27T03:32:34.296630+0000","last_fullsized":"2024-04-27T03:32:34.296630+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"27'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.668243+0000","last_change":"2024-04-27T03:33:56.668296+0000","last_active":"2024-04-27T03:33:56.668243+0000","last_peered":"2024-04-27T03:33:56.668243+0000","last_clean":"2024-04-27T03:33:56.668243+0000","last_became_active":"2024-04-27T03:30:13.385705+0000","last_became_peered":"2024-04-27T03:30:13.385705+0000","last_unstale":"2024-04-27T03:33:56.668243+0000","last_undegraded":"2024-04-27T03:33:56.668243+0000","last_fullsized":"2024-04-27T03:33:56.668243+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:56.668140+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:56.668140+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.668140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:00.786770+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.290659+0000","last_change":"2024-04-27T03:30:11.360980+0000","last_active":"2024-04-27T03:32:34.290659+0000","last_peered":"2024-04-27T03:32:34.290659+0000","last_clean":"2024-04-27T03:32:34.290659+0000","last_became_active":"2024-04-27T03:30:11.360667+0000","last_became_peered":"2024-04-27T03:30:11.360667+0000","last_unstale":"2024-04-27T03:32:34.290659+0000","last_undegraded":"2024-04-27T03:32:34.290659+0000","last_fullsized":"2024-04-27T03:32:34.290659+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"27'14","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297355+0000","last_change":"2024-04-27T03:30:13.380806+0000","last_active":"2024-04-27T03:32:39.297355+0000","last_peered":"2024-04-27T03:32:39.297355+0000","last_clean":"2024-04-27T03:32:39.297355+0000","last_became_active":"2024-04-27T03:30:13.380347+0000","last_became_peered":"2024-04-27T03:30:13.380347+0000","last_unstale":"2024-04-27T03:32:39.297355+0000","last_undegraded":"2024-04-27T03:32:39.297355+0000","last_fullsized":"2024-04-27T03:32:39.297355+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":269,"ondisk_log_size":269,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2281,"ondisk_log_size":2281,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":1888,"num_read_kb":1653,"num_write":789,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2692,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1751,"ondisk_log_size":1751,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":18,"ondisk_log_size":18,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":477,"ondisk_log_size":477,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":82,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705715,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39904,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331936,"statfs":{"total":96636764160,"available":96595902464,"internally_reserved":0,"allocated":417792,"data_stored":120948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705716,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40416,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331424,"statfs":{"total":96636764160,"available":96595378176,"internally_reserved":0,"allocated":942080,"data_stored":708075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705715,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335556,"statfs":{"total":96636764160,"available":96599609344,"internally_reserved":0,"allocated":970752,"data_stored":708832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705717,"num_pgs":134,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39868,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331972,"statfs":{"total":96636764160,"available":96595939328,"internally_reserved":0,"allocated":380928,"data_stored":119531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T03:34:12.837 INFO:tasks.ceph:pgid 3.3e last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.838 INFO:tasks.ceph:pgid 4.39 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.838 INFO:tasks.ceph:pgid 3.3d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.838 INFO:tasks.ceph:pgid 4.3a last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.838 INFO:tasks.ceph:pgid 3.3c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.838 INFO:tasks.ceph:pgid 4.3b last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.838 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.838 INFO:tasks.ceph:pgid 3.3a last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.838 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.839 INFO:tasks.ceph:pgid 3.38 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.839 INFO:tasks.ceph:pgid 3.37 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.839 INFO:tasks.ceph:pgid 3.35 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.839 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.839 INFO:tasks.ceph:pgid 3.34 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.839 INFO:tasks.ceph:pgid 3.33 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.839 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.840 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.840 INFO:tasks.ceph:pgid 4.14 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.840 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.840 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.840 INFO:tasks.ceph:pgid 3.12 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.840 INFO:tasks.ceph:pgid 4.16 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.840 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.840 INFO:tasks.ceph:pgid 8.1b last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.841 INFO:tasks.ceph:pgid 3.10 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.841 INFO:tasks.ceph:pgid 8.4 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.841 INFO:tasks.ceph:pgid 5.8 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.841 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.841 INFO:tasks.ceph:pgid 8.6 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.841 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.841 INFO:tasks.ceph:pgid 5.a last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.842 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.842 INFO:tasks.ceph:pgid 4.c last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.842 INFO:tasks.ceph:pgid 3.b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.842 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.842 INFO:tasks.ceph:pgid 3.a last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.842 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.842 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.843 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.843 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.843 INFO:tasks.ceph:pgid 3.2 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.843 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.843 INFO:tasks.ceph:pgid 4.6 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.843 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.843 INFO:tasks.ceph:pgid 3.1 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.843 INFO:tasks.ceph:pgid 7.5 last_scrub_stamp 2024-04-27T03:30:18.376286+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=18, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.844 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.844 INFO:tasks.ceph:pgid 3.0 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.844 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.844 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.844 INFO:tasks.ceph:pgid 3.3 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.844 INFO:tasks.ceph:pgid 8.8 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.844 INFO:tasks.ceph:pgid 3.5 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.844 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.845 INFO:tasks.ceph:pgid 8.e last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.845 INFO:tasks.ceph:pgid 4.3 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.845 INFO:tasks.ceph:pgid 3.4 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.845 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.845 INFO:tasks.ceph:pgid 8.f last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.845 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.845 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.846 INFO:tasks.ceph:pgid 8.c last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.846 INFO:tasks.ceph:pgid 4.1 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.846 INFO:tasks.ceph:pgid 3.6 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.846 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.846 INFO:tasks.ceph:pgid 8.1f last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.846 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.846 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.847 INFO:tasks.ceph:pgid 4.12 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.847 INFO:tasks.ceph:pgid 3.15 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.847 INFO:tasks.ceph:pgid 8.1d last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.847 INFO:tasks.ceph:pgid 4.11 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.847 INFO:tasks.ceph:pgid 3.16 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.847 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.847 INFO:tasks.ceph:pgid 3.17 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.848 INFO:tasks.ceph:pgid 4.1f last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.848 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.848 INFO:tasks.ceph:pgid 8.13 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.848 INFO:tasks.ceph:pgid 8.12 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.848 INFO:tasks.ceph:pgid 3.1a last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.848 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.848 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.848 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.849 INFO:tasks.ceph:pgid 3.1c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.849 INFO:tasks.ceph:pgid 3.1d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.849 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.849 INFO:tasks.ceph:pgid 4.19 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.849 INFO:tasks.ceph:pgid 3.1e last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.849 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.850 INFO:tasks.ceph:pgid 3.1f last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.850 INFO:tasks.ceph:pgid 8.14 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.850 INFO:tasks.ceph:pgid 3.20 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.850 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.850 INFO:tasks.ceph:pgid 3.23 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.850 INFO:tasks.ceph:pgid 4.23 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.850 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.851 INFO:tasks.ceph:pgid 3.25 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.851 INFO:tasks.ceph:pgid 4.21 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.851 INFO:tasks.ceph:pgid 3.26 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.851 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.851 INFO:tasks.ceph:pgid 4.2e last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.851 INFO:tasks.ceph:pgid 4.2d last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.851 INFO:tasks.ceph:pgid 3.2a last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.851 INFO:tasks.ceph:pgid 3.2b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.852 INFO:tasks.ceph:pgid 4.2b last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.852 INFO:tasks.ceph:pgid 3.2c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.852 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.852 INFO:tasks.ceph:pgid 4.29 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.852 INFO:tasks.ceph:pgid 3.2f last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.852 INFO:tasks.ceph:pgid 3.30 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.852 INFO:tasks.ceph:pgid 3.31 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.852 INFO:tasks.ceph:pgid 4.36 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:12.853 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-27T03:34:32.854 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T03:34:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.957+0000 7f0eb1eeb640 1 Processor -- start 2024-04-27T03:34:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.957+0000 7f0eb1eeb640 1 -- start start 2024-04-27T03:34:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.958+0000 7f0eb1eeb640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0eac06fd50 0x7f0eac133860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:34:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.958+0000 7f0eb1eeb640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0eac133da0 con 0x7f0eac06f260 2024-04-27T03:34:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.958+0000 7f0eb1eeb640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f0eac070bc0 con 0x7f0eac06fd50 2024-04-27T03:34:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.958+0000 7f0eaaffd640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0eac06fd50 0x7f0eac133860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:34:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.958+0000 7f0eab7fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0eac06f260 0x7f0eac06f640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:56216/0 (socket says 172.21.15.89:56216) 2024-04-27T03:34:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.958+0000 7f0eab7fe640 1 -- 172.21.15.89:0/2698007243 learned_addr learned my addr 172.21.15.89:0/2698007243 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:34:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.958+0000 7f0eaaffd640 1 -- 172.21.15.89:0/2698007243 >> v1:172.21.15.89:6789/0 conn(0x7f0eac06f260 legacy=0x7f0eac06f640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:34:32.964 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.958+0000 7f0eaaffd640 1 -- 172.21.15.89:0/2698007243 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac134f80 con 0x7f0eac06fd50 2024-04-27T03:34:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.959+0000 7f0eaaffd640 1 --2- 172.21.15.89:0/2698007243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0eac06fd50 0x7f0eac133860 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009720 tx=0x7f0ea002b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd4cdd75ada7eac server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:34:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.959+0000 7f0eaa7fc640 1 -- 172.21.15.89:0/2698007243 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ea00316d0 con 0x7f0eac06fd50 2024-04-27T03:34:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.959+0000 7f0eaa7fc640 1 -- 172.21.15.89:0/2698007243 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0ea0037070 con 0x7f0eac06fd50 2024-04-27T03:34:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.959+0000 7f0eaa7fc640 1 -- 172.21.15.89:0/2698007243 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0ea00319c0 con 0x7f0eac06fd50 2024-04-27T03:34:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.960+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/2698007243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0eac06fd50 msgr2=0x7f0eac133860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:34:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.960+0000 7f0eb1eeb640 1 --2- 172.21.15.89:0/2698007243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0eac06fd50 0x7f0eac133860 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009720 tx=0x7f0ea002b840 comp rx=0 tx=0).stop 2024-04-27T03:34:32.966 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.960+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/2698007243 shutdown_connections 2024-04-27T03:34:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.960+0000 7f0eb1eeb640 1 --2- 172.21.15.89:0/2698007243 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0eac06fd50 0x7f0eac133860 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:34:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.960+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/2698007243 >> 172.21.15.89:0/2698007243 conn(0x7f0eac09e170 msgr2=0x7f0eac06d5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:34:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.960+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/2698007243 shutdown_connections 2024-04-27T03:34:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.960+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/2698007243 wait complete. 2024-04-27T03:34:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.960+0000 7f0eb1eeb640 1 Processor -- start 2024-04-27T03:34:32.967 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.961+0000 7f0eb1eeb640 1 -- start start 2024-04-27T03:34:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.961+0000 7f0eb1eeb640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0eac06f260 0x7f0eac12aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:34:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.961+0000 7f0eb1eeb640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0eac134300 con 0x7f0eac06f260 2024-04-27T03:34:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.961+0000 7f0eab7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0eac06f260 0x7f0eac12aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:34:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.961+0000 7f0eab7fe640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0eac06f260 0x7f0eac12aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:54560/0 (socket says 172.21.15.89:54560) 2024-04-27T03:34:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.961+0000 7f0eab7fe640 1 -- 172.21.15.89:0/89900444 learned_addr learned my addr 172.21.15.89:0/89900444 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:34:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.961+0000 7f0eab7fe640 1 -- 172.21.15.89:0/89900444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0eac11b5e0 con 0x7f0eac06f260 2024-04-27T03:34:32.969 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.961+0000 7f0eab7fe640 1 --2- 172.21.15.89:0/89900444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0eac06f260 0x7f0eac12aa30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00c4a0 tx=0x7f0e9c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:34:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.962+0000 7f0e93fff640 1 -- 172.21.15.89:0/89900444 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e9c017070 con 0x7f0eac06f260 2024-04-27T03:34:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.962+0000 7f0e93fff640 1 -- 172.21.15.89:0/89900444 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f0e9c005c10 con 0x7f0eac06f260 2024-04-27T03:34:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.962+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/89900444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0eac11c250 con 0x7f0eac06f260 2024-04-27T03:34:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.962+0000 7f0e93fff640 1 -- 172.21.15.89:0/89900444 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0e9c007070 con 0x7f0eac06f260 2024-04-27T03:34:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.962+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/89900444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0eac11c910 con 0x7f0eac06f260 2024-04-27T03:34:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.962+0000 7f0e93fff640 1 -- 172.21.15.89:0/89900444 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f0e9c01a070 con 0x7f0eac06f260 2024-04-27T03:34:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.963+0000 7f0e93fff640 1 --2- 172.21.15.89:0/89900444 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e98038060 0x7f0e9803a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:34:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.963+0000 7f0e93fff640 1 -- 172.21.15.89:0/89900444 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f0e9c04ac20 con 0x7f0eac06f260 2024-04-27T03:34:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.963+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/89900444 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0eac0a18a0 con 0x7f0eac06f260 2024-04-27T03:34:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.963+0000 7f0eaaffd640 1 --2- 172.21.15.89:0/89900444 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e98038060 0x7f0e9803a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:34:32.970 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.963+0000 7f0eaaffd640 1 --2- 172.21.15.89:0/89900444 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e98038060 0x7f0e9803a520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009720 tx=0x7f0ea0008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:34:32.973 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:32.968+0000 7f0e93fff640 1 -- 172.21.15.89:0/89900444 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0eac0a18a0 con 0x7f0eac06f260 2024-04-27T03:34:33.145 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:33.140+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/89900444 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f0eac062da0 con 0x7f0e98038060 2024-04-27T03:34:33.161 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:33.155+0000 7f0e93fff640 1 -- 172.21.15.89:0/89900444 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+664467 (secure 0 0 0) 0x7f0eac062da0 con 0x7f0e98038060 2024-04-27T03:34:33.168 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:34:33.174 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-04-27T03:34:33.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:33.160+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/89900444 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e98038060 msgr2=0x7f0e9803a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:34:33.176 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:33.160+0000 7f0eb1eeb640 1 --2- 172.21.15.89:0/89900444 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e98038060 0x7f0e9803a520 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f0ea0009720 tx=0x7f0ea0008030 comp rx=0 tx=0).stop 2024-04-27T03:34:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:33.160+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/89900444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0eac06f260 msgr2=0x7f0eac12aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:34:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:33.160+0000 7f0eb1eeb640 1 --2- 172.21.15.89:0/89900444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0eac06f260 0x7f0eac12aa30 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c00c4a0 tx=0x7f0e9c00c960 comp rx=0 tx=0).stop 2024-04-27T03:34:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:33.160+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/89900444 shutdown_connections 2024-04-27T03:34:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:33.160+0000 7f0eb1eeb640 1 --2- 172.21.15.89:0/89900444 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f0e98038060 0x7f0e9803a520 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:34:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:33.160+0000 7f0eb1eeb640 1 --2- 172.21.15.89:0/89900444 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0eac06f260 0x7f0eac12aa30 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:34:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:33.160+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/89900444 >> 172.21.15.89:0/89900444 conn(0x7f0eac09e170 msgr2=0x7f0eac12edc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:34:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:33.160+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/89900444 shutdown_connections 2024-04-27T03:34:33.177 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:33.160+0000 7f0eb1eeb640 1 -- 172.21.15.89:0/89900444 wait complete. 2024-04-27T03:34:33.177 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":158,"stamp":"2024-04-27T03:34:32.291586+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":252,"num_object_clones":0,"num_object_copies":504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":252,"num_whiteouts":0,"num_read":5378,"num_read_kb":4932,"num_write":2369,"num_write_kb":61373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3041,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5005,"ondisk_log_size":5005,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":156472,"kb_used_data":2648,"kb_used_omap":6,"kb_used_meta":153785,"kb_avail":377330888,"statfs":{"total":386547056640,"available":386386829312,"internally_reserved":0,"allocated":2711552,"data_stored":1657386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":157476630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003855"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.834248+0000","last_change":"2024-04-27T03:34:07.834248+0000","last_active":"2024-04-27T03:34:07.834248+0000","last_peered":"2024-04-27T03:34:07.834248+0000","last_clean":"2024-04-27T03:34:07.834248+0000","last_became_active":"2024-04-27T03:30:11.362574+0000","last_became_peered":"2024-04-27T03:30:11.362574+0000","last_unstale":"2024-04-27T03:34:07.834248+0000","last_undegraded":"2024-04-27T03:34:07.834248+0000","last_fullsized":"2024-04-27T03:34:07.834248+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:07.834117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:07.834117+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.834117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:15.180175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.077803+0000","last_change":"2024-04-27T03:33:56.077828+0000","last_active":"2024-04-27T03:33:56.077803+0000","last_peered":"2024-04-27T03:33:56.077803+0000","last_clean":"2024-04-27T03:33:56.077803+0000","last_became_active":"2024-04-27T03:30:13.384580+0000","last_became_peered":"2024-04-27T03:30:13.384580+0000","last_unstale":"2024-04-27T03:33:56.077803+0000","last_undegraded":"2024-04-27T03:33:56.077803+0000","last_fullsized":"2024-04-27T03:33:56.077803+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:56.077710+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:56.077710+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.077710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:21.049969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.703599+0000","last_change":"2024-04-27T03:34:09.703599+0000","last_active":"2024-04-27T03:34:09.703599+0000","last_peered":"2024-04-27T03:34:09.703599+0000","last_clean":"2024-04-27T03:34:09.703599+0000","last_became_active":"2024-04-27T03:30:11.360857+0000","last_became_peered":"2024-04-27T03:30:11.360857+0000","last_unstale":"2024-04-27T03:34:09.703599+0000","last_undegraded":"2024-04-27T03:34:09.703599+0000","last_fullsized":"2024-04-27T03:34:09.703599+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:09.703487+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:09.703487+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.703487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:08:10.048372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"27'12","reported_seq":88,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:21.258937+0000","last_change":"2024-04-27T03:34:21.258982+0000","last_active":"2024-04-27T03:34:21.258937+0000","last_peered":"2024-04-27T03:34:21.258937+0000","last_clean":"2024-04-27T03:34:21.258937+0000","last_became_active":"2024-04-27T03:30:13.376221+0000","last_became_peered":"2024-04-27T03:30:13.376221+0000","last_unstale":"2024-04-27T03:34:21.258937+0000","last_undegraded":"2024-04-27T03:34:21.258937+0000","last_fullsized":"2024-04-27T03:34:21.258937+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-04-27T03:34:21.258835+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-04-27T03:34:21.258835+0000","last_clean_scrub_stamp":"2024-04-27T03:34:21.258835+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:22.651830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:27.334011+0000","last_change":"2024-04-27T03:34:27.334036+0000","last_active":"2024-04-27T03:34:27.334011+0000","last_peered":"2024-04-27T03:34:27.334011+0000","last_clean":"2024-04-27T03:34:27.334011+0000","last_became_active":"2024-04-27T03:30:11.362739+0000","last_became_peered":"2024-04-27T03:30:11.362739+0000","last_unstale":"2024-04-27T03:34:27.334011+0000","last_undegraded":"2024-04-27T03:34:27.334011+0000","last_fullsized":"2024-04-27T03:34:27.334011+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:27.333877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:27.333877+0000","last_clean_scrub_stamp":"2024-04-27T03:34:27.333877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:08.432542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"27'8","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:18.221789+0000","last_change":"2024-04-27T03:34:18.221821+0000","last_active":"2024-04-27T03:34:18.221789+0000","last_peered":"2024-04-27T03:34:18.221789+0000","last_clean":"2024-04-27T03:34:18.221789+0000","last_became_active":"2024-04-27T03:30:13.376524+0000","last_became_peered":"2024-04-27T03:30:13.376524+0000","last_unstale":"2024-04-27T03:34:18.221789+0000","last_undegraded":"2024-04-27T03:34:18.221789+0000","last_fullsized":"2024-04-27T03:34:18.221789+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-04-27T03:34:18.221705+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-04-27T03:34:18.221705+0000","last_clean_scrub_stamp":"2024-04-27T03:34:18.221705+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:20.194608+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:15.859998+0000","last_change":"2024-04-27T03:34:15.860023+0000","last_active":"2024-04-27T03:34:15.859998+0000","last_peered":"2024-04-27T03:34:15.859998+0000","last_clean":"2024-04-27T03:34:15.859998+0000","last_became_active":"2024-04-27T03:30:11.360188+0000","last_became_peered":"2024-04-27T03:30:11.360188+0000","last_unstale":"2024-04-27T03:34:15.859998+0000","last_undegraded":"2024-04-27T03:34:15.859998+0000","last_fullsized":"2024-04-27T03:34:15.859998+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:15.859866+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:15.859866+0000","last_clean_scrub_stamp":"2024-04-27T03:34:15.859866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:16.509376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"27'16","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:20.246700+0000","last_change":"2024-04-27T03:34:20.246757+0000","last_active":"2024-04-27T03:34:20.246700+0000","last_peered":"2024-04-27T03:34:20.246700+0000","last_clean":"2024-04-27T03:34:20.246700+0000","last_became_active":"2024-04-27T03:30:13.376288+0000","last_became_peered":"2024-04-27T03:30:13.376288+0000","last_unstale":"2024-04-27T03:34:20.246700+0000","last_undegraded":"2024-04-27T03:34:20.246700+0000","last_fullsized":"2024-04-27T03:34:20.246700+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:20.246599+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:20.246599+0000","last_clean_scrub_stamp":"2024-04-27T03:34:20.246599+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:03.552732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298576+0000","last_change":"2024-04-27T03:30:11.355274+0000","last_active":"2024-04-27T03:32:44.298576+0000","last_peered":"2024-04-27T03:32:44.298576+0000","last_clean":"2024-04-27T03:32:44.298576+0000","last_became_active":"2024-04-27T03:30:11.354989+0000","last_became_peered":"2024-04-27T03:30:11.354989+0000","last_unstale":"2024-04-27T03:32:44.298576+0000","last_undegraded":"2024-04-27T03:32:44.298576+0000","last_fullsized":"2024-04-27T03:32:44.298576+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"27'6","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.200104+0000","last_change":"2024-04-27T03:34:04.200104+0000","last_active":"2024-04-27T03:34:04.200104+0000","last_peered":"2024-04-27T03:34:04.200104+0000","last_clean":"2024-04-27T03:34:04.200104+0000","last_became_active":"2024-04-27T03:30:13.373424+0000","last_became_peered":"2024-04-27T03:30:13.373424+0000","last_unstale":"2024-04-27T03:34:04.200104+0000","last_undegraded":"2024-04-27T03:34:04.200104+0000","last_fullsized":"2024-04-27T03:34:04.200104+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:04.199924+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:04.199924+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.199924+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:59.695293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:14.842885+0000","last_change":"2024-04-27T03:34:14.842915+0000","last_active":"2024-04-27T03:34:14.842885+0000","last_peered":"2024-04-27T03:34:14.842885+0000","last_clean":"2024-04-27T03:34:14.842885+0000","last_became_active":"2024-04-27T03:30:11.360219+0000","last_became_peered":"2024-04-27T03:30:11.360219+0000","last_unstale":"2024-04-27T03:34:14.842885+0000","last_undegraded":"2024-04-27T03:34:14.842885+0000","last_fullsized":"2024-04-27T03:34:14.842885+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:14.842794+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:14.842794+0000","last_clean_scrub_stamp":"2024-04-27T03:34:14.842794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:59.247435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"25'8","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.275565+0000","last_change":"2024-04-27T03:34:05.275565+0000","last_active":"2024-04-27T03:34:05.275565+0000","last_peered":"2024-04-27T03:34:05.275565+0000","last_clean":"2024-04-27T03:34:05.275565+0000","last_became_active":"2024-04-27T03:30:13.385665+0000","last_became_peered":"2024-04-27T03:30:13.385665+0000","last_unstale":"2024-04-27T03:34:05.275565+0000","last_undegraded":"2024-04-27T03:34:05.275565+0000","last_fullsized":"2024-04-27T03:34:05.275565+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'8","last_scrub_stamp":"2024-04-27T03:34:05.275413+0000","last_deep_scrub":"25'8","last_deep_scrub_stamp":"2024-04-27T03:34:05.275413+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.275413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:20:59.991475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298491+0000","last_change":"2024-04-27T03:30:11.373049+0000","last_active":"2024-04-27T03:32:44.298491+0000","last_peered":"2024-04-27T03:32:44.298491+0000","last_clean":"2024-04-27T03:32:44.298491+0000","last_became_active":"2024-04-27T03:30:11.372704+0000","last_became_peered":"2024-04-27T03:30:11.372704+0000","last_unstale":"2024-04-27T03:32:44.298491+0000","last_undegraded":"2024-04-27T03:32:44.298491+0000","last_fullsized":"2024-04-27T03:32:44.298491+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"27'4","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.771780+0000","last_change":"2024-04-27T03:33:56.771780+0000","last_active":"2024-04-27T03:33:56.771780+0000","last_peered":"2024-04-27T03:33:56.771780+0000","last_clean":"2024-04-27T03:33:56.771780+0000","last_became_active":"2024-04-27T03:30:13.374199+0000","last_became_peered":"2024-04-27T03:30:13.374199+0000","last_unstale":"2024-04-27T03:33:56.771780+0000","last_undegraded":"2024-04-27T03:33:56.771780+0000","last_fullsized":"2024-04-27T03:33:56.771780+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:56.771668+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:56.771668+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.771668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:18.489287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:19.635377+0000","last_change":"2024-04-27T03:34:19.635403+0000","last_active":"2024-04-27T03:34:19.635377+0000","last_peered":"2024-04-27T03:34:19.635377+0000","last_clean":"2024-04-27T03:34:19.635377+0000","last_became_active":"2024-04-27T03:30:11.358181+0000","last_became_peered":"2024-04-27T03:30:11.358181+0000","last_unstale":"2024-04-27T03:34:19.635377+0000","last_undegraded":"2024-04-27T03:34:19.635377+0000","last_fullsized":"2024-04-27T03:34:19.635377+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:19.635237+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:19.635237+0000","last_clean_scrub_stamp":"2024-04-27T03:34:19.635237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:22.761966+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.810889+0000","last_change":"2024-04-27T03:34:00.810889+0000","last_active":"2024-04-27T03:34:00.810889+0000","last_peered":"2024-04-27T03:34:00.810889+0000","last_clean":"2024-04-27T03:34:00.810889+0000","last_became_active":"2024-04-27T03:30:13.379059+0000","last_became_peered":"2024-04-27T03:30:13.379059+0000","last_unstale":"2024-04-27T03:34:00.810889+0000","last_undegraded":"2024-04-27T03:34:00.810889+0000","last_fullsized":"2024-04-27T03:34:00.810889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:00.810774+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:00.810774+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.810774+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:01.151920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"27'6","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.269064+0000","last_change":"2024-04-27T03:34:06.269111+0000","last_active":"2024-04-27T03:34:06.269064+0000","last_peered":"2024-04-27T03:34:06.269064+0000","last_clean":"2024-04-27T03:34:06.269064+0000","last_became_active":"2024-04-27T03:30:13.383694+0000","last_became_peered":"2024-04-27T03:30:13.383694+0000","last_unstale":"2024-04-27T03:34:06.269064+0000","last_undegraded":"2024-04-27T03:34:06.269064+0000","last_fullsized":"2024-04-27T03:34:06.269064+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:06.268965+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:06.268965+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.268965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:49.525413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:24.057185+0000","last_change":"2024-04-27T03:34:24.057220+0000","last_active":"2024-04-27T03:34:24.057185+0000","last_peered":"2024-04-27T03:34:24.057185+0000","last_clean":"2024-04-27T03:34:24.057185+0000","last_became_active":"2024-04-27T03:30:11.370157+0000","last_became_peered":"2024-04-27T03:30:11.370157+0000","last_unstale":"2024-04-27T03:34:24.057185+0000","last_undegraded":"2024-04-27T03:34:24.057185+0000","last_fullsized":"2024-04-27T03:34:24.057185+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:24.057056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:24.057056+0000","last_clean_scrub_stamp":"2024-04-27T03:34:24.057056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:46.521120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"27'6","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.201544+0000","last_change":"2024-04-27T03:34:03.201590+0000","last_active":"2024-04-27T03:34:03.201544+0000","last_peered":"2024-04-27T03:34:03.201544+0000","last_clean":"2024-04-27T03:34:03.201544+0000","last_became_active":"2024-04-27T03:30:13.372973+0000","last_became_peered":"2024-04-27T03:30:13.372973+0000","last_unstale":"2024-04-27T03:34:03.201544+0000","last_undegraded":"2024-04-27T03:34:03.201544+0000","last_fullsized":"2024-04-27T03:34:03.201544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:03.201445+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:03.201445+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.201445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:57.765391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"27'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.817465+0000","last_change":"2024-04-27T03:34:03.817515+0000","last_active":"2024-04-27T03:34:03.817465+0000","last_peered":"2024-04-27T03:34:03.817465+0000","last_clean":"2024-04-27T03:34:03.817465+0000","last_became_active":"2024-04-27T03:30:11.363109+0000","last_became_peered":"2024-04-27T03:30:11.363109+0000","last_unstale":"2024-04-27T03:34:03.817465+0000","last_undegraded":"2024-04-27T03:34:03.817465+0000","last_fullsized":"2024-04-27T03:34:03.817465+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:03.817361+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:03.817361+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.817361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:30.353138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"27'10","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.121833+0000","last_change":"2024-04-27T03:33:58.121867+0000","last_active":"2024-04-27T03:33:58.121833+0000","last_peered":"2024-04-27T03:33:58.121833+0000","last_clean":"2024-04-27T03:33:58.121833+0000","last_became_active":"2024-04-27T03:30:13.383742+0000","last_became_peered":"2024-04-27T03:30:13.383742+0000","last_unstale":"2024-04-27T03:33:58.121833+0000","last_undegraded":"2024-04-27T03:33:58.121833+0000","last_fullsized":"2024-04-27T03:33:58.121833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:33:58.121739+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:33:58.121739+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.121739+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:37.923650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"25'4","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:26.075178+0000","last_change":"2024-04-27T03:34:26.075209+0000","last_active":"2024-04-27T03:34:26.075178+0000","last_peered":"2024-04-27T03:34:26.075178+0000","last_clean":"2024-04-27T03:34:26.075178+0000","last_became_active":"2024-04-27T03:30:11.370380+0000","last_became_peered":"2024-04-27T03:30:11.370380+0000","last_unstale":"2024-04-27T03:34:26.075178+0000","last_undegraded":"2024-04-27T03:34:26.075178+0000","last_fullsized":"2024-04-27T03:34:26.075178+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'4","last_scrub_stamp":"2024-04-27T03:34:26.075076+0000","last_deep_scrub":"25'4","last_deep_scrub_stamp":"2024-04-27T03:34:26.075076+0000","last_clean_scrub_stamp":"2024-04-27T03:34:26.075076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:58.085501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"27'6","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.016547+0000","last_change":"2024-04-27T03:31:01.446266+0000","last_active":"2024-04-27T03:33:21.016547+0000","last_peered":"2024-04-27T03:33:21.016547+0000","last_clean":"2024-04-27T03:33:21.016547+0000","last_became_active":"2024-04-27T03:31:01.440762+0000","last_became_peered":"2024-04-27T03:31:01.440762+0000","last_unstale":"2024-04-27T03:33:21.016547+0000","last_undegraded":"2024-04-27T03:33:21.016547+0000","last_fullsized":"2024-04-27T03:33:21.016547+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:21.120619+0000","last_change":"2024-04-27T03:34:21.120644+0000","last_active":"2024-04-27T03:34:21.120619+0000","last_peered":"2024-04-27T03:34:21.120619+0000","last_clean":"2024-04-27T03:34:21.120619+0000","last_became_active":"2024-04-27T03:30:11.369551+0000","last_became_peered":"2024-04-27T03:30:11.369551+0000","last_unstale":"2024-04-27T03:34:21.120619+0000","last_undegraded":"2024-04-27T03:34:21.120619+0000","last_fullsized":"2024-04-27T03:34:21.120619+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:21.120533+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:21.120533+0000","last_clean_scrub_stamp":"2024-04-27T03:34:21.120533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:54.803174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"27'2","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.850385+0000","last_change":"2024-04-27T03:34:04.850412+0000","last_active":"2024-04-27T03:34:04.850385+0000","last_peered":"2024-04-27T03:34:04.850385+0000","last_clean":"2024-04-27T03:34:04.850385+0000","last_became_active":"2024-04-27T03:30:13.381901+0000","last_became_peered":"2024-04-27T03:30:13.381901+0000","last_unstale":"2024-04-27T03:34:04.850385+0000","last_undegraded":"2024-04-27T03:34:04.850385+0000","last_fullsized":"2024-04-27T03:34:04.850385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:04.850293+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:04.850293+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.850293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:29.194843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"25'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:15.639011+0000","last_change":"2024-04-27T03:34:15.639038+0000","last_active":"2024-04-27T03:34:15.639011+0000","last_peered":"2024-04-27T03:34:15.639011+0000","last_clean":"2024-04-27T03:34:15.639011+0000","last_became_active":"2024-04-27T03:30:11.360515+0000","last_became_peered":"2024-04-27T03:30:11.360515+0000","last_unstale":"2024-04-27T03:34:15.639011+0000","last_undegraded":"2024-04-27T03:34:15.639011+0000","last_fullsized":"2024-04-27T03:34:15.639011+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:15.638909+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:15.638909+0000","last_clean_scrub_stamp":"2024-04-27T03:34:15.638909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:04.458649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"27'4","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.155138+0000","last_change":"2024-04-27T03:33:59.155166+0000","last_active":"2024-04-27T03:33:59.155138+0000","last_peered":"2024-04-27T03:33:59.155138+0000","last_clean":"2024-04-27T03:33:59.155138+0000","last_became_active":"2024-04-27T03:30:13.378932+0000","last_became_peered":"2024-04-27T03:30:13.378932+0000","last_unstale":"2024-04-27T03:33:59.155138+0000","last_undegraded":"2024-04-27T03:33:59.155138+0000","last_fullsized":"2024-04-27T03:33:59.155138+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:59.155047+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:59.155047+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.155047+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:36.999661+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296759+0000","last_change":"2024-04-27T03:30:11.372964+0000","last_active":"2024-04-27T03:32:34.296759+0000","last_peered":"2024-04-27T03:32:34.296759+0000","last_clean":"2024-04-27T03:32:34.296759+0000","last_became_active":"2024-04-27T03:30:11.372673+0000","last_became_peered":"2024-04-27T03:30:11.372673+0000","last_unstale":"2024-04-27T03:32:34.296759+0000","last_undegraded":"2024-04-27T03:32:34.296759+0000","last_fullsized":"2024-04-27T03:32:34.296759+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"27'35","reported_seq":100,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.028786+0000","last_change":"2024-04-27T03:30:21.413597+0000","last_active":"2024-04-27T03:33:21.028786+0000","last_peered":"2024-04-27T03:33:21.028786+0000","last_clean":"2024-04-27T03:33:21.028786+0000","last_became_active":"2024-04-27T03:30:21.413255+0000","last_became_peered":"2024-04-27T03:30:21.413255+0000","last_unstale":"2024-04-27T03:33:21.028786+0000","last_undegraded":"2024-04-27T03:33:21.028786+0000","last_fullsized":"2024-04-27T03:33:21.028786+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":35,"log_dups_size":0,"ondisk_log_size":35,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"27'4","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:16.259266+0000","last_change":"2024-04-27T03:34:16.259291+0000","last_active":"2024-04-27T03:34:16.259266+0000","last_peered":"2024-04-27T03:34:16.259266+0000","last_clean":"2024-04-27T03:34:16.259266+0000","last_became_active":"2024-04-27T03:30:13.377110+0000","last_became_peered":"2024-04-27T03:30:13.377110+0000","last_unstale":"2024-04-27T03:34:16.259266+0000","last_undegraded":"2024-04-27T03:34:16.259266+0000","last_fullsized":"2024-04-27T03:34:16.259266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:16.259165+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:16.259165+0000","last_clean_scrub_stamp":"2024-04-27T03:34:16.259165+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:04.414839+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296793+0000","last_change":"2024-04-27T03:30:11.360152+0000","last_active":"2024-04-27T03:32:34.296793+0000","last_peered":"2024-04-27T03:32:34.296793+0000","last_clean":"2024-04-27T03:32:34.296793+0000","last_became_active":"2024-04-27T03:30:11.359892+0000","last_became_peered":"2024-04-27T03:30:11.359892+0000","last_unstale":"2024-04-27T03:32:34.296793+0000","last_undegraded":"2024-04-27T03:32:34.296793+0000","last_fullsized":"2024-04-27T03:32:34.296793+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.148150+0000","last_change":"2024-04-27T03:33:38.148173+0000","last_active":"2024-04-27T03:33:38.148150+0000","last_peered":"2024-04-27T03:33:38.148150+0000","last_clean":"2024-04-27T03:33:38.148150+0000","last_became_active":"2024-04-27T03:30:15.380864+0000","last_became_peered":"2024-04-27T03:30:15.380864+0000","last_unstale":"2024-04-27T03:33:38.148150+0000","last_undegraded":"2024-04-27T03:33:38.148150+0000","last_fullsized":"2024-04-27T03:33:38.148150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:38.148061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:38.148061+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.148061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:53.865374+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"27'50","reported_seq":117,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.088201+0000","last_change":"2024-04-27T03:33:43.088242+0000","last_active":"2024-04-27T03:33:43.088201+0000","last_peered":"2024-04-27T03:33:43.088201+0000","last_clean":"2024-04-27T03:33:43.088201+0000","last_became_active":"2024-04-27T03:30:17.387391+0000","last_became_peered":"2024-04-27T03:30:17.387391+0000","last_unstale":"2024-04-27T03:33:43.088201+0000","last_undegraded":"2024-04-27T03:33:43.088201+0000","last_fullsized":"2024-04-27T03:33:43.088201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'50","last_scrub_stamp":"2024-04-27T03:33:43.088100+0000","last_deep_scrub":"27'50","last_deep_scrub_stamp":"2024-04-27T03:33:43.088100+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.088100+0000","objects_scrubbed":6,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:13:27.492179+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":45,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.971326+0000","last_change":"2024-04-27T03:33:44.971352+0000","last_active":"2024-04-27T03:33:44.971326+0000","last_peered":"2024-04-27T03:33:44.971326+0000","last_clean":"2024-04-27T03:33:44.971326+0000","last_became_active":"2024-04-27T03:30:19.407637+0000","last_became_peered":"2024-04-27T03:30:19.407637+0000","last_unstale":"2024-04-27T03:33:44.971326+0000","last_undegraded":"2024-04-27T03:33:44.971326+0000","last_fullsized":"2024-04-27T03:33:44.971326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:44.971239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:44.971239+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.971239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:22.448440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"27'9","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:49.299125+0000","last_change":"2024-04-27T03:30:21.414223+0000","last_active":"2024-04-27T03:32:49.299125+0000","last_peered":"2024-04-27T03:32:49.299125+0000","last_clean":"2024-04-27T03:32:49.299125+0000","last_became_active":"2024-04-27T03:30:21.413940+0000","last_became_peered":"2024-04-27T03:30:21.413940+0000","last_unstale":"2024-04-27T03:32:49.299125+0000","last_undegraded":"2024-04-27T03:32:49.299125+0000","last_fullsized":"2024-04-27T03:32:49.299125+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"27'16","reported_seq":76,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.265928+0000","last_change":"2024-04-27T03:34:09.266003+0000","last_active":"2024-04-27T03:34:09.265928+0000","last_peered":"2024-04-27T03:34:09.265928+0000","last_clean":"2024-04-27T03:34:09.265928+0000","last_became_active":"2024-04-27T03:30:13.381487+0000","last_became_peered":"2024-04-27T03:30:13.381487+0000","last_unstale":"2024-04-27T03:34:09.265928+0000","last_undegraded":"2024-04-27T03:34:09.265928+0000","last_fullsized":"2024-04-27T03:34:09.265928+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:09.265817+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:09.265817+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.265817+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:46.218428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:13.813006+0000","last_change":"2024-04-27T03:34:13.813035+0000","last_active":"2024-04-27T03:34:13.813006+0000","last_peered":"2024-04-27T03:34:13.813006+0000","last_clean":"2024-04-27T03:34:13.813006+0000","last_became_active":"2024-04-27T03:30:11.359446+0000","last_became_peered":"2024-04-27T03:30:11.359446+0000","last_unstale":"2024-04-27T03:34:13.813006+0000","last_undegraded":"2024-04-27T03:34:13.813006+0000","last_fullsized":"2024-04-27T03:34:13.813006+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:13.812877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:13.812877+0000","last_clean_scrub_stamp":"2024-04-27T03:34:13.812877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:46.859807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.671578+0000","last_change":"2024-04-27T03:33:55.671603+0000","last_active":"2024-04-27T03:33:55.671578+0000","last_peered":"2024-04-27T03:33:55.671578+0000","last_clean":"2024-04-27T03:33:55.671578+0000","last_became_active":"2024-04-27T03:30:15.380313+0000","last_became_peered":"2024-04-27T03:30:15.380313+0000","last_unstale":"2024-04-27T03:33:55.671578+0000","last_undegraded":"2024-04-27T03:33:55.671578+0000","last_fullsized":"2024-04-27T03:33:55.671578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:55.671471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:55.671471+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.671471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:20.643729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"27'16","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.841774+0000","last_change":"2024-04-27T03:33:37.841840+0000","last_active":"2024-04-27T03:33:37.841774+0000","last_peered":"2024-04-27T03:33:37.841774+0000","last_clean":"2024-04-27T03:33:37.841774+0000","last_became_active":"2024-04-27T03:30:17.389126+0000","last_became_peered":"2024-04-27T03:30:17.389126+0000","last_unstale":"2024-04-27T03:33:37.841774+0000","last_undegraded":"2024-04-27T03:33:37.841774+0000","last_fullsized":"2024-04-27T03:33:37.841774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:33:37.841643+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:33:37.841643+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.841643+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:42.665962+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.837474+0000","last_change":"2024-04-27T03:33:36.837497+0000","last_active":"2024-04-27T03:33:36.837474+0000","last_peered":"2024-04-27T03:33:36.837474+0000","last_clean":"2024-04-27T03:33:36.837474+0000","last_became_active":"2024-04-27T03:30:19.405170+0000","last_became_peered":"2024-04-27T03:30:19.405170+0000","last_unstale":"2024-04-27T03:33:36.837474+0000","last_undegraded":"2024-04-27T03:33:36.837474+0000","last_fullsized":"2024-04-27T03:33:36.837474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.837388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.837388+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.837388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:14.962289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'6","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.661566+0000","last_change":"2024-04-27T03:33:58.661623+0000","last_active":"2024-04-27T03:33:58.661566+0000","last_peered":"2024-04-27T03:33:58.661566+0000","last_clean":"2024-04-27T03:33:58.661566+0000","last_became_active":"2024-04-27T03:30:21.407917+0000","last_became_peered":"2024-04-27T03:30:21.407917+0000","last_unstale":"2024-04-27T03:33:58.661566+0000","last_undegraded":"2024-04-27T03:33:58.661566+0000","last_fullsized":"2024-04-27T03:33:58.661566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:58.661469+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:58.661469+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.661469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:52.661163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":104,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:12.193343+0000","last_change":"2024-04-27T03:34:12.193369+0000","last_active":"2024-04-27T03:34:12.193343+0000","last_peered":"2024-04-27T03:34:12.193343+0000","last_clean":"2024-04-27T03:34:12.193343+0000","last_became_active":"2024-04-27T03:30:13.377423+0000","last_became_peered":"2024-04-27T03:30:13.377423+0000","last_unstale":"2024-04-27T03:34:12.193343+0000","last_undegraded":"2024-04-27T03:34:12.193343+0000","last_fullsized":"2024-04-27T03:34:12.193343+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-04-27T03:34:12.193224+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-04-27T03:34:12.193224+0000","last_clean_scrub_stamp":"2024-04-27T03:34:12.193224+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:00:13.595186+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296713+0000","last_change":"2024-04-27T03:30:11.359542+0000","last_active":"2024-04-27T03:32:34.296713+0000","last_peered":"2024-04-27T03:32:34.296713+0000","last_clean":"2024-04-27T03:32:34.296713+0000","last_became_active":"2024-04-27T03:30:11.359261+0000","last_became_peered":"2024-04-27T03:30:11.359261+0000","last_unstale":"2024-04-27T03:32:34.296713+0000","last_undegraded":"2024-04-27T03:32:34.296713+0000","last_fullsized":"2024-04-27T03:32:34.296713+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.753269+0000","last_change":"2024-04-27T03:33:47.753294+0000","last_active":"2024-04-27T03:33:47.753269+0000","last_peered":"2024-04-27T03:33:47.753269+0000","last_clean":"2024-04-27T03:33:47.753269+0000","last_became_active":"2024-04-27T03:30:15.383618+0000","last_became_peered":"2024-04-27T03:30:15.383618+0000","last_unstale":"2024-04-27T03:33:47.753269+0000","last_undegraded":"2024-04-27T03:33:47.753269+0000","last_fullsized":"2024-04-27T03:33:47.753269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:47.753177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:47.753177+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.753177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:54.718147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"27'30","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.888348+0000","last_change":"2024-04-27T03:33:37.888412+0000","last_active":"2024-04-27T03:33:37.888348+0000","last_peered":"2024-04-27T03:33:37.888348+0000","last_clean":"2024-04-27T03:33:37.888348+0000","last_became_active":"2024-04-27T03:30:17.392223+0000","last_became_peered":"2024-04-27T03:30:17.392223+0000","last_unstale":"2024-04-27T03:33:37.888348+0000","last_undegraded":"2024-04-27T03:33:37.888348+0000","last_fullsized":"2024-04-27T03:33:37.888348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-04-27T03:33:37.888233+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-04-27T03:33:37.888233+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.888233+0000","objects_scrubbed":7,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:27:30.955280+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"27'286","reported_seq":394,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.206383+0000","last_change":"2024-04-27T03:34:03.206432+0000","last_active":"2024-04-27T03:34:03.206383+0000","last_peered":"2024-04-27T03:34:03.206383+0000","last_clean":"2024-04-27T03:34:03.206383+0000","last_became_active":"2024-04-27T03:30:19.405730+0000","last_became_peered":"2024-04-27T03:30:19.405730+0000","last_unstale":"2024-04-27T03:34:03.206383+0000","last_undegraded":"2024-04-27T03:34:03.206383+0000","last_fullsized":"2024-04-27T03:34:03.206383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'286","last_scrub_stamp":"2024-04-27T03:34:03.206276+0000","last_deep_scrub":"27'286","last_deep_scrub_stamp":"2024-04-27T03:34:03.206276+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.206276+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:25.670340+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:19.898900+0000","last_change":"2024-04-27T03:34:19.898939+0000","last_active":"2024-04-27T03:34:19.898900+0000","last_peered":"2024-04-27T03:34:19.898900+0000","last_clean":"2024-04-27T03:34:19.898900+0000","last_became_active":"2024-04-27T03:30:21.409156+0000","last_became_peered":"2024-04-27T03:30:21.409156+0000","last_unstale":"2024-04-27T03:34:19.898900+0000","last_undegraded":"2024-04-27T03:34:19.898900+0000","last_fullsized":"2024-04-27T03:34:19.898900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:19.898783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:19.898783+0000","last_clean_scrub_stamp":"2024-04-27T03:34:19.898783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:23:50.887010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"27'6","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.704469+0000","last_change":"2024-04-27T03:33:59.704499+0000","last_active":"2024-04-27T03:33:59.704469+0000","last_peered":"2024-04-27T03:33:59.704469+0000","last_clean":"2024-04-27T03:33:59.704469+0000","last_became_active":"2024-04-27T03:30:13.385491+0000","last_became_peered":"2024-04-27T03:30:13.385491+0000","last_unstale":"2024-04-27T03:33:59.704469+0000","last_undegraded":"2024-04-27T03:33:59.704469+0000","last_fullsized":"2024-04-27T03:33:59.704469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:59.704383+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:59.704383+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.704383+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:13:15.421696+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:23.050142+0000","last_change":"2024-04-27T03:34:23.050167+0000","last_active":"2024-04-27T03:34:23.050142+0000","last_peered":"2024-04-27T03:34:23.050142+0000","last_clean":"2024-04-27T03:34:23.050142+0000","last_became_active":"2024-04-27T03:30:11.369939+0000","last_became_peered":"2024-04-27T03:30:11.369939+0000","last_unstale":"2024-04-27T03:34:23.050142+0000","last_undegraded":"2024-04-27T03:34:23.050142+0000","last_fullsized":"2024-04-27T03:34:23.050142+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:23.050056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:23.050056+0000","last_clean_scrub_stamp":"2024-04-27T03:34:23.050056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:26:29.085590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.134971+0000","last_change":"2024-04-27T03:33:33.134999+0000","last_active":"2024-04-27T03:33:33.134971+0000","last_peered":"2024-04-27T03:33:33.134971+0000","last_clean":"2024-04-27T03:33:33.134971+0000","last_became_active":"2024-04-27T03:30:15.384114+0000","last_became_peered":"2024-04-27T03:30:15.384114+0000","last_unstale":"2024-04-27T03:33:33.134971+0000","last_undegraded":"2024-04-27T03:33:33.134971+0000","last_fullsized":"2024-04-27T03:33:33.134971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:33.134840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:33.134840+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.134840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:57.479389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"27'43","reported_seq":104,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.859628+0000","last_change":"2024-04-27T03:33:38.859665+0000","last_active":"2024-04-27T03:33:38.859628+0000","last_peered":"2024-04-27T03:33:38.859628+0000","last_clean":"2024-04-27T03:33:38.859628+0000","last_became_active":"2024-04-27T03:30:17.391880+0000","last_became_peered":"2024-04-27T03:30:17.391880+0000","last_unstale":"2024-04-27T03:33:38.859628+0000","last_undegraded":"2024-04-27T03:33:38.859628+0000","last_fullsized":"2024-04-27T03:33:38.859628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'43","last_scrub_stamp":"2024-04-27T03:33:38.859525+0000","last_deep_scrub":"27'43","last_deep_scrub_stamp":"2024-04-27T03:33:38.859525+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.859525+0000","objects_scrubbed":5,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:06.442676+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"27'285","reported_seq":417,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.785523+0000","last_change":"2024-04-27T03:33:35.785553+0000","last_active":"2024-04-27T03:33:35.785523+0000","last_peered":"2024-04-27T03:33:35.785523+0000","last_clean":"2024-04-27T03:33:35.785523+0000","last_became_active":"2024-04-27T03:30:19.404744+0000","last_became_peered":"2024-04-27T03:30:19.404744+0000","last_unstale":"2024-04-27T03:33:35.785523+0000","last_undegraded":"2024-04-27T03:33:35.785523+0000","last_fullsized":"2024-04-27T03:33:35.785523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:35.785437+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:35.785437+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.785437+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:23.864435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'7","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.165585+0000","last_change":"2024-04-27T03:33:49.165611+0000","last_active":"2024-04-27T03:33:49.165585+0000","last_peered":"2024-04-27T03:33:49.165585+0000","last_clean":"2024-04-27T03:33:49.165585+0000","last_became_active":"2024-04-27T03:30:13.387335+0000","last_became_peered":"2024-04-27T03:30:13.387335+0000","last_unstale":"2024-04-27T03:33:49.165585+0000","last_undegraded":"2024-04-27T03:33:49.165585+0000","last_fullsized":"2024-04-27T03:33:49.165585+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:33:49.165500+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:33:49.165500+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.165500+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:22.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.841302+0000","last_change":"2024-04-27T03:34:01.841348+0000","last_active":"2024-04-27T03:34:01.841302+0000","last_peered":"2024-04-27T03:34:01.841302+0000","last_clean":"2024-04-27T03:34:01.841302+0000","last_became_active":"2024-04-27T03:30:11.371600+0000","last_became_peered":"2024-04-27T03:30:11.371600+0000","last_unstale":"2024-04-27T03:34:01.841302+0000","last_undegraded":"2024-04-27T03:34:01.841302+0000","last_fullsized":"2024-04-27T03:34:01.841302+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:01.841193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:01.841193+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.841193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:21:13.198341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.990712+0000","last_change":"2024-04-27T03:33:46.990760+0000","last_active":"2024-04-27T03:33:46.990712+0000","last_peered":"2024-04-27T03:33:46.990712+0000","last_clean":"2024-04-27T03:33:46.990712+0000","last_became_active":"2024-04-27T03:30:15.381359+0000","last_became_peered":"2024-04-27T03:30:15.381359+0000","last_unstale":"2024-04-27T03:33:46.990712+0000","last_undegraded":"2024-04-27T03:33:46.990712+0000","last_fullsized":"2024-04-27T03:33:46.990712+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:46.990601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:46.990601+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.990601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:08.708219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"27'58","reported_seq":124,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.809607+0000","last_change":"2024-04-27T03:33:41.809671+0000","last_active":"2024-04-27T03:33:41.809607+0000","last_peered":"2024-04-27T03:33:41.809607+0000","last_clean":"2024-04-27T03:33:41.809607+0000","last_became_active":"2024-04-27T03:30:17.390021+0000","last_became_peered":"2024-04-27T03:30:17.390021+0000","last_unstale":"2024-04-27T03:33:41.809607+0000","last_undegraded":"2024-04-27T03:33:41.809607+0000","last_fullsized":"2024-04-27T03:33:41.809607+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'58","last_scrub_stamp":"2024-04-27T03:33:41.809501+0000","last_deep_scrub":"27'58","last_deep_scrub_stamp":"2024-04-27T03:33:41.809501+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.809501+0000","objects_scrubbed":10,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:46.251050+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":30,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.738236+0000","last_change":"2024-04-27T03:33:32.738255+0000","last_active":"2024-04-27T03:33:32.738236+0000","last_peered":"2024-04-27T03:33:32.738236+0000","last_clean":"2024-04-27T03:33:32.738236+0000","last_became_active":"2024-04-27T03:30:19.404948+0000","last_became_peered":"2024-04-27T03:30:19.404948+0000","last_unstale":"2024-04-27T03:33:32.738236+0000","last_undegraded":"2024-04-27T03:33:32.738236+0000","last_fullsized":"2024-04-27T03:33:32.738236+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:32.738165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:32.738165+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.738165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:19.681571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"27'3","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:21.929577+0000","last_change":"2024-04-27T03:34:21.929604+0000","last_active":"2024-04-27T03:34:21.929577+0000","last_peered":"2024-04-27T03:34:21.929577+0000","last_clean":"2024-04-27T03:34:21.929577+0000","last_became_active":"2024-04-27T03:30:21.412331+0000","last_became_peered":"2024-04-27T03:30:21.412331+0000","last_unstale":"2024-04-27T03:34:21.929577+0000","last_undegraded":"2024-04-27T03:34:21.929577+0000","last_fullsized":"2024-04-27T03:34:21.929577+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-27T03:34:21.929483+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-27T03:34:21.929483+0000","last_clean_scrub_stamp":"2024-04-27T03:34:21.929483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:01:02.076280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"25'6","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.184692+0000","last_change":"2024-04-27T03:34:06.184718+0000","last_active":"2024-04-27T03:34:06.184692+0000","last_peered":"2024-04-27T03:34:06.184692+0000","last_clean":"2024-04-27T03:34:06.184692+0000","last_became_active":"2024-04-27T03:30:13.373226+0000","last_became_peered":"2024-04-27T03:30:13.373226+0000","last_unstale":"2024-04-27T03:34:06.184692+0000","last_undegraded":"2024-04-27T03:34:06.184692+0000","last_fullsized":"2024-04-27T03:34:06.184692+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'6","last_scrub_stamp":"2024-04-27T03:34:06.184601+0000","last_deep_scrub":"25'6","last_deep_scrub_stamp":"2024-04-27T03:34:06.184601+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.184601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:06.834111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.791842+0000","last_change":"2024-04-27T03:33:59.791865+0000","last_active":"2024-04-27T03:33:59.791842+0000","last_peered":"2024-04-27T03:33:59.791842+0000","last_clean":"2024-04-27T03:33:59.791842+0000","last_became_active":"2024-04-27T03:30:11.373430+0000","last_became_peered":"2024-04-27T03:30:11.373430+0000","last_unstale":"2024-04-27T03:33:59.791842+0000","last_undegraded":"2024-04-27T03:33:59.791842+0000","last_fullsized":"2024-04-27T03:33:59.791842+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:59.791751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:59.791751+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.791751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:23.025879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"27'2","reported_seq":90,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:25.988249+0000","last_change":"2024-04-27T03:34:25.988283+0000","last_active":"2024-04-27T03:34:25.988249+0000","last_peered":"2024-04-27T03:34:25.988249+0000","last_clean":"2024-04-27T03:34:25.988249+0000","last_became_active":"2024-04-27T03:31:01.443259+0000","last_became_peered":"2024-04-27T03:31:01.443259+0000","last_unstale":"2024-04-27T03:34:25.988249+0000","last_undegraded":"2024-04-27T03:34:25.988249+0000","last_fullsized":"2024-04-27T03:34:25.988249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:25.988159+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:25.988159+0000","last_clean_scrub_stamp":"2024-04-27T03:34:25.988159+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:14.807899+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"27'47","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.230606+0000","last_change":"2024-04-27T03:34:04.230642+0000","last_active":"2024-04-27T03:34:04.230606+0000","last_peered":"2024-04-27T03:34:04.230606+0000","last_clean":"2024-04-27T03:34:04.230606+0000","last_became_active":"2024-04-27T03:30:17.388994+0000","last_became_peered":"2024-04-27T03:30:17.388994+0000","last_unstale":"2024-04-27T03:34:04.230606+0000","last_undegraded":"2024-04-27T03:34:04.230606+0000","last_fullsized":"2024-04-27T03:34:04.230606+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-04-27T03:34:04.230519+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-04-27T03:34:04.230519+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.230519+0000","objects_scrubbed":5,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:43:53.086291+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:30.805151+0000","last_change":"2024-04-27T03:33:30.805201+0000","last_active":"2024-04-27T03:33:30.805151+0000","last_peered":"2024-04-27T03:33:30.805151+0000","last_clean":"2024-04-27T03:33:30.805151+0000","last_became_active":"2024-04-27T03:30:19.405334+0000","last_became_peered":"2024-04-27T03:30:19.405334+0000","last_unstale":"2024-04-27T03:33:30.805151+0000","last_undegraded":"2024-04-27T03:33:30.805151+0000","last_fullsized":"2024-04-27T03:33:30.805151+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:30.805019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:30.805019+0000","last_clean_scrub_stamp":"2024-04-27T03:33:30.805019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:46:37.451973+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"27'4","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.099583+0000","last_change":"2024-04-27T03:33:57.099611+0000","last_active":"2024-04-27T03:33:57.099583+0000","last_peered":"2024-04-27T03:33:57.099583+0000","last_clean":"2024-04-27T03:33:57.099583+0000","last_became_active":"2024-04-27T03:30:21.409922+0000","last_became_peered":"2024-04-27T03:30:21.409922+0000","last_unstale":"2024-04-27T03:33:57.099583+0000","last_undegraded":"2024-04-27T03:33:57.099583+0000","last_fullsized":"2024-04-27T03:33:57.099583+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:57.099495+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:57.099495+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.099495+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:01.218125+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"27'6","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.819531+0000","last_change":"2024-04-27T03:34:02.819584+0000","last_active":"2024-04-27T03:34:02.819531+0000","last_peered":"2024-04-27T03:34:02.819531+0000","last_clean":"2024-04-27T03:34:02.819531+0000","last_became_active":"2024-04-27T03:30:13.381270+0000","last_became_peered":"2024-04-27T03:30:13.381270+0000","last_unstale":"2024-04-27T03:34:02.819531+0000","last_undegraded":"2024-04-27T03:34:02.819531+0000","last_fullsized":"2024-04-27T03:34:02.819531+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:02.819411+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:02.819411+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.819411+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:59.129585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295920+0000","last_change":"2024-04-27T03:30:11.372966+0000","last_active":"2024-04-27T03:32:29.295920+0000","last_peered":"2024-04-27T03:32:29.295920+0000","last_clean":"2024-04-27T03:32:29.295920+0000","last_became_active":"2024-04-27T03:30:11.372583+0000","last_became_peered":"2024-04-27T03:30:11.372583+0000","last_unstale":"2024-04-27T03:32:29.295920+0000","last_undegraded":"2024-04-27T03:32:29.295920+0000","last_fullsized":"2024-04-27T03:32:29.295920+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.709284+0000","last_change":"2024-04-27T03:33:51.709320+0000","last_active":"2024-04-27T03:33:51.709284+0000","last_peered":"2024-04-27T03:33:51.709284+0000","last_clean":"2024-04-27T03:33:51.709284+0000","last_became_active":"2024-04-27T03:30:15.382207+0000","last_became_peered":"2024-04-27T03:30:15.382207+0000","last_unstale":"2024-04-27T03:33:51.709284+0000","last_undegraded":"2024-04-27T03:33:51.709284+0000","last_fullsized":"2024-04-27T03:33:51.709284+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:51.709192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:51.709192+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.709192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:40.618465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"27'61","reported_seq":131,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.183997+0000","last_change":"2024-04-27T03:34:00.184072+0000","last_active":"2024-04-27T03:34:00.183997+0000","last_peered":"2024-04-27T03:34:00.183997+0000","last_clean":"2024-04-27T03:34:00.183997+0000","last_became_active":"2024-04-27T03:30:17.389242+0000","last_became_peered":"2024-04-27T03:30:17.389242+0000","last_unstale":"2024-04-27T03:34:00.183997+0000","last_undegraded":"2024-04-27T03:34:00.183997+0000","last_fullsized":"2024-04-27T03:34:00.183997+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'61","last_scrub_stamp":"2024-04-27T03:34:00.183892+0000","last_deep_scrub":"27'61","last_deep_scrub_stamp":"2024-04-27T03:34:00.183892+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.183892+0000","objects_scrubbed":9,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:30:17.176389+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":62,"num_read_kb":51,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"27'285","reported_seq":360,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.734569+0000","last_change":"2024-04-27T03:33:38.734598+0000","last_active":"2024-04-27T03:33:38.734569+0000","last_peered":"2024-04-27T03:33:38.734569+0000","last_clean":"2024-04-27T03:33:38.734569+0000","last_became_active":"2024-04-27T03:30:19.406364+0000","last_became_peered":"2024-04-27T03:30:19.406364+0000","last_unstale":"2024-04-27T03:33:38.734569+0000","last_undegraded":"2024-04-27T03:33:38.734569+0000","last_fullsized":"2024-04-27T03:33:38.734569+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:38.734486+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:38.734486+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.734486+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:11.608629+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:27.705515+0000","last_change":"2024-04-27T03:34:27.705542+0000","last_active":"2024-04-27T03:34:27.705515+0000","last_peered":"2024-04-27T03:34:27.705515+0000","last_clean":"2024-04-27T03:34:27.705515+0000","last_became_active":"2024-04-27T03:30:21.408168+0000","last_became_peered":"2024-04-27T03:30:21.408168+0000","last_unstale":"2024-04-27T03:34:27.705515+0000","last_undegraded":"2024-04-27T03:34:27.705515+0000","last_fullsized":"2024-04-27T03:34:27.705515+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:27.705421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:27.705421+0000","last_clean_scrub_stamp":"2024-04-27T03:34:27.705421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:10.961869+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"27'14","reported_seq":110,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.819117+0000","last_change":"2024-04-27T03:33:55.819144+0000","last_active":"2024-04-27T03:33:55.819117+0000","last_peered":"2024-04-27T03:33:55.819117+0000","last_clean":"2024-04-27T03:33:55.819117+0000","last_became_active":"2024-04-27T03:30:13.371503+0000","last_became_peered":"2024-04-27T03:30:13.371503+0000","last_unstale":"2024-04-27T03:33:55.819117+0000","last_undegraded":"2024-04-27T03:33:55.819117+0000","last_fullsized":"2024-04-27T03:33:55.819117+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:33:55.819026+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:33:55.819026+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.819026+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:53.861728+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"25'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295881+0000","last_change":"2024-04-27T03:30:11.357185+0000","last_active":"2024-04-27T03:32:29.295881+0000","last_peered":"2024-04-27T03:32:29.295881+0000","last_clean":"2024-04-27T03:32:29.295881+0000","last_became_active":"2024-04-27T03:30:11.356739+0000","last_became_peered":"2024-04-27T03:30:11.356739+0000","last_unstale":"2024-04-27T03:32:29.295881+0000","last_undegraded":"2024-04-27T03:32:29.295881+0000","last_fullsized":"2024-04-27T03:32:29.295881+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:23.927184+0000","last_change":"2024-04-27T03:34:23.927207+0000","last_active":"2024-04-27T03:34:23.927184+0000","last_peered":"2024-04-27T03:34:23.927184+0000","last_clean":"2024-04-27T03:34:23.927184+0000","last_became_active":"2024-04-27T03:31:01.443118+0000","last_became_peered":"2024-04-27T03:31:01.443118+0000","last_unstale":"2024-04-27T03:34:23.927184+0000","last_undegraded":"2024-04-27T03:34:23.927184+0000","last_fullsized":"2024-04-27T03:34:23.927184+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:23.927098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:23.927098+0000","last_clean_scrub_stamp":"2024-04-27T03:34:23.927098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:25:26.728092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"27'79","reported_seq":164,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.115580+0000","last_change":"2024-04-27T03:33:56.115644+0000","last_active":"2024-04-27T03:33:56.115580+0000","last_peered":"2024-04-27T03:33:56.115580+0000","last_clean":"2024-04-27T03:33:56.115580+0000","last_became_active":"2024-04-27T03:30:17.387318+0000","last_became_peered":"2024-04-27T03:30:17.387318+0000","last_unstale":"2024-04-27T03:33:56.115580+0000","last_undegraded":"2024-04-27T03:33:56.115580+0000","last_fullsized":"2024-04-27T03:33:56.115580+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'79","last_scrub_stamp":"2024-04-27T03:33:56.115468+0000","last_deep_scrub":"27'79","last_deep_scrub_stamp":"2024-04-27T03:33:56.115468+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.115468+0000","objects_scrubbed":7,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:06.434627+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":79,"num_read_kb":68,"num_write":34,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.701834+0000","last_change":"2024-04-27T03:33:37.701877+0000","last_active":"2024-04-27T03:33:37.701834+0000","last_peered":"2024-04-27T03:33:37.701834+0000","last_clean":"2024-04-27T03:33:37.701834+0000","last_became_active":"2024-04-27T03:30:19.404265+0000","last_became_peered":"2024-04-27T03:30:19.404265+0000","last_unstale":"2024-04-27T03:33:37.701834+0000","last_undegraded":"2024-04-27T03:33:37.701834+0000","last_fullsized":"2024-04-27T03:33:37.701834+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:37.701748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:37.701748+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.701748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:42.526067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"27'18","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:21.124420+0000","last_change":"2024-04-27T03:30:21.414109+0000","last_active":"2024-04-27T03:33:21.124420+0000","last_peered":"2024-04-27T03:33:21.124420+0000","last_clean":"2024-04-27T03:33:21.124420+0000","last_became_active":"2024-04-27T03:30:21.413757+0000","last_became_peered":"2024-04-27T03:30:21.413757+0000","last_unstale":"2024-04-27T03:33:21.124420+0000","last_undegraded":"2024-04-27T03:33:21.124420+0000","last_fullsized":"2024-04-27T03:33:21.124420+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"27'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:11.168951+0000","last_change":"2024-04-27T03:34:11.168986+0000","last_active":"2024-04-27T03:34:11.168951+0000","last_peered":"2024-04-27T03:34:11.168951+0000","last_clean":"2024-04-27T03:34:11.168951+0000","last_became_active":"2024-04-27T03:30:13.379861+0000","last_became_peered":"2024-04-27T03:30:13.379861+0000","last_unstale":"2024-04-27T03:34:11.168951+0000","last_undegraded":"2024-04-27T03:34:11.168951+0000","last_fullsized":"2024-04-27T03:34:11.168951+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:11.168855+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:11.168855+0000","last_clean_scrub_stamp":"2024-04-27T03:34:11.168855+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:36.750674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:18.628805+0000","last_change":"2024-04-27T03:34:18.628832+0000","last_active":"2024-04-27T03:34:18.628805+0000","last_peered":"2024-04-27T03:34:18.628805+0000","last_clean":"2024-04-27T03:34:18.628805+0000","last_became_active":"2024-04-27T03:30:11.357176+0000","last_became_peered":"2024-04-27T03:30:11.357176+0000","last_unstale":"2024-04-27T03:34:18.628805+0000","last_undegraded":"2024-04-27T03:34:18.628805+0000","last_fullsized":"2024-04-27T03:34:18.628805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:18.628698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:18.628698+0000","last_clean_scrub_stamp":"2024-04-27T03:34:18.628698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:24.899427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.754096+0000","last_change":"2024-04-27T03:33:49.754119+0000","last_active":"2024-04-27T03:33:49.754096+0000","last_peered":"2024-04-27T03:33:49.754096+0000","last_clean":"2024-04-27T03:33:49.754096+0000","last_became_active":"2024-04-27T03:30:15.382290+0000","last_became_peered":"2024-04-27T03:30:15.382290+0000","last_unstale":"2024-04-27T03:33:49.754096+0000","last_undegraded":"2024-04-27T03:33:49.754096+0000","last_fullsized":"2024-04-27T03:33:49.754096+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:49.754004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:49.754004+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.754004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:57.154906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"27'46","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.971592+0000","last_change":"2024-04-27T03:33:41.971630+0000","last_active":"2024-04-27T03:33:41.971592+0000","last_peered":"2024-04-27T03:33:41.971592+0000","last_clean":"2024-04-27T03:33:41.971592+0000","last_became_active":"2024-04-27T03:30:17.385902+0000","last_became_peered":"2024-04-27T03:30:17.385902+0000","last_unstale":"2024-04-27T03:33:41.971592+0000","last_undegraded":"2024-04-27T03:33:41.971592+0000","last_fullsized":"2024-04-27T03:33:41.971592+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:41.971499+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:41.971499+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.971499+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:48.621217+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.094297+0000","last_change":"2024-04-27T03:33:55.094322+0000","last_active":"2024-04-27T03:33:55.094297+0000","last_peered":"2024-04-27T03:33:55.094297+0000","last_clean":"2024-04-27T03:33:55.094297+0000","last_became_active":"2024-04-27T03:30:19.405450+0000","last_became_peered":"2024-04-27T03:30:19.405450+0000","last_unstale":"2024-04-27T03:33:55.094297+0000","last_undegraded":"2024-04-27T03:33:55.094297+0000","last_fullsized":"2024-04-27T03:33:55.094297+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:55.094207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:55.094207+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.094207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:33.101519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"27'9","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.757256+0000","last_change":"2024-04-27T03:30:21.415884+0000","last_active":"2024-04-27T03:33:31.757256+0000","last_peered":"2024-04-27T03:33:31.757256+0000","last_clean":"2024-04-27T03:33:31.757256+0000","last_became_active":"2024-04-27T03:30:21.415494+0000","last_became_peered":"2024-04-27T03:30:21.415494+0000","last_unstale":"2024-04-27T03:33:31.757256+0000","last_undegraded":"2024-04-27T03:33:31.757256+0000","last_fullsized":"2024-04-27T03:33:31.757256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'6","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.201209+0000","last_change":"2024-04-27T03:33:48.201236+0000","last_active":"2024-04-27T03:33:48.201209+0000","last_peered":"2024-04-27T03:33:48.201209+0000","last_clean":"2024-04-27T03:33:48.201209+0000","last_became_active":"2024-04-27T03:30:13.387533+0000","last_became_peered":"2024-04-27T03:30:13.387533+0000","last_unstale":"2024-04-27T03:33:48.201209+0000","last_undegraded":"2024-04-27T03:33:48.201209+0000","last_fullsized":"2024-04-27T03:33:48.201209+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:48.201118+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:48.201118+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.201118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:48.546003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:22.293367+0000","last_change":"2024-04-27T03:34:22.293400+0000","last_active":"2024-04-27T03:34:22.293367+0000","last_peered":"2024-04-27T03:34:22.293367+0000","last_clean":"2024-04-27T03:34:22.293367+0000","last_became_active":"2024-04-27T03:30:11.373440+0000","last_became_peered":"2024-04-27T03:30:11.373440+0000","last_unstale":"2024-04-27T03:34:22.293367+0000","last_undegraded":"2024-04-27T03:34:22.293367+0000","last_fullsized":"2024-04-27T03:34:22.293367+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:22.293281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:22.293281+0000","last_clean_scrub_stamp":"2024-04-27T03:34:22.293281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:13:00.638888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.730998+0000","last_change":"2024-04-27T03:33:48.731024+0000","last_active":"2024-04-27T03:33:48.730998+0000","last_peered":"2024-04-27T03:33:48.730998+0000","last_clean":"2024-04-27T03:33:48.730998+0000","last_became_active":"2024-04-27T03:30:15.382913+0000","last_became_peered":"2024-04-27T03:30:15.382913+0000","last_unstale":"2024-04-27T03:33:48.730998+0000","last_undegraded":"2024-04-27T03:33:48.730998+0000","last_fullsized":"2024-04-27T03:33:48.730998+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.730891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.730891+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.730891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:49.302463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"27'45","reported_seq":103,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.809157+0000","last_change":"2024-04-27T03:33:45.809220+0000","last_active":"2024-04-27T03:33:45.809157+0000","last_peered":"2024-04-27T03:33:45.809157+0000","last_clean":"2024-04-27T03:33:45.809157+0000","last_became_active":"2024-04-27T03:30:17.381106+0000","last_became_peered":"2024-04-27T03:30:17.381106+0000","last_unstale":"2024-04-27T03:33:45.809157+0000","last_undegraded":"2024-04-27T03:33:45.809157+0000","last_fullsized":"2024-04-27T03:33:45.809157+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'45","last_scrub_stamp":"2024-04-27T03:33:45.809061+0000","last_deep_scrub":"27'45","last_deep_scrub_stamp":"2024-04-27T03:33:45.809061+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.809061+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:33.888060+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"27'285","reported_seq":415,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.864438+0000","last_change":"2024-04-27T03:33:31.864480+0000","last_active":"2024-04-27T03:33:31.864438+0000","last_peered":"2024-04-27T03:33:31.864438+0000","last_clean":"2024-04-27T03:33:31.864438+0000","last_became_active":"2024-04-27T03:30:19.403413+0000","last_became_peered":"2024-04-27T03:30:19.403413+0000","last_unstale":"2024-04-27T03:33:31.864438+0000","last_undegraded":"2024-04-27T03:33:31.864438+0000","last_fullsized":"2024-04-27T03:33:31.864438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:31.864348+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:31.864348+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.864348+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:27.874206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"27'15","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.705541+0000","last_change":"2024-04-27T03:33:54.705571+0000","last_active":"2024-04-27T03:33:54.705541+0000","last_peered":"2024-04-27T03:33:54.705541+0000","last_clean":"2024-04-27T03:33:54.705541+0000","last_became_active":"2024-04-27T03:30:21.402102+0000","last_became_peered":"2024-04-27T03:30:21.402102+0000","last_unstale":"2024-04-27T03:33:54.705541+0000","last_undegraded":"2024-04-27T03:33:54.705541+0000","last_fullsized":"2024-04-27T03:33:54.705541+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-04-27T03:33:54.705410+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-04-27T03:33:54.705410+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.705410+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:00.713477+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":23,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'12","reported_seq":71,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.884557+0000","last_change":"2024-04-27T03:34:08.884586+0000","last_active":"2024-04-27T03:34:08.884557+0000","last_peered":"2024-04-27T03:34:08.884557+0000","last_clean":"2024-04-27T03:34:08.884557+0000","last_became_active":"2024-04-27T03:30:13.382899+0000","last_became_peered":"2024-04-27T03:30:13.382899+0000","last_unstale":"2024-04-27T03:34:08.884557+0000","last_undegraded":"2024-04-27T03:34:08.884557+0000","last_fullsized":"2024-04-27T03:34:08.884557+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-04-27T03:34:08.884470+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-04-27T03:34:08.884470+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.884470+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:46.729084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'4","reported_seq":19,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.926560+0000","last_change":"2024-04-27T03:31:01.444153+0000","last_active":"2024-04-27T03:33:20.926560+0000","last_peered":"2024-04-27T03:33:20.926560+0000","last_clean":"2024-04-27T03:33:20.926560+0000","last_became_active":"2024-04-27T03:31:01.439776+0000","last_became_peered":"2024-04-27T03:31:01.439776+0000","last_unstale":"2024-04-27T03:33:20.926560+0000","last_undegraded":"2024-04-27T03:33:20.926560+0000","last_fullsized":"2024-04-27T03:33:20.926560+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"27'5","reported_seq":428,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.950334+0000","last_change":"2024-04-27T03:33:43.950380+0000","last_active":"2024-04-27T03:33:43.950334+0000","last_peered":"2024-04-27T03:33:43.950334+0000","last_clean":"2024-04-27T03:33:43.950334+0000","last_became_active":"2024-04-27T03:30:15.382502+0000","last_became_peered":"2024-04-27T03:30:15.382502+0000","last_unstale":"2024-04-27T03:33:43.950334+0000","last_undegraded":"2024-04-27T03:33:43.950334+0000","last_fullsized":"2024-04-27T03:33:43.950334+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:43.950258+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:43.950258+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.950258+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:48.355472+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"27'15","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.197694+0000","last_change":"2024-04-27T03:33:58.197757+0000","last_active":"2024-04-27T03:33:58.197694+0000","last_peered":"2024-04-27T03:33:58.197694+0000","last_clean":"2024-04-27T03:33:58.197694+0000","last_became_active":"2024-04-27T03:30:17.387739+0000","last_became_peered":"2024-04-27T03:30:17.387739+0000","last_unstale":"2024-04-27T03:33:58.197694+0000","last_undegraded":"2024-04-27T03:33:58.197694+0000","last_fullsized":"2024-04-27T03:33:58.197694+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-04-27T03:33:58.197595+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-04-27T03:33:58.197595+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.197595+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:01.324323+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"27'285","reported_seq":522,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.933904+0000","last_change":"2024-04-27T03:33:32.933929+0000","last_active":"2024-04-27T03:33:32.933904+0000","last_peered":"2024-04-27T03:33:32.933904+0000","last_clean":"2024-04-27T03:33:32.933904+0000","last_became_active":"2024-04-27T03:30:19.402484+0000","last_became_peered":"2024-04-27T03:30:19.402484+0000","last_unstale":"2024-04-27T03:33:32.933904+0000","last_undegraded":"2024-04-27T03:33:32.933904+0000","last_fullsized":"2024-04-27T03:33:32.933904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:32.933800+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:32.933800+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.933800+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:19.924355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.778511+0000","last_change":"2024-04-27T03:33:52.778552+0000","last_active":"2024-04-27T03:33:52.778511+0000","last_peered":"2024-04-27T03:33:52.778511+0000","last_clean":"2024-04-27T03:33:52.778511+0000","last_became_active":"2024-04-27T03:30:21.408512+0000","last_became_peered":"2024-04-27T03:30:21.408512+0000","last_unstale":"2024-04-27T03:33:52.778511+0000","last_undegraded":"2024-04-27T03:33:52.778511+0000","last_fullsized":"2024-04-27T03:33:52.778511+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:52.778420+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:52.778420+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.778420+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:57.183634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'14","reported_seq":79,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.153948+0000","last_change":"2024-04-27T03:34:02.153985+0000","last_active":"2024-04-27T03:34:02.153948+0000","last_peered":"2024-04-27T03:34:02.153948+0000","last_clean":"2024-04-27T03:34:02.153948+0000","last_became_active":"2024-04-27T03:30:13.380540+0000","last_became_peered":"2024-04-27T03:30:13.380540+0000","last_unstale":"2024-04-27T03:34:02.153948+0000","last_undegraded":"2024-04-27T03:34:02.153948+0000","last_fullsized":"2024-04-27T03:34:02.153948+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:34:02.153863+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:34:02.153863+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.153863+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:47.534998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:24.295052+0000","last_change":"2024-04-27T03:30:11.372470+0000","last_active":"2024-04-27T03:32:24.295052+0000","last_peered":"2024-04-27T03:32:24.295052+0000","last_clean":"2024-04-27T03:32:24.295052+0000","last_became_active":"2024-04-27T03:30:11.372193+0000","last_became_peered":"2024-04-27T03:30:11.372193+0000","last_unstale":"2024-04-27T03:32:24.295052+0000","last_undegraded":"2024-04-27T03:32:24.295052+0000","last_fullsized":"2024-04-27T03:32:24.295052+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.115468+0000","last_change":"2024-04-27T03:33:35.115491+0000","last_active":"2024-04-27T03:33:35.115468+0000","last_peered":"2024-04-27T03:33:35.115468+0000","last_clean":"2024-04-27T03:33:35.115468+0000","last_became_active":"2024-04-27T03:30:15.383924+0000","last_became_peered":"2024-04-27T03:30:15.383924+0000","last_unstale":"2024-04-27T03:33:35.115468+0000","last_undegraded":"2024-04-27T03:33:35.115468+0000","last_fullsized":"2024-04-27T03:33:35.115468+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:35.115383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:35.115383+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.115383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:37.564842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"27'48","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.883186+0000","last_change":"2024-04-27T03:33:38.883223+0000","last_active":"2024-04-27T03:33:38.883186+0000","last_peered":"2024-04-27T03:33:38.883186+0000","last_clean":"2024-04-27T03:33:38.883186+0000","last_became_active":"2024-04-27T03:30:17.387578+0000","last_became_peered":"2024-04-27T03:30:17.387578+0000","last_unstale":"2024-04-27T03:33:38.883186+0000","last_undegraded":"2024-04-27T03:33:38.883186+0000","last_fullsized":"2024-04-27T03:33:38.883186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-04-27T03:33:38.883110+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-04-27T03:33:38.883110+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.883110+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:11.757254+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.856113+0000","last_change":"2024-04-27T03:33:32.856137+0000","last_active":"2024-04-27T03:33:32.856113+0000","last_peered":"2024-04-27T03:33:32.856113+0000","last_clean":"2024-04-27T03:33:32.856113+0000","last_became_active":"2024-04-27T03:30:19.405512+0000","last_became_peered":"2024-04-27T03:30:19.405512+0000","last_unstale":"2024-04-27T03:33:32.856113+0000","last_undegraded":"2024-04-27T03:33:32.856113+0000","last_fullsized":"2024-04-27T03:33:32.856113+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:32.856020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:32.856020+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.856020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:34.194673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"27'14","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.990549+0000","last_change":"2024-04-27T03:33:48.990573+0000","last_active":"2024-04-27T03:33:48.990549+0000","last_peered":"2024-04-27T03:33:48.990549+0000","last_clean":"2024-04-27T03:33:48.990549+0000","last_became_active":"2024-04-27T03:30:21.406883+0000","last_became_peered":"2024-04-27T03:30:21.406883+0000","last_unstale":"2024-04-27T03:33:48.990549+0000","last_undegraded":"2024-04-27T03:33:48.990549+0000","last_fullsized":"2024-04-27T03:33:48.990549+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:33:48.990468+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:33:48.990468+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.990468+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:14:34.099716+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":59,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'20","reported_seq":89,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.148613+0000","last_change":"2024-04-27T03:33:52.148613+0000","last_active":"2024-04-27T03:33:52.148613+0000","last_peered":"2024-04-27T03:33:52.148613+0000","last_clean":"2024-04-27T03:33:52.148613+0000","last_became_active":"2024-04-27T03:30:13.387180+0000","last_became_peered":"2024-04-27T03:30:13.387180+0000","last_unstale":"2024-04-27T03:33:52.148613+0000","last_undegraded":"2024-04-27T03:33:52.148613+0000","last_fullsized":"2024-04-27T03:33:52.148613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-04-27T03:33:52.148491+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-04-27T03:33:52.148491+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.148491+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:24:54.949485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":205,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:19.121523+0000","last_change":"2024-04-27T03:34:19.121563+0000","last_active":"2024-04-27T03:34:19.121523+0000","last_peered":"2024-04-27T03:34:19.121523+0000","last_clean":"2024-04-27T03:34:19.121523+0000","last_became_active":"2024-04-27T03:30:06.102893+0000","last_became_peered":"2024-04-27T03:30:06.102893+0000","last_unstale":"2024-04-27T03:34:19.121523+0000","last_undegraded":"2024-04-27T03:34:19.121523+0000","last_fullsized":"2024-04-27T03:34:19.121523+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-04-27T03:34:19.121425+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-04-27T03:34:19.121425+0000","last_clean_scrub_stamp":"2024-04-27T03:34:19.121425+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:22.248153+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.755816+0000","last_change":"2024-04-27T03:30:06.106875+0000","last_active":"2024-04-27T03:33:31.755816+0000","last_peered":"2024-04-27T03:33:31.755816+0000","last_clean":"2024-04-27T03:33:31.755816+0000","last_became_active":"2024-04-27T03:30:06.106511+0000","last_became_peered":"2024-04-27T03:30:06.106511+0000","last_unstale":"2024-04-27T03:33:31.755816+0000","last_undegraded":"2024-04-27T03:33:31.755816+0000","last_fullsized":"2024-04-27T03:33:31.755816+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:28.327417+0000","last_change":"2024-04-27T03:34:28.327451+0000","last_active":"2024-04-27T03:34:28.327417+0000","last_peered":"2024-04-27T03:34:28.327417+0000","last_clean":"2024-04-27T03:34:28.327417+0000","last_became_active":"2024-04-27T03:30:11.362445+0000","last_became_peered":"2024-04-27T03:30:11.362445+0000","last_unstale":"2024-04-27T03:34:28.327417+0000","last_undegraded":"2024-04-27T03:34:28.327417+0000","last_fullsized":"2024-04-27T03:34:28.327417+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:28.327321+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:28.327321+0000","last_clean_scrub_stamp":"2024-04-27T03:34:28.327321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:37.622290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:30.911615+0000","last_change":"2024-04-27T03:31:01.441027+0000","last_active":"2024-04-27T03:33:30.911615+0000","last_peered":"2024-04-27T03:33:30.911615+0000","last_clean":"2024-04-27T03:33:30.911615+0000","last_became_active":"2024-04-27T03:31:01.440678+0000","last_became_peered":"2024-04-27T03:31:01.440678+0000","last_unstale":"2024-04-27T03:33:30.911615+0000","last_undegraded":"2024-04-27T03:33:30.911615+0000","last_fullsized":"2024-04-27T03:33:30.911615+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:14.357542+0000","last_clean_scrub_stamp":"2024-04-27T03:30:14.357542+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"27'95","reported_seq":191,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.176299+0000","last_change":"2024-04-27T03:33:45.176369+0000","last_active":"2024-04-27T03:33:45.176299+0000","last_peered":"2024-04-27T03:33:45.176299+0000","last_clean":"2024-04-27T03:33:45.176299+0000","last_became_active":"2024-04-27T03:30:17.387707+0000","last_became_peered":"2024-04-27T03:30:17.387707+0000","last_unstale":"2024-04-27T03:33:45.176299+0000","last_undegraded":"2024-04-27T03:33:45.176299+0000","last_fullsized":"2024-04-27T03:33:45.176299+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'95","last_scrub_stamp":"2024-04-27T03:33:45.176195+0000","last_deep_scrub":"27'95","last_deep_scrub_stamp":"2024-04-27T03:33:45.176195+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.176195+0000","objects_scrubbed":6,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:45.184076+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":93,"num_read_kb":81,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.779872+0000","last_change":"2024-04-27T03:33:31.779891+0000","last_active":"2024-04-27T03:33:31.779872+0000","last_peered":"2024-04-27T03:33:31.779872+0000","last_clean":"2024-04-27T03:33:31.779872+0000","last_became_active":"2024-04-27T03:30:19.405359+0000","last_became_peered":"2024-04-27T03:30:19.405359+0000","last_unstale":"2024-04-27T03:33:31.779872+0000","last_undegraded":"2024-04-27T03:33:31.779872+0000","last_fullsized":"2024-04-27T03:33:31.779872+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:31.779786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:31.779786+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.779786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:45.852820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"27'81","reported_seq":171,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.838882+0000","last_change":"2024-04-27T03:33:44.838907+0000","last_active":"2024-04-27T03:33:44.838882+0000","last_peered":"2024-04-27T03:33:44.838882+0000","last_clean":"2024-04-27T03:33:44.838882+0000","last_became_active":"2024-04-27T03:30:21.404300+0000","last_became_peered":"2024-04-27T03:30:21.404300+0000","last_unstale":"2024-04-27T03:33:44.838882+0000","last_undegraded":"2024-04-27T03:33:44.838882+0000","last_fullsized":"2024-04-27T03:33:44.838882+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'81","last_scrub_stamp":"2024-04-27T03:33:44.838760+0000","last_deep_scrub":"27'81","last_deep_scrub_stamp":"2024-04-27T03:33:44.838760+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.838760+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:18.504969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:10.245195+0000","last_change":"2024-04-27T03:34:10.245276+0000","last_active":"2024-04-27T03:34:10.245195+0000","last_peered":"2024-04-27T03:34:10.245195+0000","last_clean":"2024-04-27T03:34:10.245195+0000","last_became_active":"2024-04-27T03:30:13.377713+0000","last_became_peered":"2024-04-27T03:30:13.377713+0000","last_unstale":"2024-04-27T03:34:10.245195+0000","last_undegraded":"2024-04-27T03:34:10.245195+0000","last_fullsized":"2024-04-27T03:34:10.245195+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:10.245096+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:10.245096+0000","last_clean_scrub_stamp":"2024-04-27T03:34:10.245096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:32.051573+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.755713+0000","last_change":"2024-04-27T03:30:06.106890+0000","last_active":"2024-04-27T03:33:31.755713+0000","last_peered":"2024-04-27T03:33:31.755713+0000","last_clean":"2024-04-27T03:33:31.755713+0000","last_became_active":"2024-04-27T03:30:06.106474+0000","last_became_peered":"2024-04-27T03:30:06.106474+0000","last_unstale":"2024-04-27T03:33:31.755713+0000","last_undegraded":"2024-04-27T03:33:31.755713+0000","last_fullsized":"2024-04-27T03:33:31.755713+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:20.655637+0000","last_change":"2024-04-27T03:34:20.655637+0000","last_active":"2024-04-27T03:34:20.655637+0000","last_peered":"2024-04-27T03:34:20.655637+0000","last_clean":"2024-04-27T03:34:20.655637+0000","last_became_active":"2024-04-27T03:30:11.360244+0000","last_became_peered":"2024-04-27T03:30:11.360244+0000","last_unstale":"2024-04-27T03:34:20.655637+0000","last_undegraded":"2024-04-27T03:34:20.655637+0000","last_fullsized":"2024-04-27T03:34:20.655637+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:20.655487+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:20.655487+0000","last_clean_scrub_stamp":"2024-04-27T03:34:20.655487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:53:31.032767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.098611+0000","last_change":"2024-04-27T03:33:42.098651+0000","last_active":"2024-04-27T03:33:42.098611+0000","last_peered":"2024-04-27T03:33:42.098611+0000","last_clean":"2024-04-27T03:33:42.098611+0000","last_became_active":"2024-04-27T03:30:15.379448+0000","last_became_peered":"2024-04-27T03:30:15.379448+0000","last_unstale":"2024-04-27T03:33:42.098611+0000","last_undegraded":"2024-04-27T03:33:42.098611+0000","last_fullsized":"2024-04-27T03:33:42.098611+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.098519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.098519+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.098519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:34:23.337627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"27'297","reported_seq":529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.877345+0000","last_change":"2024-04-27T03:33:35.877345+0000","last_active":"2024-04-27T03:33:35.877345+0000","last_peered":"2024-04-27T03:33:35.877345+0000","last_clean":"2024-04-27T03:33:35.877345+0000","last_became_active":"2024-04-27T03:30:17.390884+0000","last_became_peered":"2024-04-27T03:30:17.390884+0000","last_unstale":"2024-04-27T03:33:35.877345+0000","last_undegraded":"2024-04-27T03:33:35.877345+0000","last_fullsized":"2024-04-27T03:33:35.877345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'297","last_scrub_stamp":"2024-04-27T03:33:35.877210+0000","last_deep_scrub":"27'297","last_deep_scrub_stamp":"2024-04-27T03:33:35.877210+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.877210+0000","objects_scrubbed":11,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:23.956209+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":387,"num_read_kb":371,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:11.241230+0000","last_change":"2024-04-27T03:34:11.241230+0000","last_active":"2024-04-27T03:34:11.241230+0000","last_peered":"2024-04-27T03:34:11.241230+0000","last_clean":"2024-04-27T03:34:11.241230+0000","last_became_active":"2024-04-27T03:30:19.405777+0000","last_became_peered":"2024-04-27T03:30:19.405777+0000","last_unstale":"2024-04-27T03:34:11.241230+0000","last_undegraded":"2024-04-27T03:34:11.241230+0000","last_fullsized":"2024-04-27T03:34:11.241230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:11.241087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:11.241087+0000","last_clean_scrub_stamp":"2024-04-27T03:34:11.241087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:19.362648+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.973284+0000","last_change":"2024-04-27T03:33:52.973335+0000","last_active":"2024-04-27T03:33:52.973284+0000","last_peered":"2024-04-27T03:33:52.973284+0000","last_clean":"2024-04-27T03:33:52.973284+0000","last_became_active":"2024-04-27T03:30:21.409359+0000","last_became_peered":"2024-04-27T03:30:21.409359+0000","last_unstale":"2024-04-27T03:33:52.973284+0000","last_undegraded":"2024-04-27T03:33:52.973284+0000","last_fullsized":"2024-04-27T03:33:52.973284+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:33:52.973192+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:33:52.973192+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.973192+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:49.283366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"25'10","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.632263+0000","last_change":"2024-04-27T03:33:57.632293+0000","last_active":"2024-04-27T03:33:57.632263+0000","last_peered":"2024-04-27T03:33:57.632263+0000","last_clean":"2024-04-27T03:33:57.632263+0000","last_became_active":"2024-04-27T03:30:13.385521+0000","last_became_peered":"2024-04-27T03:30:13.385521+0000","last_unstale":"2024-04-27T03:33:57.632263+0000","last_undegraded":"2024-04-27T03:33:57.632263+0000","last_fullsized":"2024-04-27T03:33:57.632263+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'10","last_scrub_stamp":"2024-04-27T03:33:57.632133+0000","last_deep_scrub":"25'10","last_deep_scrub_stamp":"2024-04-27T03:33:57.632133+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.632133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:37.434045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:10.860062+0000","last_change":"2024-04-27T03:34:10.860062+0000","last_active":"2024-04-27T03:34:10.860062+0000","last_peered":"2024-04-27T03:34:10.860062+0000","last_clean":"2024-04-27T03:34:10.860062+0000","last_became_active":"2024-04-27T03:30:06.103441+0000","last_became_peered":"2024-04-27T03:30:06.103441+0000","last_unstale":"2024-04-27T03:34:10.860062+0000","last_undegraded":"2024-04-27T03:34:10.860062+0000","last_fullsized":"2024-04-27T03:34:10.860062+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:10.859905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:10.859905+0000","last_clean_scrub_stamp":"2024-04-27T03:34:10.859905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:58.011918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:21.646692+0000","last_change":"2024-04-27T03:34:21.646749+0000","last_active":"2024-04-27T03:34:21.646692+0000","last_peered":"2024-04-27T03:34:21.646692+0000","last_clean":"2024-04-27T03:34:21.646692+0000","last_became_active":"2024-04-27T03:30:11.360443+0000","last_became_peered":"2024-04-27T03:30:11.360443+0000","last_unstale":"2024-04-27T03:34:21.646692+0000","last_undegraded":"2024-04-27T03:34:21.646692+0000","last_fullsized":"2024-04-27T03:34:21.646692+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:21.646559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:21.646559+0000","last_clean_scrub_stamp":"2024-04-27T03:34:21.646559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:30:38.639056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.772880+0000","last_change":"2024-04-27T03:33:46.772935+0000","last_active":"2024-04-27T03:33:46.772880+0000","last_peered":"2024-04-27T03:33:46.772880+0000","last_clean":"2024-04-27T03:33:46.772880+0000","last_became_active":"2024-04-27T03:30:15.383213+0000","last_became_peered":"2024-04-27T03:30:15.383213+0000","last_unstale":"2024-04-27T03:33:46.772880+0000","last_undegraded":"2024-04-27T03:33:46.772880+0000","last_fullsized":"2024-04-27T03:33:46.772880+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:46.772765+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:46.772765+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.772765+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:24.897665+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"27'23","reported_seq":101,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.859533+0000","last_change":"2024-04-27T03:33:42.859533+0000","last_active":"2024-04-27T03:33:42.859533+0000","last_peered":"2024-04-27T03:33:42.859533+0000","last_clean":"2024-04-27T03:33:42.859533+0000","last_became_active":"2024-04-27T03:30:17.388588+0000","last_became_peered":"2024-04-27T03:30:17.388588+0000","last_unstale":"2024-04-27T03:33:42.859533+0000","last_undegraded":"2024-04-27T03:33:42.859533+0000","last_fullsized":"2024-04-27T03:33:42.859533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-04-27T03:33:42.859384+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-04-27T03:33:42.859384+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.859384+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:29.849938+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.729982+0000","last_change":"2024-04-27T03:33:46.729982+0000","last_active":"2024-04-27T03:33:46.729982+0000","last_peered":"2024-04-27T03:33:46.729982+0000","last_clean":"2024-04-27T03:33:46.729982+0000","last_became_active":"2024-04-27T03:30:19.407004+0000","last_became_peered":"2024-04-27T03:30:19.407004+0000","last_unstale":"2024-04-27T03:33:46.729982+0000","last_undegraded":"2024-04-27T03:33:46.729982+0000","last_fullsized":"2024-04-27T03:33:46.729982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:46.729851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:46.729851+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.729851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:36.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"25'2","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297632+0000","last_change":"2024-04-27T03:30:21.417022+0000","last_active":"2024-04-27T03:32:39.297632+0000","last_peered":"2024-04-27T03:32:39.297632+0000","last_clean":"2024-04-27T03:32:39.297632+0000","last_became_active":"2024-04-27T03:30:21.416389+0000","last_became_peered":"2024-04-27T03:30:21.416389+0000","last_unstale":"2024-04-27T03:32:39.297632+0000","last_undegraded":"2024-04-27T03:32:39.297632+0000","last_fullsized":"2024-04-27T03:32:39.297632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'8","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.675570+0000","last_change":"2024-04-27T03:34:00.675570+0000","last_active":"2024-04-27T03:34:00.675570+0000","last_peered":"2024-04-27T03:34:00.675570+0000","last_clean":"2024-04-27T03:34:00.675570+0000","last_became_active":"2024-04-27T03:30:13.369638+0000","last_became_peered":"2024-04-27T03:30:13.369638+0000","last_unstale":"2024-04-27T03:34:00.675570+0000","last_undegraded":"2024-04-27T03:34:00.675570+0000","last_fullsized":"2024-04-27T03:34:00.675570+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-04-27T03:34:00.675392+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-04-27T03:34:00.675392+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.675392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:47.827405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.755719+0000","last_change":"2024-04-27T03:30:06.107772+0000","last_active":"2024-04-27T03:33:31.755719+0000","last_peered":"2024-04-27T03:33:31.755719+0000","last_clean":"2024-04-27T03:33:31.755719+0000","last_became_active":"2024-04-27T03:30:06.107356+0000","last_became_peered":"2024-04-27T03:30:06.107356+0000","last_unstale":"2024-04-27T03:33:31.755719+0000","last_undegraded":"2024-04-27T03:33:31.755719+0000","last_fullsized":"2024-04-27T03:33:31.755719+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:11.852227+0000","last_change":"2024-04-27T03:34:11.852227+0000","last_active":"2024-04-27T03:34:11.852227+0000","last_peered":"2024-04-27T03:34:11.852227+0000","last_clean":"2024-04-27T03:34:11.852227+0000","last_became_active":"2024-04-27T03:30:11.356685+0000","last_became_peered":"2024-04-27T03:30:11.356685+0000","last_unstale":"2024-04-27T03:34:11.852227+0000","last_undegraded":"2024-04-27T03:34:11.852227+0000","last_fullsized":"2024-04-27T03:34:11.852227+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:11.852112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:11.852112+0000","last_clean_scrub_stamp":"2024-04-27T03:34:11.852112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:57.233248+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.706693+0000","last_change":"2024-04-27T03:33:49.706693+0000","last_active":"2024-04-27T03:33:49.706693+0000","last_peered":"2024-04-27T03:33:49.706693+0000","last_clean":"2024-04-27T03:33:49.706693+0000","last_became_active":"2024-04-27T03:30:15.382326+0000","last_became_peered":"2024-04-27T03:30:15.382326+0000","last_unstale":"2024-04-27T03:33:49.706693+0000","last_undegraded":"2024-04-27T03:33:49.706693+0000","last_fullsized":"2024-04-27T03:33:49.706693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:49.706574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:49.706574+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.706574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:45.223380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"27'20","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.220468+0000","last_change":"2024-04-27T03:34:01.220511+0000","last_active":"2024-04-27T03:34:01.220468+0000","last_peered":"2024-04-27T03:34:01.220468+0000","last_clean":"2024-04-27T03:34:01.220468+0000","last_became_active":"2024-04-27T03:30:17.389373+0000","last_became_peered":"2024-04-27T03:30:17.389373+0000","last_unstale":"2024-04-27T03:34:01.220468+0000","last_undegraded":"2024-04-27T03:34:01.220468+0000","last_fullsized":"2024-04-27T03:34:01.220468+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-04-27T03:34:01.220376+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-04-27T03:34:01.220376+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.220376+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:49.020148+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":350,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.297507+0000","last_change":"2024-04-27T03:34:07.297531+0000","last_active":"2024-04-27T03:34:07.297507+0000","last_peered":"2024-04-27T03:34:07.297507+0000","last_clean":"2024-04-27T03:34:07.297507+0000","last_became_active":"2024-04-27T03:30:19.405953+0000","last_became_peered":"2024-04-27T03:30:19.405953+0000","last_unstale":"2024-04-27T03:34:07.297507+0000","last_undegraded":"2024-04-27T03:34:07.297507+0000","last_fullsized":"2024-04-27T03:34:07.297507+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:07.297411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:07.297411+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.297411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:38.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"27'7","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:28.029515+0000","last_change":"2024-04-27T03:34:28.029576+0000","last_active":"2024-04-27T03:34:28.029515+0000","last_peered":"2024-04-27T03:34:28.029515+0000","last_clean":"2024-04-27T03:34:28.029515+0000","last_became_active":"2024-04-27T03:30:21.412790+0000","last_became_peered":"2024-04-27T03:30:21.412790+0000","last_unstale":"2024-04-27T03:34:28.029515+0000","last_undegraded":"2024-04-27T03:34:28.029515+0000","last_fullsized":"2024-04-27T03:34:28.029515+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:34:28.029410+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:34:28.029410+0000","last_clean_scrub_stamp":"2024-04-27T03:34:28.029410+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:10:15.508139+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'18","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.201621+0000","last_change":"2024-04-27T03:34:00.201621+0000","last_active":"2024-04-27T03:34:00.201621+0000","last_peered":"2024-04-27T03:34:00.201621+0000","last_clean":"2024-04-27T03:34:00.201621+0000","last_became_active":"2024-04-27T03:30:13.380888+0000","last_became_peered":"2024-04-27T03:30:13.380888+0000","last_unstale":"2024-04-27T03:34:00.201621+0000","last_undegraded":"2024-04-27T03:34:00.201621+0000","last_fullsized":"2024-04-27T03:34:00.201621+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-04-27T03:34:00.201503+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-04-27T03:34:00.201503+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.201503+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:13:15.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":73,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.878323+0000","last_change":"2024-04-27T03:34:09.878323+0000","last_active":"2024-04-27T03:34:09.878323+0000","last_peered":"2024-04-27T03:34:09.878323+0000","last_clean":"2024-04-27T03:34:09.878323+0000","last_became_active":"2024-04-27T03:30:11.361786+0000","last_became_peered":"2024-04-27T03:30:11.361786+0000","last_unstale":"2024-04-27T03:34:09.878323+0000","last_undegraded":"2024-04-27T03:34:09.878323+0000","last_fullsized":"2024-04-27T03:34:09.878323+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:09.878163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:09.878163+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.878163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:13:25.595476+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:17.617492+0000","last_change":"2024-04-27T03:34:17.617492+0000","last_active":"2024-04-27T03:34:17.617492+0000","last_peered":"2024-04-27T03:34:17.617492+0000","last_clean":"2024-04-27T03:34:17.617492+0000","last_became_active":"2024-04-27T03:30:06.107594+0000","last_became_peered":"2024-04-27T03:30:06.107594+0000","last_unstale":"2024-04-27T03:34:17.617492+0000","last_undegraded":"2024-04-27T03:34:17.617492+0000","last_fullsized":"2024-04-27T03:34:17.617492+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:17.617335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:17.617335+0000","last_clean_scrub_stamp":"2024-04-27T03:34:17.617335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:27.936494+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"27'2","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:53.688679+0000","last_change":"2024-04-27T03:33:53.688679+0000","last_active":"2024-04-27T03:33:53.688679+0000","last_peered":"2024-04-27T03:33:53.688679+0000","last_clean":"2024-04-27T03:33:53.688679+0000","last_became_active":"2024-04-27T03:30:15.381392+0000","last_became_peered":"2024-04-27T03:30:15.381392+0000","last_unstale":"2024-04-27T03:33:53.688679+0000","last_undegraded":"2024-04-27T03:33:53.688679+0000","last_fullsized":"2024-04-27T03:33:53.688679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:53.688558+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:53.688558+0000","last_clean_scrub_stamp":"2024-04-27T03:33:53.688558+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:08:53.046929+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"27'87","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.739991+0000","last_change":"2024-04-27T03:33:39.739991+0000","last_active":"2024-04-27T03:33:39.739991+0000","last_peered":"2024-04-27T03:33:39.739991+0000","last_clean":"2024-04-27T03:33:39.739991+0000","last_became_active":"2024-04-27T03:30:17.390409+0000","last_became_peered":"2024-04-27T03:30:17.390409+0000","last_unstale":"2024-04-27T03:33:39.739991+0000","last_undegraded":"2024-04-27T03:33:39.739991+0000","last_fullsized":"2024-04-27T03:33:39.739991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'87","last_scrub_stamp":"2024-04-27T03:33:39.739797+0000","last_deep_scrub":"27'87","last_deep_scrub_stamp":"2024-04-27T03:33:39.739797+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.739797+0000","objects_scrubbed":11,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:27:02.606163+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":89,"num_read_kb":75,"num_write":42,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":292,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.814624+0000","last_change":"2024-04-27T03:33:33.814624+0000","last_active":"2024-04-27T03:33:33.814624+0000","last_peered":"2024-04-27T03:33:33.814624+0000","last_clean":"2024-04-27T03:33:33.814624+0000","last_became_active":"2024-04-27T03:30:19.402977+0000","last_became_peered":"2024-04-27T03:30:19.402977+0000","last_unstale":"2024-04-27T03:33:33.814624+0000","last_undegraded":"2024-04-27T03:33:33.814624+0000","last_fullsized":"2024-04-27T03:33:33.814624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:33.814493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:33.814493+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.814493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:14:40.755714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"27'11","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:28.666118+0000","last_change":"2024-04-27T03:34:28.666146+0000","last_active":"2024-04-27T03:34:28.666118+0000","last_peered":"2024-04-27T03:34:28.666118+0000","last_clean":"2024-04-27T03:34:28.666118+0000","last_became_active":"2024-04-27T03:30:21.410511+0000","last_became_peered":"2024-04-27T03:30:21.410511+0000","last_unstale":"2024-04-27T03:34:28.666118+0000","last_undegraded":"2024-04-27T03:34:28.666118+0000","last_fullsized":"2024-04-27T03:34:28.666118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-04-27T03:34:28.665996+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-04-27T03:34:28.665996+0000","last_clean_scrub_stamp":"2024-04-27T03:34:28.665996+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:00.104554+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"27'4","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:13.162744+0000","last_change":"2024-04-27T03:34:13.162744+0000","last_active":"2024-04-27T03:34:13.162744+0000","last_peered":"2024-04-27T03:34:13.162744+0000","last_clean":"2024-04-27T03:34:13.162744+0000","last_became_active":"2024-04-27T03:30:13.379985+0000","last_became_peered":"2024-04-27T03:30:13.379985+0000","last_unstale":"2024-04-27T03:34:13.162744+0000","last_undegraded":"2024-04-27T03:34:13.162744+0000","last_fullsized":"2024-04-27T03:34:13.162744+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:13.162618+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:13.162618+0000","last_clean_scrub_stamp":"2024-04-27T03:34:13.162618+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:56.317292+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:13.657044+0000","last_change":"2024-04-27T03:34:13.657044+0000","last_active":"2024-04-27T03:34:13.657044+0000","last_peered":"2024-04-27T03:34:13.657044+0000","last_clean":"2024-04-27T03:34:13.657044+0000","last_became_active":"2024-04-27T03:30:11.360352+0000","last_became_peered":"2024-04-27T03:30:11.360352+0000","last_unstale":"2024-04-27T03:34:13.657044+0000","last_undegraded":"2024-04-27T03:34:13.657044+0000","last_fullsized":"2024-04-27T03:34:13.657044+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:13.656944+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:13.656944+0000","last_clean_scrub_stamp":"2024-04-27T03:34:13.656944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:25:16.457938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.755719+0000","last_change":"2024-04-27T03:30:06.106816+0000","last_active":"2024-04-27T03:33:31.755719+0000","last_peered":"2024-04-27T03:33:31.755719+0000","last_clean":"2024-04-27T03:33:31.755719+0000","last_became_active":"2024-04-27T03:30:06.106475+0000","last_became_peered":"2024-04-27T03:30:06.106475+0000","last_unstale":"2024-04-27T03:33:31.755719+0000","last_undegraded":"2024-04-27T03:33:31.755719+0000","last_fullsized":"2024-04-27T03:33:31.755719+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.089706+0000","last_change":"2024-04-27T03:33:41.089706+0000","last_active":"2024-04-27T03:33:41.089706+0000","last_peered":"2024-04-27T03:33:41.089706+0000","last_clean":"2024-04-27T03:33:41.089706+0000","last_became_active":"2024-04-27T03:30:15.380030+0000","last_became_peered":"2024-04-27T03:30:15.380030+0000","last_unstale":"2024-04-27T03:33:41.089706+0000","last_undegraded":"2024-04-27T03:33:41.089706+0000","last_fullsized":"2024-04-27T03:33:41.089706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:41.089578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:41.089578+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.089578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:36.584947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"27'28","reported_seq":76,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.880920+0000","last_change":"2024-04-27T03:33:34.880920+0000","last_active":"2024-04-27T03:33:34.880920+0000","last_peered":"2024-04-27T03:33:34.880920+0000","last_clean":"2024-04-27T03:33:34.880920+0000","last_became_active":"2024-04-27T03:30:17.393253+0000","last_became_peered":"2024-04-27T03:30:17.393253+0000","last_unstale":"2024-04-27T03:33:34.880920+0000","last_undegraded":"2024-04-27T03:33:34.880920+0000","last_fullsized":"2024-04-27T03:33:34.880920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-04-27T03:33:34.880792+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-04-27T03:33:34.880792+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.880792+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:08.547001+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"27'285","reported_seq":399,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.921440+0000","last_change":"2024-04-27T03:33:33.921440+0000","last_active":"2024-04-27T03:33:33.921440+0000","last_peered":"2024-04-27T03:33:33.921440+0000","last_clean":"2024-04-27T03:33:33.921440+0000","last_became_active":"2024-04-27T03:30:19.402319+0000","last_became_peered":"2024-04-27T03:30:19.402319+0000","last_unstale":"2024-04-27T03:33:33.921440+0000","last_undegraded":"2024-04-27T03:33:33.921440+0000","last_fullsized":"2024-04-27T03:33:33.921440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:33.921301+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:33.921301+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.921301+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:42.843568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:15.236481+0000","last_change":"2024-04-27T03:34:15.236528+0000","last_active":"2024-04-27T03:34:15.236481+0000","last_peered":"2024-04-27T03:34:15.236481+0000","last_clean":"2024-04-27T03:34:15.236481+0000","last_became_active":"2024-04-27T03:30:21.414109+0000","last_became_peered":"2024-04-27T03:30:21.414109+0000","last_unstale":"2024-04-27T03:34:15.236481+0000","last_undegraded":"2024-04-27T03:34:15.236481+0000","last_fullsized":"2024-04-27T03:34:15.236481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:15.236366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:15.236366+0000","last_clean_scrub_stamp":"2024-04-27T03:34:15.236366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:40.927977+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"25'2","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.192314+0000","last_change":"2024-04-27T03:33:46.192314+0000","last_active":"2024-04-27T03:33:46.192314+0000","last_peered":"2024-04-27T03:33:46.192314+0000","last_clean":"2024-04-27T03:33:46.192314+0000","last_became_active":"2024-04-27T03:30:13.387764+0000","last_became_peered":"2024-04-27T03:30:13.387764+0000","last_unstale":"2024-04-27T03:33:46.192314+0000","last_undegraded":"2024-04-27T03:33:46.192314+0000","last_fullsized":"2024-04-27T03:33:46.192314+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:33:46.192183+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:33:46.192183+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.192183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:21:54.976582+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.921434+0000","last_change":"2024-04-27T03:30:11.356555+0000","last_active":"2024-04-27T03:33:20.921434+0000","last_peered":"2024-04-27T03:33:20.921434+0000","last_clean":"2024-04-27T03:33:20.921434+0000","last_became_active":"2024-04-27T03:30:11.355704+0000","last_became_peered":"2024-04-27T03:30:11.355704+0000","last_unstale":"2024-04-27T03:33:20.921434+0000","last_undegraded":"2024-04-27T03:33:20.921434+0000","last_fullsized":"2024-04-27T03:33:20.921434+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:16.597669+0000","last_change":"2024-04-27T03:34:16.597669+0000","last_active":"2024-04-27T03:34:16.597669+0000","last_peered":"2024-04-27T03:34:16.597669+0000","last_clean":"2024-04-27T03:34:16.597669+0000","last_became_active":"2024-04-27T03:30:06.103759+0000","last_became_peered":"2024-04-27T03:30:06.103759+0000","last_unstale":"2024-04-27T03:34:16.597669+0000","last_undegraded":"2024-04-27T03:34:16.597669+0000","last_fullsized":"2024-04-27T03:34:16.597669+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:16.597501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:16.597501+0000","last_clean_scrub_stamp":"2024-04-27T03:34:16.597501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:54.604812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"27'5","reported_seq":239,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.180895+0000","last_change":"2024-04-27T03:33:32.180895+0000","last_active":"2024-04-27T03:33:32.180895+0000","last_peered":"2024-04-27T03:33:32.180895+0000","last_clean":"2024-04-27T03:33:32.180895+0000","last_became_active":"2024-04-27T03:30:15.384372+0000","last_became_peered":"2024-04-27T03:30:15.384372+0000","last_unstale":"2024-04-27T03:33:32.180895+0000","last_undegraded":"2024-04-27T03:33:32.180895+0000","last_fullsized":"2024-04-27T03:33:32.180895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:32.180778+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:32.180778+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.180778+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:01:22.337645+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'26","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.104813+0000","last_change":"2024-04-27T03:33:51.104813+0000","last_active":"2024-04-27T03:33:51.104813+0000","last_peered":"2024-04-27T03:33:51.104813+0000","last_clean":"2024-04-27T03:33:51.104813+0000","last_became_active":"2024-04-27T03:30:17.389154+0000","last_became_peered":"2024-04-27T03:30:17.389154+0000","last_unstale":"2024-04-27T03:33:51.104813+0000","last_undegraded":"2024-04-27T03:33:51.104813+0000","last_fullsized":"2024-04-27T03:33:51.104813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-04-27T03:33:51.104672+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-04-27T03:33:51.104672+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.104672+0000","objects_scrubbed":6,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:09.096082+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"27'285","reported_seq":411,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.780327+0000","last_change":"2024-04-27T03:33:33.780327+0000","last_active":"2024-04-27T03:33:33.780327+0000","last_peered":"2024-04-27T03:33:33.780327+0000","last_clean":"2024-04-27T03:33:33.780327+0000","last_became_active":"2024-04-27T03:30:19.404867+0000","last_became_peered":"2024-04-27T03:30:19.404867+0000","last_unstale":"2024-04-27T03:33:33.780327+0000","last_undegraded":"2024-04-27T03:33:33.780327+0000","last_fullsized":"2024-04-27T03:33:33.780327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:33.780200+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:33.780200+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.780200+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:42.702468+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"25'3","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:29.070874+0000","last_change":"2024-04-27T03:34:29.070899+0000","last_active":"2024-04-27T03:34:29.070874+0000","last_peered":"2024-04-27T03:34:29.070874+0000","last_clean":"2024-04-27T03:34:29.070874+0000","last_became_active":"2024-04-27T03:30:21.413264+0000","last_became_peered":"2024-04-27T03:30:21.413264+0000","last_unstale":"2024-04-27T03:34:29.070874+0000","last_undegraded":"2024-04-27T03:34:29.070874+0000","last_fullsized":"2024-04-27T03:34:29.070874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'3","last_scrub_stamp":"2024-04-27T03:34:29.070741+0000","last_deep_scrub":"25'3","last_deep_scrub_stamp":"2024-04-27T03:34:29.070741+0000","last_clean_scrub_stamp":"2024-04-27T03:34:29.070741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:05:54.556761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"27'4","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:18.852257+0000","last_change":"2024-04-27T03:34:18.852257+0000","last_active":"2024-04-27T03:34:18.852257+0000","last_peered":"2024-04-27T03:34:18.852257+0000","last_clean":"2024-04-27T03:34:18.852257+0000","last_became_active":"2024-04-27T03:30:13.383208+0000","last_became_peered":"2024-04-27T03:30:13.383208+0000","last_unstale":"2024-04-27T03:34:18.852257+0000","last_undegraded":"2024-04-27T03:34:18.852257+0000","last_fullsized":"2024-04-27T03:34:18.852257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:18.852125+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:18.852125+0000","last_clean_scrub_stamp":"2024-04-27T03:34:18.852125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:50.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:18.128058+0000","last_change":"2024-04-27T03:34:18.128080+0000","last_active":"2024-04-27T03:34:18.128058+0000","last_peered":"2024-04-27T03:34:18.128058+0000","last_clean":"2024-04-27T03:34:18.128058+0000","last_became_active":"2024-04-27T03:30:11.369568+0000","last_became_peered":"2024-04-27T03:30:11.369568+0000","last_unstale":"2024-04-27T03:34:18.128058+0000","last_undegraded":"2024-04-27T03:34:18.128058+0000","last_fullsized":"2024-04-27T03:34:18.128058+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:18.127955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:18.127955+0000","last_clean_scrub_stamp":"2024-04-27T03:34:18.127955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:40.539908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.755754+0000","last_change":"2024-04-27T03:30:06.107954+0000","last_active":"2024-04-27T03:33:31.755754+0000","last_peered":"2024-04-27T03:33:31.755754+0000","last_clean":"2024-04-27T03:33:31.755754+0000","last_became_active":"2024-04-27T03:30:06.107600+0000","last_became_peered":"2024-04-27T03:30:06.107600+0000","last_unstale":"2024-04-27T03:33:31.755754+0000","last_undegraded":"2024-04-27T03:33:31.755754+0000","last_fullsized":"2024-04-27T03:33:31.755754+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:05.096302+0000","last_clean_scrub_stamp":"2024-04-27T03:30:05.096302+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.124794+0000","last_change":"2024-04-27T03:33:40.124794+0000","last_active":"2024-04-27T03:33:40.124794+0000","last_peered":"2024-04-27T03:33:40.124794+0000","last_clean":"2024-04-27T03:33:40.124794+0000","last_became_active":"2024-04-27T03:30:15.380598+0000","last_became_peered":"2024-04-27T03:30:15.380598+0000","last_unstale":"2024-04-27T03:33:40.124794+0000","last_undegraded":"2024-04-27T03:33:40.124794+0000","last_fullsized":"2024-04-27T03:33:40.124794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:40.124684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:40.124684+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.124684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:12.385924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"27'104","reported_seq":204,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.834973+0000","last_change":"2024-04-27T03:33:39.834973+0000","last_active":"2024-04-27T03:33:39.834973+0000","last_peered":"2024-04-27T03:33:39.834973+0000","last_clean":"2024-04-27T03:33:39.834973+0000","last_became_active":"2024-04-27T03:30:17.391191+0000","last_became_peered":"2024-04-27T03:30:17.391191+0000","last_unstale":"2024-04-27T03:33:39.834973+0000","last_undegraded":"2024-04-27T03:33:39.834973+0000","last_fullsized":"2024-04-27T03:33:39.834973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'104","last_scrub_stamp":"2024-04-27T03:33:39.834804+0000","last_deep_scrub":"27'104","last_deep_scrub_stamp":"2024-04-27T03:33:39.834804+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.834804+0000","objects_scrubbed":9,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:46.431245+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":102,"num_read_kb":87,"num_write":40,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.744784+0000","last_change":"2024-04-27T03:33:36.744784+0000","last_active":"2024-04-27T03:33:36.744784+0000","last_peered":"2024-04-27T03:33:36.744784+0000","last_clean":"2024-04-27T03:33:36.744784+0000","last_became_active":"2024-04-27T03:30:19.404187+0000","last_became_peered":"2024-04-27T03:30:19.404187+0000","last_unstale":"2024-04-27T03:33:36.744784+0000","last_undegraded":"2024-04-27T03:33:36.744784+0000","last_fullsized":"2024-04-27T03:33:36.744784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.744626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.744626+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.744626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:14.869527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.769625+0000","last_change":"2024-04-27T03:33:48.769648+0000","last_active":"2024-04-27T03:33:48.769625+0000","last_peered":"2024-04-27T03:33:48.769625+0000","last_clean":"2024-04-27T03:33:48.769625+0000","last_became_active":"2024-04-27T03:30:21.407857+0000","last_became_peered":"2024-04-27T03:30:21.407857+0000","last_unstale":"2024-04-27T03:33:48.769625+0000","last_undegraded":"2024-04-27T03:33:48.769625+0000","last_fullsized":"2024-04-27T03:33:48.769625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.769538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.769538+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.769538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:21.643681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"25'2","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:20.938331+0000","last_change":"2024-04-27T03:34:20.938358+0000","last_active":"2024-04-27T03:34:20.938331+0000","last_peered":"2024-04-27T03:34:20.938331+0000","last_clean":"2024-04-27T03:34:20.938331+0000","last_became_active":"2024-04-27T03:30:21.411054+0000","last_became_peered":"2024-04-27T03:30:21.411054+0000","last_unstale":"2024-04-27T03:34:20.938331+0000","last_undegraded":"2024-04-27T03:34:20.938331+0000","last_fullsized":"2024-04-27T03:34:20.938331+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:20.938237+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:20.938237+0000","last_clean_scrub_stamp":"2024-04-27T03:34:20.938237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:13:39.173999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"27'6","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.813031+0000","last_change":"2024-04-27T03:34:06.813056+0000","last_active":"2024-04-27T03:34:06.813031+0000","last_peered":"2024-04-27T03:34:06.813031+0000","last_clean":"2024-04-27T03:34:06.813031+0000","last_became_active":"2024-04-27T03:30:13.382675+0000","last_became_peered":"2024-04-27T03:30:13.382675+0000","last_unstale":"2024-04-27T03:34:06.813031+0000","last_undegraded":"2024-04-27T03:34:06.813031+0000","last_fullsized":"2024-04-27T03:34:06.813031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:06.812933+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:06.812933+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.812933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:10.931563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:34.296814+0000","last_change":"2024-04-27T03:30:11.359531+0000","last_active":"2024-04-27T03:32:34.296814+0000","last_peered":"2024-04-27T03:32:34.296814+0000","last_clean":"2024-04-27T03:32:34.296814+0000","last_became_active":"2024-04-27T03:30:11.359195+0000","last_became_peered":"2024-04-27T03:30:11.359195+0000","last_unstale":"2024-04-27T03:32:34.296814+0000","last_undegraded":"2024-04-27T03:32:34.296814+0000","last_fullsized":"2024-04-27T03:32:34.296814+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.994229+0000","last_change":"2024-04-27T03:33:50.994275+0000","last_active":"2024-04-27T03:33:50.994229+0000","last_peered":"2024-04-27T03:33:50.994229+0000","last_clean":"2024-04-27T03:33:50.994229+0000","last_became_active":"2024-04-27T03:30:15.379541+0000","last_became_peered":"2024-04-27T03:30:15.379541+0000","last_unstale":"2024-04-27T03:33:50.994229+0000","last_undegraded":"2024-04-27T03:33:50.994229+0000","last_fullsized":"2024-04-27T03:33:50.994229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:50.994123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:50.994123+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.994123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:51.335268+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"27'46","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.177061+0000","last_change":"2024-04-27T03:33:37.177105+0000","last_active":"2024-04-27T03:33:37.177061+0000","last_peered":"2024-04-27T03:33:37.177061+0000","last_clean":"2024-04-27T03:33:37.177061+0000","last_became_active":"2024-04-27T03:30:17.386496+0000","last_became_peered":"2024-04-27T03:30:17.386496+0000","last_unstale":"2024-04-27T03:33:37.177061+0000","last_undegraded":"2024-04-27T03:33:37.177061+0000","last_fullsized":"2024-04-27T03:33:37.177061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:37.176970+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:37.176970+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.176970+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:15.021583+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:53.149637+0000","last_change":"2024-04-27T03:33:53.149659+0000","last_active":"2024-04-27T03:33:53.149637+0000","last_peered":"2024-04-27T03:33:53.149637+0000","last_clean":"2024-04-27T03:33:53.149637+0000","last_became_active":"2024-04-27T03:30:19.405189+0000","last_became_peered":"2024-04-27T03:30:19.405189+0000","last_unstale":"2024-04-27T03:33:53.149637+0000","last_undegraded":"2024-04-27T03:33:53.149637+0000","last_fullsized":"2024-04-27T03:33:53.149637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:53.149540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:53.149540+0000","last_clean_scrub_stamp":"2024-04-27T03:33:53.149540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:17:47.405355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"27'3","reported_seq":23,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:30.487447+0000","last_change":"2024-04-27T03:30:21.414203+0000","last_active":"2024-04-27T03:33:30.487447+0000","last_peered":"2024-04-27T03:33:30.487447+0000","last_clean":"2024-04-27T03:33:30.487447+0000","last_became_active":"2024-04-27T03:30:21.413920+0000","last_became_peered":"2024-04-27T03:30:21.413920+0000","last_unstale":"2024-04-27T03:33:30.487447+0000","last_undegraded":"2024-04-27T03:33:30.487447+0000","last_fullsized":"2024-04-27T03:33:30.487447+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"27'16","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:27.036989+0000","last_change":"2024-04-27T03:34:27.037018+0000","last_active":"2024-04-27T03:34:27.036989+0000","last_peered":"2024-04-27T03:34:27.036989+0000","last_clean":"2024-04-27T03:34:27.036989+0000","last_became_active":"2024-04-27T03:30:13.380240+0000","last_became_peered":"2024-04-27T03:30:13.380240+0000","last_unstale":"2024-04-27T03:34:27.036989+0000","last_undegraded":"2024-04-27T03:34:27.036989+0000","last_fullsized":"2024-04-27T03:34:27.036989+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:27.036880+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:27.036880+0000","last_clean_scrub_stamp":"2024-04-27T03:34:27.036880+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:55:53.303121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":31,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"25'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:16.846118+0000","last_change":"2024-04-27T03:34:16.846162+0000","last_active":"2024-04-27T03:34:16.846118+0000","last_peered":"2024-04-27T03:34:16.846118+0000","last_clean":"2024-04-27T03:34:16.846118+0000","last_became_active":"2024-04-27T03:30:11.360217+0000","last_became_peered":"2024-04-27T03:30:11.360217+0000","last_unstale":"2024-04-27T03:34:16.846118+0000","last_undegraded":"2024-04-27T03:34:16.846118+0000","last_fullsized":"2024-04-27T03:34:16.846118+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:16.846006+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:16.846006+0000","last_clean_scrub_stamp":"2024-04-27T03:34:16.846006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:54:35.473558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.125251+0000","last_change":"2024-04-27T03:33:36.125275+0000","last_active":"2024-04-27T03:33:36.125251+0000","last_peered":"2024-04-27T03:33:36.125251+0000","last_clean":"2024-04-27T03:33:36.125251+0000","last_became_active":"2024-04-27T03:30:15.381507+0000","last_became_peered":"2024-04-27T03:30:15.381507+0000","last_unstale":"2024-04-27T03:33:36.125251+0000","last_undegraded":"2024-04-27T03:33:36.125251+0000","last_fullsized":"2024-04-27T03:33:36.125251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.125165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.125165+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.125165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:43.471223+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"27'9","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.702947+0000","last_change":"2024-04-27T03:33:44.703001+0000","last_active":"2024-04-27T03:33:44.702947+0000","last_peered":"2024-04-27T03:33:44.702947+0000","last_clean":"2024-04-27T03:33:44.702947+0000","last_became_active":"2024-04-27T03:30:17.385318+0000","last_became_peered":"2024-04-27T03:30:17.385318+0000","last_unstale":"2024-04-27T03:33:44.702947+0000","last_undegraded":"2024-04-27T03:33:44.702947+0000","last_fullsized":"2024-04-27T03:33:44.702947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-04-27T03:33:44.702827+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-04-27T03:33:44.702827+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.702827+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:55.754975+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.873119+0000","last_change":"2024-04-27T03:33:36.873142+0000","last_active":"2024-04-27T03:33:36.873119+0000","last_peered":"2024-04-27T03:33:36.873119+0000","last_clean":"2024-04-27T03:33:36.873119+0000","last_became_active":"2024-04-27T03:30:19.406736+0000","last_became_peered":"2024-04-27T03:30:19.406736+0000","last_unstale":"2024-04-27T03:33:36.873119+0000","last_undegraded":"2024-04-27T03:33:36.873119+0000","last_fullsized":"2024-04-27T03:33:36.873119+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.873031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.873031+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.873031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:57.751682+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:26.716914+0000","last_change":"2024-04-27T03:34:26.716984+0000","last_active":"2024-04-27T03:34:26.716914+0000","last_peered":"2024-04-27T03:34:26.716914+0000","last_clean":"2024-04-27T03:34:26.716914+0000","last_became_active":"2024-04-27T03:30:21.408079+0000","last_became_peered":"2024-04-27T03:30:21.408079+0000","last_unstale":"2024-04-27T03:34:26.716914+0000","last_undegraded":"2024-04-27T03:34:26.716914+0000","last_fullsized":"2024-04-27T03:34:26.716914+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:26.716795+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:26.716795+0000","last_clean_scrub_stamp":"2024-04-27T03:34:26.716795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:21:21.432858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"27'4","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:12.698736+0000","last_change":"2024-04-27T03:34:12.698757+0000","last_active":"2024-04-27T03:34:12.698736+0000","last_peered":"2024-04-27T03:34:12.698736+0000","last_clean":"2024-04-27T03:34:12.698736+0000","last_became_active":"2024-04-27T03:30:13.375350+0000","last_became_peered":"2024-04-27T03:30:13.375350+0000","last_unstale":"2024-04-27T03:34:12.698736+0000","last_undegraded":"2024-04-27T03:34:12.698736+0000","last_fullsized":"2024-04-27T03:34:12.698736+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:12.698671+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:12.698671+0000","last_clean_scrub_stamp":"2024-04-27T03:34:12.698671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:30.690081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:17.813624+0000","last_change":"2024-04-27T03:34:17.813663+0000","last_active":"2024-04-27T03:34:17.813624+0000","last_peered":"2024-04-27T03:34:17.813624+0000","last_clean":"2024-04-27T03:34:17.813624+0000","last_became_active":"2024-04-27T03:30:11.360938+0000","last_became_peered":"2024-04-27T03:30:11.360938+0000","last_unstale":"2024-04-27T03:34:17.813624+0000","last_undegraded":"2024-04-27T03:34:17.813624+0000","last_fullsized":"2024-04-27T03:34:17.813624+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:17.813510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:17.813510+0000","last_clean_scrub_stamp":"2024-04-27T03:34:17.813510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:25.064062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.767320+0000","last_change":"2024-04-27T03:33:47.767359+0000","last_active":"2024-04-27T03:33:47.767320+0000","last_peered":"2024-04-27T03:33:47.767320+0000","last_clean":"2024-04-27T03:33:47.767320+0000","last_became_active":"2024-04-27T03:30:15.382287+0000","last_became_peered":"2024-04-27T03:30:15.382287+0000","last_unstale":"2024-04-27T03:33:47.767320+0000","last_undegraded":"2024-04-27T03:33:47.767320+0000","last_fullsized":"2024-04-27T03:33:47.767320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:47.767216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:47.767216+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.767216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:52.591536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"27'47","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.768186+0000","last_change":"2024-04-27T03:33:41.768233+0000","last_active":"2024-04-27T03:33:41.768186+0000","last_peered":"2024-04-27T03:33:41.768186+0000","last_clean":"2024-04-27T03:33:41.768186+0000","last_became_active":"2024-04-27T03:30:17.388260+0000","last_became_peered":"2024-04-27T03:30:17.388260+0000","last_unstale":"2024-04-27T03:33:41.768186+0000","last_undegraded":"2024-04-27T03:33:41.768186+0000","last_fullsized":"2024-04-27T03:33:41.768186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-04-27T03:33:41.768105+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-04-27T03:33:41.768105+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.768105+0000","objects_scrubbed":6,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:48.417823+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":662,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.927890+0000","last_change":"2024-04-27T03:33:31.927916+0000","last_active":"2024-04-27T03:33:31.927890+0000","last_peered":"2024-04-27T03:33:31.927890+0000","last_clean":"2024-04-27T03:33:31.927890+0000","last_became_active":"2024-04-27T03:30:19.402430+0000","last_became_peered":"2024-04-27T03:30:19.402430+0000","last_unstale":"2024-04-27T03:33:31.927890+0000","last_undegraded":"2024-04-27T03:33:31.927890+0000","last_fullsized":"2024-04-27T03:33:31.927890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:31.927798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:31.927798+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.927798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:36.369347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:54.300287+0000","last_change":"2024-04-27T03:30:21.416316+0000","last_active":"2024-04-27T03:32:54.300287+0000","last_peered":"2024-04-27T03:32:54.300287+0000","last_clean":"2024-04-27T03:32:54.300287+0000","last_became_active":"2024-04-27T03:30:21.415984+0000","last_became_peered":"2024-04-27T03:30:21.415984+0000","last_unstale":"2024-04-27T03:32:54.300287+0000","last_undegraded":"2024-04-27T03:32:54.300287+0000","last_fullsized":"2024-04-27T03:32:54.300287+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"27'5","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.144432+0000","last_change":"2024-04-27T03:33:50.144460+0000","last_active":"2024-04-27T03:33:50.144432+0000","last_peered":"2024-04-27T03:33:50.144432+0000","last_clean":"2024-04-27T03:33:50.144432+0000","last_became_active":"2024-04-27T03:30:13.386985+0000","last_became_peered":"2024-04-27T03:30:13.386985+0000","last_unstale":"2024-04-27T03:33:50.144432+0000","last_undegraded":"2024-04-27T03:33:50.144432+0000","last_fullsized":"2024-04-27T03:33:50.144432+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:50.144340+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:50.144340+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.144340+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:15.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:22.086952+0000","last_change":"2024-04-27T03:34:22.087002+0000","last_active":"2024-04-27T03:34:22.086952+0000","last_peered":"2024-04-27T03:34:22.086952+0000","last_clean":"2024-04-27T03:34:22.086952+0000","last_became_active":"2024-04-27T03:30:11.369824+0000","last_became_peered":"2024-04-27T03:30:11.369824+0000","last_unstale":"2024-04-27T03:34:22.086952+0000","last_undegraded":"2024-04-27T03:34:22.086952+0000","last_fullsized":"2024-04-27T03:34:22.086952+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:22.086864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:22.086864+0000","last_clean_scrub_stamp":"2024-04-27T03:34:22.086864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:09.886636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.154995+0000","last_change":"2024-04-27T03:33:34.155037+0000","last_active":"2024-04-27T03:33:34.154995+0000","last_peered":"2024-04-27T03:33:34.154995+0000","last_clean":"2024-04-27T03:33:34.154995+0000","last_became_active":"2024-04-27T03:30:15.383771+0000","last_became_peered":"2024-04-27T03:30:15.383771+0000","last_unstale":"2024-04-27T03:33:34.154995+0000","last_undegraded":"2024-04-27T03:33:34.154995+0000","last_fullsized":"2024-04-27T03:33:34.154995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.154879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.154879+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.154879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:59.127137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"27'36","reported_seq":91,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.840074+0000","last_change":"2024-04-27T03:33:43.840113+0000","last_active":"2024-04-27T03:33:43.840074+0000","last_peered":"2024-04-27T03:33:43.840074+0000","last_clean":"2024-04-27T03:33:43.840074+0000","last_became_active":"2024-04-27T03:30:17.385488+0000","last_became_peered":"2024-04-27T03:30:17.385488+0000","last_unstale":"2024-04-27T03:33:43.840074+0000","last_undegraded":"2024-04-27T03:33:43.840074+0000","last_fullsized":"2024-04-27T03:33:43.840074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-04-27T03:33:43.839983+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-04-27T03:33:43.839983+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.839983+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:52.762250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.844652+0000","last_change":"2024-04-27T03:33:35.844680+0000","last_active":"2024-04-27T03:33:35.844652+0000","last_peered":"2024-04-27T03:33:35.844652+0000","last_clean":"2024-04-27T03:33:35.844652+0000","last_became_active":"2024-04-27T03:30:19.398484+0000","last_became_peered":"2024-04-27T03:30:19.398484+0000","last_unstale":"2024-04-27T03:33:35.844652+0000","last_undegraded":"2024-04-27T03:33:35.844652+0000","last_fullsized":"2024-04-27T03:33:35.844652+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:35.844545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:35.844545+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.844545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:17.570301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"25'2","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:14.212832+0000","last_change":"2024-04-27T03:34:14.212832+0000","last_active":"2024-04-27T03:34:14.212832+0000","last_peered":"2024-04-27T03:34:14.212832+0000","last_clean":"2024-04-27T03:34:14.212832+0000","last_became_active":"2024-04-27T03:30:13.377217+0000","last_became_peered":"2024-04-27T03:30:13.377217+0000","last_unstale":"2024-04-27T03:34:14.212832+0000","last_undegraded":"2024-04-27T03:34:14.212832+0000","last_fullsized":"2024-04-27T03:34:14.212832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:14.212665+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:14.212665+0000","last_clean_scrub_stamp":"2024-04-27T03:34:14.212665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:40:43.816639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"25'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297519+0000","last_change":"2024-04-27T03:30:11.370820+0000","last_active":"2024-04-27T03:32:39.297519+0000","last_peered":"2024-04-27T03:32:39.297519+0000","last_clean":"2024-04-27T03:32:39.297519+0000","last_became_active":"2024-04-27T03:30:11.370547+0000","last_became_peered":"2024-04-27T03:30:11.370547+0000","last_unstale":"2024-04-27T03:32:39.297519+0000","last_undegraded":"2024-04-27T03:32:39.297519+0000","last_fullsized":"2024-04-27T03:32:39.297519+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.700581+0000","last_change":"2024-04-27T03:33:52.700581+0000","last_active":"2024-04-27T03:33:52.700581+0000","last_peered":"2024-04-27T03:33:52.700581+0000","last_clean":"2024-04-27T03:33:52.700581+0000","last_became_active":"2024-04-27T03:30:15.381956+0000","last_became_peered":"2024-04-27T03:30:15.381956+0000","last_unstale":"2024-04-27T03:33:52.700581+0000","last_undegraded":"2024-04-27T03:33:52.700581+0000","last_fullsized":"2024-04-27T03:33:52.700581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:52.700486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:52.700486+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.700486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:49.010661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"27'70","reported_seq":149,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.743749+0000","last_change":"2024-04-27T03:33:45.743793+0000","last_active":"2024-04-27T03:33:45.743749+0000","last_peered":"2024-04-27T03:33:45.743749+0000","last_clean":"2024-04-27T03:33:45.743749+0000","last_became_active":"2024-04-27T03:30:17.383178+0000","last_became_peered":"2024-04-27T03:30:17.383178+0000","last_unstale":"2024-04-27T03:33:45.743749+0000","last_undegraded":"2024-04-27T03:33:45.743749+0000","last_fullsized":"2024-04-27T03:33:45.743749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'70","last_scrub_stamp":"2024-04-27T03:33:45.743668+0000","last_deep_scrub":"27'70","last_deep_scrub_stamp":"2024-04-27T03:33:45.743668+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.743668+0000","objects_scrubbed":6,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:37.293903+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":28,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.812908+0000","last_change":"2024-04-27T03:33:34.812933+0000","last_active":"2024-04-27T03:33:34.812908+0000","last_peered":"2024-04-27T03:33:34.812908+0000","last_clean":"2024-04-27T03:33:34.812908+0000","last_became_active":"2024-04-27T03:30:19.402781+0000","last_became_peered":"2024-04-27T03:30:19.402781+0000","last_unstale":"2024-04-27T03:33:34.812908+0000","last_undegraded":"2024-04-27T03:33:34.812908+0000","last_fullsized":"2024-04-27T03:33:34.812908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.812814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.812814+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.812814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:12.666738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:25.726821+0000","last_change":"2024-04-27T03:34:25.726878+0000","last_active":"2024-04-27T03:34:25.726821+0000","last_peered":"2024-04-27T03:34:25.726821+0000","last_clean":"2024-04-27T03:34:25.726821+0000","last_became_active":"2024-04-27T03:30:21.408049+0000","last_became_peered":"2024-04-27T03:30:21.408049+0000","last_unstale":"2024-04-27T03:34:25.726821+0000","last_undegraded":"2024-04-27T03:34:25.726821+0000","last_fullsized":"2024-04-27T03:34:25.726821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:25.726721+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:25.726721+0000","last_clean_scrub_stamp":"2024-04-27T03:34:25.726721+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:44:14.582492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":21,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.693523+0000","last_change":"2024-04-27T03:34:08.693523+0000","last_active":"2024-04-27T03:34:08.693523+0000","last_peered":"2024-04-27T03:34:08.693523+0000","last_clean":"2024-04-27T03:34:08.693523+0000","last_became_active":"2024-04-27T03:30:13.373257+0000","last_became_peered":"2024-04-27T03:30:13.373257+0000","last_unstale":"2024-04-27T03:34:08.693523+0000","last_undegraded":"2024-04-27T03:34:08.693523+0000","last_fullsized":"2024-04-27T03:34:08.693523+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:08.693374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:08.693374+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.693374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:22.810525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.222541+0000","last_change":"2024-04-27T03:34:08.222565+0000","last_active":"2024-04-27T03:34:08.222541+0000","last_peered":"2024-04-27T03:34:08.222541+0000","last_clean":"2024-04-27T03:34:08.222541+0000","last_became_active":"2024-04-27T03:30:11.370091+0000","last_became_peered":"2024-04-27T03:30:11.370091+0000","last_unstale":"2024-04-27T03:34:08.222541+0000","last_undegraded":"2024-04-27T03:34:08.222541+0000","last_fullsized":"2024-04-27T03:34:08.222541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:08.222454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:08.222454+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.222454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:22:17.006853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.756557+0000","last_change":"2024-04-27T03:33:51.756557+0000","last_active":"2024-04-27T03:33:51.756557+0000","last_peered":"2024-04-27T03:33:51.756557+0000","last_clean":"2024-04-27T03:33:51.756557+0000","last_became_active":"2024-04-27T03:30:15.379800+0000","last_became_peered":"2024-04-27T03:30:15.379800+0000","last_unstale":"2024-04-27T03:33:51.756557+0000","last_undegraded":"2024-04-27T03:33:51.756557+0000","last_fullsized":"2024-04-27T03:33:51.756557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:51.756438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:51.756438+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.756438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:14:49.565231+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"27'46","reported_seq":104,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.217186+0000","last_change":"2024-04-27T03:34:02.217229+0000","last_active":"2024-04-27T03:34:02.217186+0000","last_peered":"2024-04-27T03:34:02.217186+0000","last_clean":"2024-04-27T03:34:02.217186+0000","last_became_active":"2024-04-27T03:30:17.388246+0000","last_became_peered":"2024-04-27T03:30:17.388246+0000","last_unstale":"2024-04-27T03:34:02.217186+0000","last_undegraded":"2024-04-27T03:34:02.217186+0000","last_fullsized":"2024-04-27T03:34:02.217186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:34:02.217099+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:34:02.217099+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.217099+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:17:44.534386+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.948322+0000","last_change":"2024-04-27T03:33:40.948348+0000","last_active":"2024-04-27T03:33:40.948322+0000","last_peered":"2024-04-27T03:33:40.948322+0000","last_clean":"2024-04-27T03:33:40.948322+0000","last_became_active":"2024-04-27T03:30:19.406707+0000","last_became_peered":"2024-04-27T03:30:19.406707+0000","last_unstale":"2024-04-27T03:33:40.948322+0000","last_undegraded":"2024-04-27T03:33:40.948322+0000","last_fullsized":"2024-04-27T03:33:40.948322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:40.948229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:40.948229+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.948229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:11.173852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:23.317810+0000","last_change":"2024-04-27T03:34:23.317834+0000","last_active":"2024-04-27T03:34:23.317810+0000","last_peered":"2024-04-27T03:34:23.317810+0000","last_clean":"2024-04-27T03:34:23.317810+0000","last_became_active":"2024-04-27T03:30:21.415441+0000","last_became_peered":"2024-04-27T03:30:21.415441+0000","last_unstale":"2024-04-27T03:34:23.317810+0000","last_undegraded":"2024-04-27T03:34:23.317810+0000","last_fullsized":"2024-04-27T03:34:23.317810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:23.317720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:23.317720+0000","last_clean_scrub_stamp":"2024-04-27T03:34:23.317720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:31.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.127154+0000","last_change":"2024-04-27T03:33:44.127210+0000","last_active":"2024-04-27T03:33:44.127154+0000","last_peered":"2024-04-27T03:33:44.127154+0000","last_clean":"2024-04-27T03:33:44.127154+0000","last_became_active":"2024-04-27T03:30:13.387397+0000","last_became_peered":"2024-04-27T03:30:13.387397+0000","last_unstale":"2024-04-27T03:33:44.127154+0000","last_undegraded":"2024-04-27T03:33:44.127154+0000","last_fullsized":"2024-04-27T03:33:44.127154+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:44.127050+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:44.127050+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.127050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:24.841845+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:16.127749+0000","last_change":"2024-04-27T03:34:16.127771+0000","last_active":"2024-04-27T03:34:16.127749+0000","last_peered":"2024-04-27T03:34:16.127749+0000","last_clean":"2024-04-27T03:34:16.127749+0000","last_became_active":"2024-04-27T03:30:11.369799+0000","last_became_peered":"2024-04-27T03:30:11.369799+0000","last_unstale":"2024-04-27T03:34:16.127749+0000","last_undegraded":"2024-04-27T03:34:16.127749+0000","last_fullsized":"2024-04-27T03:34:16.127749+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:16.127666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:16.127666+0000","last_clean_scrub_stamp":"2024-04-27T03:34:16.127666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:04.947407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:53.786343+0000","last_change":"2024-04-27T03:33:53.786369+0000","last_active":"2024-04-27T03:33:53.786343+0000","last_peered":"2024-04-27T03:33:53.786343+0000","last_clean":"2024-04-27T03:33:53.786343+0000","last_became_active":"2024-04-27T03:30:15.379873+0000","last_became_peered":"2024-04-27T03:30:15.379873+0000","last_unstale":"2024-04-27T03:33:53.786343+0000","last_undegraded":"2024-04-27T03:33:53.786343+0000","last_fullsized":"2024-04-27T03:33:53.786343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:53.786220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:53.786220+0000","last_clean_scrub_stamp":"2024-04-27T03:33:53.786220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:31.263421+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"27'48","reported_seq":111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.722022+0000","last_change":"2024-04-27T03:33:43.722022+0000","last_active":"2024-04-27T03:33:43.722022+0000","last_peered":"2024-04-27T03:33:43.722022+0000","last_clean":"2024-04-27T03:33:43.722022+0000","last_became_active":"2024-04-27T03:30:17.387205+0000","last_became_peered":"2024-04-27T03:30:17.387205+0000","last_unstale":"2024-04-27T03:33:43.722022+0000","last_undegraded":"2024-04-27T03:33:43.722022+0000","last_fullsized":"2024-04-27T03:33:43.722022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-04-27T03:33:43.721859+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-04-27T03:33:43.721859+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.721859+0000","objects_scrubbed":6,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:42.507737+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":63,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.822276+0000","last_change":"2024-04-27T03:33:34.822276+0000","last_active":"2024-04-27T03:33:34.822276+0000","last_peered":"2024-04-27T03:33:34.822276+0000","last_clean":"2024-04-27T03:33:34.822276+0000","last_became_active":"2024-04-27T03:30:19.405462+0000","last_became_peered":"2024-04-27T03:30:19.405462+0000","last_unstale":"2024-04-27T03:33:34.822276+0000","last_undegraded":"2024-04-27T03:33:34.822276+0000","last_fullsized":"2024-04-27T03:33:34.822276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.822124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.822124+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.822124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:04:28.428726+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"27'7","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298466+0000","last_change":"2024-04-27T03:30:21.416894+0000","last_active":"2024-04-27T03:32:44.298466+0000","last_peered":"2024-04-27T03:32:44.298466+0000","last_clean":"2024-04-27T03:32:44.298466+0000","last_became_active":"2024-04-27T03:30:21.416592+0000","last_became_peered":"2024-04-27T03:30:21.416592+0000","last_unstale":"2024-04-27T03:32:44.298466+0000","last_undegraded":"2024-04-27T03:32:44.298466+0000","last_fullsized":"2024-04-27T03:32:44.298466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"27'6","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.718193+0000","last_change":"2024-04-27T03:34:04.718246+0000","last_active":"2024-04-27T03:34:04.718193+0000","last_peered":"2024-04-27T03:34:04.718193+0000","last_clean":"2024-04-27T03:34:04.718193+0000","last_became_active":"2024-04-27T03:30:13.369643+0000","last_became_peered":"2024-04-27T03:30:13.369643+0000","last_unstale":"2024-04-27T03:34:04.718193+0000","last_undegraded":"2024-04-27T03:34:04.718193+0000","last_fullsized":"2024-04-27T03:34:04.718193+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:04.718068+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:04.718068+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.718068+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:13:49.122146+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2","reported_seq":40,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:27.688098+0000","last_change":"2024-04-27T03:30:11.372867+0000","last_active":"2024-04-27T03:33:27.688098+0000","last_peered":"2024-04-27T03:33:27.688098+0000","last_clean":"2024-04-27T03:33:27.688098+0000","last_became_active":"2024-04-27T03:30:11.372220+0000","last_became_peered":"2024-04-27T03:30:11.372220+0000","last_unstale":"2024-04-27T03:33:27.688098+0000","last_undegraded":"2024-04-27T03:33:27.688098+0000","last_fullsized":"2024-04-27T03:33:27.688098+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.972519+0000","last_change":"2024-04-27T03:33:49.972545+0000","last_active":"2024-04-27T03:33:49.972519+0000","last_peered":"2024-04-27T03:33:49.972519+0000","last_clean":"2024-04-27T03:33:49.972519+0000","last_became_active":"2024-04-27T03:30:15.380573+0000","last_became_peered":"2024-04-27T03:30:15.380573+0000","last_unstale":"2024-04-27T03:33:49.972519+0000","last_undegraded":"2024-04-27T03:33:49.972519+0000","last_fullsized":"2024-04-27T03:33:49.972519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:49.972421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:49.972421+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.972421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:13.206549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"27'22","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.170926+0000","last_change":"2024-04-27T03:33:47.170926+0000","last_active":"2024-04-27T03:33:47.170926+0000","last_peered":"2024-04-27T03:33:47.170926+0000","last_clean":"2024-04-27T03:33:47.170926+0000","last_became_active":"2024-04-27T03:30:17.388289+0000","last_became_peered":"2024-04-27T03:30:17.388289+0000","last_unstale":"2024-04-27T03:33:47.170926+0000","last_undegraded":"2024-04-27T03:33:47.170926+0000","last_fullsized":"2024-04-27T03:33:47.170926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-04-27T03:33:47.170743+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-04-27T03:33:47.170743+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.170743+0000","objects_scrubbed":6,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:01.287895+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"27'285","reported_seq":385,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.829103+0000","last_change":"2024-04-27T03:33:40.829103+0000","last_active":"2024-04-27T03:33:40.829103+0000","last_peered":"2024-04-27T03:33:40.829103+0000","last_clean":"2024-04-27T03:33:40.829103+0000","last_became_active":"2024-04-27T03:30:19.406854+0000","last_became_peered":"2024-04-27T03:30:19.406854+0000","last_unstale":"2024-04-27T03:33:40.829103+0000","last_undegraded":"2024-04-27T03:33:40.829103+0000","last_fullsized":"2024-04-27T03:33:40.829103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:40.828943+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:40.828943+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.828943+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:59.341953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298415+0000","last_change":"2024-04-27T03:30:21.415992+0000","last_active":"2024-04-27T03:32:44.298415+0000","last_peered":"2024-04-27T03:32:44.298415+0000","last_clean":"2024-04-27T03:32:44.298415+0000","last_became_active":"2024-04-27T03:30:21.415534+0000","last_became_peered":"2024-04-27T03:30:21.415534+0000","last_unstale":"2024-04-27T03:32:44.298415+0000","last_undegraded":"2024-04-27T03:32:44.298415+0000","last_fullsized":"2024-04-27T03:32:44.298415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"27'24","reported_seq":111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.819891+0000","last_change":"2024-04-27T03:33:57.819934+0000","last_active":"2024-04-27T03:33:57.819891+0000","last_peered":"2024-04-27T03:33:57.819891+0000","last_clean":"2024-04-27T03:33:57.819891+0000","last_became_active":"2024-04-27T03:30:13.371158+0000","last_became_peered":"2024-04-27T03:30:13.371158+0000","last_unstale":"2024-04-27T03:33:57.819891+0000","last_undegraded":"2024-04-27T03:33:57.819891+0000","last_fullsized":"2024-04-27T03:33:57.819891+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-04-27T03:33:57.819758+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-04-27T03:33:57.819758+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.819758+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:10:53.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":88,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:20.150149+0000","last_change":"2024-04-27T03:34:20.150172+0000","last_active":"2024-04-27T03:34:20.150149+0000","last_peered":"2024-04-27T03:34:20.150149+0000","last_clean":"2024-04-27T03:34:20.150149+0000","last_became_active":"2024-04-27T03:30:11.358893+0000","last_became_peered":"2024-04-27T03:30:11.358893+0000","last_unstale":"2024-04-27T03:34:20.150149+0000","last_undegraded":"2024-04-27T03:34:20.150149+0000","last_fullsized":"2024-04-27T03:34:20.150149+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:20.150063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:20.150063+0000","last_clean_scrub_stamp":"2024-04-27T03:34:20.150063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:19:31.126632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.016872+0000","last_change":"2024-04-27T03:33:48.016893+0000","last_active":"2024-04-27T03:33:48.016872+0000","last_peered":"2024-04-27T03:33:48.016872+0000","last_clean":"2024-04-27T03:33:48.016872+0000","last_became_active":"2024-04-27T03:30:15.381175+0000","last_became_peered":"2024-04-27T03:30:15.381175+0000","last_unstale":"2024-04-27T03:33:48.016872+0000","last_undegraded":"2024-04-27T03:33:48.016872+0000","last_fullsized":"2024-04-27T03:33:48.016872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.016786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.016786+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.016786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:39:37.651908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"27'73","reported_seq":155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.916431+0000","last_change":"2024-04-27T03:33:39.916480+0000","last_active":"2024-04-27T03:33:39.916431+0000","last_peered":"2024-04-27T03:33:39.916431+0000","last_clean":"2024-04-27T03:33:39.916431+0000","last_became_active":"2024-04-27T03:30:17.386673+0000","last_became_peered":"2024-04-27T03:30:17.386673+0000","last_unstale":"2024-04-27T03:33:39.916431+0000","last_undegraded":"2024-04-27T03:33:39.916431+0000","last_fullsized":"2024-04-27T03:33:39.916431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'73","last_scrub_stamp":"2024-04-27T03:33:39.916339+0000","last_deep_scrub":"27'73","last_deep_scrub_stamp":"2024-04-27T03:33:39.916339+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.916339+0000","objects_scrubbed":11,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:47.317240+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":80,"num_read_kb":67,"num_write":44,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.711840+0000","last_change":"2024-04-27T03:33:50.711878+0000","last_active":"2024-04-27T03:33:50.711840+0000","last_peered":"2024-04-27T03:33:50.711840+0000","last_clean":"2024-04-27T03:33:50.711840+0000","last_became_active":"2024-04-27T03:30:19.406990+0000","last_became_peered":"2024-04-27T03:30:19.406990+0000","last_unstale":"2024-04-27T03:33:50.711840+0000","last_undegraded":"2024-04-27T03:33:50.711840+0000","last_fullsized":"2024-04-27T03:33:50.711840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:50.711713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:50.711713+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.711713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:29.194718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.179312+0000","last_change":"2024-04-27T03:34:01.179339+0000","last_active":"2024-04-27T03:34:01.179312+0000","last_peered":"2024-04-27T03:34:01.179312+0000","last_clean":"2024-04-27T03:34:01.179312+0000","last_became_active":"2024-04-27T03:30:21.410060+0000","last_became_peered":"2024-04-27T03:30:21.410060+0000","last_unstale":"2024-04-27T03:34:01.179312+0000","last_undegraded":"2024-04-27T03:34:01.179312+0000","last_fullsized":"2024-04-27T03:34:01.179312+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:01.179224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:01.179224+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.179224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:48.331237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"27'10","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.805577+0000","last_change":"2024-04-27T03:33:54.805600+0000","last_active":"2024-04-27T03:33:54.805577+0000","last_peered":"2024-04-27T03:33:54.805577+0000","last_clean":"2024-04-27T03:33:54.805577+0000","last_became_active":"2024-04-27T03:30:13.371866+0000","last_became_peered":"2024-04-27T03:30:13.371866+0000","last_unstale":"2024-04-27T03:33:54.805577+0000","last_undegraded":"2024-04-27T03:33:54.805577+0000","last_fullsized":"2024-04-27T03:33:54.805577+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:33:54.805478+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:33:54.805478+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.805478+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:21:58.032925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"27'4","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:26.311334+0000","last_change":"2024-04-27T03:34:26.311360+0000","last_active":"2024-04-27T03:34:26.311334+0000","last_peered":"2024-04-27T03:34:26.311334+0000","last_clean":"2024-04-27T03:34:26.311334+0000","last_became_active":"2024-04-27T03:30:11.362420+0000","last_became_peered":"2024-04-27T03:30:11.362420+0000","last_unstale":"2024-04-27T03:34:26.311334+0000","last_undegraded":"2024-04-27T03:34:26.311334+0000","last_fullsized":"2024-04-27T03:34:26.311334+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:26.311245+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:26.311245+0000","last_clean_scrub_stamp":"2024-04-27T03:34:26.311245+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:05.338841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.769662+0000","last_change":"2024-04-27T03:33:50.769691+0000","last_active":"2024-04-27T03:33:50.769662+0000","last_peered":"2024-04-27T03:33:50.769662+0000","last_clean":"2024-04-27T03:33:50.769662+0000","last_became_active":"2024-04-27T03:30:15.382305+0000","last_became_peered":"2024-04-27T03:30:15.382305+0000","last_unstale":"2024-04-27T03:33:50.769662+0000","last_undegraded":"2024-04-27T03:33:50.769662+0000","last_fullsized":"2024-04-27T03:33:50.769662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:50.769565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:50.769565+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.769565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:30.717513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"27'46","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.785014+0000","last_change":"2024-04-27T03:33:40.785057+0000","last_active":"2024-04-27T03:33:40.785014+0000","last_peered":"2024-04-27T03:33:40.785014+0000","last_clean":"2024-04-27T03:33:40.785014+0000","last_became_active":"2024-04-27T03:30:17.390281+0000","last_became_peered":"2024-04-27T03:30:17.390281+0000","last_unstale":"2024-04-27T03:33:40.785014+0000","last_undegraded":"2024-04-27T03:33:40.785014+0000","last_fullsized":"2024-04-27T03:33:40.785014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:40.784918+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:40.784918+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.784918+0000","objects_scrubbed":5,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:11.010541+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":50,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.154146+0000","last_change":"2024-04-27T03:33:59.154184+0000","last_active":"2024-04-27T03:33:59.154146+0000","last_peered":"2024-04-27T03:33:59.154146+0000","last_clean":"2024-04-27T03:33:59.154146+0000","last_became_active":"2024-04-27T03:30:19.405353+0000","last_became_peered":"2024-04-27T03:30:19.405353+0000","last_unstale":"2024-04-27T03:33:59.154146+0000","last_undegraded":"2024-04-27T03:33:59.154146+0000","last_fullsized":"2024-04-27T03:33:59.154146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:59.154022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:59.154022+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.154022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:53:09.531303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:49.299534+0000","last_change":"2024-04-27T03:30:21.416876+0000","last_active":"2024-04-27T03:32:49.299534+0000","last_peered":"2024-04-27T03:32:49.299534+0000","last_clean":"2024-04-27T03:32:49.299534+0000","last_became_active":"2024-04-27T03:30:21.416276+0000","last_became_peered":"2024-04-27T03:30:21.416276+0000","last_unstale":"2024-04-27T03:32:49.299534+0000","last_undegraded":"2024-04-27T03:32:49.299534+0000","last_fullsized":"2024-04-27T03:32:49.299534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'24","reported_seq":110,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:13.170815+0000","last_change":"2024-04-27T03:34:13.170839+0000","last_active":"2024-04-27T03:34:13.170815+0000","last_peered":"2024-04-27T03:34:13.170815+0000","last_clean":"2024-04-27T03:34:13.170815+0000","last_became_active":"2024-04-27T03:30:13.376415+0000","last_became_peered":"2024-04-27T03:30:13.376415+0000","last_unstale":"2024-04-27T03:34:13.170815+0000","last_undegraded":"2024-04-27T03:34:13.170815+0000","last_fullsized":"2024-04-27T03:34:13.170815+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-04-27T03:34:13.170716+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-04-27T03:34:13.170716+0000","last_clean_scrub_stamp":"2024-04-27T03:34:13.170716+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:49:02.946260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:11.675077+0000","last_change":"2024-04-27T03:34:11.675077+0000","last_active":"2024-04-27T03:34:11.675077+0000","last_peered":"2024-04-27T03:34:11.675077+0000","last_clean":"2024-04-27T03:34:11.675077+0000","last_became_active":"2024-04-27T03:30:11.360590+0000","last_became_peered":"2024-04-27T03:30:11.360590+0000","last_unstale":"2024-04-27T03:34:11.675077+0000","last_undegraded":"2024-04-27T03:34:11.675077+0000","last_fullsized":"2024-04-27T03:34:11.675077+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:11.674894+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:11.674894+0000","last_clean_scrub_stamp":"2024-04-27T03:34:11.674894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:37.256713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.996324+0000","last_change":"2024-04-27T03:33:42.996361+0000","last_active":"2024-04-27T03:33:42.996324+0000","last_peered":"2024-04-27T03:33:42.996324+0000","last_clean":"2024-04-27T03:33:42.996324+0000","last_became_active":"2024-04-27T03:30:15.385614+0000","last_became_peered":"2024-04-27T03:30:15.385614+0000","last_unstale":"2024-04-27T03:33:42.996324+0000","last_undegraded":"2024-04-27T03:33:42.996324+0000","last_fullsized":"2024-04-27T03:33:42.996324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.996219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.996219+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.996219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:58.172124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"27'37","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.164208+0000","last_change":"2024-04-27T03:33:39.164242+0000","last_active":"2024-04-27T03:33:39.164208+0000","last_peered":"2024-04-27T03:33:39.164208+0000","last_clean":"2024-04-27T03:33:39.164208+0000","last_became_active":"2024-04-27T03:30:17.386983+0000","last_became_peered":"2024-04-27T03:30:17.386983+0000","last_unstale":"2024-04-27T03:33:39.164208+0000","last_undegraded":"2024-04-27T03:33:39.164208+0000","last_fullsized":"2024-04-27T03:33:39.164208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-04-27T03:33:39.164121+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-04-27T03:33:39.164121+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.164121+0000","objects_scrubbed":3,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:26.316134+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":33,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.113482+0000","last_change":"2024-04-27T03:33:54.113533+0000","last_active":"2024-04-27T03:33:54.113482+0000","last_peered":"2024-04-27T03:33:54.113482+0000","last_clean":"2024-04-27T03:33:54.113482+0000","last_became_active":"2024-04-27T03:30:19.402652+0000","last_became_peered":"2024-04-27T03:30:19.402652+0000","last_unstale":"2024-04-27T03:33:54.113482+0000","last_undegraded":"2024-04-27T03:33:54.113482+0000","last_fullsized":"2024-04-27T03:33:54.113482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:54.113385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:54.113385+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.113385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:04:42.933126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"27'6","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.937826+0000","last_change":"2024-04-27T03:30:21.420988+0000","last_active":"2024-04-27T03:33:20.937826+0000","last_peered":"2024-04-27T03:33:20.937826+0000","last_clean":"2024-04-27T03:33:20.937826+0000","last_became_active":"2024-04-27T03:30:21.420725+0000","last_became_peered":"2024-04-27T03:30:21.420725+0000","last_unstale":"2024-04-27T03:33:20.937826+0000","last_undegraded":"2024-04-27T03:33:20.937826+0000","last_fullsized":"2024-04-27T03:33:20.937826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.788128+0000","last_change":"2024-04-27T03:33:58.788151+0000","last_active":"2024-04-27T03:33:58.788128+0000","last_peered":"2024-04-27T03:33:58.788128+0000","last_clean":"2024-04-27T03:33:58.788128+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:33:58.788128+0000","last_undegraded":"2024-04-27T03:33:58.788128+0000","last_fullsized":"2024-04-27T03:33:58.788128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:58.788034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:58.788034+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.788034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:14:43.897282+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:25.076472+0000","last_change":"2024-04-27T03:34:25.076512+0000","last_active":"2024-04-27T03:34:25.076472+0000","last_peered":"2024-04-27T03:34:25.076472+0000","last_clean":"2024-04-27T03:34:25.076472+0000","last_became_active":"2024-04-27T03:30:11.369877+0000","last_became_peered":"2024-04-27T03:30:11.369877+0000","last_unstale":"2024-04-27T03:34:25.076472+0000","last_undegraded":"2024-04-27T03:34:25.076472+0000","last_fullsized":"2024-04-27T03:34:25.076472+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:25.076378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:25.076378+0000","last_clean_scrub_stamp":"2024-04-27T03:34:25.076378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:44:13.932150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"27'3","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.974605+0000","last_change":"2024-04-27T03:33:45.974632+0000","last_active":"2024-04-27T03:33:45.974605+0000","last_peered":"2024-04-27T03:33:45.974605+0000","last_clean":"2024-04-27T03:33:45.974605+0000","last_became_active":"2024-04-27T03:30:15.381574+0000","last_became_peered":"2024-04-27T03:30:15.381574+0000","last_unstale":"2024-04-27T03:33:45.974605+0000","last_undegraded":"2024-04-27T03:33:45.974605+0000","last_fullsized":"2024-04-27T03:33:45.974605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-27T03:33:45.974523+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-27T03:33:45.974523+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.974523+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:37.524758+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"27'53","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.159896+0000","last_change":"2024-04-27T03:33:57.159964+0000","last_active":"2024-04-27T03:33:57.159896+0000","last_peered":"2024-04-27T03:33:57.159896+0000","last_clean":"2024-04-27T03:33:57.159896+0000","last_became_active":"2024-04-27T03:30:17.388277+0000","last_became_peered":"2024-04-27T03:30:17.388277+0000","last_unstale":"2024-04-27T03:33:57.159896+0000","last_undegraded":"2024-04-27T03:33:57.159896+0000","last_fullsized":"2024-04-27T03:33:57.159896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'53","last_scrub_stamp":"2024-04-27T03:33:57.159788+0000","last_deep_scrub":"27'53","last_deep_scrub_stamp":"2024-04-27T03:33:57.159788+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.159788+0000","objects_scrubbed":7,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:03.430517+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":52,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1138,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.728511+0000","last_change":"2024-04-27T03:33:42.728536+0000","last_active":"2024-04-27T03:33:42.728511+0000","last_peered":"2024-04-27T03:33:42.728511+0000","last_clean":"2024-04-27T03:33:42.728511+0000","last_became_active":"2024-04-27T03:30:19.406565+0000","last_became_peered":"2024-04-27T03:30:19.406565+0000","last_unstale":"2024-04-27T03:33:42.728511+0000","last_undegraded":"2024-04-27T03:33:42.728511+0000","last_fullsized":"2024-04-27T03:33:42.728511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.728420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.728420+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.728420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:57.904325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"27'4","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:19.270206+0000","last_change":"2024-04-27T03:34:19.270233+0000","last_active":"2024-04-27T03:34:19.270206+0000","last_peered":"2024-04-27T03:34:19.270206+0000","last_clean":"2024-04-27T03:34:19.270206+0000","last_became_active":"2024-04-27T03:30:21.414510+0000","last_became_peered":"2024-04-27T03:30:21.414510+0000","last_unstale":"2024-04-27T03:34:19.270206+0000","last_undegraded":"2024-04-27T03:34:19.270206+0000","last_fullsized":"2024-04-27T03:34:19.270206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:19.270117+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:19.270117+0000","last_clean_scrub_stamp":"2024-04-27T03:34:19.270117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:43:26.209785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"25'8","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.006545+0000","last_change":"2024-04-27T03:33:54.006570+0000","last_active":"2024-04-27T03:33:54.006545+0000","last_peered":"2024-04-27T03:33:54.006545+0000","last_clean":"2024-04-27T03:33:54.006545+0000","last_became_active":"2024-04-27T03:30:13.387149+0000","last_became_peered":"2024-04-27T03:30:13.387149+0000","last_unstale":"2024-04-27T03:33:54.006545+0000","last_undegraded":"2024-04-27T03:33:54.006545+0000","last_fullsized":"2024-04-27T03:33:54.006545+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'8","last_scrub_stamp":"2024-04-27T03:33:54.006451+0000","last_deep_scrub":"25'8","last_deep_scrub_stamp":"2024-04-27T03:33:54.006451+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.006451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:20.542228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"25'6","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:12.836026+0000","last_change":"2024-04-27T03:34:12.836026+0000","last_active":"2024-04-27T03:34:12.836026+0000","last_peered":"2024-04-27T03:34:12.836026+0000","last_clean":"2024-04-27T03:34:12.836026+0000","last_became_active":"2024-04-27T03:30:11.357878+0000","last_became_peered":"2024-04-27T03:30:11.357878+0000","last_unstale":"2024-04-27T03:34:12.836026+0000","last_undegraded":"2024-04-27T03:34:12.836026+0000","last_fullsized":"2024-04-27T03:34:12.836026+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'6","last_scrub_stamp":"2024-04-27T03:34:12.835816+0000","last_deep_scrub":"25'6","last_deep_scrub_stamp":"2024-04-27T03:34:12.835816+0000","last_clean_scrub_stamp":"2024-04-27T03:34:12.835816+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:53.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"27'4","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.707627+0000","last_change":"2024-04-27T03:34:02.707653+0000","last_active":"2024-04-27T03:34:02.707627+0000","last_peered":"2024-04-27T03:34:02.707627+0000","last_clean":"2024-04-27T03:34:02.707627+0000","last_became_active":"2024-04-27T03:30:13.369681+0000","last_became_peered":"2024-04-27T03:30:13.369681+0000","last_unstale":"2024-04-27T03:34:02.707627+0000","last_undegraded":"2024-04-27T03:34:02.707627+0000","last_fullsized":"2024-04-27T03:34:02.707627+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:02.707538+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:02.707538+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.707538+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:58.202906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"25'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.696795+0000","last_change":"2024-04-27T03:34:01.696795+0000","last_active":"2024-04-27T03:34:01.696795+0000","last_peered":"2024-04-27T03:34:01.696795+0000","last_clean":"2024-04-27T03:34:01.696795+0000","last_became_active":"2024-04-27T03:30:11.373186+0000","last_became_peered":"2024-04-27T03:30:11.373186+0000","last_unstale":"2024-04-27T03:34:01.696795+0000","last_undegraded":"2024-04-27T03:34:01.696795+0000","last_fullsized":"2024-04-27T03:34:01.696795+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:01.696661+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:01.696661+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.696661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:33.957902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.249733+0000","last_change":"2024-04-27T03:34:08.249759+0000","last_active":"2024-04-27T03:34:08.249733+0000","last_peered":"2024-04-27T03:34:08.249733+0000","last_clean":"2024-04-27T03:34:08.249733+0000","last_became_active":"2024-04-27T03:30:13.382069+0000","last_became_peered":"2024-04-27T03:30:13.382069+0000","last_unstale":"2024-04-27T03:34:08.249733+0000","last_undegraded":"2024-04-27T03:34:08.249733+0000","last_fullsized":"2024-04-27T03:34:08.249733+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:08.249643+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:08.249643+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.249643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:17:08.768291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"27'3","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.756036+0000","last_change":"2024-04-27T03:30:11.350547+0000","last_active":"2024-04-27T03:33:31.756036+0000","last_peered":"2024-04-27T03:33:31.756036+0000","last_clean":"2024-04-27T03:33:31.756036+0000","last_became_active":"2024-04-27T03:30:11.350295+0000","last_became_peered":"2024-04-27T03:30:11.350295+0000","last_unstale":"2024-04-27T03:33:31.756036+0000","last_undegraded":"2024-04-27T03:33:31.756036+0000","last_fullsized":"2024-04-27T03:33:31.756036+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"27'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.676368+0000","last_change":"2024-04-27T03:34:06.676435+0000","last_active":"2024-04-27T03:34:06.676368+0000","last_peered":"2024-04-27T03:34:06.676368+0000","last_clean":"2024-04-27T03:34:06.676368+0000","last_became_active":"2024-04-27T03:30:13.370066+0000","last_became_peered":"2024-04-27T03:30:13.370066+0000","last_unstale":"2024-04-27T03:34:06.676368+0000","last_undegraded":"2024-04-27T03:34:06.676368+0000","last_fullsized":"2024-04-27T03:34:06.676368+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:06.676262+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:06.676262+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.676262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:06.684143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"27'4","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:22.693546+0000","last_change":"2024-04-27T03:34:22.693546+0000","last_active":"2024-04-27T03:34:22.693546+0000","last_peered":"2024-04-27T03:34:22.693546+0000","last_clean":"2024-04-27T03:34:22.693546+0000","last_became_active":"2024-04-27T03:30:11.360376+0000","last_became_peered":"2024-04-27T03:30:11.360376+0000","last_unstale":"2024-04-27T03:34:22.693546+0000","last_undegraded":"2024-04-27T03:34:22.693546+0000","last_fullsized":"2024-04-27T03:34:22.693546+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:22.693417+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:22.693417+0000","last_clean_scrub_stamp":"2024-04-27T03:34:22.693417+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:10.493189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"27'4","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:24.695215+0000","last_change":"2024-04-27T03:34:24.695265+0000","last_active":"2024-04-27T03:34:24.695215+0000","last_peered":"2024-04-27T03:34:24.695215+0000","last_clean":"2024-04-27T03:34:24.695215+0000","last_became_active":"2024-04-27T03:30:13.378368+0000","last_became_peered":"2024-04-27T03:30:13.378368+0000","last_unstale":"2024-04-27T03:34:24.695215+0000","last_undegraded":"2024-04-27T03:34:24.695215+0000","last_fullsized":"2024-04-27T03:34:24.695215+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:24.695112+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:24.695112+0000","last_clean_scrub_stamp":"2024-04-27T03:34:24.695112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:47.159176+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.669173+0000","last_change":"2024-04-27T03:34:05.669200+0000","last_active":"2024-04-27T03:34:05.669173+0000","last_peered":"2024-04-27T03:34:05.669173+0000","last_clean":"2024-04-27T03:34:05.669173+0000","last_became_active":"2024-04-27T03:30:11.371553+0000","last_became_peered":"2024-04-27T03:30:11.371553+0000","last_unstale":"2024-04-27T03:34:05.669173+0000","last_undegraded":"2024-04-27T03:34:05.669173+0000","last_fullsized":"2024-04-27T03:34:05.669173+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:05.669082+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:05.669082+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.669082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:46.383877+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"27'10","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:30.911453+0000","last_change":"2024-04-27T03:31:01.440910+0000","last_active":"2024-04-27T03:33:30.911453+0000","last_peered":"2024-04-27T03:33:30.911453+0000","last_clean":"2024-04-27T03:33:30.911453+0000","last_became_active":"2024-04-27T03:31:01.440576+0000","last_became_peered":"2024-04-27T03:31:01.440576+0000","last_unstale":"2024-04-27T03:33:30.911453+0000","last_undegraded":"2024-04-27T03:33:30.911453+0000","last_fullsized":"2024-04-27T03:33:30.911453+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:12.346115+0000","last_clean_scrub_stamp":"2024-04-27T03:30:12.346115+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:12.120741+0000","last_change":"2024-04-27T03:34:12.120768+0000","last_active":"2024-04-27T03:34:12.120741+0000","last_peered":"2024-04-27T03:34:12.120741+0000","last_clean":"2024-04-27T03:34:12.120741+0000","last_became_active":"2024-04-27T03:30:11.369517+0000","last_became_peered":"2024-04-27T03:30:11.369517+0000","last_unstale":"2024-04-27T03:34:12.120741+0000","last_undegraded":"2024-04-27T03:34:12.120741+0000","last_fullsized":"2024-04-27T03:34:12.120741+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:12.120650+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:12.120650+0000","last_clean_scrub_stamp":"2024-04-27T03:34:12.120650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:30.112060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"27'4","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:10.688469+0000","last_change":"2024-04-27T03:34:10.688469+0000","last_active":"2024-04-27T03:34:10.688469+0000","last_peered":"2024-04-27T03:34:10.688469+0000","last_clean":"2024-04-27T03:34:10.688469+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:34:10.688469+0000","last_undegraded":"2024-04-27T03:34:10.688469+0000","last_fullsized":"2024-04-27T03:34:10.688469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:10.688326+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:10.688326+0000","last_clean_scrub_stamp":"2024-04-27T03:34:10.688326+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:44.126012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:22.880972+0000","last_change":"2024-04-27T03:34:22.881026+0000","last_active":"2024-04-27T03:34:22.880972+0000","last_peered":"2024-04-27T03:34:22.880972+0000","last_clean":"2024-04-27T03:34:22.880972+0000","last_became_active":"2024-04-27T03:31:01.441159+0000","last_became_peered":"2024-04-27T03:31:01.441159+0000","last_unstale":"2024-04-27T03:34:22.880972+0000","last_undegraded":"2024-04-27T03:34:22.880972+0000","last_fullsized":"2024-04-27T03:34:22.880972+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:22.880861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:22.880861+0000","last_clean_scrub_stamp":"2024-04-27T03:34:22.880861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:40.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"27'4","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.181941+0000","last_change":"2024-04-27T03:34:07.181941+0000","last_active":"2024-04-27T03:34:07.181941+0000","last_peered":"2024-04-27T03:34:07.181941+0000","last_clean":"2024-04-27T03:34:07.181941+0000","last_became_active":"2024-04-27T03:30:13.377821+0000","last_became_peered":"2024-04-27T03:30:13.377821+0000","last_unstale":"2024-04-27T03:34:07.181941+0000","last_undegraded":"2024-04-27T03:34:07.181941+0000","last_fullsized":"2024-04-27T03:34:07.181941+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:07.181834+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:07.181834+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.181834+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:07.189714+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.697093+0000","last_change":"2024-04-27T03:34:03.697117+0000","last_active":"2024-04-27T03:34:03.697093+0000","last_peered":"2024-04-27T03:34:03.697093+0000","last_clean":"2024-04-27T03:34:03.697093+0000","last_became_active":"2024-04-27T03:30:11.371604+0000","last_became_peered":"2024-04-27T03:30:11.371604+0000","last_unstale":"2024-04-27T03:34:03.697093+0000","last_undegraded":"2024-04-27T03:34:03.697093+0000","last_fullsized":"2024-04-27T03:34:03.697093+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:03.697000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:03.697000+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.697000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:34:44.936107+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"27'8","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.956162+0000","last_change":"2024-04-27T03:33:51.956191+0000","last_active":"2024-04-27T03:33:51.956162+0000","last_peered":"2024-04-27T03:33:51.956162+0000","last_clean":"2024-04-27T03:33:51.956162+0000","last_became_active":"2024-04-27T03:30:13.388421+0000","last_became_peered":"2024-04-27T03:30:13.388421+0000","last_unstale":"2024-04-27T03:33:51.956162+0000","last_undegraded":"2024-04-27T03:33:51.956162+0000","last_fullsized":"2024-04-27T03:33:51.956162+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-04-27T03:33:51.956075+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-04-27T03:33:51.956075+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.956075+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:21:03.313222+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.680852+0000","last_change":"2024-04-27T03:30:11.355453+0000","last_active":"2024-04-27T03:33:20.680852+0000","last_peered":"2024-04-27T03:33:20.680852+0000","last_clean":"2024-04-27T03:33:20.680852+0000","last_became_active":"2024-04-27T03:30:11.355183+0000","last_became_peered":"2024-04-27T03:30:11.355183+0000","last_unstale":"2024-04-27T03:33:20.680852+0000","last_undegraded":"2024-04-27T03:33:20.680852+0000","last_fullsized":"2024-04-27T03:33:20.680852+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:17.237435+0000","last_change":"2024-04-27T03:34:17.237459+0000","last_active":"2024-04-27T03:34:17.237435+0000","last_peered":"2024-04-27T03:34:17.237435+0000","last_clean":"2024-04-27T03:34:17.237435+0000","last_became_active":"2024-04-27T03:30:13.376875+0000","last_became_peered":"2024-04-27T03:30:13.376875+0000","last_unstale":"2024-04-27T03:34:17.237435+0000","last_undegraded":"2024-04-27T03:34:17.237435+0000","last_fullsized":"2024-04-27T03:34:17.237435+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:17.237351+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:17.237351+0000","last_clean_scrub_stamp":"2024-04-27T03:34:17.237351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:05.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"25'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.648387+0000","last_change":"2024-04-27T03:34:07.648432+0000","last_active":"2024-04-27T03:34:07.648387+0000","last_peered":"2024-04-27T03:34:07.648387+0000","last_clean":"2024-04-27T03:34:07.648387+0000","last_became_active":"2024-04-27T03:30:11.360299+0000","last_became_peered":"2024-04-27T03:30:11.360299+0000","last_unstale":"2024-04-27T03:34:07.648387+0000","last_undegraded":"2024-04-27T03:34:07.648387+0000","last_fullsized":"2024-04-27T03:34:07.648387+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:07.648243+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:07.648243+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.648243+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:22:16.432642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"27'10","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:14.663971+0000","last_change":"2024-04-27T03:34:14.664018+0000","last_active":"2024-04-27T03:34:14.663971+0000","last_peered":"2024-04-27T03:34:14.663971+0000","last_clean":"2024-04-27T03:34:14.663971+0000","last_became_active":"2024-04-27T03:30:13.376087+0000","last_became_peered":"2024-04-27T03:30:13.376087+0000","last_unstale":"2024-04-27T03:34:14.663971+0000","last_undegraded":"2024-04-27T03:34:14.663971+0000","last_fullsized":"2024-04-27T03:34:14.663971+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:34:14.663832+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:34:14.663832+0000","last_clean_scrub_stamp":"2024-04-27T03:34:14.663832+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:08.919647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"25'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:10.162497+0000","last_change":"2024-04-27T03:34:10.162522+0000","last_active":"2024-04-27T03:34:10.162497+0000","last_peered":"2024-04-27T03:34:10.162497+0000","last_clean":"2024-04-27T03:34:10.162497+0000","last_became_active":"2024-04-27T03:30:11.370837+0000","last_became_peered":"2024-04-27T03:30:11.370837+0000","last_unstale":"2024-04-27T03:34:10.162497+0000","last_undegraded":"2024-04-27T03:34:10.162497+0000","last_fullsized":"2024-04-27T03:34:10.162497+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:10.162390+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:10.162390+0000","last_clean_scrub_stamp":"2024-04-27T03:34:10.162390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:13:28.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"27'6","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.179298+0000","last_change":"2024-04-27T03:34:05.179323+0000","last_active":"2024-04-27T03:34:05.179298+0000","last_peered":"2024-04-27T03:34:05.179298+0000","last_clean":"2024-04-27T03:34:05.179298+0000","last_became_active":"2024-04-27T03:30:13.373511+0000","last_became_peered":"2024-04-27T03:30:13.373511+0000","last_unstale":"2024-04-27T03:34:05.179298+0000","last_undegraded":"2024-04-27T03:34:05.179298+0000","last_fullsized":"2024-04-27T03:34:05.179298+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:05.179213+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:05.179213+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.179213+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:49.583854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:14.167253+0000","last_change":"2024-04-27T03:34:14.167281+0000","last_active":"2024-04-27T03:34:14.167253+0000","last_peered":"2024-04-27T03:34:14.167253+0000","last_clean":"2024-04-27T03:34:14.167253+0000","last_became_active":"2024-04-27T03:30:11.370283+0000","last_became_peered":"2024-04-27T03:30:11.370283+0000","last_unstale":"2024-04-27T03:34:14.167253+0000","last_undegraded":"2024-04-27T03:34:14.167253+0000","last_fullsized":"2024-04-27T03:34:14.167253+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:14.167157+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:14.167157+0000","last_clean_scrub_stamp":"2024-04-27T03:34:14.167157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:24.417880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"27'12","reported_seq":69,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:17.133759+0000","last_change":"2024-04-27T03:34:17.133790+0000","last_active":"2024-04-27T03:34:17.133759+0000","last_peered":"2024-04-27T03:34:17.133759+0000","last_clean":"2024-04-27T03:34:17.133759+0000","last_became_active":"2024-04-27T03:30:13.380404+0000","last_became_peered":"2024-04-27T03:30:13.380404+0000","last_unstale":"2024-04-27T03:34:17.133759+0000","last_undegraded":"2024-04-27T03:34:17.133759+0000","last_fullsized":"2024-04-27T03:34:17.133759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-04-27T03:34:17.133674+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-04-27T03:34:17.133674+0000","last_clean_scrub_stamp":"2024-04-27T03:34:17.133674+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:57:39.000345+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:24.348373+0000","last_change":"2024-04-27T03:34:24.348395+0000","last_active":"2024-04-27T03:34:24.348373+0000","last_peered":"2024-04-27T03:34:24.348373+0000","last_clean":"2024-04-27T03:34:24.348373+0000","last_became_active":"2024-04-27T03:30:11.373026+0000","last_became_peered":"2024-04-27T03:30:11.373026+0000","last_unstale":"2024-04-27T03:34:24.348373+0000","last_undegraded":"2024-04-27T03:34:24.348373+0000","last_fullsized":"2024-04-27T03:34:24.348373+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:24.348277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:24.348277+0000","last_clean_scrub_stamp":"2024-04-27T03:34:24.348277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:23.175329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"27'2","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.031529+0000","last_change":"2024-04-27T03:33:55.031559+0000","last_active":"2024-04-27T03:33:55.031529+0000","last_peered":"2024-04-27T03:33:55.031529+0000","last_clean":"2024-04-27T03:33:55.031529+0000","last_became_active":"2024-04-27T03:30:13.385791+0000","last_became_peered":"2024-04-27T03:30:13.385791+0000","last_unstale":"2024-04-27T03:33:55.031529+0000","last_undegraded":"2024-04-27T03:33:55.031529+0000","last_fullsized":"2024-04-27T03:33:55.031529+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:55.031445+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:55.031445+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.031445+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:01.039513+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:29.295932+0000","last_change":"2024-04-27T03:30:11.358239+0000","last_active":"2024-04-27T03:32:29.295932+0000","last_peered":"2024-04-27T03:32:29.295932+0000","last_clean":"2024-04-27T03:32:29.295932+0000","last_became_active":"2024-04-27T03:30:11.357898+0000","last_became_peered":"2024-04-27T03:30:11.357898+0000","last_unstale":"2024-04-27T03:32:29.295932+0000","last_undegraded":"2024-04-27T03:32:29.295932+0000","last_fullsized":"2024-04-27T03:32:29.295932+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"27'6","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:24.943886+0000","last_change":"2024-04-27T03:34:24.943920+0000","last_active":"2024-04-27T03:34:24.943886+0000","last_peered":"2024-04-27T03:34:24.943886+0000","last_clean":"2024-04-27T03:34:24.943886+0000","last_became_active":"2024-04-27T03:31:01.440820+0000","last_became_peered":"2024-04-27T03:31:01.440820+0000","last_unstale":"2024-04-27T03:34:24.943886+0000","last_undegraded":"2024-04-27T03:34:24.943886+0000","last_fullsized":"2024-04-27T03:34:24.943886+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:24.943789+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:24.943789+0000","last_clean_scrub_stamp":"2024-04-27T03:34:24.943789+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:19.199603+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.846496+0000","last_change":"2024-04-27T03:34:05.846517+0000","last_active":"2024-04-27T03:34:05.846496+0000","last_peered":"2024-04-27T03:34:05.846496+0000","last_clean":"2024-04-27T03:34:05.846496+0000","last_became_active":"2024-04-27T03:30:11.362788+0000","last_became_peered":"2024-04-27T03:30:11.362788+0000","last_unstale":"2024-04-27T03:34:05.846496+0000","last_undegraded":"2024-04-27T03:34:05.846496+0000","last_fullsized":"2024-04-27T03:34:05.846496+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:05.846403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:05.846403+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.846403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:30.818661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"25'2","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.209177+0000","last_change":"2024-04-27T03:34:09.209205+0000","last_active":"2024-04-27T03:34:09.209177+0000","last_peered":"2024-04-27T03:34:09.209177+0000","last_clean":"2024-04-27T03:34:09.209177+0000","last_became_active":"2024-04-27T03:30:13.377845+0000","last_became_peered":"2024-04-27T03:30:13.377845+0000","last_unstale":"2024-04-27T03:34:09.209177+0000","last_undegraded":"2024-04-27T03:34:09.209177+0000","last_fullsized":"2024-04-27T03:34:09.209177+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:09.209073+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:09.209073+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.209073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:40.804195+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:25.348774+0000","last_change":"2024-04-27T03:34:25.348820+0000","last_active":"2024-04-27T03:34:25.348774+0000","last_peered":"2024-04-27T03:34:25.348774+0000","last_clean":"2024-04-27T03:34:25.348774+0000","last_became_active":"2024-04-27T03:30:11.362559+0000","last_became_peered":"2024-04-27T03:30:11.362559+0000","last_unstale":"2024-04-27T03:34:25.348774+0000","last_undegraded":"2024-04-27T03:34:25.348774+0000","last_fullsized":"2024-04-27T03:34:25.348774+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:25.348662+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:25.348662+0000","last_clean_scrub_stamp":"2024-04-27T03:34:25.348662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:25.577657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:29.307800+0000","last_change":"2024-04-27T03:34:29.307826+0000","last_active":"2024-04-27T03:34:29.307800+0000","last_peered":"2024-04-27T03:34:29.307800+0000","last_clean":"2024-04-27T03:34:29.307800+0000","last_became_active":"2024-04-27T03:30:11.363340+0000","last_became_peered":"2024-04-27T03:30:11.363340+0000","last_unstale":"2024-04-27T03:34:29.307800+0000","last_undegraded":"2024-04-27T03:34:29.307800+0000","last_fullsized":"2024-04-27T03:34:29.307800+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:29.307702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:29.307702+0000","last_clean_scrub_stamp":"2024-04-27T03:34:29.307702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:45.266144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"27'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.668243+0000","last_change":"2024-04-27T03:33:56.668296+0000","last_active":"2024-04-27T03:33:56.668243+0000","last_peered":"2024-04-27T03:33:56.668243+0000","last_clean":"2024-04-27T03:33:56.668243+0000","last_became_active":"2024-04-27T03:30:13.385705+0000","last_became_peered":"2024-04-27T03:30:13.385705+0000","last_unstale":"2024-04-27T03:33:56.668243+0000","last_undegraded":"2024-04-27T03:33:56.668243+0000","last_fullsized":"2024-04-27T03:33:56.668243+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:56.668140+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:56.668140+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.668140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:00.786770+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"25'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:23.731875+0000","last_change":"2024-04-27T03:34:23.731902+0000","last_active":"2024-04-27T03:34:23.731875+0000","last_peered":"2024-04-27T03:34:23.731875+0000","last_clean":"2024-04-27T03:34:23.731875+0000","last_became_active":"2024-04-27T03:30:11.360667+0000","last_became_peered":"2024-04-27T03:30:11.360667+0000","last_unstale":"2024-04-27T03:34:23.731875+0000","last_undegraded":"2024-04-27T03:34:23.731875+0000","last_fullsized":"2024-04-27T03:34:23.731875+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:23.731763+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:23.731763+0000","last_clean_scrub_stamp":"2024-04-27T03:34:23.731763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:18:06.049050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"27'14","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:15.127191+0000","last_change":"2024-04-27T03:34:15.127239+0000","last_active":"2024-04-27T03:34:15.127191+0000","last_peered":"2024-04-27T03:34:15.127191+0000","last_clean":"2024-04-27T03:34:15.127191+0000","last_became_active":"2024-04-27T03:30:13.380347+0000","last_became_peered":"2024-04-27T03:30:13.380347+0000","last_unstale":"2024-04-27T03:34:15.127191+0000","last_undegraded":"2024-04-27T03:34:15.127191+0000","last_fullsized":"2024-04-27T03:34:15.127191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:34:15.127095+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:34:15.127095+0000","last_clean_scrub_stamp":"2024-04-27T03:34:15.127095+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:03.309204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":270,"ondisk_log_size":270,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2281,"ondisk_log_size":2281,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":1888,"num_read_kb":1653,"num_write":789,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2692,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1751,"ondisk_log_size":1751,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":482,"ondisk_log_size":482,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":82,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705719,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39904,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331936,"statfs":{"total":96636764160,"available":96595902464,"internally_reserved":0,"allocated":417792,"data_stored":120948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705720,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40416,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331424,"statfs":{"total":96636764160,"available":96595378176,"internally_reserved":0,"allocated":942080,"data_stored":708075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705719,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335556,"statfs":{"total":96636764160,"available":96599609344,"internally_reserved":0,"allocated":970752,"data_stored":708832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705721,"num_pgs":134,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39868,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331972,"statfs":{"total":96636764160,"available":96595939328,"internally_reserved":0,"allocated":380928,"data_stored":119531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T03:34:33.187 INFO:tasks.ceph:pgid 3.3b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.188 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.188 INFO:tasks.ceph:pgid 4.33 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.188 INFO:tasks.ceph:pgid 3.32 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.188 INFO:tasks.ceph:pgid 8.18 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.188 INFO:tasks.ceph:pgid 3.13 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.188 INFO:tasks.ceph:pgid 8.19 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.189 INFO:tasks.ceph:pgid 3.11 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.189 INFO:tasks.ceph:pgid 3.d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.189 INFO:tasks.ceph:pgid 3.c last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.189 INFO:tasks.ceph:pgid 8.7 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.189 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.190 INFO:tasks.ceph:pgid 3.9 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.190 INFO:tasks.ceph:pgid 3.8 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.190 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.190 INFO:tasks.ceph:pgid 5.4 last_scrub_stamp 2024-04-27T03:30:14.357542+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=14, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.190 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.191 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.191 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.191 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.191 INFO:tasks.ceph:pgid 3.7 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.191 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2024-04-27T03:30:05.096302+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=5, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.192 INFO:tasks.ceph:pgid 3.14 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.192 INFO:tasks.ceph:pgid 8.1e last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.192 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.192 INFO:tasks.ceph:pgid 3.18 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.193 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.193 INFO:tasks.ceph:pgid 3.1b last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.193 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.193 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.193 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.194 INFO:tasks.ceph:pgid 3.22 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.194 INFO:tasks.ceph:pgid 4.22 last_scrub_stamp 2024-04-27T03:30:12.346115+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=12, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.194 INFO:tasks.ceph:pgid 3.28 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.194 INFO:tasks.ceph:pgid 3.2d last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:33.195 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-27T03:34:53.196 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T03:34:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.298+0000 7f8f03ae2640 1 Processor -- start 2024-04-27T03:34:53.304 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.298+0000 7f8f03ae2640 1 -- start start 2024-04-27T03:34:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.299+0000 7f8f03ae2640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8efc1287d0 0x7f8efc130360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:34:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.299+0000 7f8f03ae2640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f8efc127b00 con 0x7f8efc127ce0 2024-04-27T03:34:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.299+0000 7f8f03ae2640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f8efc0708e0 con 0x7f8efc1287d0 2024-04-27T03:34:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.299+0000 7f8f0107d640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f8efc1287d0 0x7f8efc130360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:34:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.299+0000 7f8f0187e640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f8efc127ce0 0x7f8efc1280c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:57678/0 (socket says 172.21.15.89:57678) 2024-04-27T03:34:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.299+0000 7f8f0187e640 1 -- 172.21.15.89:0/1456715026 learned_addr learned my addr 172.21.15.89:0/1456715026 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:34:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.299+0000 7f8f0107d640 1 -- 172.21.15.89:0/1456715026 >> v1:172.21.15.89:6789/0 conn(0x7f8efc127ce0 legacy=0x7f8efc1280c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:34:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.299+0000 7f8f0107d640 1 -- 172.21.15.89:0/1456715026 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc098800 con 0x7f8efc1287d0 2024-04-27T03:34:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.300+0000 7f8f0107d640 1 --2- 172.21.15.89:0/1456715026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8efc1287d0 0x7f8efc130360 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009750 tx=0x7f8eec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef6fe19af6cd4f8b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:34:53.306 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.300+0000 7f8f0087c640 1 -- 172.21.15.89:0/1456715026 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eec0316d0 con 0x7f8efc1287d0 2024-04-27T03:34:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.300+0000 7f8f0087c640 1 -- 172.21.15.89:0/1456715026 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8eec037070 con 0x7f8efc1287d0 2024-04-27T03:34:53.307 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.300+0000 7f8f0087c640 1 -- 172.21.15.89:0/1456715026 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8eec0319c0 con 0x7f8efc1287d0 2024-04-27T03:34:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.301+0000 7f8f03ae2640 1 -- 172.21.15.89:0/1456715026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8efc1287d0 msgr2=0x7f8efc130360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:34:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.301+0000 7f8f03ae2640 1 --2- 172.21.15.89:0/1456715026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8efc1287d0 0x7f8efc130360 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f8eec009750 tx=0x7f8eec02b840 comp rx=0 tx=0).stop 2024-04-27T03:34:53.309 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.301+0000 7f8f03ae2640 1 -- 172.21.15.89:0/1456715026 shutdown_connections 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.301+0000 7f8f03ae2640 1 --2- 172.21.15.89:0/1456715026 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8efc1287d0 0x7f8efc130360 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.301+0000 7f8f03ae2640 1 -- 172.21.15.89:0/1456715026 >> 172.21.15.89:0/1456715026 conn(0x7f8efc099680 msgr2=0x7f8efc099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.301+0000 7f8f03ae2640 1 -- 172.21.15.89:0/1456715026 shutdown_connections 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.301+0000 7f8f03ae2640 1 -- 172.21.15.89:0/1456715026 wait complete. 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.302+0000 7f8f03ae2640 1 Processor -- start 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.302+0000 7f8f03ae2640 1 -- start start 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.302+0000 7f8f03ae2640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8efc127ce0 0x7f8efc1c0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.302+0000 7f8f03ae2640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f8efc06faa0 con 0x7f8efc127ce0 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.302+0000 7f8f0187e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8efc127ce0 0x7f8efc1c0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.302+0000 7f8f0187e640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8efc127ce0 0x7f8efc1c0dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:36572/0 (socket says 172.21.15.89:36572) 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.302+0000 7f8f0187e640 1 -- 172.21.15.89:0/645406826 learned_addr learned my addr 172.21.15.89:0/645406826 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.302+0000 7f8f0187e640 1 -- 172.21.15.89:0/645406826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8efc125c80 con 0x7f8efc127ce0 2024-04-27T03:34:53.310 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.302+0000 7f8f0187e640 1 --2- 172.21.15.89:0/645406826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8efc127ce0 0x7f8efc1c0dd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800c4a0 tx=0x7f8ef800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:34:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.303+0000 7f8ef27fc640 1 -- 172.21.15.89:0/645406826 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ef8017070 con 0x7f8efc127ce0 2024-04-27T03:34:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.303+0000 7f8ef27fc640 1 -- 172.21.15.89:0/645406826 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f8ef8005c10 con 0x7f8efc127ce0 2024-04-27T03:34:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.303+0000 7f8f03ae2640 1 -- 172.21.15.89:0/645406826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8efc06d6c0 con 0x7f8efc127ce0 2024-04-27T03:34:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.303+0000 7f8ef27fc640 1 -- 172.21.15.89:0/645406826 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f8ef8007070 con 0x7f8efc127ce0 2024-04-27T03:34:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.303+0000 7f8f03ae2640 1 -- 172.21.15.89:0/645406826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8efc125170 con 0x7f8efc127ce0 2024-04-27T03:34:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.303+0000 7f8ef27fc640 1 -- 172.21.15.89:0/645406826 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f8ef801a070 con 0x7f8efc127ce0 2024-04-27T03:34:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.304+0000 7f8ef27fc640 1 --2- 172.21.15.89:0/645406826 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8ee0038060 0x7f8ee003a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:34:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.304+0000 7f8ef27fc640 1 -- 172.21.15.89:0/645406826 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f8ef804ab60 con 0x7f8efc127ce0 2024-04-27T03:34:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.304+0000 7f8f03ae2640 1 -- 172.21.15.89:0/645406826 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8efc062da0 con 0x7f8efc127ce0 2024-04-27T03:34:53.311 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.304+0000 7f8f0107d640 1 --2- 172.21.15.89:0/645406826 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8ee0038060 0x7f8ee003a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:34:53.312 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.304+0000 7f8f0107d640 1 --2- 172.21.15.89:0/645406826 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8ee0038060 0x7f8ee003a520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8eec036000 tx=0x7f8eec008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:34:53.314 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.308+0000 7f8ef27fc640 1 -- 172.21.15.89:0/645406826 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f8efc127b00 con 0x7f8efc127ce0 2024-04-27T03:34:53.489 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.483+0000 7f8f03ae2640 1 -- 172.21.15.89:0/645406826 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f8efc06c170 con 0x7f8ee0038060 2024-04-27T03:34:53.504 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.498+0000 7f8ef27fc640 1 -- 172.21.15.89:0/645406826 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665500 (secure 0 0 0) 0x7f8efc06c170 con 0x7f8ee0038060 2024-04-27T03:34:53.511 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:34:53.517 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-04-27T03:34:53.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.503+0000 7f8f03ae2640 1 -- 172.21.15.89:0/645406826 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8ee0038060 msgr2=0x7f8ee003a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:34:53.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.503+0000 7f8f03ae2640 1 --2- 172.21.15.89:0/645406826 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8ee0038060 0x7f8ee003a520 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7f8eec036000 tx=0x7f8eec008030 comp rx=0 tx=0).stop 2024-04-27T03:34:53.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.503+0000 7f8f03ae2640 1 -- 172.21.15.89:0/645406826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8efc127ce0 msgr2=0x7f8efc1c0dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:34:53.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.503+0000 7f8f03ae2640 1 --2- 172.21.15.89:0/645406826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8efc127ce0 0x7f8efc1c0dd0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f8ef800c4a0 tx=0x7f8ef800c960 comp rx=0 tx=0).stop 2024-04-27T03:34:53.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.503+0000 7f8f03ae2640 1 -- 172.21.15.89:0/645406826 shutdown_connections 2024-04-27T03:34:53.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.503+0000 7f8f03ae2640 1 --2- 172.21.15.89:0/645406826 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f8ee0038060 0x7f8ee003a520 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:34:53.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.503+0000 7f8f03ae2640 1 --2- 172.21.15.89:0/645406826 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f8efc127ce0 0x7f8efc1c0dd0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:34:53.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.503+0000 7f8f03ae2640 1 -- 172.21.15.89:0/645406826 >> 172.21.15.89:0/645406826 conn(0x7f8efc099680 msgr2=0x7f8efc12ae10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:34:53.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.503+0000 7f8f03ae2640 1 -- 172.21.15.89:0/645406826 shutdown_connections 2024-04-27T03:34:53.520 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:34:53.503+0000 7f8f03ae2640 1 -- 172.21.15.89:0/645406826 wait complete. 2024-04-27T03:34:53.521 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":168,"stamp":"2024-04-27T03:34:52.298063+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":252,"num_object_clones":0,"num_object_copies":504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":252,"num_whiteouts":0,"num_read":5378,"num_read_kb":4932,"num_write":2369,"num_write_kb":61373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3041,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5009,"ondisk_log_size":5009,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":156472,"kb_used_data":2648,"kb_used_omap":6,"kb_used_meta":153785,"kb_avail":377330888,"statfs":{"total":386547056640,"available":386386829312,"internally_reserved":0,"allocated":2711552,"data_stored":1657386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":157476630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004207"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.834248+0000","last_change":"2024-04-27T03:34:07.834248+0000","last_active":"2024-04-27T03:34:07.834248+0000","last_peered":"2024-04-27T03:34:07.834248+0000","last_clean":"2024-04-27T03:34:07.834248+0000","last_became_active":"2024-04-27T03:30:11.362574+0000","last_became_peered":"2024-04-27T03:30:11.362574+0000","last_unstale":"2024-04-27T03:34:07.834248+0000","last_undegraded":"2024-04-27T03:34:07.834248+0000","last_fullsized":"2024-04-27T03:34:07.834248+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:07.834117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:07.834117+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.834117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:15.180175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.077803+0000","last_change":"2024-04-27T03:33:56.077828+0000","last_active":"2024-04-27T03:33:56.077803+0000","last_peered":"2024-04-27T03:33:56.077803+0000","last_clean":"2024-04-27T03:33:56.077803+0000","last_became_active":"2024-04-27T03:30:13.384580+0000","last_became_peered":"2024-04-27T03:30:13.384580+0000","last_unstale":"2024-04-27T03:33:56.077803+0000","last_undegraded":"2024-04-27T03:33:56.077803+0000","last_fullsized":"2024-04-27T03:33:56.077803+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:56.077710+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:56.077710+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.077710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:21.049969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.703599+0000","last_change":"2024-04-27T03:34:09.703599+0000","last_active":"2024-04-27T03:34:09.703599+0000","last_peered":"2024-04-27T03:34:09.703599+0000","last_clean":"2024-04-27T03:34:09.703599+0000","last_became_active":"2024-04-27T03:30:11.360857+0000","last_became_peered":"2024-04-27T03:30:11.360857+0000","last_unstale":"2024-04-27T03:34:09.703599+0000","last_undegraded":"2024-04-27T03:34:09.703599+0000","last_fullsized":"2024-04-27T03:34:09.703599+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:09.703487+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:09.703487+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.703487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:08:10.048372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"27'12","reported_seq":88,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:21.258937+0000","last_change":"2024-04-27T03:34:21.258982+0000","last_active":"2024-04-27T03:34:21.258937+0000","last_peered":"2024-04-27T03:34:21.258937+0000","last_clean":"2024-04-27T03:34:21.258937+0000","last_became_active":"2024-04-27T03:30:13.376221+0000","last_became_peered":"2024-04-27T03:30:13.376221+0000","last_unstale":"2024-04-27T03:34:21.258937+0000","last_undegraded":"2024-04-27T03:34:21.258937+0000","last_fullsized":"2024-04-27T03:34:21.258937+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-04-27T03:34:21.258835+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-04-27T03:34:21.258835+0000","last_clean_scrub_stamp":"2024-04-27T03:34:21.258835+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:22.651830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:27.334011+0000","last_change":"2024-04-27T03:34:27.334036+0000","last_active":"2024-04-27T03:34:27.334011+0000","last_peered":"2024-04-27T03:34:27.334011+0000","last_clean":"2024-04-27T03:34:27.334011+0000","last_became_active":"2024-04-27T03:30:11.362739+0000","last_became_peered":"2024-04-27T03:30:11.362739+0000","last_unstale":"2024-04-27T03:34:27.334011+0000","last_undegraded":"2024-04-27T03:34:27.334011+0000","last_fullsized":"2024-04-27T03:34:27.334011+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:27.333877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:27.333877+0000","last_clean_scrub_stamp":"2024-04-27T03:34:27.333877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:08.432542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"27'8","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:18.221789+0000","last_change":"2024-04-27T03:34:18.221821+0000","last_active":"2024-04-27T03:34:18.221789+0000","last_peered":"2024-04-27T03:34:18.221789+0000","last_clean":"2024-04-27T03:34:18.221789+0000","last_became_active":"2024-04-27T03:30:13.376524+0000","last_became_peered":"2024-04-27T03:30:13.376524+0000","last_unstale":"2024-04-27T03:34:18.221789+0000","last_undegraded":"2024-04-27T03:34:18.221789+0000","last_fullsized":"2024-04-27T03:34:18.221789+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-04-27T03:34:18.221705+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-04-27T03:34:18.221705+0000","last_clean_scrub_stamp":"2024-04-27T03:34:18.221705+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:20.194608+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:15.859998+0000","last_change":"2024-04-27T03:34:15.860023+0000","last_active":"2024-04-27T03:34:15.859998+0000","last_peered":"2024-04-27T03:34:15.859998+0000","last_clean":"2024-04-27T03:34:15.859998+0000","last_became_active":"2024-04-27T03:30:11.360188+0000","last_became_peered":"2024-04-27T03:30:11.360188+0000","last_unstale":"2024-04-27T03:34:15.859998+0000","last_undegraded":"2024-04-27T03:34:15.859998+0000","last_fullsized":"2024-04-27T03:34:15.859998+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:15.859866+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:15.859866+0000","last_clean_scrub_stamp":"2024-04-27T03:34:15.859866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:16.509376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"27'16","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:20.246700+0000","last_change":"2024-04-27T03:34:20.246757+0000","last_active":"2024-04-27T03:34:20.246700+0000","last_peered":"2024-04-27T03:34:20.246700+0000","last_clean":"2024-04-27T03:34:20.246700+0000","last_became_active":"2024-04-27T03:30:13.376288+0000","last_became_peered":"2024-04-27T03:30:13.376288+0000","last_unstale":"2024-04-27T03:34:20.246700+0000","last_undegraded":"2024-04-27T03:34:20.246700+0000","last_fullsized":"2024-04-27T03:34:20.246700+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:20.246599+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:20.246599+0000","last_clean_scrub_stamp":"2024-04-27T03:34:20.246599+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:03.552732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:47.331517+0000","last_change":"2024-04-27T03:34:47.331559+0000","last_active":"2024-04-27T03:34:47.331517+0000","last_peered":"2024-04-27T03:34:47.331517+0000","last_clean":"2024-04-27T03:34:47.331517+0000","last_became_active":"2024-04-27T03:30:11.354989+0000","last_became_peered":"2024-04-27T03:30:11.354989+0000","last_unstale":"2024-04-27T03:34:47.331517+0000","last_undegraded":"2024-04-27T03:34:47.331517+0000","last_fullsized":"2024-04-27T03:34:47.331517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:47.331390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:47.331390+0000","last_clean_scrub_stamp":"2024-04-27T03:34:47.331390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:56.995689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"27'6","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.200104+0000","last_change":"2024-04-27T03:34:04.200104+0000","last_active":"2024-04-27T03:34:04.200104+0000","last_peered":"2024-04-27T03:34:04.200104+0000","last_clean":"2024-04-27T03:34:04.200104+0000","last_became_active":"2024-04-27T03:30:13.373424+0000","last_became_peered":"2024-04-27T03:30:13.373424+0000","last_unstale":"2024-04-27T03:34:04.200104+0000","last_undegraded":"2024-04-27T03:34:04.200104+0000","last_fullsized":"2024-04-27T03:34:04.200104+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:04.199924+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:04.199924+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.199924+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:59.695293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:14.842885+0000","last_change":"2024-04-27T03:34:14.842915+0000","last_active":"2024-04-27T03:34:14.842885+0000","last_peered":"2024-04-27T03:34:14.842885+0000","last_clean":"2024-04-27T03:34:14.842885+0000","last_became_active":"2024-04-27T03:30:11.360219+0000","last_became_peered":"2024-04-27T03:30:11.360219+0000","last_unstale":"2024-04-27T03:34:14.842885+0000","last_undegraded":"2024-04-27T03:34:14.842885+0000","last_fullsized":"2024-04-27T03:34:14.842885+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:14.842794+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:14.842794+0000","last_clean_scrub_stamp":"2024-04-27T03:34:14.842794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:59.247435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"25'8","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.275565+0000","last_change":"2024-04-27T03:34:05.275565+0000","last_active":"2024-04-27T03:34:05.275565+0000","last_peered":"2024-04-27T03:34:05.275565+0000","last_clean":"2024-04-27T03:34:05.275565+0000","last_became_active":"2024-04-27T03:30:13.385665+0000","last_became_peered":"2024-04-27T03:30:13.385665+0000","last_unstale":"2024-04-27T03:34:05.275565+0000","last_undegraded":"2024-04-27T03:34:05.275565+0000","last_fullsized":"2024-04-27T03:34:05.275565+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'8","last_scrub_stamp":"2024-04-27T03:34:05.275413+0000","last_deep_scrub":"25'8","last_deep_scrub_stamp":"2024-04-27T03:34:05.275413+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.275413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:20:59.991475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298491+0000","last_change":"2024-04-27T03:30:11.373049+0000","last_active":"2024-04-27T03:32:44.298491+0000","last_peered":"2024-04-27T03:32:44.298491+0000","last_clean":"2024-04-27T03:32:44.298491+0000","last_became_active":"2024-04-27T03:30:11.372704+0000","last_became_peered":"2024-04-27T03:30:11.372704+0000","last_unstale":"2024-04-27T03:32:44.298491+0000","last_undegraded":"2024-04-27T03:32:44.298491+0000","last_fullsized":"2024-04-27T03:32:44.298491+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:10.339743+0000","last_clean_scrub_stamp":"2024-04-27T03:30:10.339743+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"27'4","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.771780+0000","last_change":"2024-04-27T03:33:56.771780+0000","last_active":"2024-04-27T03:33:56.771780+0000","last_peered":"2024-04-27T03:33:56.771780+0000","last_clean":"2024-04-27T03:33:56.771780+0000","last_became_active":"2024-04-27T03:30:13.374199+0000","last_became_peered":"2024-04-27T03:30:13.374199+0000","last_unstale":"2024-04-27T03:33:56.771780+0000","last_undegraded":"2024-04-27T03:33:56.771780+0000","last_fullsized":"2024-04-27T03:33:56.771780+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:56.771668+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:56.771668+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.771668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:18.489287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:19.635377+0000","last_change":"2024-04-27T03:34:19.635403+0000","last_active":"2024-04-27T03:34:19.635377+0000","last_peered":"2024-04-27T03:34:19.635377+0000","last_clean":"2024-04-27T03:34:19.635377+0000","last_became_active":"2024-04-27T03:30:11.358181+0000","last_became_peered":"2024-04-27T03:30:11.358181+0000","last_unstale":"2024-04-27T03:34:19.635377+0000","last_undegraded":"2024-04-27T03:34:19.635377+0000","last_fullsized":"2024-04-27T03:34:19.635377+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:19.635237+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:19.635237+0000","last_clean_scrub_stamp":"2024-04-27T03:34:19.635237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:22.761966+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.810889+0000","last_change":"2024-04-27T03:34:00.810889+0000","last_active":"2024-04-27T03:34:00.810889+0000","last_peered":"2024-04-27T03:34:00.810889+0000","last_clean":"2024-04-27T03:34:00.810889+0000","last_became_active":"2024-04-27T03:30:13.379059+0000","last_became_peered":"2024-04-27T03:30:13.379059+0000","last_unstale":"2024-04-27T03:34:00.810889+0000","last_undegraded":"2024-04-27T03:34:00.810889+0000","last_fullsized":"2024-04-27T03:34:00.810889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:00.810774+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:00.810774+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.810774+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:01.151920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"27'6","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.269064+0000","last_change":"2024-04-27T03:34:06.269111+0000","last_active":"2024-04-27T03:34:06.269064+0000","last_peered":"2024-04-27T03:34:06.269064+0000","last_clean":"2024-04-27T03:34:06.269064+0000","last_became_active":"2024-04-27T03:30:13.383694+0000","last_became_peered":"2024-04-27T03:30:13.383694+0000","last_unstale":"2024-04-27T03:34:06.269064+0000","last_undegraded":"2024-04-27T03:34:06.269064+0000","last_fullsized":"2024-04-27T03:34:06.269064+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:06.268965+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:06.268965+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.268965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:49.525413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:24.057185+0000","last_change":"2024-04-27T03:34:24.057220+0000","last_active":"2024-04-27T03:34:24.057185+0000","last_peered":"2024-04-27T03:34:24.057185+0000","last_clean":"2024-04-27T03:34:24.057185+0000","last_became_active":"2024-04-27T03:30:11.370157+0000","last_became_peered":"2024-04-27T03:30:11.370157+0000","last_unstale":"2024-04-27T03:34:24.057185+0000","last_undegraded":"2024-04-27T03:34:24.057185+0000","last_fullsized":"2024-04-27T03:34:24.057185+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:24.057056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:24.057056+0000","last_clean_scrub_stamp":"2024-04-27T03:34:24.057056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:46.521120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"27'6","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.201544+0000","last_change":"2024-04-27T03:34:03.201590+0000","last_active":"2024-04-27T03:34:03.201544+0000","last_peered":"2024-04-27T03:34:03.201544+0000","last_clean":"2024-04-27T03:34:03.201544+0000","last_became_active":"2024-04-27T03:30:13.372973+0000","last_became_peered":"2024-04-27T03:30:13.372973+0000","last_unstale":"2024-04-27T03:34:03.201544+0000","last_undegraded":"2024-04-27T03:34:03.201544+0000","last_fullsized":"2024-04-27T03:34:03.201544+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:03.201445+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:03.201445+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.201445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:57.765391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"27'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.817465+0000","last_change":"2024-04-27T03:34:03.817515+0000","last_active":"2024-04-27T03:34:03.817465+0000","last_peered":"2024-04-27T03:34:03.817465+0000","last_clean":"2024-04-27T03:34:03.817465+0000","last_became_active":"2024-04-27T03:30:11.363109+0000","last_became_peered":"2024-04-27T03:30:11.363109+0000","last_unstale":"2024-04-27T03:34:03.817465+0000","last_undegraded":"2024-04-27T03:34:03.817465+0000","last_fullsized":"2024-04-27T03:34:03.817465+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:03.817361+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:03.817361+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.817361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:30.353138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"27'10","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.121833+0000","last_change":"2024-04-27T03:33:58.121867+0000","last_active":"2024-04-27T03:33:58.121833+0000","last_peered":"2024-04-27T03:33:58.121833+0000","last_clean":"2024-04-27T03:33:58.121833+0000","last_became_active":"2024-04-27T03:30:13.383742+0000","last_became_peered":"2024-04-27T03:30:13.383742+0000","last_unstale":"2024-04-27T03:33:58.121833+0000","last_undegraded":"2024-04-27T03:33:58.121833+0000","last_fullsized":"2024-04-27T03:33:58.121833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:33:58.121739+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:33:58.121739+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.121739+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:37.923650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"25'4","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:26.075178+0000","last_change":"2024-04-27T03:34:26.075209+0000","last_active":"2024-04-27T03:34:26.075178+0000","last_peered":"2024-04-27T03:34:26.075178+0000","last_clean":"2024-04-27T03:34:26.075178+0000","last_became_active":"2024-04-27T03:30:11.370380+0000","last_became_peered":"2024-04-27T03:30:11.370380+0000","last_unstale":"2024-04-27T03:34:26.075178+0000","last_undegraded":"2024-04-27T03:34:26.075178+0000","last_fullsized":"2024-04-27T03:34:26.075178+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'4","last_scrub_stamp":"2024-04-27T03:34:26.075076+0000","last_deep_scrub":"25'4","last_deep_scrub_stamp":"2024-04-27T03:34:26.075076+0000","last_clean_scrub_stamp":"2024-04-27T03:34:26.075076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:58.085501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"27'6","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:31.721773+0000","last_change":"2024-04-27T03:34:31.721798+0000","last_active":"2024-04-27T03:34:31.721773+0000","last_peered":"2024-04-27T03:34:31.721773+0000","last_clean":"2024-04-27T03:34:31.721773+0000","last_became_active":"2024-04-27T03:31:01.440762+0000","last_became_peered":"2024-04-27T03:31:01.440762+0000","last_unstale":"2024-04-27T03:34:31.721773+0000","last_undegraded":"2024-04-27T03:34:31.721773+0000","last_fullsized":"2024-04-27T03:34:31.721773+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:31.721686+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:31.721686+0000","last_clean_scrub_stamp":"2024-04-27T03:34:31.721686+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:14.945006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:21.120619+0000","last_change":"2024-04-27T03:34:21.120644+0000","last_active":"2024-04-27T03:34:21.120619+0000","last_peered":"2024-04-27T03:34:21.120619+0000","last_clean":"2024-04-27T03:34:21.120619+0000","last_became_active":"2024-04-27T03:30:11.369551+0000","last_became_peered":"2024-04-27T03:30:11.369551+0000","last_unstale":"2024-04-27T03:34:21.120619+0000","last_undegraded":"2024-04-27T03:34:21.120619+0000","last_fullsized":"2024-04-27T03:34:21.120619+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:21.120533+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:21.120533+0000","last_clean_scrub_stamp":"2024-04-27T03:34:21.120533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:54.803174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"27'2","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.850385+0000","last_change":"2024-04-27T03:34:04.850412+0000","last_active":"2024-04-27T03:34:04.850385+0000","last_peered":"2024-04-27T03:34:04.850385+0000","last_clean":"2024-04-27T03:34:04.850385+0000","last_became_active":"2024-04-27T03:30:13.381901+0000","last_became_peered":"2024-04-27T03:30:13.381901+0000","last_unstale":"2024-04-27T03:34:04.850385+0000","last_undegraded":"2024-04-27T03:34:04.850385+0000","last_fullsized":"2024-04-27T03:34:04.850385+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:04.850293+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:04.850293+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.850293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:29.194843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"25'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:15.639011+0000","last_change":"2024-04-27T03:34:15.639038+0000","last_active":"2024-04-27T03:34:15.639011+0000","last_peered":"2024-04-27T03:34:15.639011+0000","last_clean":"2024-04-27T03:34:15.639011+0000","last_became_active":"2024-04-27T03:30:11.360515+0000","last_became_peered":"2024-04-27T03:30:11.360515+0000","last_unstale":"2024-04-27T03:34:15.639011+0000","last_undegraded":"2024-04-27T03:34:15.639011+0000","last_fullsized":"2024-04-27T03:34:15.639011+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:15.638909+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:15.638909+0000","last_clean_scrub_stamp":"2024-04-27T03:34:15.638909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:04.458649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"27'4","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.155138+0000","last_change":"2024-04-27T03:33:59.155166+0000","last_active":"2024-04-27T03:33:59.155138+0000","last_peered":"2024-04-27T03:33:59.155138+0000","last_clean":"2024-04-27T03:33:59.155138+0000","last_became_active":"2024-04-27T03:30:13.378932+0000","last_became_peered":"2024-04-27T03:30:13.378932+0000","last_unstale":"2024-04-27T03:33:59.155138+0000","last_undegraded":"2024-04-27T03:33:59.155138+0000","last_fullsized":"2024-04-27T03:33:59.155138+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:59.155047+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:59.155047+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.155047+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:36.999661+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:37.333033+0000","last_change":"2024-04-27T03:34:37.333059+0000","last_active":"2024-04-27T03:34:37.333033+0000","last_peered":"2024-04-27T03:34:37.333033+0000","last_clean":"2024-04-27T03:34:37.333033+0000","last_became_active":"2024-04-27T03:30:11.372673+0000","last_became_peered":"2024-04-27T03:30:11.372673+0000","last_unstale":"2024-04-27T03:34:37.333033+0000","last_undegraded":"2024-04-27T03:34:37.333033+0000","last_fullsized":"2024-04-27T03:34:37.333033+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:37.332923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:37.332923+0000","last_clean_scrub_stamp":"2024-04-27T03:34:37.332923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:48.977095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"27'36","reported_seq":109,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:30.101093+0000","last_change":"2024-04-27T03:34:30.101126+0000","last_active":"2024-04-27T03:34:30.101093+0000","last_peered":"2024-04-27T03:34:30.101093+0000","last_clean":"2024-04-27T03:34:30.101093+0000","last_became_active":"2024-04-27T03:30:21.413255+0000","last_became_peered":"2024-04-27T03:30:21.413255+0000","last_unstale":"2024-04-27T03:34:30.101093+0000","last_undegraded":"2024-04-27T03:34:30.101093+0000","last_fullsized":"2024-04-27T03:34:30.101093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-04-27T03:34:30.100998+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-04-27T03:34:30.100998+0000","last_clean_scrub_stamp":"2024-04-27T03:34:30.100998+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:07:40.938805+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"27'4","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:16.259266+0000","last_change":"2024-04-27T03:34:16.259291+0000","last_active":"2024-04-27T03:34:16.259266+0000","last_peered":"2024-04-27T03:34:16.259266+0000","last_clean":"2024-04-27T03:34:16.259266+0000","last_became_active":"2024-04-27T03:30:13.377110+0000","last_became_peered":"2024-04-27T03:30:13.377110+0000","last_unstale":"2024-04-27T03:34:16.259266+0000","last_undegraded":"2024-04-27T03:34:16.259266+0000","last_fullsized":"2024-04-27T03:34:16.259266+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:16.259165+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:16.259165+0000","last_clean_scrub_stamp":"2024-04-27T03:34:16.259165+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:04.414839+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:30.301680+0000","last_change":"2024-04-27T03:34:30.301717+0000","last_active":"2024-04-27T03:34:30.301680+0000","last_peered":"2024-04-27T03:34:30.301680+0000","last_clean":"2024-04-27T03:34:30.301680+0000","last_became_active":"2024-04-27T03:30:11.359892+0000","last_became_peered":"2024-04-27T03:30:11.359892+0000","last_unstale":"2024-04-27T03:34:30.301680+0000","last_undegraded":"2024-04-27T03:34:30.301680+0000","last_fullsized":"2024-04-27T03:34:30.301680+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:30.301573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:30.301573+0000","last_clean_scrub_stamp":"2024-04-27T03:34:30.301573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:05:34.290404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.148150+0000","last_change":"2024-04-27T03:33:38.148173+0000","last_active":"2024-04-27T03:33:38.148150+0000","last_peered":"2024-04-27T03:33:38.148150+0000","last_clean":"2024-04-27T03:33:38.148150+0000","last_became_active":"2024-04-27T03:30:15.380864+0000","last_became_peered":"2024-04-27T03:30:15.380864+0000","last_unstale":"2024-04-27T03:33:38.148150+0000","last_undegraded":"2024-04-27T03:33:38.148150+0000","last_fullsized":"2024-04-27T03:33:38.148150+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:38.148061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:38.148061+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.148061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:53.865374+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"27'50","reported_seq":117,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.088201+0000","last_change":"2024-04-27T03:33:43.088242+0000","last_active":"2024-04-27T03:33:43.088201+0000","last_peered":"2024-04-27T03:33:43.088201+0000","last_clean":"2024-04-27T03:33:43.088201+0000","last_became_active":"2024-04-27T03:30:17.387391+0000","last_became_peered":"2024-04-27T03:30:17.387391+0000","last_unstale":"2024-04-27T03:33:43.088201+0000","last_undegraded":"2024-04-27T03:33:43.088201+0000","last_fullsized":"2024-04-27T03:33:43.088201+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'50","last_scrub_stamp":"2024-04-27T03:33:43.088100+0000","last_deep_scrub":"27'50","last_deep_scrub_stamp":"2024-04-27T03:33:43.088100+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.088100+0000","objects_scrubbed":6,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:13:27.492179+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":45,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.971326+0000","last_change":"2024-04-27T03:33:44.971352+0000","last_active":"2024-04-27T03:33:44.971326+0000","last_peered":"2024-04-27T03:33:44.971326+0000","last_clean":"2024-04-27T03:33:44.971326+0000","last_became_active":"2024-04-27T03:30:19.407637+0000","last_became_peered":"2024-04-27T03:30:19.407637+0000","last_unstale":"2024-04-27T03:33:44.971326+0000","last_undegraded":"2024-04-27T03:33:44.971326+0000","last_fullsized":"2024-04-27T03:33:44.971326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:44.971239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:44.971239+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.971239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:22.448440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"27'9","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:31.139111+0000","last_change":"2024-04-27T03:34:31.139137+0000","last_active":"2024-04-27T03:34:31.139111+0000","last_peered":"2024-04-27T03:34:31.139111+0000","last_clean":"2024-04-27T03:34:31.139111+0000","last_became_active":"2024-04-27T03:30:21.413940+0000","last_became_peered":"2024-04-27T03:30:21.413940+0000","last_unstale":"2024-04-27T03:34:31.139111+0000","last_undegraded":"2024-04-27T03:34:31.139111+0000","last_fullsized":"2024-04-27T03:34:31.139111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-04-27T03:34:31.139021+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-04-27T03:34:31.139021+0000","last_clean_scrub_stamp":"2024-04-27T03:34:31.139021+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:52.945498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"27'16","reported_seq":76,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.265928+0000","last_change":"2024-04-27T03:34:09.266003+0000","last_active":"2024-04-27T03:34:09.265928+0000","last_peered":"2024-04-27T03:34:09.265928+0000","last_clean":"2024-04-27T03:34:09.265928+0000","last_became_active":"2024-04-27T03:30:13.381487+0000","last_became_peered":"2024-04-27T03:30:13.381487+0000","last_unstale":"2024-04-27T03:34:09.265928+0000","last_undegraded":"2024-04-27T03:34:09.265928+0000","last_fullsized":"2024-04-27T03:34:09.265928+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:09.265817+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:09.265817+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.265817+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:46.218428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:13.813006+0000","last_change":"2024-04-27T03:34:13.813035+0000","last_active":"2024-04-27T03:34:13.813006+0000","last_peered":"2024-04-27T03:34:13.813006+0000","last_clean":"2024-04-27T03:34:13.813006+0000","last_became_active":"2024-04-27T03:30:11.359446+0000","last_became_peered":"2024-04-27T03:30:11.359446+0000","last_unstale":"2024-04-27T03:34:13.813006+0000","last_undegraded":"2024-04-27T03:34:13.813006+0000","last_fullsized":"2024-04-27T03:34:13.813006+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:13.812877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:13.812877+0000","last_clean_scrub_stamp":"2024-04-27T03:34:13.812877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:46.859807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.671578+0000","last_change":"2024-04-27T03:33:55.671603+0000","last_active":"2024-04-27T03:33:55.671578+0000","last_peered":"2024-04-27T03:33:55.671578+0000","last_clean":"2024-04-27T03:33:55.671578+0000","last_became_active":"2024-04-27T03:30:15.380313+0000","last_became_peered":"2024-04-27T03:30:15.380313+0000","last_unstale":"2024-04-27T03:33:55.671578+0000","last_undegraded":"2024-04-27T03:33:55.671578+0000","last_fullsized":"2024-04-27T03:33:55.671578+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:55.671471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:55.671471+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.671471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:20.643729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"27'16","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.841774+0000","last_change":"2024-04-27T03:33:37.841840+0000","last_active":"2024-04-27T03:33:37.841774+0000","last_peered":"2024-04-27T03:33:37.841774+0000","last_clean":"2024-04-27T03:33:37.841774+0000","last_became_active":"2024-04-27T03:30:17.389126+0000","last_became_peered":"2024-04-27T03:30:17.389126+0000","last_unstale":"2024-04-27T03:33:37.841774+0000","last_undegraded":"2024-04-27T03:33:37.841774+0000","last_fullsized":"2024-04-27T03:33:37.841774+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:33:37.841643+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:33:37.841643+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.841643+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:42.665962+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.837474+0000","last_change":"2024-04-27T03:33:36.837497+0000","last_active":"2024-04-27T03:33:36.837474+0000","last_peered":"2024-04-27T03:33:36.837474+0000","last_clean":"2024-04-27T03:33:36.837474+0000","last_became_active":"2024-04-27T03:30:19.405170+0000","last_became_peered":"2024-04-27T03:30:19.405170+0000","last_unstale":"2024-04-27T03:33:36.837474+0000","last_undegraded":"2024-04-27T03:33:36.837474+0000","last_fullsized":"2024-04-27T03:33:36.837474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.837388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.837388+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.837388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:14.962289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'6","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.661566+0000","last_change":"2024-04-27T03:33:58.661623+0000","last_active":"2024-04-27T03:33:58.661566+0000","last_peered":"2024-04-27T03:33:58.661566+0000","last_clean":"2024-04-27T03:33:58.661566+0000","last_became_active":"2024-04-27T03:30:21.407917+0000","last_became_peered":"2024-04-27T03:30:21.407917+0000","last_unstale":"2024-04-27T03:33:58.661566+0000","last_undegraded":"2024-04-27T03:33:58.661566+0000","last_fullsized":"2024-04-27T03:33:58.661566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:58.661469+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:58.661469+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.661469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:52.661163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":104,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:12.193343+0000","last_change":"2024-04-27T03:34:12.193369+0000","last_active":"2024-04-27T03:34:12.193343+0000","last_peered":"2024-04-27T03:34:12.193343+0000","last_clean":"2024-04-27T03:34:12.193343+0000","last_became_active":"2024-04-27T03:30:13.377423+0000","last_became_peered":"2024-04-27T03:30:13.377423+0000","last_unstale":"2024-04-27T03:34:12.193343+0000","last_undegraded":"2024-04-27T03:34:12.193343+0000","last_fullsized":"2024-04-27T03:34:12.193343+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-04-27T03:34:12.193224+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-04-27T03:34:12.193224+0000","last_clean_scrub_stamp":"2024-04-27T03:34:12.193224+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:00:13.595186+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:32.332980+0000","last_change":"2024-04-27T03:34:32.333004+0000","last_active":"2024-04-27T03:34:32.332980+0000","last_peered":"2024-04-27T03:34:32.332980+0000","last_clean":"2024-04-27T03:34:32.332980+0000","last_became_active":"2024-04-27T03:30:11.359261+0000","last_became_peered":"2024-04-27T03:30:11.359261+0000","last_unstale":"2024-04-27T03:34:32.332980+0000","last_undegraded":"2024-04-27T03:34:32.332980+0000","last_fullsized":"2024-04-27T03:34:32.332980+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:32.332847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:32.332847+0000","last_clean_scrub_stamp":"2024-04-27T03:34:32.332847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:01:05.240389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.753269+0000","last_change":"2024-04-27T03:33:47.753294+0000","last_active":"2024-04-27T03:33:47.753269+0000","last_peered":"2024-04-27T03:33:47.753269+0000","last_clean":"2024-04-27T03:33:47.753269+0000","last_became_active":"2024-04-27T03:30:15.383618+0000","last_became_peered":"2024-04-27T03:30:15.383618+0000","last_unstale":"2024-04-27T03:33:47.753269+0000","last_undegraded":"2024-04-27T03:33:47.753269+0000","last_fullsized":"2024-04-27T03:33:47.753269+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:47.753177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:47.753177+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.753177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:54.718147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"27'30","reported_seq":81,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.888348+0000","last_change":"2024-04-27T03:33:37.888412+0000","last_active":"2024-04-27T03:33:37.888348+0000","last_peered":"2024-04-27T03:33:37.888348+0000","last_clean":"2024-04-27T03:33:37.888348+0000","last_became_active":"2024-04-27T03:30:17.392223+0000","last_became_peered":"2024-04-27T03:30:17.392223+0000","last_unstale":"2024-04-27T03:33:37.888348+0000","last_undegraded":"2024-04-27T03:33:37.888348+0000","last_fullsized":"2024-04-27T03:33:37.888348+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-04-27T03:33:37.888233+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-04-27T03:33:37.888233+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.888233+0000","objects_scrubbed":7,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:27:30.955280+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"27'286","reported_seq":394,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.206383+0000","last_change":"2024-04-27T03:34:03.206432+0000","last_active":"2024-04-27T03:34:03.206383+0000","last_peered":"2024-04-27T03:34:03.206383+0000","last_clean":"2024-04-27T03:34:03.206383+0000","last_became_active":"2024-04-27T03:30:19.405730+0000","last_became_peered":"2024-04-27T03:30:19.405730+0000","last_unstale":"2024-04-27T03:34:03.206383+0000","last_undegraded":"2024-04-27T03:34:03.206383+0000","last_fullsized":"2024-04-27T03:34:03.206383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'286","last_scrub_stamp":"2024-04-27T03:34:03.206276+0000","last_deep_scrub":"27'286","last_deep_scrub_stamp":"2024-04-27T03:34:03.206276+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.206276+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:25.670340+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:19.898900+0000","last_change":"2024-04-27T03:34:19.898939+0000","last_active":"2024-04-27T03:34:19.898900+0000","last_peered":"2024-04-27T03:34:19.898900+0000","last_clean":"2024-04-27T03:34:19.898900+0000","last_became_active":"2024-04-27T03:30:21.409156+0000","last_became_peered":"2024-04-27T03:30:21.409156+0000","last_unstale":"2024-04-27T03:34:19.898900+0000","last_undegraded":"2024-04-27T03:34:19.898900+0000","last_fullsized":"2024-04-27T03:34:19.898900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:19.898783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:19.898783+0000","last_clean_scrub_stamp":"2024-04-27T03:34:19.898783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:23:50.887010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"27'6","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.704469+0000","last_change":"2024-04-27T03:33:59.704499+0000","last_active":"2024-04-27T03:33:59.704469+0000","last_peered":"2024-04-27T03:33:59.704469+0000","last_clean":"2024-04-27T03:33:59.704469+0000","last_became_active":"2024-04-27T03:30:13.385491+0000","last_became_peered":"2024-04-27T03:30:13.385491+0000","last_unstale":"2024-04-27T03:33:59.704469+0000","last_undegraded":"2024-04-27T03:33:59.704469+0000","last_fullsized":"2024-04-27T03:33:59.704469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:59.704383+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:59.704383+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.704383+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:13:15.421696+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:23.050142+0000","last_change":"2024-04-27T03:34:23.050167+0000","last_active":"2024-04-27T03:34:23.050142+0000","last_peered":"2024-04-27T03:34:23.050142+0000","last_clean":"2024-04-27T03:34:23.050142+0000","last_became_active":"2024-04-27T03:30:11.369939+0000","last_became_peered":"2024-04-27T03:30:11.369939+0000","last_unstale":"2024-04-27T03:34:23.050142+0000","last_undegraded":"2024-04-27T03:34:23.050142+0000","last_fullsized":"2024-04-27T03:34:23.050142+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:23.050056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:23.050056+0000","last_clean_scrub_stamp":"2024-04-27T03:34:23.050056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:26:29.085590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.134971+0000","last_change":"2024-04-27T03:33:33.134999+0000","last_active":"2024-04-27T03:33:33.134971+0000","last_peered":"2024-04-27T03:33:33.134971+0000","last_clean":"2024-04-27T03:33:33.134971+0000","last_became_active":"2024-04-27T03:30:15.384114+0000","last_became_peered":"2024-04-27T03:30:15.384114+0000","last_unstale":"2024-04-27T03:33:33.134971+0000","last_undegraded":"2024-04-27T03:33:33.134971+0000","last_fullsized":"2024-04-27T03:33:33.134971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:33.134840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:33.134840+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.134840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:57.479389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"27'43","reported_seq":104,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.859628+0000","last_change":"2024-04-27T03:33:38.859665+0000","last_active":"2024-04-27T03:33:38.859628+0000","last_peered":"2024-04-27T03:33:38.859628+0000","last_clean":"2024-04-27T03:33:38.859628+0000","last_became_active":"2024-04-27T03:30:17.391880+0000","last_became_peered":"2024-04-27T03:30:17.391880+0000","last_unstale":"2024-04-27T03:33:38.859628+0000","last_undegraded":"2024-04-27T03:33:38.859628+0000","last_fullsized":"2024-04-27T03:33:38.859628+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'43","last_scrub_stamp":"2024-04-27T03:33:38.859525+0000","last_deep_scrub":"27'43","last_deep_scrub_stamp":"2024-04-27T03:33:38.859525+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.859525+0000","objects_scrubbed":5,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:06.442676+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"27'285","reported_seq":417,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.785523+0000","last_change":"2024-04-27T03:33:35.785553+0000","last_active":"2024-04-27T03:33:35.785523+0000","last_peered":"2024-04-27T03:33:35.785523+0000","last_clean":"2024-04-27T03:33:35.785523+0000","last_became_active":"2024-04-27T03:30:19.404744+0000","last_became_peered":"2024-04-27T03:30:19.404744+0000","last_unstale":"2024-04-27T03:33:35.785523+0000","last_undegraded":"2024-04-27T03:33:35.785523+0000","last_fullsized":"2024-04-27T03:33:35.785523+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:35.785437+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:35.785437+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.785437+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:23.864435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'7","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.165611+0000","last_change":"2024-04-27T03:33:49.165611+0000","last_active":"2024-04-27T03:33:49.165611+0000","last_peered":"2024-04-27T03:33:49.165611+0000","last_clean":"2024-04-27T03:33:49.165611+0000","last_became_active":"2024-04-27T03:30:13.387335+0000","last_became_peered":"2024-04-27T03:30:13.387335+0000","last_unstale":"2024-04-27T03:33:49.165611+0000","last_undegraded":"2024-04-27T03:33:49.165611+0000","last_fullsized":"2024-04-27T03:33:49.165611+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:33:49.165500+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:33:49.165500+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.165500+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:22.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.841348+0000","last_change":"2024-04-27T03:34:01.841348+0000","last_active":"2024-04-27T03:34:01.841348+0000","last_peered":"2024-04-27T03:34:01.841348+0000","last_clean":"2024-04-27T03:34:01.841348+0000","last_became_active":"2024-04-27T03:30:11.371600+0000","last_became_peered":"2024-04-27T03:30:11.371600+0000","last_unstale":"2024-04-27T03:34:01.841348+0000","last_undegraded":"2024-04-27T03:34:01.841348+0000","last_fullsized":"2024-04-27T03:34:01.841348+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:01.841193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:01.841193+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.841193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:21:13.198341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.990760+0000","last_change":"2024-04-27T03:33:46.990760+0000","last_active":"2024-04-27T03:33:46.990760+0000","last_peered":"2024-04-27T03:33:46.990760+0000","last_clean":"2024-04-27T03:33:46.990760+0000","last_became_active":"2024-04-27T03:30:15.381359+0000","last_became_peered":"2024-04-27T03:30:15.381359+0000","last_unstale":"2024-04-27T03:33:46.990760+0000","last_undegraded":"2024-04-27T03:33:46.990760+0000","last_fullsized":"2024-04-27T03:33:46.990760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:46.990601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:46.990601+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.990601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:08.708219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"27'58","reported_seq":125,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.809671+0000","last_change":"2024-04-27T03:33:41.809671+0000","last_active":"2024-04-27T03:33:41.809671+0000","last_peered":"2024-04-27T03:33:41.809671+0000","last_clean":"2024-04-27T03:33:41.809671+0000","last_became_active":"2024-04-27T03:30:17.390021+0000","last_became_peered":"2024-04-27T03:30:17.390021+0000","last_unstale":"2024-04-27T03:33:41.809671+0000","last_undegraded":"2024-04-27T03:33:41.809671+0000","last_fullsized":"2024-04-27T03:33:41.809671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'58","last_scrub_stamp":"2024-04-27T03:33:41.809501+0000","last_deep_scrub":"27'58","last_deep_scrub_stamp":"2024-04-27T03:33:41.809501+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.809501+0000","objects_scrubbed":10,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:46.251050+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":30,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.738255+0000","last_change":"2024-04-27T03:33:32.738255+0000","last_active":"2024-04-27T03:33:32.738255+0000","last_peered":"2024-04-27T03:33:32.738255+0000","last_clean":"2024-04-27T03:33:32.738255+0000","last_became_active":"2024-04-27T03:30:19.404948+0000","last_became_peered":"2024-04-27T03:30:19.404948+0000","last_unstale":"2024-04-27T03:33:32.738255+0000","last_undegraded":"2024-04-27T03:33:32.738255+0000","last_fullsized":"2024-04-27T03:33:32.738255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:32.738165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:32.738165+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.738165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:19.681571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"27'3","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:21.929604+0000","last_change":"2024-04-27T03:34:21.929604+0000","last_active":"2024-04-27T03:34:21.929604+0000","last_peered":"2024-04-27T03:34:21.929604+0000","last_clean":"2024-04-27T03:34:21.929604+0000","last_became_active":"2024-04-27T03:30:21.412331+0000","last_became_peered":"2024-04-27T03:30:21.412331+0000","last_unstale":"2024-04-27T03:34:21.929604+0000","last_undegraded":"2024-04-27T03:34:21.929604+0000","last_fullsized":"2024-04-27T03:34:21.929604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-27T03:34:21.929483+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-27T03:34:21.929483+0000","last_clean_scrub_stamp":"2024-04-27T03:34:21.929483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:01:02.076280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"25'6","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.184718+0000","last_change":"2024-04-27T03:34:06.184718+0000","last_active":"2024-04-27T03:34:06.184718+0000","last_peered":"2024-04-27T03:34:06.184718+0000","last_clean":"2024-04-27T03:34:06.184718+0000","last_became_active":"2024-04-27T03:30:13.373226+0000","last_became_peered":"2024-04-27T03:30:13.373226+0000","last_unstale":"2024-04-27T03:34:06.184718+0000","last_undegraded":"2024-04-27T03:34:06.184718+0000","last_fullsized":"2024-04-27T03:34:06.184718+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'6","last_scrub_stamp":"2024-04-27T03:34:06.184601+0000","last_deep_scrub":"25'6","last_deep_scrub_stamp":"2024-04-27T03:34:06.184601+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.184601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:06.834111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.791865+0000","last_change":"2024-04-27T03:33:59.791865+0000","last_active":"2024-04-27T03:33:59.791865+0000","last_peered":"2024-04-27T03:33:59.791865+0000","last_clean":"2024-04-27T03:33:59.791865+0000","last_became_active":"2024-04-27T03:30:11.373430+0000","last_became_peered":"2024-04-27T03:30:11.373430+0000","last_unstale":"2024-04-27T03:33:59.791865+0000","last_undegraded":"2024-04-27T03:33:59.791865+0000","last_fullsized":"2024-04-27T03:33:59.791865+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:59.791751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:59.791751+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.791751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:23.025879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"27'2","reported_seq":90,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:25.988249+0000","last_change":"2024-04-27T03:34:25.988283+0000","last_active":"2024-04-27T03:34:25.988249+0000","last_peered":"2024-04-27T03:34:25.988249+0000","last_clean":"2024-04-27T03:34:25.988249+0000","last_became_active":"2024-04-27T03:31:01.443259+0000","last_became_peered":"2024-04-27T03:31:01.443259+0000","last_unstale":"2024-04-27T03:34:25.988249+0000","last_undegraded":"2024-04-27T03:34:25.988249+0000","last_fullsized":"2024-04-27T03:34:25.988249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:25.988159+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:25.988159+0000","last_clean_scrub_stamp":"2024-04-27T03:34:25.988159+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:14.807899+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"27'47","reported_seq":109,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.230642+0000","last_change":"2024-04-27T03:34:04.230642+0000","last_active":"2024-04-27T03:34:04.230642+0000","last_peered":"2024-04-27T03:34:04.230642+0000","last_clean":"2024-04-27T03:34:04.230642+0000","last_became_active":"2024-04-27T03:30:17.388994+0000","last_became_peered":"2024-04-27T03:30:17.388994+0000","last_unstale":"2024-04-27T03:34:04.230642+0000","last_undegraded":"2024-04-27T03:34:04.230642+0000","last_fullsized":"2024-04-27T03:34:04.230642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-04-27T03:34:04.230519+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-04-27T03:34:04.230519+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.230519+0000","objects_scrubbed":5,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:43:53.086291+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:30.805201+0000","last_change":"2024-04-27T03:33:30.805201+0000","last_active":"2024-04-27T03:33:30.805201+0000","last_peered":"2024-04-27T03:33:30.805201+0000","last_clean":"2024-04-27T03:33:30.805201+0000","last_became_active":"2024-04-27T03:30:19.405334+0000","last_became_peered":"2024-04-27T03:30:19.405334+0000","last_unstale":"2024-04-27T03:33:30.805201+0000","last_undegraded":"2024-04-27T03:33:30.805201+0000","last_fullsized":"2024-04-27T03:33:30.805201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:30.805019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:30.805019+0000","last_clean_scrub_stamp":"2024-04-27T03:33:30.805019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:46:37.451973+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.099611+0000","last_change":"2024-04-27T03:33:57.099611+0000","last_active":"2024-04-27T03:33:57.099611+0000","last_peered":"2024-04-27T03:33:57.099611+0000","last_clean":"2024-04-27T03:33:57.099611+0000","last_became_active":"2024-04-27T03:30:21.409922+0000","last_became_peered":"2024-04-27T03:30:21.409922+0000","last_unstale":"2024-04-27T03:33:57.099611+0000","last_undegraded":"2024-04-27T03:33:57.099611+0000","last_fullsized":"2024-04-27T03:33:57.099611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:57.099495+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:57.099495+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.099495+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:01.218125+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"27'6","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.819584+0000","last_change":"2024-04-27T03:34:02.819584+0000","last_active":"2024-04-27T03:34:02.819584+0000","last_peered":"2024-04-27T03:34:02.819584+0000","last_clean":"2024-04-27T03:34:02.819584+0000","last_became_active":"2024-04-27T03:30:13.381270+0000","last_became_peered":"2024-04-27T03:30:13.381270+0000","last_unstale":"2024-04-27T03:34:02.819584+0000","last_undegraded":"2024-04-27T03:34:02.819584+0000","last_fullsized":"2024-04-27T03:34:02.819584+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:02.819411+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:02.819411+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.819411+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:59.129585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:39.379731+0000","last_change":"2024-04-27T03:34:39.379731+0000","last_active":"2024-04-27T03:34:39.379731+0000","last_peered":"2024-04-27T03:34:39.379731+0000","last_clean":"2024-04-27T03:34:39.379731+0000","last_became_active":"2024-04-27T03:30:11.372583+0000","last_became_peered":"2024-04-27T03:30:11.372583+0000","last_unstale":"2024-04-27T03:34:39.379731+0000","last_undegraded":"2024-04-27T03:34:39.379731+0000","last_fullsized":"2024-04-27T03:34:39.379731+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:39.379616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:39.379616+0000","last_clean_scrub_stamp":"2024-04-27T03:34:39.379616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:03:33.468387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.709320+0000","last_change":"2024-04-27T03:33:51.709320+0000","last_active":"2024-04-27T03:33:51.709320+0000","last_peered":"2024-04-27T03:33:51.709320+0000","last_clean":"2024-04-27T03:33:51.709320+0000","last_became_active":"2024-04-27T03:30:15.382207+0000","last_became_peered":"2024-04-27T03:30:15.382207+0000","last_unstale":"2024-04-27T03:33:51.709320+0000","last_undegraded":"2024-04-27T03:33:51.709320+0000","last_fullsized":"2024-04-27T03:33:51.709320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:51.709192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:51.709192+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.709192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:40.618465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"27'61","reported_seq":132,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.184072+0000","last_change":"2024-04-27T03:34:00.184072+0000","last_active":"2024-04-27T03:34:00.184072+0000","last_peered":"2024-04-27T03:34:00.184072+0000","last_clean":"2024-04-27T03:34:00.184072+0000","last_became_active":"2024-04-27T03:30:17.389242+0000","last_became_peered":"2024-04-27T03:30:17.389242+0000","last_unstale":"2024-04-27T03:34:00.184072+0000","last_undegraded":"2024-04-27T03:34:00.184072+0000","last_fullsized":"2024-04-27T03:34:00.184072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'61","last_scrub_stamp":"2024-04-27T03:34:00.183892+0000","last_deep_scrub":"27'61","last_deep_scrub_stamp":"2024-04-27T03:34:00.183892+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.183892+0000","objects_scrubbed":9,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:30:17.176389+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":62,"num_read_kb":51,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"27'285","reported_seq":361,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.734598+0000","last_change":"2024-04-27T03:33:38.734598+0000","last_active":"2024-04-27T03:33:38.734598+0000","last_peered":"2024-04-27T03:33:38.734598+0000","last_clean":"2024-04-27T03:33:38.734598+0000","last_became_active":"2024-04-27T03:30:19.406364+0000","last_became_peered":"2024-04-27T03:30:19.406364+0000","last_unstale":"2024-04-27T03:33:38.734598+0000","last_undegraded":"2024-04-27T03:33:38.734598+0000","last_fullsized":"2024-04-27T03:33:38.734598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:38.734486+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:38.734486+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.734486+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:11.608629+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:27.705542+0000","last_change":"2024-04-27T03:34:27.705542+0000","last_active":"2024-04-27T03:34:27.705542+0000","last_peered":"2024-04-27T03:34:27.705542+0000","last_clean":"2024-04-27T03:34:27.705542+0000","last_became_active":"2024-04-27T03:30:21.408168+0000","last_became_peered":"2024-04-27T03:30:21.408168+0000","last_unstale":"2024-04-27T03:34:27.705542+0000","last_undegraded":"2024-04-27T03:34:27.705542+0000","last_fullsized":"2024-04-27T03:34:27.705542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:27.705421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:27.705421+0000","last_clean_scrub_stamp":"2024-04-27T03:34:27.705421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:10.961869+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"27'14","reported_seq":111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.819144+0000","last_change":"2024-04-27T03:33:55.819144+0000","last_active":"2024-04-27T03:33:55.819144+0000","last_peered":"2024-04-27T03:33:55.819144+0000","last_clean":"2024-04-27T03:33:55.819144+0000","last_became_active":"2024-04-27T03:30:13.371503+0000","last_became_peered":"2024-04-27T03:30:13.371503+0000","last_unstale":"2024-04-27T03:33:55.819144+0000","last_undegraded":"2024-04-27T03:33:55.819144+0000","last_fullsized":"2024-04-27T03:33:55.819144+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:33:55.819026+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:33:55.819026+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.819026+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:53.861728+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:38.360071+0000","last_change":"2024-04-27T03:34:38.360071+0000","last_active":"2024-04-27T03:34:38.360071+0000","last_peered":"2024-04-27T03:34:38.360071+0000","last_clean":"2024-04-27T03:34:38.360071+0000","last_became_active":"2024-04-27T03:30:11.356739+0000","last_became_peered":"2024-04-27T03:30:11.356739+0000","last_unstale":"2024-04-27T03:34:38.360071+0000","last_undegraded":"2024-04-27T03:34:38.360071+0000","last_fullsized":"2024-04-27T03:34:38.360071+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:38.359934+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:38.359934+0000","last_clean_scrub_stamp":"2024-04-27T03:34:38.359934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:24:19.694594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:23.927184+0000","last_change":"2024-04-27T03:34:23.927207+0000","last_active":"2024-04-27T03:34:23.927184+0000","last_peered":"2024-04-27T03:34:23.927184+0000","last_clean":"2024-04-27T03:34:23.927184+0000","last_became_active":"2024-04-27T03:31:01.443118+0000","last_became_peered":"2024-04-27T03:31:01.443118+0000","last_unstale":"2024-04-27T03:34:23.927184+0000","last_undegraded":"2024-04-27T03:34:23.927184+0000","last_fullsized":"2024-04-27T03:34:23.927184+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:23.927098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:23.927098+0000","last_clean_scrub_stamp":"2024-04-27T03:34:23.927098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:25:26.728092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"27'79","reported_seq":165,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.115644+0000","last_change":"2024-04-27T03:33:56.115644+0000","last_active":"2024-04-27T03:33:56.115644+0000","last_peered":"2024-04-27T03:33:56.115644+0000","last_clean":"2024-04-27T03:33:56.115644+0000","last_became_active":"2024-04-27T03:30:17.387318+0000","last_became_peered":"2024-04-27T03:30:17.387318+0000","last_unstale":"2024-04-27T03:33:56.115644+0000","last_undegraded":"2024-04-27T03:33:56.115644+0000","last_fullsized":"2024-04-27T03:33:56.115644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'79","last_scrub_stamp":"2024-04-27T03:33:56.115468+0000","last_deep_scrub":"27'79","last_deep_scrub_stamp":"2024-04-27T03:33:56.115468+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.115468+0000","objects_scrubbed":7,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:06.434627+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":79,"num_read_kb":68,"num_write":34,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.701877+0000","last_change":"2024-04-27T03:33:37.701877+0000","last_active":"2024-04-27T03:33:37.701877+0000","last_peered":"2024-04-27T03:33:37.701877+0000","last_clean":"2024-04-27T03:33:37.701877+0000","last_became_active":"2024-04-27T03:30:19.404265+0000","last_became_peered":"2024-04-27T03:30:19.404265+0000","last_unstale":"2024-04-27T03:33:37.701877+0000","last_undegraded":"2024-04-27T03:33:37.701877+0000","last_fullsized":"2024-04-27T03:33:37.701877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:37.701748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:37.701748+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.701748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:42.526067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"27'21","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:29.689488+0000","last_change":"2024-04-27T03:34:29.689488+0000","last_active":"2024-04-27T03:34:29.689488+0000","last_peered":"2024-04-27T03:34:29.689488+0000","last_clean":"2024-04-27T03:34:29.689488+0000","last_became_active":"2024-04-27T03:30:21.413757+0000","last_became_peered":"2024-04-27T03:30:21.413757+0000","last_unstale":"2024-04-27T03:34:29.689488+0000","last_undegraded":"2024-04-27T03:34:29.689488+0000","last_fullsized":"2024-04-27T03:34:29.689488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-04-27T03:34:29.689358+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-04-27T03:34:29.689358+0000","last_clean_scrub_stamp":"2024-04-27T03:34:29.689358+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:17:30.208007+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"27'4","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:11.168986+0000","last_change":"2024-04-27T03:34:11.168986+0000","last_active":"2024-04-27T03:34:11.168986+0000","last_peered":"2024-04-27T03:34:11.168986+0000","last_clean":"2024-04-27T03:34:11.168986+0000","last_became_active":"2024-04-27T03:30:13.379861+0000","last_became_peered":"2024-04-27T03:30:13.379861+0000","last_unstale":"2024-04-27T03:34:11.168986+0000","last_undegraded":"2024-04-27T03:34:11.168986+0000","last_fullsized":"2024-04-27T03:34:11.168986+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:11.168855+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:11.168855+0000","last_clean_scrub_stamp":"2024-04-27T03:34:11.168855+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:36.750674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:18.628832+0000","last_change":"2024-04-27T03:34:18.628832+0000","last_active":"2024-04-27T03:34:18.628832+0000","last_peered":"2024-04-27T03:34:18.628832+0000","last_clean":"2024-04-27T03:34:18.628832+0000","last_became_active":"2024-04-27T03:30:11.357176+0000","last_became_peered":"2024-04-27T03:30:11.357176+0000","last_unstale":"2024-04-27T03:34:18.628832+0000","last_undegraded":"2024-04-27T03:34:18.628832+0000","last_fullsized":"2024-04-27T03:34:18.628832+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:18.628698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:18.628698+0000","last_clean_scrub_stamp":"2024-04-27T03:34:18.628698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:24.899427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.754119+0000","last_change":"2024-04-27T03:33:49.754119+0000","last_active":"2024-04-27T03:33:49.754119+0000","last_peered":"2024-04-27T03:33:49.754119+0000","last_clean":"2024-04-27T03:33:49.754119+0000","last_became_active":"2024-04-27T03:30:15.382290+0000","last_became_peered":"2024-04-27T03:30:15.382290+0000","last_unstale":"2024-04-27T03:33:49.754119+0000","last_undegraded":"2024-04-27T03:33:49.754119+0000","last_fullsized":"2024-04-27T03:33:49.754119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:49.754004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:49.754004+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.754004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:57.154906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"27'46","reported_seq":109,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.971630+0000","last_change":"2024-04-27T03:33:41.971630+0000","last_active":"2024-04-27T03:33:41.971630+0000","last_peered":"2024-04-27T03:33:41.971630+0000","last_clean":"2024-04-27T03:33:41.971630+0000","last_became_active":"2024-04-27T03:30:17.385902+0000","last_became_peered":"2024-04-27T03:30:17.385902+0000","last_unstale":"2024-04-27T03:33:41.971630+0000","last_undegraded":"2024-04-27T03:33:41.971630+0000","last_fullsized":"2024-04-27T03:33:41.971630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:41.971499+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:41.971499+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.971499+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:48.621217+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.094322+0000","last_change":"2024-04-27T03:33:55.094322+0000","last_active":"2024-04-27T03:33:55.094322+0000","last_peered":"2024-04-27T03:33:55.094322+0000","last_clean":"2024-04-27T03:33:55.094322+0000","last_became_active":"2024-04-27T03:30:19.405450+0000","last_became_peered":"2024-04-27T03:30:19.405450+0000","last_unstale":"2024-04-27T03:33:55.094322+0000","last_undegraded":"2024-04-27T03:33:55.094322+0000","last_fullsized":"2024-04-27T03:33:55.094322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:55.094207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:55.094207+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.094207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:33.101519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"27'9","reported_seq":28,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.757256+0000","last_change":"2024-04-27T03:30:21.415884+0000","last_active":"2024-04-27T03:33:31.757256+0000","last_peered":"2024-04-27T03:33:31.757256+0000","last_clean":"2024-04-27T03:33:31.757256+0000","last_became_active":"2024-04-27T03:30:21.415494+0000","last_became_peered":"2024-04-27T03:30:21.415494+0000","last_unstale":"2024-04-27T03:33:31.757256+0000","last_undegraded":"2024-04-27T03:33:31.757256+0000","last_fullsized":"2024-04-27T03:33:31.757256+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'6","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.201236+0000","last_change":"2024-04-27T03:33:48.201236+0000","last_active":"2024-04-27T03:33:48.201236+0000","last_peered":"2024-04-27T03:33:48.201236+0000","last_clean":"2024-04-27T03:33:48.201236+0000","last_became_active":"2024-04-27T03:30:13.387533+0000","last_became_peered":"2024-04-27T03:30:13.387533+0000","last_unstale":"2024-04-27T03:33:48.201236+0000","last_undegraded":"2024-04-27T03:33:48.201236+0000","last_fullsized":"2024-04-27T03:33:48.201236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:48.201118+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:48.201118+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.201118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:48.546003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:22.293400+0000","last_change":"2024-04-27T03:34:22.293400+0000","last_active":"2024-04-27T03:34:22.293400+0000","last_peered":"2024-04-27T03:34:22.293400+0000","last_clean":"2024-04-27T03:34:22.293400+0000","last_became_active":"2024-04-27T03:30:11.373440+0000","last_became_peered":"2024-04-27T03:30:11.373440+0000","last_unstale":"2024-04-27T03:34:22.293400+0000","last_undegraded":"2024-04-27T03:34:22.293400+0000","last_fullsized":"2024-04-27T03:34:22.293400+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:22.293281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:22.293281+0000","last_clean_scrub_stamp":"2024-04-27T03:34:22.293281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:13:00.638888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.731024+0000","last_change":"2024-04-27T03:33:48.731024+0000","last_active":"2024-04-27T03:33:48.731024+0000","last_peered":"2024-04-27T03:33:48.731024+0000","last_clean":"2024-04-27T03:33:48.731024+0000","last_became_active":"2024-04-27T03:30:15.382913+0000","last_became_peered":"2024-04-27T03:30:15.382913+0000","last_unstale":"2024-04-27T03:33:48.731024+0000","last_undegraded":"2024-04-27T03:33:48.731024+0000","last_fullsized":"2024-04-27T03:33:48.731024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.730891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.730891+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.730891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:49.302463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"27'45","reported_seq":104,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.809220+0000","last_change":"2024-04-27T03:33:45.809220+0000","last_active":"2024-04-27T03:33:45.809220+0000","last_peered":"2024-04-27T03:33:45.809220+0000","last_clean":"2024-04-27T03:33:45.809220+0000","last_became_active":"2024-04-27T03:30:17.381106+0000","last_became_peered":"2024-04-27T03:30:17.381106+0000","last_unstale":"2024-04-27T03:33:45.809220+0000","last_undegraded":"2024-04-27T03:33:45.809220+0000","last_fullsized":"2024-04-27T03:33:45.809220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'45","last_scrub_stamp":"2024-04-27T03:33:45.809061+0000","last_deep_scrub":"27'45","last_deep_scrub_stamp":"2024-04-27T03:33:45.809061+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.809061+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:33.888060+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"27'285","reported_seq":416,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.864480+0000","last_change":"2024-04-27T03:33:31.864480+0000","last_active":"2024-04-27T03:33:31.864480+0000","last_peered":"2024-04-27T03:33:31.864480+0000","last_clean":"2024-04-27T03:33:31.864480+0000","last_became_active":"2024-04-27T03:30:19.403413+0000","last_became_peered":"2024-04-27T03:30:19.403413+0000","last_unstale":"2024-04-27T03:33:31.864480+0000","last_undegraded":"2024-04-27T03:33:31.864480+0000","last_fullsized":"2024-04-27T03:33:31.864480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:31.864348+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:31.864348+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.864348+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:27.874206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"27'15","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.705571+0000","last_change":"2024-04-27T03:33:54.705571+0000","last_active":"2024-04-27T03:33:54.705571+0000","last_peered":"2024-04-27T03:33:54.705571+0000","last_clean":"2024-04-27T03:33:54.705571+0000","last_became_active":"2024-04-27T03:30:21.402102+0000","last_became_peered":"2024-04-27T03:30:21.402102+0000","last_unstale":"2024-04-27T03:33:54.705571+0000","last_undegraded":"2024-04-27T03:33:54.705571+0000","last_fullsized":"2024-04-27T03:33:54.705571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-04-27T03:33:54.705410+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-04-27T03:33:54.705410+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.705410+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:00.713477+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":23,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'12","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.884586+0000","last_change":"2024-04-27T03:34:08.884586+0000","last_active":"2024-04-27T03:34:08.884586+0000","last_peered":"2024-04-27T03:34:08.884586+0000","last_clean":"2024-04-27T03:34:08.884586+0000","last_became_active":"2024-04-27T03:30:13.382899+0000","last_became_peered":"2024-04-27T03:30:13.382899+0000","last_unstale":"2024-04-27T03:34:08.884586+0000","last_undegraded":"2024-04-27T03:34:08.884586+0000","last_fullsized":"2024-04-27T03:34:08.884586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-04-27T03:34:08.884470+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-04-27T03:34:08.884470+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.884470+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:46.729084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'4","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:34.054235+0000","last_change":"2024-04-27T03:34:34.054287+0000","last_active":"2024-04-27T03:34:34.054235+0000","last_peered":"2024-04-27T03:34:34.054235+0000","last_clean":"2024-04-27T03:34:34.054235+0000","last_became_active":"2024-04-27T03:31:01.439776+0000","last_became_peered":"2024-04-27T03:31:01.439776+0000","last_unstale":"2024-04-27T03:34:34.054235+0000","last_undegraded":"2024-04-27T03:34:34.054235+0000","last_fullsized":"2024-04-27T03:34:34.054235+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:34.054128+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:34.054128+0000","last_clean_scrub_stamp":"2024-04-27T03:34:34.054128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:49:23.829672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"27'5","reported_seq":429,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.950380+0000","last_change":"2024-04-27T03:33:43.950380+0000","last_active":"2024-04-27T03:33:43.950380+0000","last_peered":"2024-04-27T03:33:43.950380+0000","last_clean":"2024-04-27T03:33:43.950380+0000","last_became_active":"2024-04-27T03:30:15.382502+0000","last_became_peered":"2024-04-27T03:30:15.382502+0000","last_unstale":"2024-04-27T03:33:43.950380+0000","last_undegraded":"2024-04-27T03:33:43.950380+0000","last_fullsized":"2024-04-27T03:33:43.950380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:43.950258+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:43.950258+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.950258+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:48.355472+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"27'15","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.197757+0000","last_change":"2024-04-27T03:33:58.197757+0000","last_active":"2024-04-27T03:33:58.197757+0000","last_peered":"2024-04-27T03:33:58.197757+0000","last_clean":"2024-04-27T03:33:58.197757+0000","last_became_active":"2024-04-27T03:30:17.387739+0000","last_became_peered":"2024-04-27T03:30:17.387739+0000","last_unstale":"2024-04-27T03:33:58.197757+0000","last_undegraded":"2024-04-27T03:33:58.197757+0000","last_fullsized":"2024-04-27T03:33:58.197757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-04-27T03:33:58.197595+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-04-27T03:33:58.197595+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.197595+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:01.324323+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"27'285","reported_seq":523,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.933929+0000","last_change":"2024-04-27T03:33:32.933929+0000","last_active":"2024-04-27T03:33:32.933929+0000","last_peered":"2024-04-27T03:33:32.933929+0000","last_clean":"2024-04-27T03:33:32.933929+0000","last_became_active":"2024-04-27T03:30:19.402484+0000","last_became_peered":"2024-04-27T03:30:19.402484+0000","last_unstale":"2024-04-27T03:33:32.933929+0000","last_undegraded":"2024-04-27T03:33:32.933929+0000","last_fullsized":"2024-04-27T03:33:32.933929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:32.933800+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:32.933800+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.933800+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:19.924355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"27'4","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.778552+0000","last_change":"2024-04-27T03:33:52.778552+0000","last_active":"2024-04-27T03:33:52.778552+0000","last_peered":"2024-04-27T03:33:52.778552+0000","last_clean":"2024-04-27T03:33:52.778552+0000","last_became_active":"2024-04-27T03:30:21.408512+0000","last_became_peered":"2024-04-27T03:30:21.408512+0000","last_unstale":"2024-04-27T03:33:52.778552+0000","last_undegraded":"2024-04-27T03:33:52.778552+0000","last_fullsized":"2024-04-27T03:33:52.778552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:52.778420+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:52.778420+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.778420+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:57.183634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'14","reported_seq":80,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.153985+0000","last_change":"2024-04-27T03:34:02.153985+0000","last_active":"2024-04-27T03:34:02.153985+0000","last_peered":"2024-04-27T03:34:02.153985+0000","last_clean":"2024-04-27T03:34:02.153985+0000","last_became_active":"2024-04-27T03:30:13.380540+0000","last_became_peered":"2024-04-27T03:30:13.380540+0000","last_unstale":"2024-04-27T03:34:02.153985+0000","last_undegraded":"2024-04-27T03:34:02.153985+0000","last_fullsized":"2024-04-27T03:34:02.153985+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:34:02.153863+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:34:02.153863+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.153863+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:47.534998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:33.346199+0000","last_change":"2024-04-27T03:34:33.346199+0000","last_active":"2024-04-27T03:34:33.346199+0000","last_peered":"2024-04-27T03:34:33.346199+0000","last_clean":"2024-04-27T03:34:33.346199+0000","last_became_active":"2024-04-27T03:30:11.372193+0000","last_became_peered":"2024-04-27T03:30:11.372193+0000","last_unstale":"2024-04-27T03:34:33.346199+0000","last_undegraded":"2024-04-27T03:34:33.346199+0000","last_fullsized":"2024-04-27T03:34:33.346199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:33.346065+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:33.346065+0000","last_clean_scrub_stamp":"2024-04-27T03:34:33.346065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:54.581545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.115491+0000","last_change":"2024-04-27T03:33:35.115491+0000","last_active":"2024-04-27T03:33:35.115491+0000","last_peered":"2024-04-27T03:33:35.115491+0000","last_clean":"2024-04-27T03:33:35.115491+0000","last_became_active":"2024-04-27T03:30:15.383924+0000","last_became_peered":"2024-04-27T03:30:15.383924+0000","last_unstale":"2024-04-27T03:33:35.115491+0000","last_undegraded":"2024-04-27T03:33:35.115491+0000","last_fullsized":"2024-04-27T03:33:35.115491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:35.115383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:35.115383+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.115383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:37.564842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"27'48","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.883223+0000","last_change":"2024-04-27T03:33:38.883223+0000","last_active":"2024-04-27T03:33:38.883223+0000","last_peered":"2024-04-27T03:33:38.883223+0000","last_clean":"2024-04-27T03:33:38.883223+0000","last_became_active":"2024-04-27T03:30:17.387578+0000","last_became_peered":"2024-04-27T03:30:17.387578+0000","last_unstale":"2024-04-27T03:33:38.883223+0000","last_undegraded":"2024-04-27T03:33:38.883223+0000","last_fullsized":"2024-04-27T03:33:38.883223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-04-27T03:33:38.883110+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-04-27T03:33:38.883110+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.883110+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:11.757254+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.856137+0000","last_change":"2024-04-27T03:33:32.856137+0000","last_active":"2024-04-27T03:33:32.856137+0000","last_peered":"2024-04-27T03:33:32.856137+0000","last_clean":"2024-04-27T03:33:32.856137+0000","last_became_active":"2024-04-27T03:30:19.405512+0000","last_became_peered":"2024-04-27T03:30:19.405512+0000","last_unstale":"2024-04-27T03:33:32.856137+0000","last_undegraded":"2024-04-27T03:33:32.856137+0000","last_fullsized":"2024-04-27T03:33:32.856137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:32.856020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:32.856020+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.856020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:34.194673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"27'14","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.990573+0000","last_change":"2024-04-27T03:33:48.990573+0000","last_active":"2024-04-27T03:33:48.990573+0000","last_peered":"2024-04-27T03:33:48.990573+0000","last_clean":"2024-04-27T03:33:48.990573+0000","last_became_active":"2024-04-27T03:30:21.406883+0000","last_became_peered":"2024-04-27T03:30:21.406883+0000","last_unstale":"2024-04-27T03:33:48.990573+0000","last_undegraded":"2024-04-27T03:33:48.990573+0000","last_fullsized":"2024-04-27T03:33:48.990573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:33:48.990468+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:33:48.990468+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.990468+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:14:34.099716+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":59,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'20","reported_seq":89,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.148613+0000","last_change":"2024-04-27T03:33:52.148613+0000","last_active":"2024-04-27T03:33:52.148613+0000","last_peered":"2024-04-27T03:33:52.148613+0000","last_clean":"2024-04-27T03:33:52.148613+0000","last_became_active":"2024-04-27T03:30:13.387180+0000","last_became_peered":"2024-04-27T03:30:13.387180+0000","last_unstale":"2024-04-27T03:33:52.148613+0000","last_undegraded":"2024-04-27T03:33:52.148613+0000","last_fullsized":"2024-04-27T03:33:52.148613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-04-27T03:33:52.148491+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-04-27T03:33:52.148491+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.148491+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:24:54.949485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":205,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:19.121523+0000","last_change":"2024-04-27T03:34:19.121563+0000","last_active":"2024-04-27T03:34:19.121523+0000","last_peered":"2024-04-27T03:34:19.121523+0000","last_clean":"2024-04-27T03:34:19.121523+0000","last_became_active":"2024-04-27T03:30:06.102893+0000","last_became_peered":"2024-04-27T03:30:06.102893+0000","last_unstale":"2024-04-27T03:34:19.121523+0000","last_undegraded":"2024-04-27T03:34:19.121523+0000","last_fullsized":"2024-04-27T03:34:19.121523+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-04-27T03:34:19.121425+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-04-27T03:34:19.121425+0000","last_clean_scrub_stamp":"2024-04-27T03:34:19.121425+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:22.248153+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:42.415967+0000","last_change":"2024-04-27T03:34:42.416007+0000","last_active":"2024-04-27T03:34:42.415967+0000","last_peered":"2024-04-27T03:34:42.415967+0000","last_clean":"2024-04-27T03:34:42.415967+0000","last_became_active":"2024-04-27T03:30:06.106511+0000","last_became_peered":"2024-04-27T03:30:06.106511+0000","last_unstale":"2024-04-27T03:34:42.415967+0000","last_undegraded":"2024-04-27T03:34:42.415967+0000","last_fullsized":"2024-04-27T03:34:42.415967+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:42.415847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:42.415847+0000","last_clean_scrub_stamp":"2024-04-27T03:34:42.415847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:33.081995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:28.327417+0000","last_change":"2024-04-27T03:34:28.327451+0000","last_active":"2024-04-27T03:34:28.327417+0000","last_peered":"2024-04-27T03:34:28.327417+0000","last_clean":"2024-04-27T03:34:28.327417+0000","last_became_active":"2024-04-27T03:30:11.362445+0000","last_became_peered":"2024-04-27T03:30:11.362445+0000","last_unstale":"2024-04-27T03:34:28.327417+0000","last_undegraded":"2024-04-27T03:34:28.327417+0000","last_fullsized":"2024-04-27T03:34:28.327417+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:28.327321+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:28.327321+0000","last_clean_scrub_stamp":"2024-04-27T03:34:28.327321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:37.622290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:33.076789+0000","last_change":"2024-04-27T03:34:33.076831+0000","last_active":"2024-04-27T03:34:33.076789+0000","last_peered":"2024-04-27T03:34:33.076789+0000","last_clean":"2024-04-27T03:34:33.076789+0000","last_became_active":"2024-04-27T03:31:01.440678+0000","last_became_peered":"2024-04-27T03:31:01.440678+0000","last_unstale":"2024-04-27T03:34:33.076789+0000","last_undegraded":"2024-04-27T03:34:33.076789+0000","last_fullsized":"2024-04-27T03:34:33.076789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:33.076696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:33.076696+0000","last_clean_scrub_stamp":"2024-04-27T03:34:33.076696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:00:34.478658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"27'95","reported_seq":192,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.176369+0000","last_change":"2024-04-27T03:33:45.176369+0000","last_active":"2024-04-27T03:33:45.176369+0000","last_peered":"2024-04-27T03:33:45.176369+0000","last_clean":"2024-04-27T03:33:45.176369+0000","last_became_active":"2024-04-27T03:30:17.387707+0000","last_became_peered":"2024-04-27T03:30:17.387707+0000","last_unstale":"2024-04-27T03:33:45.176369+0000","last_undegraded":"2024-04-27T03:33:45.176369+0000","last_fullsized":"2024-04-27T03:33:45.176369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'95","last_scrub_stamp":"2024-04-27T03:33:45.176195+0000","last_deep_scrub":"27'95","last_deep_scrub_stamp":"2024-04-27T03:33:45.176195+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.176195+0000","objects_scrubbed":6,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:45.184076+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":93,"num_read_kb":81,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.779891+0000","last_change":"2024-04-27T03:33:31.779891+0000","last_active":"2024-04-27T03:33:31.779891+0000","last_peered":"2024-04-27T03:33:31.779891+0000","last_clean":"2024-04-27T03:33:31.779891+0000","last_became_active":"2024-04-27T03:30:19.405359+0000","last_became_peered":"2024-04-27T03:30:19.405359+0000","last_unstale":"2024-04-27T03:33:31.779891+0000","last_undegraded":"2024-04-27T03:33:31.779891+0000","last_fullsized":"2024-04-27T03:33:31.779891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:31.779786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:31.779786+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.779786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:45.852820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"27'81","reported_seq":172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.838907+0000","last_change":"2024-04-27T03:33:44.838907+0000","last_active":"2024-04-27T03:33:44.838907+0000","last_peered":"2024-04-27T03:33:44.838907+0000","last_clean":"2024-04-27T03:33:44.838907+0000","last_became_active":"2024-04-27T03:30:21.404300+0000","last_became_peered":"2024-04-27T03:30:21.404300+0000","last_unstale":"2024-04-27T03:33:44.838907+0000","last_undegraded":"2024-04-27T03:33:44.838907+0000","last_fullsized":"2024-04-27T03:33:44.838907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'81","last_scrub_stamp":"2024-04-27T03:33:44.838760+0000","last_deep_scrub":"27'81","last_deep_scrub_stamp":"2024-04-27T03:33:44.838760+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.838760+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:18.504969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"25'2","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:10.245276+0000","last_change":"2024-04-27T03:34:10.245276+0000","last_active":"2024-04-27T03:34:10.245276+0000","last_peered":"2024-04-27T03:34:10.245276+0000","last_clean":"2024-04-27T03:34:10.245276+0000","last_became_active":"2024-04-27T03:30:13.377713+0000","last_became_peered":"2024-04-27T03:30:13.377713+0000","last_unstale":"2024-04-27T03:34:10.245276+0000","last_undegraded":"2024-04-27T03:34:10.245276+0000","last_fullsized":"2024-04-27T03:34:10.245276+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:10.245096+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:10.245096+0000","last_clean_scrub_stamp":"2024-04-27T03:34:10.245096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:32.051573+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:43.412993+0000","last_change":"2024-04-27T03:34:43.413018+0000","last_active":"2024-04-27T03:34:43.412993+0000","last_peered":"2024-04-27T03:34:43.412993+0000","last_clean":"2024-04-27T03:34:43.412993+0000","last_became_active":"2024-04-27T03:30:06.106474+0000","last_became_peered":"2024-04-27T03:30:06.106474+0000","last_unstale":"2024-04-27T03:34:43.412993+0000","last_undegraded":"2024-04-27T03:34:43.412993+0000","last_fullsized":"2024-04-27T03:34:43.412993+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:43.412878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:43.412878+0000","last_clean_scrub_stamp":"2024-04-27T03:34:43.412878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:59:27.850579+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:20.655637+0000","last_change":"2024-04-27T03:34:20.655637+0000","last_active":"2024-04-27T03:34:20.655637+0000","last_peered":"2024-04-27T03:34:20.655637+0000","last_clean":"2024-04-27T03:34:20.655637+0000","last_became_active":"2024-04-27T03:30:11.360244+0000","last_became_peered":"2024-04-27T03:30:11.360244+0000","last_unstale":"2024-04-27T03:34:20.655637+0000","last_undegraded":"2024-04-27T03:34:20.655637+0000","last_fullsized":"2024-04-27T03:34:20.655637+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:20.655487+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:20.655487+0000","last_clean_scrub_stamp":"2024-04-27T03:34:20.655487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:53:31.032767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.098651+0000","last_change":"2024-04-27T03:33:42.098651+0000","last_active":"2024-04-27T03:33:42.098651+0000","last_peered":"2024-04-27T03:33:42.098651+0000","last_clean":"2024-04-27T03:33:42.098651+0000","last_became_active":"2024-04-27T03:30:15.379448+0000","last_became_peered":"2024-04-27T03:30:15.379448+0000","last_unstale":"2024-04-27T03:33:42.098651+0000","last_undegraded":"2024-04-27T03:33:42.098651+0000","last_fullsized":"2024-04-27T03:33:42.098651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.098519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.098519+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.098519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:34:23.337627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"27'297","reported_seq":529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.877345+0000","last_change":"2024-04-27T03:33:35.877345+0000","last_active":"2024-04-27T03:33:35.877345+0000","last_peered":"2024-04-27T03:33:35.877345+0000","last_clean":"2024-04-27T03:33:35.877345+0000","last_became_active":"2024-04-27T03:30:17.390884+0000","last_became_peered":"2024-04-27T03:30:17.390884+0000","last_unstale":"2024-04-27T03:33:35.877345+0000","last_undegraded":"2024-04-27T03:33:35.877345+0000","last_fullsized":"2024-04-27T03:33:35.877345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'297","last_scrub_stamp":"2024-04-27T03:33:35.877210+0000","last_deep_scrub":"27'297","last_deep_scrub_stamp":"2024-04-27T03:33:35.877210+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.877210+0000","objects_scrubbed":11,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:23.956209+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":387,"num_read_kb":371,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:11.241230+0000","last_change":"2024-04-27T03:34:11.241230+0000","last_active":"2024-04-27T03:34:11.241230+0000","last_peered":"2024-04-27T03:34:11.241230+0000","last_clean":"2024-04-27T03:34:11.241230+0000","last_became_active":"2024-04-27T03:30:19.405777+0000","last_became_peered":"2024-04-27T03:30:19.405777+0000","last_unstale":"2024-04-27T03:34:11.241230+0000","last_undegraded":"2024-04-27T03:34:11.241230+0000","last_fullsized":"2024-04-27T03:34:11.241230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:11.241087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:11.241087+0000","last_clean_scrub_stamp":"2024-04-27T03:34:11.241087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:19.362648+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"27'7","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.973335+0000","last_change":"2024-04-27T03:33:52.973335+0000","last_active":"2024-04-27T03:33:52.973335+0000","last_peered":"2024-04-27T03:33:52.973335+0000","last_clean":"2024-04-27T03:33:52.973335+0000","last_became_active":"2024-04-27T03:30:21.409359+0000","last_became_peered":"2024-04-27T03:30:21.409359+0000","last_unstale":"2024-04-27T03:33:52.973335+0000","last_undegraded":"2024-04-27T03:33:52.973335+0000","last_fullsized":"2024-04-27T03:33:52.973335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:33:52.973192+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:33:52.973192+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.973192+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:49.283366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"25'10","reported_seq":82,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.632293+0000","last_change":"2024-04-27T03:33:57.632293+0000","last_active":"2024-04-27T03:33:57.632293+0000","last_peered":"2024-04-27T03:33:57.632293+0000","last_clean":"2024-04-27T03:33:57.632293+0000","last_became_active":"2024-04-27T03:30:13.385521+0000","last_became_peered":"2024-04-27T03:30:13.385521+0000","last_unstale":"2024-04-27T03:33:57.632293+0000","last_undegraded":"2024-04-27T03:33:57.632293+0000","last_fullsized":"2024-04-27T03:33:57.632293+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'10","last_scrub_stamp":"2024-04-27T03:33:57.632133+0000","last_deep_scrub":"25'10","last_deep_scrub_stamp":"2024-04-27T03:33:57.632133+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.632133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:37.434045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:10.860062+0000","last_change":"2024-04-27T03:34:10.860062+0000","last_active":"2024-04-27T03:34:10.860062+0000","last_peered":"2024-04-27T03:34:10.860062+0000","last_clean":"2024-04-27T03:34:10.860062+0000","last_became_active":"2024-04-27T03:30:06.103441+0000","last_became_peered":"2024-04-27T03:30:06.103441+0000","last_unstale":"2024-04-27T03:34:10.860062+0000","last_undegraded":"2024-04-27T03:34:10.860062+0000","last_fullsized":"2024-04-27T03:34:10.860062+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:10.859905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:10.859905+0000","last_clean_scrub_stamp":"2024-04-27T03:34:10.859905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:58.011918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:21.646692+0000","last_change":"2024-04-27T03:34:21.646749+0000","last_active":"2024-04-27T03:34:21.646692+0000","last_peered":"2024-04-27T03:34:21.646692+0000","last_clean":"2024-04-27T03:34:21.646692+0000","last_became_active":"2024-04-27T03:30:11.360443+0000","last_became_peered":"2024-04-27T03:30:11.360443+0000","last_unstale":"2024-04-27T03:34:21.646692+0000","last_undegraded":"2024-04-27T03:34:21.646692+0000","last_fullsized":"2024-04-27T03:34:21.646692+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:21.646559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:21.646559+0000","last_clean_scrub_stamp":"2024-04-27T03:34:21.646559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:30:38.639056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.772935+0000","last_change":"2024-04-27T03:33:46.772935+0000","last_active":"2024-04-27T03:33:46.772935+0000","last_peered":"2024-04-27T03:33:46.772935+0000","last_clean":"2024-04-27T03:33:46.772935+0000","last_became_active":"2024-04-27T03:30:15.383213+0000","last_became_peered":"2024-04-27T03:30:15.383213+0000","last_unstale":"2024-04-27T03:33:46.772935+0000","last_undegraded":"2024-04-27T03:33:46.772935+0000","last_fullsized":"2024-04-27T03:33:46.772935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:46.772765+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:46.772765+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.772765+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:24.897665+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"27'23","reported_seq":101,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.859533+0000","last_change":"2024-04-27T03:33:42.859533+0000","last_active":"2024-04-27T03:33:42.859533+0000","last_peered":"2024-04-27T03:33:42.859533+0000","last_clean":"2024-04-27T03:33:42.859533+0000","last_became_active":"2024-04-27T03:30:17.388588+0000","last_became_peered":"2024-04-27T03:30:17.388588+0000","last_unstale":"2024-04-27T03:33:42.859533+0000","last_undegraded":"2024-04-27T03:33:42.859533+0000","last_fullsized":"2024-04-27T03:33:42.859533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-04-27T03:33:42.859384+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-04-27T03:33:42.859384+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.859384+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:29.849938+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.729982+0000","last_change":"2024-04-27T03:33:46.729982+0000","last_active":"2024-04-27T03:33:46.729982+0000","last_peered":"2024-04-27T03:33:46.729982+0000","last_clean":"2024-04-27T03:33:46.729982+0000","last_became_active":"2024-04-27T03:30:19.407004+0000","last_became_peered":"2024-04-27T03:30:19.407004+0000","last_unstale":"2024-04-27T03:33:46.729982+0000","last_undegraded":"2024-04-27T03:33:46.729982+0000","last_fullsized":"2024-04-27T03:33:46.729982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:46.729851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:46.729851+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.729851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:36.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"25'2","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:39.297632+0000","last_change":"2024-04-27T03:30:21.417022+0000","last_active":"2024-04-27T03:32:39.297632+0000","last_peered":"2024-04-27T03:32:39.297632+0000","last_clean":"2024-04-27T03:32:39.297632+0000","last_became_active":"2024-04-27T03:30:21.416389+0000","last_became_peered":"2024-04-27T03:30:21.416389+0000","last_unstale":"2024-04-27T03:32:39.297632+0000","last_undegraded":"2024-04-27T03:32:39.297632+0000","last_fullsized":"2024-04-27T03:32:39.297632+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'8","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.675570+0000","last_change":"2024-04-27T03:34:00.675570+0000","last_active":"2024-04-27T03:34:00.675570+0000","last_peered":"2024-04-27T03:34:00.675570+0000","last_clean":"2024-04-27T03:34:00.675570+0000","last_became_active":"2024-04-27T03:30:13.369638+0000","last_became_peered":"2024-04-27T03:30:13.369638+0000","last_unstale":"2024-04-27T03:34:00.675570+0000","last_undegraded":"2024-04-27T03:34:00.675570+0000","last_fullsized":"2024-04-27T03:34:00.675570+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-04-27T03:34:00.675392+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-04-27T03:34:00.675392+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.675392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:47.827405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:44.424819+0000","last_change":"2024-04-27T03:34:44.424842+0000","last_active":"2024-04-27T03:34:44.424819+0000","last_peered":"2024-04-27T03:34:44.424819+0000","last_clean":"2024-04-27T03:34:44.424819+0000","last_became_active":"2024-04-27T03:30:06.107356+0000","last_became_peered":"2024-04-27T03:30:06.107356+0000","last_unstale":"2024-04-27T03:34:44.424819+0000","last_undegraded":"2024-04-27T03:34:44.424819+0000","last_fullsized":"2024-04-27T03:34:44.424819+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:44.424721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:44.424721+0000","last_clean_scrub_stamp":"2024-04-27T03:34:44.424721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:19:05.627341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:11.852227+0000","last_change":"2024-04-27T03:34:11.852227+0000","last_active":"2024-04-27T03:34:11.852227+0000","last_peered":"2024-04-27T03:34:11.852227+0000","last_clean":"2024-04-27T03:34:11.852227+0000","last_became_active":"2024-04-27T03:30:11.356685+0000","last_became_peered":"2024-04-27T03:30:11.356685+0000","last_unstale":"2024-04-27T03:34:11.852227+0000","last_undegraded":"2024-04-27T03:34:11.852227+0000","last_fullsized":"2024-04-27T03:34:11.852227+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:11.852112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:11.852112+0000","last_clean_scrub_stamp":"2024-04-27T03:34:11.852112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:57.233248+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.706693+0000","last_change":"2024-04-27T03:33:49.706693+0000","last_active":"2024-04-27T03:33:49.706693+0000","last_peered":"2024-04-27T03:33:49.706693+0000","last_clean":"2024-04-27T03:33:49.706693+0000","last_became_active":"2024-04-27T03:30:15.382326+0000","last_became_peered":"2024-04-27T03:30:15.382326+0000","last_unstale":"2024-04-27T03:33:49.706693+0000","last_undegraded":"2024-04-27T03:33:49.706693+0000","last_fullsized":"2024-04-27T03:33:49.706693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:49.706574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:49.706574+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.706574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:45.223380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"27'20","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.220511+0000","last_change":"2024-04-27T03:34:01.220511+0000","last_active":"2024-04-27T03:34:01.220511+0000","last_peered":"2024-04-27T03:34:01.220511+0000","last_clean":"2024-04-27T03:34:01.220511+0000","last_became_active":"2024-04-27T03:30:17.389373+0000","last_became_peered":"2024-04-27T03:30:17.389373+0000","last_unstale":"2024-04-27T03:34:01.220511+0000","last_undegraded":"2024-04-27T03:34:01.220511+0000","last_fullsized":"2024-04-27T03:34:01.220511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-04-27T03:34:01.220376+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-04-27T03:34:01.220376+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.220376+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:49.020148+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":350,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.297531+0000","last_change":"2024-04-27T03:34:07.297531+0000","last_active":"2024-04-27T03:34:07.297531+0000","last_peered":"2024-04-27T03:34:07.297531+0000","last_clean":"2024-04-27T03:34:07.297531+0000","last_became_active":"2024-04-27T03:30:19.405953+0000","last_became_peered":"2024-04-27T03:30:19.405953+0000","last_unstale":"2024-04-27T03:34:07.297531+0000","last_undegraded":"2024-04-27T03:34:07.297531+0000","last_fullsized":"2024-04-27T03:34:07.297531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:07.297411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:07.297411+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.297411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:38.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"27'7","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:28.029576+0000","last_change":"2024-04-27T03:34:28.029576+0000","last_active":"2024-04-27T03:34:28.029576+0000","last_peered":"2024-04-27T03:34:28.029576+0000","last_clean":"2024-04-27T03:34:28.029576+0000","last_became_active":"2024-04-27T03:30:21.412790+0000","last_became_peered":"2024-04-27T03:30:21.412790+0000","last_unstale":"2024-04-27T03:34:28.029576+0000","last_undegraded":"2024-04-27T03:34:28.029576+0000","last_fullsized":"2024-04-27T03:34:28.029576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:34:28.029410+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:34:28.029410+0000","last_clean_scrub_stamp":"2024-04-27T03:34:28.029410+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:10:15.508139+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'18","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.201621+0000","last_change":"2024-04-27T03:34:00.201621+0000","last_active":"2024-04-27T03:34:00.201621+0000","last_peered":"2024-04-27T03:34:00.201621+0000","last_clean":"2024-04-27T03:34:00.201621+0000","last_became_active":"2024-04-27T03:30:13.380888+0000","last_became_peered":"2024-04-27T03:30:13.380888+0000","last_unstale":"2024-04-27T03:34:00.201621+0000","last_undegraded":"2024-04-27T03:34:00.201621+0000","last_fullsized":"2024-04-27T03:34:00.201621+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-04-27T03:34:00.201503+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-04-27T03:34:00.201503+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.201503+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:13:15.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":73,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.878323+0000","last_change":"2024-04-27T03:34:09.878323+0000","last_active":"2024-04-27T03:34:09.878323+0000","last_peered":"2024-04-27T03:34:09.878323+0000","last_clean":"2024-04-27T03:34:09.878323+0000","last_became_active":"2024-04-27T03:30:11.361786+0000","last_became_peered":"2024-04-27T03:30:11.361786+0000","last_unstale":"2024-04-27T03:34:09.878323+0000","last_undegraded":"2024-04-27T03:34:09.878323+0000","last_fullsized":"2024-04-27T03:34:09.878323+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:09.878163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:09.878163+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.878163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:13:25.595476+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:17.617492+0000","last_change":"2024-04-27T03:34:17.617492+0000","last_active":"2024-04-27T03:34:17.617492+0000","last_peered":"2024-04-27T03:34:17.617492+0000","last_clean":"2024-04-27T03:34:17.617492+0000","last_became_active":"2024-04-27T03:30:06.107594+0000","last_became_peered":"2024-04-27T03:30:06.107594+0000","last_unstale":"2024-04-27T03:34:17.617492+0000","last_undegraded":"2024-04-27T03:34:17.617492+0000","last_fullsized":"2024-04-27T03:34:17.617492+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:17.617335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:17.617335+0000","last_clean_scrub_stamp":"2024-04-27T03:34:17.617335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:27.936494+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"27'2","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:53.688679+0000","last_change":"2024-04-27T03:33:53.688679+0000","last_active":"2024-04-27T03:33:53.688679+0000","last_peered":"2024-04-27T03:33:53.688679+0000","last_clean":"2024-04-27T03:33:53.688679+0000","last_became_active":"2024-04-27T03:30:15.381392+0000","last_became_peered":"2024-04-27T03:30:15.381392+0000","last_unstale":"2024-04-27T03:33:53.688679+0000","last_undegraded":"2024-04-27T03:33:53.688679+0000","last_fullsized":"2024-04-27T03:33:53.688679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:53.688558+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:53.688558+0000","last_clean_scrub_stamp":"2024-04-27T03:33:53.688558+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:08:53.046929+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"27'87","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.739991+0000","last_change":"2024-04-27T03:33:39.739991+0000","last_active":"2024-04-27T03:33:39.739991+0000","last_peered":"2024-04-27T03:33:39.739991+0000","last_clean":"2024-04-27T03:33:39.739991+0000","last_became_active":"2024-04-27T03:30:17.390409+0000","last_became_peered":"2024-04-27T03:30:17.390409+0000","last_unstale":"2024-04-27T03:33:39.739991+0000","last_undegraded":"2024-04-27T03:33:39.739991+0000","last_fullsized":"2024-04-27T03:33:39.739991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'87","last_scrub_stamp":"2024-04-27T03:33:39.739797+0000","last_deep_scrub":"27'87","last_deep_scrub_stamp":"2024-04-27T03:33:39.739797+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.739797+0000","objects_scrubbed":11,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:27:02.606163+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":89,"num_read_kb":75,"num_write":42,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":292,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.814624+0000","last_change":"2024-04-27T03:33:33.814624+0000","last_active":"2024-04-27T03:33:33.814624+0000","last_peered":"2024-04-27T03:33:33.814624+0000","last_clean":"2024-04-27T03:33:33.814624+0000","last_became_active":"2024-04-27T03:30:19.402977+0000","last_became_peered":"2024-04-27T03:30:19.402977+0000","last_unstale":"2024-04-27T03:33:33.814624+0000","last_undegraded":"2024-04-27T03:33:33.814624+0000","last_fullsized":"2024-04-27T03:33:33.814624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:33.814493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:33.814493+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.814493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:14:40.755714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"27'11","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:28.666118+0000","last_change":"2024-04-27T03:34:28.666146+0000","last_active":"2024-04-27T03:34:28.666118+0000","last_peered":"2024-04-27T03:34:28.666118+0000","last_clean":"2024-04-27T03:34:28.666118+0000","last_became_active":"2024-04-27T03:30:21.410511+0000","last_became_peered":"2024-04-27T03:30:21.410511+0000","last_unstale":"2024-04-27T03:34:28.666118+0000","last_undegraded":"2024-04-27T03:34:28.666118+0000","last_fullsized":"2024-04-27T03:34:28.666118+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-04-27T03:34:28.665996+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-04-27T03:34:28.665996+0000","last_clean_scrub_stamp":"2024-04-27T03:34:28.665996+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:00.104554+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"27'4","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:13.162744+0000","last_change":"2024-04-27T03:34:13.162744+0000","last_active":"2024-04-27T03:34:13.162744+0000","last_peered":"2024-04-27T03:34:13.162744+0000","last_clean":"2024-04-27T03:34:13.162744+0000","last_became_active":"2024-04-27T03:30:13.379985+0000","last_became_peered":"2024-04-27T03:30:13.379985+0000","last_unstale":"2024-04-27T03:34:13.162744+0000","last_undegraded":"2024-04-27T03:34:13.162744+0000","last_fullsized":"2024-04-27T03:34:13.162744+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:13.162618+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:13.162618+0000","last_clean_scrub_stamp":"2024-04-27T03:34:13.162618+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:56.317292+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:13.657044+0000","last_change":"2024-04-27T03:34:13.657044+0000","last_active":"2024-04-27T03:34:13.657044+0000","last_peered":"2024-04-27T03:34:13.657044+0000","last_clean":"2024-04-27T03:34:13.657044+0000","last_became_active":"2024-04-27T03:30:11.360352+0000","last_became_peered":"2024-04-27T03:30:11.360352+0000","last_unstale":"2024-04-27T03:34:13.657044+0000","last_undegraded":"2024-04-27T03:34:13.657044+0000","last_fullsized":"2024-04-27T03:34:13.657044+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:13.656944+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:13.656944+0000","last_clean_scrub_stamp":"2024-04-27T03:34:13.656944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:25:16.457938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:41.401887+0000","last_change":"2024-04-27T03:34:41.401910+0000","last_active":"2024-04-27T03:34:41.401887+0000","last_peered":"2024-04-27T03:34:41.401887+0000","last_clean":"2024-04-27T03:34:41.401887+0000","last_became_active":"2024-04-27T03:30:06.106475+0000","last_became_peered":"2024-04-27T03:30:06.106475+0000","last_unstale":"2024-04-27T03:34:41.401887+0000","last_undegraded":"2024-04-27T03:34:41.401887+0000","last_fullsized":"2024-04-27T03:34:41.401887+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:41.401790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:41.401790+0000","last_clean_scrub_stamp":"2024-04-27T03:34:41.401790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:15.329806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.089706+0000","last_change":"2024-04-27T03:33:41.089706+0000","last_active":"2024-04-27T03:33:41.089706+0000","last_peered":"2024-04-27T03:33:41.089706+0000","last_clean":"2024-04-27T03:33:41.089706+0000","last_became_active":"2024-04-27T03:30:15.380030+0000","last_became_peered":"2024-04-27T03:30:15.380030+0000","last_unstale":"2024-04-27T03:33:41.089706+0000","last_undegraded":"2024-04-27T03:33:41.089706+0000","last_fullsized":"2024-04-27T03:33:41.089706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:41.089578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:41.089578+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.089578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:36.584947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"27'28","reported_seq":76,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.880920+0000","last_change":"2024-04-27T03:33:34.880920+0000","last_active":"2024-04-27T03:33:34.880920+0000","last_peered":"2024-04-27T03:33:34.880920+0000","last_clean":"2024-04-27T03:33:34.880920+0000","last_became_active":"2024-04-27T03:30:17.393253+0000","last_became_peered":"2024-04-27T03:30:17.393253+0000","last_unstale":"2024-04-27T03:33:34.880920+0000","last_undegraded":"2024-04-27T03:33:34.880920+0000","last_fullsized":"2024-04-27T03:33:34.880920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-04-27T03:33:34.880792+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-04-27T03:33:34.880792+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.880792+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:08.547001+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"27'285","reported_seq":399,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.921440+0000","last_change":"2024-04-27T03:33:33.921440+0000","last_active":"2024-04-27T03:33:33.921440+0000","last_peered":"2024-04-27T03:33:33.921440+0000","last_clean":"2024-04-27T03:33:33.921440+0000","last_became_active":"2024-04-27T03:30:19.402319+0000","last_became_peered":"2024-04-27T03:30:19.402319+0000","last_unstale":"2024-04-27T03:33:33.921440+0000","last_undegraded":"2024-04-27T03:33:33.921440+0000","last_fullsized":"2024-04-27T03:33:33.921440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:33.921301+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:33.921301+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.921301+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:42.843568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:15.236481+0000","last_change":"2024-04-27T03:34:15.236528+0000","last_active":"2024-04-27T03:34:15.236481+0000","last_peered":"2024-04-27T03:34:15.236481+0000","last_clean":"2024-04-27T03:34:15.236481+0000","last_became_active":"2024-04-27T03:30:21.414109+0000","last_became_peered":"2024-04-27T03:30:21.414109+0000","last_unstale":"2024-04-27T03:34:15.236481+0000","last_undegraded":"2024-04-27T03:34:15.236481+0000","last_fullsized":"2024-04-27T03:34:15.236481+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:15.236366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:15.236366+0000","last_clean_scrub_stamp":"2024-04-27T03:34:15.236366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:40.927977+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"25'2","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.192314+0000","last_change":"2024-04-27T03:33:46.192314+0000","last_active":"2024-04-27T03:33:46.192314+0000","last_peered":"2024-04-27T03:33:46.192314+0000","last_clean":"2024-04-27T03:33:46.192314+0000","last_became_active":"2024-04-27T03:30:13.387764+0000","last_became_peered":"2024-04-27T03:30:13.387764+0000","last_unstale":"2024-04-27T03:33:46.192314+0000","last_undegraded":"2024-04-27T03:33:46.192314+0000","last_fullsized":"2024-04-27T03:33:46.192314+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:33:46.192183+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:33:46.192183+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.192183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:21:54.976582+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"27'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:40.401033+0000","last_change":"2024-04-27T03:34:40.401060+0000","last_active":"2024-04-27T03:34:40.401033+0000","last_peered":"2024-04-27T03:34:40.401033+0000","last_clean":"2024-04-27T03:34:40.401033+0000","last_became_active":"2024-04-27T03:30:11.355704+0000","last_became_peered":"2024-04-27T03:30:11.355704+0000","last_unstale":"2024-04-27T03:34:40.401033+0000","last_undegraded":"2024-04-27T03:34:40.401033+0000","last_fullsized":"2024-04-27T03:34:40.401033+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:40.400920+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:40.400920+0000","last_clean_scrub_stamp":"2024-04-27T03:34:40.400920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:24.872425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:16.597669+0000","last_change":"2024-04-27T03:34:16.597669+0000","last_active":"2024-04-27T03:34:16.597669+0000","last_peered":"2024-04-27T03:34:16.597669+0000","last_clean":"2024-04-27T03:34:16.597669+0000","last_became_active":"2024-04-27T03:30:06.103759+0000","last_became_peered":"2024-04-27T03:30:06.103759+0000","last_unstale":"2024-04-27T03:34:16.597669+0000","last_undegraded":"2024-04-27T03:34:16.597669+0000","last_fullsized":"2024-04-27T03:34:16.597669+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:16.597501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:16.597501+0000","last_clean_scrub_stamp":"2024-04-27T03:34:16.597501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:54.604812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"27'5","reported_seq":239,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.180895+0000","last_change":"2024-04-27T03:33:32.180895+0000","last_active":"2024-04-27T03:33:32.180895+0000","last_peered":"2024-04-27T03:33:32.180895+0000","last_clean":"2024-04-27T03:33:32.180895+0000","last_became_active":"2024-04-27T03:30:15.384372+0000","last_became_peered":"2024-04-27T03:30:15.384372+0000","last_unstale":"2024-04-27T03:33:32.180895+0000","last_undegraded":"2024-04-27T03:33:32.180895+0000","last_fullsized":"2024-04-27T03:33:32.180895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:32.180778+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:32.180778+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.180778+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:01:22.337645+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'26","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.104813+0000","last_change":"2024-04-27T03:33:51.104813+0000","last_active":"2024-04-27T03:33:51.104813+0000","last_peered":"2024-04-27T03:33:51.104813+0000","last_clean":"2024-04-27T03:33:51.104813+0000","last_became_active":"2024-04-27T03:30:17.389154+0000","last_became_peered":"2024-04-27T03:30:17.389154+0000","last_unstale":"2024-04-27T03:33:51.104813+0000","last_undegraded":"2024-04-27T03:33:51.104813+0000","last_fullsized":"2024-04-27T03:33:51.104813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-04-27T03:33:51.104672+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-04-27T03:33:51.104672+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.104672+0000","objects_scrubbed":6,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:09.096082+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"27'285","reported_seq":411,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.780327+0000","last_change":"2024-04-27T03:33:33.780327+0000","last_active":"2024-04-27T03:33:33.780327+0000","last_peered":"2024-04-27T03:33:33.780327+0000","last_clean":"2024-04-27T03:33:33.780327+0000","last_became_active":"2024-04-27T03:30:19.404867+0000","last_became_peered":"2024-04-27T03:30:19.404867+0000","last_unstale":"2024-04-27T03:33:33.780327+0000","last_undegraded":"2024-04-27T03:33:33.780327+0000","last_fullsized":"2024-04-27T03:33:33.780327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:33.780200+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:33.780200+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.780200+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:42.702468+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"25'3","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:29.070874+0000","last_change":"2024-04-27T03:34:29.070899+0000","last_active":"2024-04-27T03:34:29.070874+0000","last_peered":"2024-04-27T03:34:29.070874+0000","last_clean":"2024-04-27T03:34:29.070874+0000","last_became_active":"2024-04-27T03:30:21.413264+0000","last_became_peered":"2024-04-27T03:30:21.413264+0000","last_unstale":"2024-04-27T03:34:29.070874+0000","last_undegraded":"2024-04-27T03:34:29.070874+0000","last_fullsized":"2024-04-27T03:34:29.070874+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'3","last_scrub_stamp":"2024-04-27T03:34:29.070741+0000","last_deep_scrub":"25'3","last_deep_scrub_stamp":"2024-04-27T03:34:29.070741+0000","last_clean_scrub_stamp":"2024-04-27T03:34:29.070741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:05:54.556761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"27'4","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:18.852257+0000","last_change":"2024-04-27T03:34:18.852257+0000","last_active":"2024-04-27T03:34:18.852257+0000","last_peered":"2024-04-27T03:34:18.852257+0000","last_clean":"2024-04-27T03:34:18.852257+0000","last_became_active":"2024-04-27T03:30:13.383208+0000","last_became_peered":"2024-04-27T03:30:13.383208+0000","last_unstale":"2024-04-27T03:34:18.852257+0000","last_undegraded":"2024-04-27T03:34:18.852257+0000","last_fullsized":"2024-04-27T03:34:18.852257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:18.852125+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:18.852125+0000","last_clean_scrub_stamp":"2024-04-27T03:34:18.852125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:50.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:18.128080+0000","last_change":"2024-04-27T03:34:18.128080+0000","last_active":"2024-04-27T03:34:18.128080+0000","last_peered":"2024-04-27T03:34:18.128080+0000","last_clean":"2024-04-27T03:34:18.128080+0000","last_became_active":"2024-04-27T03:30:11.369568+0000","last_became_peered":"2024-04-27T03:30:11.369568+0000","last_unstale":"2024-04-27T03:34:18.128080+0000","last_undegraded":"2024-04-27T03:34:18.128080+0000","last_fullsized":"2024-04-27T03:34:18.128080+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:18.127955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:18.127955+0000","last_clean_scrub_stamp":"2024-04-27T03:34:18.127955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:40.539908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:45.394350+0000","last_change":"2024-04-27T03:34:45.394377+0000","last_active":"2024-04-27T03:34:45.394350+0000","last_peered":"2024-04-27T03:34:45.394350+0000","last_clean":"2024-04-27T03:34:45.394350+0000","last_became_active":"2024-04-27T03:30:06.107600+0000","last_became_peered":"2024-04-27T03:30:06.107600+0000","last_unstale":"2024-04-27T03:34:45.394350+0000","last_undegraded":"2024-04-27T03:34:45.394350+0000","last_fullsized":"2024-04-27T03:34:45.394350+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:45.394262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:45.394262+0000","last_clean_scrub_stamp":"2024-04-27T03:34:45.394262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:26.023255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.124794+0000","last_change":"2024-04-27T03:33:40.124794+0000","last_active":"2024-04-27T03:33:40.124794+0000","last_peered":"2024-04-27T03:33:40.124794+0000","last_clean":"2024-04-27T03:33:40.124794+0000","last_became_active":"2024-04-27T03:30:15.380598+0000","last_became_peered":"2024-04-27T03:30:15.380598+0000","last_unstale":"2024-04-27T03:33:40.124794+0000","last_undegraded":"2024-04-27T03:33:40.124794+0000","last_fullsized":"2024-04-27T03:33:40.124794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:40.124684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:40.124684+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.124684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:12.385924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"27'104","reported_seq":204,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.834973+0000","last_change":"2024-04-27T03:33:39.834973+0000","last_active":"2024-04-27T03:33:39.834973+0000","last_peered":"2024-04-27T03:33:39.834973+0000","last_clean":"2024-04-27T03:33:39.834973+0000","last_became_active":"2024-04-27T03:30:17.391191+0000","last_became_peered":"2024-04-27T03:30:17.391191+0000","last_unstale":"2024-04-27T03:33:39.834973+0000","last_undegraded":"2024-04-27T03:33:39.834973+0000","last_fullsized":"2024-04-27T03:33:39.834973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'104","last_scrub_stamp":"2024-04-27T03:33:39.834804+0000","last_deep_scrub":"27'104","last_deep_scrub_stamp":"2024-04-27T03:33:39.834804+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.834804+0000","objects_scrubbed":9,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:46.431245+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":102,"num_read_kb":87,"num_write":40,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.744784+0000","last_change":"2024-04-27T03:33:36.744784+0000","last_active":"2024-04-27T03:33:36.744784+0000","last_peered":"2024-04-27T03:33:36.744784+0000","last_clean":"2024-04-27T03:33:36.744784+0000","last_became_active":"2024-04-27T03:30:19.404187+0000","last_became_peered":"2024-04-27T03:30:19.404187+0000","last_unstale":"2024-04-27T03:33:36.744784+0000","last_undegraded":"2024-04-27T03:33:36.744784+0000","last_fullsized":"2024-04-27T03:33:36.744784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.744626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.744626+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.744626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:14.869527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.769625+0000","last_change":"2024-04-27T03:33:48.769648+0000","last_active":"2024-04-27T03:33:48.769625+0000","last_peered":"2024-04-27T03:33:48.769625+0000","last_clean":"2024-04-27T03:33:48.769625+0000","last_became_active":"2024-04-27T03:30:21.407857+0000","last_became_peered":"2024-04-27T03:30:21.407857+0000","last_unstale":"2024-04-27T03:33:48.769625+0000","last_undegraded":"2024-04-27T03:33:48.769625+0000","last_fullsized":"2024-04-27T03:33:48.769625+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.769538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.769538+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.769538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:21.643681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"25'2","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:20.938358+0000","last_change":"2024-04-27T03:34:20.938358+0000","last_active":"2024-04-27T03:34:20.938358+0000","last_peered":"2024-04-27T03:34:20.938358+0000","last_clean":"2024-04-27T03:34:20.938358+0000","last_became_active":"2024-04-27T03:30:21.411054+0000","last_became_peered":"2024-04-27T03:30:21.411054+0000","last_unstale":"2024-04-27T03:34:20.938358+0000","last_undegraded":"2024-04-27T03:34:20.938358+0000","last_fullsized":"2024-04-27T03:34:20.938358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:20.938237+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:20.938237+0000","last_clean_scrub_stamp":"2024-04-27T03:34:20.938237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:13:39.173999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"27'6","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.813031+0000","last_change":"2024-04-27T03:34:06.813056+0000","last_active":"2024-04-27T03:34:06.813031+0000","last_peered":"2024-04-27T03:34:06.813031+0000","last_clean":"2024-04-27T03:34:06.813031+0000","last_became_active":"2024-04-27T03:30:13.382675+0000","last_became_peered":"2024-04-27T03:30:13.382675+0000","last_unstale":"2024-04-27T03:34:06.813031+0000","last_undegraded":"2024-04-27T03:34:06.813031+0000","last_fullsized":"2024-04-27T03:34:06.813031+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:06.812933+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:06.812933+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.812933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:10.931563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:34.339293+0000","last_change":"2024-04-27T03:34:34.339362+0000","last_active":"2024-04-27T03:34:34.339293+0000","last_peered":"2024-04-27T03:34:34.339293+0000","last_clean":"2024-04-27T03:34:34.339293+0000","last_became_active":"2024-04-27T03:30:11.359195+0000","last_became_peered":"2024-04-27T03:30:11.359195+0000","last_unstale":"2024-04-27T03:34:34.339293+0000","last_undegraded":"2024-04-27T03:34:34.339293+0000","last_fullsized":"2024-04-27T03:34:34.339293+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:34.339184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:34.339184+0000","last_clean_scrub_stamp":"2024-04-27T03:34:34.339184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:46.068821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.994229+0000","last_change":"2024-04-27T03:33:50.994275+0000","last_active":"2024-04-27T03:33:50.994229+0000","last_peered":"2024-04-27T03:33:50.994229+0000","last_clean":"2024-04-27T03:33:50.994229+0000","last_became_active":"2024-04-27T03:30:15.379541+0000","last_became_peered":"2024-04-27T03:30:15.379541+0000","last_unstale":"2024-04-27T03:33:50.994229+0000","last_undegraded":"2024-04-27T03:33:50.994229+0000","last_fullsized":"2024-04-27T03:33:50.994229+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:50.994123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:50.994123+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.994123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:51.335268+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"27'46","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.177061+0000","last_change":"2024-04-27T03:33:37.177105+0000","last_active":"2024-04-27T03:33:37.177061+0000","last_peered":"2024-04-27T03:33:37.177061+0000","last_clean":"2024-04-27T03:33:37.177061+0000","last_became_active":"2024-04-27T03:30:17.386496+0000","last_became_peered":"2024-04-27T03:30:17.386496+0000","last_unstale":"2024-04-27T03:33:37.177061+0000","last_undegraded":"2024-04-27T03:33:37.177061+0000","last_fullsized":"2024-04-27T03:33:37.177061+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:37.176970+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:37.176970+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.176970+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:15.021583+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:53.149637+0000","last_change":"2024-04-27T03:33:53.149659+0000","last_active":"2024-04-27T03:33:53.149637+0000","last_peered":"2024-04-27T03:33:53.149637+0000","last_clean":"2024-04-27T03:33:53.149637+0000","last_became_active":"2024-04-27T03:30:19.405189+0000","last_became_peered":"2024-04-27T03:30:19.405189+0000","last_unstale":"2024-04-27T03:33:53.149637+0000","last_undegraded":"2024-04-27T03:33:53.149637+0000","last_fullsized":"2024-04-27T03:33:53.149637+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:53.149540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:53.149540+0000","last_clean_scrub_stamp":"2024-04-27T03:33:53.149540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:17:47.405355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"27'3","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:30.679029+0000","last_change":"2024-04-27T03:34:30.679053+0000","last_active":"2024-04-27T03:34:30.679029+0000","last_peered":"2024-04-27T03:34:30.679029+0000","last_clean":"2024-04-27T03:34:30.679029+0000","last_became_active":"2024-04-27T03:30:21.413920+0000","last_became_peered":"2024-04-27T03:30:21.413920+0000","last_unstale":"2024-04-27T03:34:30.679029+0000","last_undegraded":"2024-04-27T03:34:30.679029+0000","last_fullsized":"2024-04-27T03:34:30.679029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-27T03:34:30.678930+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-27T03:34:30.678930+0000","last_clean_scrub_stamp":"2024-04-27T03:34:30.678930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:50:07.631541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"27'16","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:27.036989+0000","last_change":"2024-04-27T03:34:27.037018+0000","last_active":"2024-04-27T03:34:27.036989+0000","last_peered":"2024-04-27T03:34:27.036989+0000","last_clean":"2024-04-27T03:34:27.036989+0000","last_became_active":"2024-04-27T03:30:13.380240+0000","last_became_peered":"2024-04-27T03:30:13.380240+0000","last_unstale":"2024-04-27T03:34:27.036989+0000","last_undegraded":"2024-04-27T03:34:27.036989+0000","last_fullsized":"2024-04-27T03:34:27.036989+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:27.036880+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:27.036880+0000","last_clean_scrub_stamp":"2024-04-27T03:34:27.036880+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:55:53.303121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":31,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"25'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:16.846118+0000","last_change":"2024-04-27T03:34:16.846162+0000","last_active":"2024-04-27T03:34:16.846118+0000","last_peered":"2024-04-27T03:34:16.846118+0000","last_clean":"2024-04-27T03:34:16.846118+0000","last_became_active":"2024-04-27T03:30:11.360217+0000","last_became_peered":"2024-04-27T03:30:11.360217+0000","last_unstale":"2024-04-27T03:34:16.846118+0000","last_undegraded":"2024-04-27T03:34:16.846118+0000","last_fullsized":"2024-04-27T03:34:16.846118+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:16.846006+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:16.846006+0000","last_clean_scrub_stamp":"2024-04-27T03:34:16.846006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:54:35.473558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.125251+0000","last_change":"2024-04-27T03:33:36.125275+0000","last_active":"2024-04-27T03:33:36.125251+0000","last_peered":"2024-04-27T03:33:36.125251+0000","last_clean":"2024-04-27T03:33:36.125251+0000","last_became_active":"2024-04-27T03:30:15.381507+0000","last_became_peered":"2024-04-27T03:30:15.381507+0000","last_unstale":"2024-04-27T03:33:36.125251+0000","last_undegraded":"2024-04-27T03:33:36.125251+0000","last_fullsized":"2024-04-27T03:33:36.125251+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.125165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.125165+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.125165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:43.471223+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"27'9","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.702947+0000","last_change":"2024-04-27T03:33:44.703001+0000","last_active":"2024-04-27T03:33:44.702947+0000","last_peered":"2024-04-27T03:33:44.702947+0000","last_clean":"2024-04-27T03:33:44.702947+0000","last_became_active":"2024-04-27T03:30:17.385318+0000","last_became_peered":"2024-04-27T03:30:17.385318+0000","last_unstale":"2024-04-27T03:33:44.702947+0000","last_undegraded":"2024-04-27T03:33:44.702947+0000","last_fullsized":"2024-04-27T03:33:44.702947+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-04-27T03:33:44.702827+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-04-27T03:33:44.702827+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.702827+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:55.754975+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.873119+0000","last_change":"2024-04-27T03:33:36.873142+0000","last_active":"2024-04-27T03:33:36.873119+0000","last_peered":"2024-04-27T03:33:36.873119+0000","last_clean":"2024-04-27T03:33:36.873119+0000","last_became_active":"2024-04-27T03:30:19.406736+0000","last_became_peered":"2024-04-27T03:30:19.406736+0000","last_unstale":"2024-04-27T03:33:36.873119+0000","last_undegraded":"2024-04-27T03:33:36.873119+0000","last_fullsized":"2024-04-27T03:33:36.873119+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.873031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.873031+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.873031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:57.751682+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:26.716914+0000","last_change":"2024-04-27T03:34:26.716984+0000","last_active":"2024-04-27T03:34:26.716914+0000","last_peered":"2024-04-27T03:34:26.716914+0000","last_clean":"2024-04-27T03:34:26.716914+0000","last_became_active":"2024-04-27T03:30:21.408079+0000","last_became_peered":"2024-04-27T03:30:21.408079+0000","last_unstale":"2024-04-27T03:34:26.716914+0000","last_undegraded":"2024-04-27T03:34:26.716914+0000","last_fullsized":"2024-04-27T03:34:26.716914+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:26.716795+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:26.716795+0000","last_clean_scrub_stamp":"2024-04-27T03:34:26.716795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:21:21.432858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"27'4","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:12.698736+0000","last_change":"2024-04-27T03:34:12.698757+0000","last_active":"2024-04-27T03:34:12.698736+0000","last_peered":"2024-04-27T03:34:12.698736+0000","last_clean":"2024-04-27T03:34:12.698736+0000","last_became_active":"2024-04-27T03:30:13.375350+0000","last_became_peered":"2024-04-27T03:30:13.375350+0000","last_unstale":"2024-04-27T03:34:12.698736+0000","last_undegraded":"2024-04-27T03:34:12.698736+0000","last_fullsized":"2024-04-27T03:34:12.698736+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:12.698671+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:12.698671+0000","last_clean_scrub_stamp":"2024-04-27T03:34:12.698671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:30.690081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:17.813624+0000","last_change":"2024-04-27T03:34:17.813663+0000","last_active":"2024-04-27T03:34:17.813624+0000","last_peered":"2024-04-27T03:34:17.813624+0000","last_clean":"2024-04-27T03:34:17.813624+0000","last_became_active":"2024-04-27T03:30:11.360938+0000","last_became_peered":"2024-04-27T03:30:11.360938+0000","last_unstale":"2024-04-27T03:34:17.813624+0000","last_undegraded":"2024-04-27T03:34:17.813624+0000","last_fullsized":"2024-04-27T03:34:17.813624+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:17.813510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:17.813510+0000","last_clean_scrub_stamp":"2024-04-27T03:34:17.813510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:25.064062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.767320+0000","last_change":"2024-04-27T03:33:47.767359+0000","last_active":"2024-04-27T03:33:47.767320+0000","last_peered":"2024-04-27T03:33:47.767320+0000","last_clean":"2024-04-27T03:33:47.767320+0000","last_became_active":"2024-04-27T03:30:15.382287+0000","last_became_peered":"2024-04-27T03:30:15.382287+0000","last_unstale":"2024-04-27T03:33:47.767320+0000","last_undegraded":"2024-04-27T03:33:47.767320+0000","last_fullsized":"2024-04-27T03:33:47.767320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:47.767216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:47.767216+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.767216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:52.591536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"27'47","reported_seq":119,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.768186+0000","last_change":"2024-04-27T03:33:41.768233+0000","last_active":"2024-04-27T03:33:41.768186+0000","last_peered":"2024-04-27T03:33:41.768186+0000","last_clean":"2024-04-27T03:33:41.768186+0000","last_became_active":"2024-04-27T03:30:17.388260+0000","last_became_peered":"2024-04-27T03:30:17.388260+0000","last_unstale":"2024-04-27T03:33:41.768186+0000","last_undegraded":"2024-04-27T03:33:41.768186+0000","last_fullsized":"2024-04-27T03:33:41.768186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-04-27T03:33:41.768105+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-04-27T03:33:41.768105+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.768105+0000","objects_scrubbed":6,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:48.417823+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":662,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.927890+0000","last_change":"2024-04-27T03:33:31.927916+0000","last_active":"2024-04-27T03:33:31.927890+0000","last_peered":"2024-04-27T03:33:31.927890+0000","last_clean":"2024-04-27T03:33:31.927890+0000","last_became_active":"2024-04-27T03:30:19.402430+0000","last_became_peered":"2024-04-27T03:30:19.402430+0000","last_unstale":"2024-04-27T03:33:31.927890+0000","last_undegraded":"2024-04-27T03:33:31.927890+0000","last_fullsized":"2024-04-27T03:33:31.927890+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:31.927798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:31.927798+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.927798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:36.369347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:54.300287+0000","last_change":"2024-04-27T03:30:21.416316+0000","last_active":"2024-04-27T03:32:54.300287+0000","last_peered":"2024-04-27T03:32:54.300287+0000","last_clean":"2024-04-27T03:32:54.300287+0000","last_became_active":"2024-04-27T03:30:21.415984+0000","last_became_peered":"2024-04-27T03:30:21.415984+0000","last_unstale":"2024-04-27T03:32:54.300287+0000","last_undegraded":"2024-04-27T03:32:54.300287+0000","last_fullsized":"2024-04-27T03:32:54.300287+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"27'5","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.144432+0000","last_change":"2024-04-27T03:33:50.144460+0000","last_active":"2024-04-27T03:33:50.144432+0000","last_peered":"2024-04-27T03:33:50.144432+0000","last_clean":"2024-04-27T03:33:50.144432+0000","last_became_active":"2024-04-27T03:30:13.386985+0000","last_became_peered":"2024-04-27T03:30:13.386985+0000","last_unstale":"2024-04-27T03:33:50.144432+0000","last_undegraded":"2024-04-27T03:33:50.144432+0000","last_fullsized":"2024-04-27T03:33:50.144432+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:50.144340+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:50.144340+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.144340+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:15.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:22.086952+0000","last_change":"2024-04-27T03:34:22.087002+0000","last_active":"2024-04-27T03:34:22.086952+0000","last_peered":"2024-04-27T03:34:22.086952+0000","last_clean":"2024-04-27T03:34:22.086952+0000","last_became_active":"2024-04-27T03:30:11.369824+0000","last_became_peered":"2024-04-27T03:30:11.369824+0000","last_unstale":"2024-04-27T03:34:22.086952+0000","last_undegraded":"2024-04-27T03:34:22.086952+0000","last_fullsized":"2024-04-27T03:34:22.086952+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:22.086864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:22.086864+0000","last_clean_scrub_stamp":"2024-04-27T03:34:22.086864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:09.886636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.154995+0000","last_change":"2024-04-27T03:33:34.155037+0000","last_active":"2024-04-27T03:33:34.154995+0000","last_peered":"2024-04-27T03:33:34.154995+0000","last_clean":"2024-04-27T03:33:34.154995+0000","last_became_active":"2024-04-27T03:30:15.383771+0000","last_became_peered":"2024-04-27T03:30:15.383771+0000","last_unstale":"2024-04-27T03:33:34.154995+0000","last_undegraded":"2024-04-27T03:33:34.154995+0000","last_fullsized":"2024-04-27T03:33:34.154995+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.154879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.154879+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.154879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:59.127137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"27'36","reported_seq":91,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.840074+0000","last_change":"2024-04-27T03:33:43.840113+0000","last_active":"2024-04-27T03:33:43.840074+0000","last_peered":"2024-04-27T03:33:43.840074+0000","last_clean":"2024-04-27T03:33:43.840074+0000","last_became_active":"2024-04-27T03:30:17.385488+0000","last_became_peered":"2024-04-27T03:30:17.385488+0000","last_unstale":"2024-04-27T03:33:43.840074+0000","last_undegraded":"2024-04-27T03:33:43.840074+0000","last_fullsized":"2024-04-27T03:33:43.840074+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-04-27T03:33:43.839983+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-04-27T03:33:43.839983+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.839983+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:52.762250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.844652+0000","last_change":"2024-04-27T03:33:35.844680+0000","last_active":"2024-04-27T03:33:35.844652+0000","last_peered":"2024-04-27T03:33:35.844652+0000","last_clean":"2024-04-27T03:33:35.844652+0000","last_became_active":"2024-04-27T03:30:19.398484+0000","last_became_peered":"2024-04-27T03:30:19.398484+0000","last_unstale":"2024-04-27T03:33:35.844652+0000","last_undegraded":"2024-04-27T03:33:35.844652+0000","last_fullsized":"2024-04-27T03:33:35.844652+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:35.844545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:35.844545+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.844545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:17.570301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"25'2","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:14.212832+0000","last_change":"2024-04-27T03:34:14.212832+0000","last_active":"2024-04-27T03:34:14.212832+0000","last_peered":"2024-04-27T03:34:14.212832+0000","last_clean":"2024-04-27T03:34:14.212832+0000","last_became_active":"2024-04-27T03:30:13.377217+0000","last_became_peered":"2024-04-27T03:30:13.377217+0000","last_unstale":"2024-04-27T03:34:14.212832+0000","last_undegraded":"2024-04-27T03:34:14.212832+0000","last_fullsized":"2024-04-27T03:34:14.212832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:14.212665+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:14.212665+0000","last_clean_scrub_stamp":"2024-04-27T03:34:14.212665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:40:43.816639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"25'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:31.330604+0000","last_change":"2024-04-27T03:34:31.330660+0000","last_active":"2024-04-27T03:34:31.330604+0000","last_peered":"2024-04-27T03:34:31.330604+0000","last_clean":"2024-04-27T03:34:31.330604+0000","last_became_active":"2024-04-27T03:30:11.370547+0000","last_became_peered":"2024-04-27T03:30:11.370547+0000","last_unstale":"2024-04-27T03:34:31.330604+0000","last_undegraded":"2024-04-27T03:34:31.330604+0000","last_fullsized":"2024-04-27T03:34:31.330604+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:31.330465+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:31.330465+0000","last_clean_scrub_stamp":"2024-04-27T03:34:31.330465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:57.298339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.700581+0000","last_change":"2024-04-27T03:33:52.700581+0000","last_active":"2024-04-27T03:33:52.700581+0000","last_peered":"2024-04-27T03:33:52.700581+0000","last_clean":"2024-04-27T03:33:52.700581+0000","last_became_active":"2024-04-27T03:30:15.381956+0000","last_became_peered":"2024-04-27T03:30:15.381956+0000","last_unstale":"2024-04-27T03:33:52.700581+0000","last_undegraded":"2024-04-27T03:33:52.700581+0000","last_fullsized":"2024-04-27T03:33:52.700581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:52.700486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:52.700486+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.700486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:49.010661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"27'70","reported_seq":149,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.743749+0000","last_change":"2024-04-27T03:33:45.743793+0000","last_active":"2024-04-27T03:33:45.743749+0000","last_peered":"2024-04-27T03:33:45.743749+0000","last_clean":"2024-04-27T03:33:45.743749+0000","last_became_active":"2024-04-27T03:30:17.383178+0000","last_became_peered":"2024-04-27T03:30:17.383178+0000","last_unstale":"2024-04-27T03:33:45.743749+0000","last_undegraded":"2024-04-27T03:33:45.743749+0000","last_fullsized":"2024-04-27T03:33:45.743749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'70","last_scrub_stamp":"2024-04-27T03:33:45.743668+0000","last_deep_scrub":"27'70","last_deep_scrub_stamp":"2024-04-27T03:33:45.743668+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.743668+0000","objects_scrubbed":6,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:37.293903+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":28,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.812908+0000","last_change":"2024-04-27T03:33:34.812933+0000","last_active":"2024-04-27T03:33:34.812908+0000","last_peered":"2024-04-27T03:33:34.812908+0000","last_clean":"2024-04-27T03:33:34.812908+0000","last_became_active":"2024-04-27T03:30:19.402781+0000","last_became_peered":"2024-04-27T03:30:19.402781+0000","last_unstale":"2024-04-27T03:33:34.812908+0000","last_undegraded":"2024-04-27T03:33:34.812908+0000","last_fullsized":"2024-04-27T03:33:34.812908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.812814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.812814+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.812814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:12.666738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"27'16","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:25.726821+0000","last_change":"2024-04-27T03:34:25.726878+0000","last_active":"2024-04-27T03:34:25.726821+0000","last_peered":"2024-04-27T03:34:25.726821+0000","last_clean":"2024-04-27T03:34:25.726821+0000","last_became_active":"2024-04-27T03:30:21.408049+0000","last_became_peered":"2024-04-27T03:30:21.408049+0000","last_unstale":"2024-04-27T03:34:25.726821+0000","last_undegraded":"2024-04-27T03:34:25.726821+0000","last_fullsized":"2024-04-27T03:34:25.726821+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:25.726721+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:25.726721+0000","last_clean_scrub_stamp":"2024-04-27T03:34:25.726721+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:44:14.582492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":21,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.693523+0000","last_change":"2024-04-27T03:34:08.693523+0000","last_active":"2024-04-27T03:34:08.693523+0000","last_peered":"2024-04-27T03:34:08.693523+0000","last_clean":"2024-04-27T03:34:08.693523+0000","last_became_active":"2024-04-27T03:30:13.373257+0000","last_became_peered":"2024-04-27T03:30:13.373257+0000","last_unstale":"2024-04-27T03:34:08.693523+0000","last_undegraded":"2024-04-27T03:34:08.693523+0000","last_fullsized":"2024-04-27T03:34:08.693523+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:08.693374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:08.693374+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.693374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:22.810525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.222541+0000","last_change":"2024-04-27T03:34:08.222565+0000","last_active":"2024-04-27T03:34:08.222541+0000","last_peered":"2024-04-27T03:34:08.222541+0000","last_clean":"2024-04-27T03:34:08.222541+0000","last_became_active":"2024-04-27T03:30:11.370091+0000","last_became_peered":"2024-04-27T03:30:11.370091+0000","last_unstale":"2024-04-27T03:34:08.222541+0000","last_undegraded":"2024-04-27T03:34:08.222541+0000","last_fullsized":"2024-04-27T03:34:08.222541+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:08.222454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:08.222454+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.222454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:22:17.006853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.756557+0000","last_change":"2024-04-27T03:33:51.756557+0000","last_active":"2024-04-27T03:33:51.756557+0000","last_peered":"2024-04-27T03:33:51.756557+0000","last_clean":"2024-04-27T03:33:51.756557+0000","last_became_active":"2024-04-27T03:30:15.379800+0000","last_became_peered":"2024-04-27T03:30:15.379800+0000","last_unstale":"2024-04-27T03:33:51.756557+0000","last_undegraded":"2024-04-27T03:33:51.756557+0000","last_fullsized":"2024-04-27T03:33:51.756557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:51.756438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:51.756438+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.756438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:14:49.565231+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"27'46","reported_seq":104,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.217186+0000","last_change":"2024-04-27T03:34:02.217229+0000","last_active":"2024-04-27T03:34:02.217186+0000","last_peered":"2024-04-27T03:34:02.217186+0000","last_clean":"2024-04-27T03:34:02.217186+0000","last_became_active":"2024-04-27T03:30:17.388246+0000","last_became_peered":"2024-04-27T03:30:17.388246+0000","last_unstale":"2024-04-27T03:34:02.217186+0000","last_undegraded":"2024-04-27T03:34:02.217186+0000","last_fullsized":"2024-04-27T03:34:02.217186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:34:02.217099+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:34:02.217099+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.217099+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:17:44.534386+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.948322+0000","last_change":"2024-04-27T03:33:40.948348+0000","last_active":"2024-04-27T03:33:40.948322+0000","last_peered":"2024-04-27T03:33:40.948322+0000","last_clean":"2024-04-27T03:33:40.948322+0000","last_became_active":"2024-04-27T03:30:19.406707+0000","last_became_peered":"2024-04-27T03:30:19.406707+0000","last_unstale":"2024-04-27T03:33:40.948322+0000","last_undegraded":"2024-04-27T03:33:40.948322+0000","last_fullsized":"2024-04-27T03:33:40.948322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:40.948229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:40.948229+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.948229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:11.173852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:23.317810+0000","last_change":"2024-04-27T03:34:23.317834+0000","last_active":"2024-04-27T03:34:23.317810+0000","last_peered":"2024-04-27T03:34:23.317810+0000","last_clean":"2024-04-27T03:34:23.317810+0000","last_became_active":"2024-04-27T03:30:21.415441+0000","last_became_peered":"2024-04-27T03:30:21.415441+0000","last_unstale":"2024-04-27T03:34:23.317810+0000","last_undegraded":"2024-04-27T03:34:23.317810+0000","last_fullsized":"2024-04-27T03:34:23.317810+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:23.317720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:23.317720+0000","last_clean_scrub_stamp":"2024-04-27T03:34:23.317720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:31.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.127154+0000","last_change":"2024-04-27T03:33:44.127210+0000","last_active":"2024-04-27T03:33:44.127154+0000","last_peered":"2024-04-27T03:33:44.127154+0000","last_clean":"2024-04-27T03:33:44.127154+0000","last_became_active":"2024-04-27T03:30:13.387397+0000","last_became_peered":"2024-04-27T03:30:13.387397+0000","last_unstale":"2024-04-27T03:33:44.127154+0000","last_undegraded":"2024-04-27T03:33:44.127154+0000","last_fullsized":"2024-04-27T03:33:44.127154+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:44.127050+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:44.127050+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.127050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:24.841845+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:16.127749+0000","last_change":"2024-04-27T03:34:16.127771+0000","last_active":"2024-04-27T03:34:16.127749+0000","last_peered":"2024-04-27T03:34:16.127749+0000","last_clean":"2024-04-27T03:34:16.127749+0000","last_became_active":"2024-04-27T03:30:11.369799+0000","last_became_peered":"2024-04-27T03:30:11.369799+0000","last_unstale":"2024-04-27T03:34:16.127749+0000","last_undegraded":"2024-04-27T03:34:16.127749+0000","last_fullsized":"2024-04-27T03:34:16.127749+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:16.127666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:16.127666+0000","last_clean_scrub_stamp":"2024-04-27T03:34:16.127666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:04.947407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:53.786343+0000","last_change":"2024-04-27T03:33:53.786369+0000","last_active":"2024-04-27T03:33:53.786343+0000","last_peered":"2024-04-27T03:33:53.786343+0000","last_clean":"2024-04-27T03:33:53.786343+0000","last_became_active":"2024-04-27T03:30:15.379873+0000","last_became_peered":"2024-04-27T03:30:15.379873+0000","last_unstale":"2024-04-27T03:33:53.786343+0000","last_undegraded":"2024-04-27T03:33:53.786343+0000","last_fullsized":"2024-04-27T03:33:53.786343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:53.786220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:53.786220+0000","last_clean_scrub_stamp":"2024-04-27T03:33:53.786220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:31.263421+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"27'48","reported_seq":111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.722022+0000","last_change":"2024-04-27T03:33:43.722022+0000","last_active":"2024-04-27T03:33:43.722022+0000","last_peered":"2024-04-27T03:33:43.722022+0000","last_clean":"2024-04-27T03:33:43.722022+0000","last_became_active":"2024-04-27T03:30:17.387205+0000","last_became_peered":"2024-04-27T03:30:17.387205+0000","last_unstale":"2024-04-27T03:33:43.722022+0000","last_undegraded":"2024-04-27T03:33:43.722022+0000","last_fullsized":"2024-04-27T03:33:43.722022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-04-27T03:33:43.721859+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-04-27T03:33:43.721859+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.721859+0000","objects_scrubbed":6,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:42.507737+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":63,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.822276+0000","last_change":"2024-04-27T03:33:34.822276+0000","last_active":"2024-04-27T03:33:34.822276+0000","last_peered":"2024-04-27T03:33:34.822276+0000","last_clean":"2024-04-27T03:33:34.822276+0000","last_became_active":"2024-04-27T03:30:19.405462+0000","last_became_peered":"2024-04-27T03:30:19.405462+0000","last_unstale":"2024-04-27T03:33:34.822276+0000","last_undegraded":"2024-04-27T03:33:34.822276+0000","last_fullsized":"2024-04-27T03:33:34.822276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.822124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.822124+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.822124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:04:28.428726+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"27'7","reported_seq":25,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298466+0000","last_change":"2024-04-27T03:30:21.416894+0000","last_active":"2024-04-27T03:32:44.298466+0000","last_peered":"2024-04-27T03:32:44.298466+0000","last_clean":"2024-04-27T03:32:44.298466+0000","last_became_active":"2024-04-27T03:30:21.416592+0000","last_became_peered":"2024-04-27T03:30:21.416592+0000","last_unstale":"2024-04-27T03:32:44.298466+0000","last_undegraded":"2024-04-27T03:32:44.298466+0000","last_fullsized":"2024-04-27T03:32:44.298466+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"27'6","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.718193+0000","last_change":"2024-04-27T03:34:04.718246+0000","last_active":"2024-04-27T03:34:04.718193+0000","last_peered":"2024-04-27T03:34:04.718193+0000","last_clean":"2024-04-27T03:34:04.718193+0000","last_became_active":"2024-04-27T03:30:13.369643+0000","last_became_peered":"2024-04-27T03:30:13.369643+0000","last_unstale":"2024-04-27T03:34:04.718193+0000","last_undegraded":"2024-04-27T03:34:04.718193+0000","last_fullsized":"2024-04-27T03:34:04.718193+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:04.718068+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:04.718068+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.718068+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:13:49.122146+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:35.349265+0000","last_change":"2024-04-27T03:34:35.349290+0000","last_active":"2024-04-27T03:34:35.349265+0000","last_peered":"2024-04-27T03:34:35.349265+0000","last_clean":"2024-04-27T03:34:35.349265+0000","last_became_active":"2024-04-27T03:30:11.372220+0000","last_became_peered":"2024-04-27T03:30:11.372220+0000","last_unstale":"2024-04-27T03:34:35.349265+0000","last_undegraded":"2024-04-27T03:34:35.349265+0000","last_fullsized":"2024-04-27T03:34:35.349265+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:35.349174+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:35.349174+0000","last_clean_scrub_stamp":"2024-04-27T03:34:35.349174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:22.579508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.972519+0000","last_change":"2024-04-27T03:33:49.972545+0000","last_active":"2024-04-27T03:33:49.972519+0000","last_peered":"2024-04-27T03:33:49.972519+0000","last_clean":"2024-04-27T03:33:49.972519+0000","last_became_active":"2024-04-27T03:30:15.380573+0000","last_became_peered":"2024-04-27T03:30:15.380573+0000","last_unstale":"2024-04-27T03:33:49.972519+0000","last_undegraded":"2024-04-27T03:33:49.972519+0000","last_fullsized":"2024-04-27T03:33:49.972519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:49.972421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:49.972421+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.972421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:13.206549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"27'22","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.170926+0000","last_change":"2024-04-27T03:33:47.170926+0000","last_active":"2024-04-27T03:33:47.170926+0000","last_peered":"2024-04-27T03:33:47.170926+0000","last_clean":"2024-04-27T03:33:47.170926+0000","last_became_active":"2024-04-27T03:30:17.388289+0000","last_became_peered":"2024-04-27T03:30:17.388289+0000","last_unstale":"2024-04-27T03:33:47.170926+0000","last_undegraded":"2024-04-27T03:33:47.170926+0000","last_fullsized":"2024-04-27T03:33:47.170926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-04-27T03:33:47.170743+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-04-27T03:33:47.170743+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.170743+0000","objects_scrubbed":6,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:01.287895+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"27'285","reported_seq":385,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.829103+0000","last_change":"2024-04-27T03:33:40.829103+0000","last_active":"2024-04-27T03:33:40.829103+0000","last_peered":"2024-04-27T03:33:40.829103+0000","last_clean":"2024-04-27T03:33:40.829103+0000","last_became_active":"2024-04-27T03:30:19.406854+0000","last_became_peered":"2024-04-27T03:30:19.406854+0000","last_unstale":"2024-04-27T03:33:40.829103+0000","last_undegraded":"2024-04-27T03:33:40.829103+0000","last_fullsized":"2024-04-27T03:33:40.829103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:40.828943+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:40.828943+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.828943+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:59.341953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:44.298415+0000","last_change":"2024-04-27T03:30:21.415992+0000","last_active":"2024-04-27T03:32:44.298415+0000","last_peered":"2024-04-27T03:32:44.298415+0000","last_clean":"2024-04-27T03:32:44.298415+0000","last_became_active":"2024-04-27T03:30:21.415534+0000","last_became_peered":"2024-04-27T03:30:21.415534+0000","last_unstale":"2024-04-27T03:32:44.298415+0000","last_undegraded":"2024-04-27T03:32:44.298415+0000","last_fullsized":"2024-04-27T03:32:44.298415+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"27'24","reported_seq":111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.819891+0000","last_change":"2024-04-27T03:33:57.819934+0000","last_active":"2024-04-27T03:33:57.819891+0000","last_peered":"2024-04-27T03:33:57.819891+0000","last_clean":"2024-04-27T03:33:57.819891+0000","last_became_active":"2024-04-27T03:30:13.371158+0000","last_became_peered":"2024-04-27T03:30:13.371158+0000","last_unstale":"2024-04-27T03:33:57.819891+0000","last_undegraded":"2024-04-27T03:33:57.819891+0000","last_fullsized":"2024-04-27T03:33:57.819891+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-04-27T03:33:57.819758+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-04-27T03:33:57.819758+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.819758+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:10:53.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":88,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:20.150149+0000","last_change":"2024-04-27T03:34:20.150172+0000","last_active":"2024-04-27T03:34:20.150149+0000","last_peered":"2024-04-27T03:34:20.150149+0000","last_clean":"2024-04-27T03:34:20.150149+0000","last_became_active":"2024-04-27T03:30:11.358893+0000","last_became_peered":"2024-04-27T03:30:11.358893+0000","last_unstale":"2024-04-27T03:34:20.150149+0000","last_undegraded":"2024-04-27T03:34:20.150149+0000","last_fullsized":"2024-04-27T03:34:20.150149+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:20.150063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:20.150063+0000","last_clean_scrub_stamp":"2024-04-27T03:34:20.150063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:19:31.126632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.016872+0000","last_change":"2024-04-27T03:33:48.016893+0000","last_active":"2024-04-27T03:33:48.016872+0000","last_peered":"2024-04-27T03:33:48.016872+0000","last_clean":"2024-04-27T03:33:48.016872+0000","last_became_active":"2024-04-27T03:30:15.381175+0000","last_became_peered":"2024-04-27T03:30:15.381175+0000","last_unstale":"2024-04-27T03:33:48.016872+0000","last_undegraded":"2024-04-27T03:33:48.016872+0000","last_fullsized":"2024-04-27T03:33:48.016872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.016786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.016786+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.016786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:39:37.651908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"27'73","reported_seq":155,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.916431+0000","last_change":"2024-04-27T03:33:39.916480+0000","last_active":"2024-04-27T03:33:39.916431+0000","last_peered":"2024-04-27T03:33:39.916431+0000","last_clean":"2024-04-27T03:33:39.916431+0000","last_became_active":"2024-04-27T03:30:17.386673+0000","last_became_peered":"2024-04-27T03:30:17.386673+0000","last_unstale":"2024-04-27T03:33:39.916431+0000","last_undegraded":"2024-04-27T03:33:39.916431+0000","last_fullsized":"2024-04-27T03:33:39.916431+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'73","last_scrub_stamp":"2024-04-27T03:33:39.916339+0000","last_deep_scrub":"27'73","last_deep_scrub_stamp":"2024-04-27T03:33:39.916339+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.916339+0000","objects_scrubbed":11,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:47.317240+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":80,"num_read_kb":67,"num_write":44,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.711840+0000","last_change":"2024-04-27T03:33:50.711878+0000","last_active":"2024-04-27T03:33:50.711840+0000","last_peered":"2024-04-27T03:33:50.711840+0000","last_clean":"2024-04-27T03:33:50.711840+0000","last_became_active":"2024-04-27T03:30:19.406990+0000","last_became_peered":"2024-04-27T03:30:19.406990+0000","last_unstale":"2024-04-27T03:33:50.711840+0000","last_undegraded":"2024-04-27T03:33:50.711840+0000","last_fullsized":"2024-04-27T03:33:50.711840+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:50.711713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:50.711713+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.711713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:29.194718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.179312+0000","last_change":"2024-04-27T03:34:01.179339+0000","last_active":"2024-04-27T03:34:01.179312+0000","last_peered":"2024-04-27T03:34:01.179312+0000","last_clean":"2024-04-27T03:34:01.179312+0000","last_became_active":"2024-04-27T03:30:21.410060+0000","last_became_peered":"2024-04-27T03:30:21.410060+0000","last_unstale":"2024-04-27T03:34:01.179312+0000","last_undegraded":"2024-04-27T03:34:01.179312+0000","last_fullsized":"2024-04-27T03:34:01.179312+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:01.179224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:01.179224+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.179224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:48.331237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"27'10","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.805577+0000","last_change":"2024-04-27T03:33:54.805600+0000","last_active":"2024-04-27T03:33:54.805577+0000","last_peered":"2024-04-27T03:33:54.805577+0000","last_clean":"2024-04-27T03:33:54.805577+0000","last_became_active":"2024-04-27T03:30:13.371866+0000","last_became_peered":"2024-04-27T03:30:13.371866+0000","last_unstale":"2024-04-27T03:33:54.805577+0000","last_undegraded":"2024-04-27T03:33:54.805577+0000","last_fullsized":"2024-04-27T03:33:54.805577+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:33:54.805478+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:33:54.805478+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.805478+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:21:58.032925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"27'4","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:26.311334+0000","last_change":"2024-04-27T03:34:26.311360+0000","last_active":"2024-04-27T03:34:26.311334+0000","last_peered":"2024-04-27T03:34:26.311334+0000","last_clean":"2024-04-27T03:34:26.311334+0000","last_became_active":"2024-04-27T03:30:11.362420+0000","last_became_peered":"2024-04-27T03:30:11.362420+0000","last_unstale":"2024-04-27T03:34:26.311334+0000","last_undegraded":"2024-04-27T03:34:26.311334+0000","last_fullsized":"2024-04-27T03:34:26.311334+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:26.311245+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:26.311245+0000","last_clean_scrub_stamp":"2024-04-27T03:34:26.311245+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:05.338841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.769662+0000","last_change":"2024-04-27T03:33:50.769691+0000","last_active":"2024-04-27T03:33:50.769662+0000","last_peered":"2024-04-27T03:33:50.769662+0000","last_clean":"2024-04-27T03:33:50.769662+0000","last_became_active":"2024-04-27T03:30:15.382305+0000","last_became_peered":"2024-04-27T03:30:15.382305+0000","last_unstale":"2024-04-27T03:33:50.769662+0000","last_undegraded":"2024-04-27T03:33:50.769662+0000","last_fullsized":"2024-04-27T03:33:50.769662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:50.769565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:50.769565+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.769565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:30.717513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"27'46","reported_seq":107,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.785014+0000","last_change":"2024-04-27T03:33:40.785057+0000","last_active":"2024-04-27T03:33:40.785014+0000","last_peered":"2024-04-27T03:33:40.785014+0000","last_clean":"2024-04-27T03:33:40.785014+0000","last_became_active":"2024-04-27T03:30:17.390281+0000","last_became_peered":"2024-04-27T03:30:17.390281+0000","last_unstale":"2024-04-27T03:33:40.785014+0000","last_undegraded":"2024-04-27T03:33:40.785014+0000","last_fullsized":"2024-04-27T03:33:40.785014+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:40.784918+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:40.784918+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.784918+0000","objects_scrubbed":5,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:11.010541+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":50,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.154146+0000","last_change":"2024-04-27T03:33:59.154184+0000","last_active":"2024-04-27T03:33:59.154146+0000","last_peered":"2024-04-27T03:33:59.154146+0000","last_clean":"2024-04-27T03:33:59.154146+0000","last_became_active":"2024-04-27T03:30:19.405353+0000","last_became_peered":"2024-04-27T03:30:19.405353+0000","last_unstale":"2024-04-27T03:33:59.154146+0000","last_undegraded":"2024-04-27T03:33:59.154146+0000","last_fullsized":"2024-04-27T03:33:59.154146+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:59.154022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:59.154022+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.154022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:53:09.531303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":18,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:32:49.299534+0000","last_change":"2024-04-27T03:30:21.416876+0000","last_active":"2024-04-27T03:32:49.299534+0000","last_peered":"2024-04-27T03:32:49.299534+0000","last_clean":"2024-04-27T03:32:49.299534+0000","last_became_active":"2024-04-27T03:30:21.416276+0000","last_became_peered":"2024-04-27T03:30:21.416276+0000","last_unstale":"2024-04-27T03:32:49.299534+0000","last_undegraded":"2024-04-27T03:32:49.299534+0000","last_fullsized":"2024-04-27T03:32:49.299534+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'24","reported_seq":110,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:13.170815+0000","last_change":"2024-04-27T03:34:13.170839+0000","last_active":"2024-04-27T03:34:13.170815+0000","last_peered":"2024-04-27T03:34:13.170815+0000","last_clean":"2024-04-27T03:34:13.170815+0000","last_became_active":"2024-04-27T03:30:13.376415+0000","last_became_peered":"2024-04-27T03:30:13.376415+0000","last_unstale":"2024-04-27T03:34:13.170815+0000","last_undegraded":"2024-04-27T03:34:13.170815+0000","last_fullsized":"2024-04-27T03:34:13.170815+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-04-27T03:34:13.170716+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-04-27T03:34:13.170716+0000","last_clean_scrub_stamp":"2024-04-27T03:34:13.170716+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:49:02.946260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:11.675077+0000","last_change":"2024-04-27T03:34:11.675077+0000","last_active":"2024-04-27T03:34:11.675077+0000","last_peered":"2024-04-27T03:34:11.675077+0000","last_clean":"2024-04-27T03:34:11.675077+0000","last_became_active":"2024-04-27T03:30:11.360590+0000","last_became_peered":"2024-04-27T03:30:11.360590+0000","last_unstale":"2024-04-27T03:34:11.675077+0000","last_undegraded":"2024-04-27T03:34:11.675077+0000","last_fullsized":"2024-04-27T03:34:11.675077+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:11.674894+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:11.674894+0000","last_clean_scrub_stamp":"2024-04-27T03:34:11.674894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:37.256713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.996324+0000","last_change":"2024-04-27T03:33:42.996361+0000","last_active":"2024-04-27T03:33:42.996324+0000","last_peered":"2024-04-27T03:33:42.996324+0000","last_clean":"2024-04-27T03:33:42.996324+0000","last_became_active":"2024-04-27T03:30:15.385614+0000","last_became_peered":"2024-04-27T03:30:15.385614+0000","last_unstale":"2024-04-27T03:33:42.996324+0000","last_undegraded":"2024-04-27T03:33:42.996324+0000","last_fullsized":"2024-04-27T03:33:42.996324+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.996219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.996219+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.996219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:58.172124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"27'37","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.164208+0000","last_change":"2024-04-27T03:33:39.164242+0000","last_active":"2024-04-27T03:33:39.164208+0000","last_peered":"2024-04-27T03:33:39.164208+0000","last_clean":"2024-04-27T03:33:39.164208+0000","last_became_active":"2024-04-27T03:30:17.386983+0000","last_became_peered":"2024-04-27T03:30:17.386983+0000","last_unstale":"2024-04-27T03:33:39.164208+0000","last_undegraded":"2024-04-27T03:33:39.164208+0000","last_fullsized":"2024-04-27T03:33:39.164208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-04-27T03:33:39.164121+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-04-27T03:33:39.164121+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.164121+0000","objects_scrubbed":3,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:26.316134+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":33,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.113482+0000","last_change":"2024-04-27T03:33:54.113533+0000","last_active":"2024-04-27T03:33:54.113482+0000","last_peered":"2024-04-27T03:33:54.113482+0000","last_clean":"2024-04-27T03:33:54.113482+0000","last_became_active":"2024-04-27T03:30:19.402652+0000","last_became_peered":"2024-04-27T03:30:19.402652+0000","last_unstale":"2024-04-27T03:33:54.113482+0000","last_undegraded":"2024-04-27T03:33:54.113482+0000","last_fullsized":"2024-04-27T03:33:54.113482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:54.113385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:54.113385+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.113385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:04:42.933126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"27'6","reported_seq":24,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:20.937826+0000","last_change":"2024-04-27T03:30:21.420988+0000","last_active":"2024-04-27T03:33:20.937826+0000","last_peered":"2024-04-27T03:33:20.937826+0000","last_clean":"2024-04-27T03:33:20.937826+0000","last_became_active":"2024-04-27T03:30:21.420725+0000","last_became_peered":"2024-04-27T03:30:21.420725+0000","last_unstale":"2024-04-27T03:33:20.937826+0000","last_undegraded":"2024-04-27T03:33:20.937826+0000","last_fullsized":"2024-04-27T03:33:20.937826+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:30:20.391893+0000","last_clean_scrub_stamp":"2024-04-27T03:30:20.391893+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"queued for deep scrub","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.788128+0000","last_change":"2024-04-27T03:33:58.788151+0000","last_active":"2024-04-27T03:33:58.788128+0000","last_peered":"2024-04-27T03:33:58.788128+0000","last_clean":"2024-04-27T03:33:58.788128+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:33:58.788128+0000","last_undegraded":"2024-04-27T03:33:58.788128+0000","last_fullsized":"2024-04-27T03:33:58.788128+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:58.788034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:58.788034+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.788034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:14:43.897282+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:25.076472+0000","last_change":"2024-04-27T03:34:25.076512+0000","last_active":"2024-04-27T03:34:25.076472+0000","last_peered":"2024-04-27T03:34:25.076472+0000","last_clean":"2024-04-27T03:34:25.076472+0000","last_became_active":"2024-04-27T03:30:11.369877+0000","last_became_peered":"2024-04-27T03:30:11.369877+0000","last_unstale":"2024-04-27T03:34:25.076472+0000","last_undegraded":"2024-04-27T03:34:25.076472+0000","last_fullsized":"2024-04-27T03:34:25.076472+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:25.076378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:25.076378+0000","last_clean_scrub_stamp":"2024-04-27T03:34:25.076378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:44:13.932150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"27'3","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.974605+0000","last_change":"2024-04-27T03:33:45.974632+0000","last_active":"2024-04-27T03:33:45.974605+0000","last_peered":"2024-04-27T03:33:45.974605+0000","last_clean":"2024-04-27T03:33:45.974605+0000","last_became_active":"2024-04-27T03:30:15.381574+0000","last_became_peered":"2024-04-27T03:30:15.381574+0000","last_unstale":"2024-04-27T03:33:45.974605+0000","last_undegraded":"2024-04-27T03:33:45.974605+0000","last_fullsized":"2024-04-27T03:33:45.974605+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-27T03:33:45.974523+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-27T03:33:45.974523+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.974523+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:37.524758+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"27'53","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.159896+0000","last_change":"2024-04-27T03:33:57.159964+0000","last_active":"2024-04-27T03:33:57.159896+0000","last_peered":"2024-04-27T03:33:57.159896+0000","last_clean":"2024-04-27T03:33:57.159896+0000","last_became_active":"2024-04-27T03:30:17.388277+0000","last_became_peered":"2024-04-27T03:30:17.388277+0000","last_unstale":"2024-04-27T03:33:57.159896+0000","last_undegraded":"2024-04-27T03:33:57.159896+0000","last_fullsized":"2024-04-27T03:33:57.159896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'53","last_scrub_stamp":"2024-04-27T03:33:57.159788+0000","last_deep_scrub":"27'53","last_deep_scrub_stamp":"2024-04-27T03:33:57.159788+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.159788+0000","objects_scrubbed":7,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:03.430517+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":52,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1138,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.728511+0000","last_change":"2024-04-27T03:33:42.728536+0000","last_active":"2024-04-27T03:33:42.728511+0000","last_peered":"2024-04-27T03:33:42.728511+0000","last_clean":"2024-04-27T03:33:42.728511+0000","last_became_active":"2024-04-27T03:30:19.406565+0000","last_became_peered":"2024-04-27T03:30:19.406565+0000","last_unstale":"2024-04-27T03:33:42.728511+0000","last_undegraded":"2024-04-27T03:33:42.728511+0000","last_fullsized":"2024-04-27T03:33:42.728511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.728420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.728420+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.728420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:57.904325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"27'4","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:19.270206+0000","last_change":"2024-04-27T03:34:19.270233+0000","last_active":"2024-04-27T03:34:19.270206+0000","last_peered":"2024-04-27T03:34:19.270206+0000","last_clean":"2024-04-27T03:34:19.270206+0000","last_became_active":"2024-04-27T03:30:21.414510+0000","last_became_peered":"2024-04-27T03:30:21.414510+0000","last_unstale":"2024-04-27T03:34:19.270206+0000","last_undegraded":"2024-04-27T03:34:19.270206+0000","last_fullsized":"2024-04-27T03:34:19.270206+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:19.270117+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:19.270117+0000","last_clean_scrub_stamp":"2024-04-27T03:34:19.270117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:43:26.209785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"25'8","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.006570+0000","last_change":"2024-04-27T03:33:54.006570+0000","last_active":"2024-04-27T03:33:54.006570+0000","last_peered":"2024-04-27T03:33:54.006570+0000","last_clean":"2024-04-27T03:33:54.006570+0000","last_became_active":"2024-04-27T03:30:13.387149+0000","last_became_peered":"2024-04-27T03:30:13.387149+0000","last_unstale":"2024-04-27T03:33:54.006570+0000","last_undegraded":"2024-04-27T03:33:54.006570+0000","last_fullsized":"2024-04-27T03:33:54.006570+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'8","last_scrub_stamp":"2024-04-27T03:33:54.006451+0000","last_deep_scrub":"25'8","last_deep_scrub_stamp":"2024-04-27T03:33:54.006451+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.006451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:20.542228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"25'6","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:12.836026+0000","last_change":"2024-04-27T03:34:12.836026+0000","last_active":"2024-04-27T03:34:12.836026+0000","last_peered":"2024-04-27T03:34:12.836026+0000","last_clean":"2024-04-27T03:34:12.836026+0000","last_became_active":"2024-04-27T03:30:11.357878+0000","last_became_peered":"2024-04-27T03:30:11.357878+0000","last_unstale":"2024-04-27T03:34:12.836026+0000","last_undegraded":"2024-04-27T03:34:12.836026+0000","last_fullsized":"2024-04-27T03:34:12.836026+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'6","last_scrub_stamp":"2024-04-27T03:34:12.835816+0000","last_deep_scrub":"25'6","last_deep_scrub_stamp":"2024-04-27T03:34:12.835816+0000","last_clean_scrub_stamp":"2024-04-27T03:34:12.835816+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:53.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"27'4","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.707653+0000","last_change":"2024-04-27T03:34:02.707653+0000","last_active":"2024-04-27T03:34:02.707653+0000","last_peered":"2024-04-27T03:34:02.707653+0000","last_clean":"2024-04-27T03:34:02.707653+0000","last_became_active":"2024-04-27T03:30:13.369681+0000","last_became_peered":"2024-04-27T03:30:13.369681+0000","last_unstale":"2024-04-27T03:34:02.707653+0000","last_undegraded":"2024-04-27T03:34:02.707653+0000","last_fullsized":"2024-04-27T03:34:02.707653+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:02.707538+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:02.707538+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.707538+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:58.202906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"25'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.696795+0000","last_change":"2024-04-27T03:34:01.696795+0000","last_active":"2024-04-27T03:34:01.696795+0000","last_peered":"2024-04-27T03:34:01.696795+0000","last_clean":"2024-04-27T03:34:01.696795+0000","last_became_active":"2024-04-27T03:30:11.373186+0000","last_became_peered":"2024-04-27T03:30:11.373186+0000","last_unstale":"2024-04-27T03:34:01.696795+0000","last_undegraded":"2024-04-27T03:34:01.696795+0000","last_fullsized":"2024-04-27T03:34:01.696795+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:01.696661+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:01.696661+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.696661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:33.957902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"27'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.249759+0000","last_change":"2024-04-27T03:34:08.249759+0000","last_active":"2024-04-27T03:34:08.249759+0000","last_peered":"2024-04-27T03:34:08.249759+0000","last_clean":"2024-04-27T03:34:08.249759+0000","last_became_active":"2024-04-27T03:30:13.382069+0000","last_became_peered":"2024-04-27T03:30:13.382069+0000","last_unstale":"2024-04-27T03:34:08.249759+0000","last_undegraded":"2024-04-27T03:34:08.249759+0000","last_fullsized":"2024-04-27T03:34:08.249759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:08.249643+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:08.249643+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.249643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:17:08.768291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"27'3","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:46.362974+0000","last_change":"2024-04-27T03:34:46.363030+0000","last_active":"2024-04-27T03:34:46.362974+0000","last_peered":"2024-04-27T03:34:46.362974+0000","last_clean":"2024-04-27T03:34:46.362974+0000","last_became_active":"2024-04-27T03:30:11.350295+0000","last_became_peered":"2024-04-27T03:30:11.350295+0000","last_unstale":"2024-04-27T03:34:46.362974+0000","last_undegraded":"2024-04-27T03:34:46.362974+0000","last_fullsized":"2024-04-27T03:34:46.362974+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-27T03:34:46.362862+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-27T03:34:46.362862+0000","last_clean_scrub_stamp":"2024-04-27T03:34:46.362862+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:34.791681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"27'4","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.676435+0000","last_change":"2024-04-27T03:34:06.676435+0000","last_active":"2024-04-27T03:34:06.676435+0000","last_peered":"2024-04-27T03:34:06.676435+0000","last_clean":"2024-04-27T03:34:06.676435+0000","last_became_active":"2024-04-27T03:30:13.370066+0000","last_became_peered":"2024-04-27T03:30:13.370066+0000","last_unstale":"2024-04-27T03:34:06.676435+0000","last_undegraded":"2024-04-27T03:34:06.676435+0000","last_fullsized":"2024-04-27T03:34:06.676435+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:06.676262+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:06.676262+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.676262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:06.684143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"27'4","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:22.693546+0000","last_change":"2024-04-27T03:34:22.693546+0000","last_active":"2024-04-27T03:34:22.693546+0000","last_peered":"2024-04-27T03:34:22.693546+0000","last_clean":"2024-04-27T03:34:22.693546+0000","last_became_active":"2024-04-27T03:30:11.360376+0000","last_became_peered":"2024-04-27T03:30:11.360376+0000","last_unstale":"2024-04-27T03:34:22.693546+0000","last_undegraded":"2024-04-27T03:34:22.693546+0000","last_fullsized":"2024-04-27T03:34:22.693546+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:22.693417+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:22.693417+0000","last_clean_scrub_stamp":"2024-04-27T03:34:22.693417+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:10.493189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"27'4","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:24.695215+0000","last_change":"2024-04-27T03:34:24.695265+0000","last_active":"2024-04-27T03:34:24.695215+0000","last_peered":"2024-04-27T03:34:24.695215+0000","last_clean":"2024-04-27T03:34:24.695215+0000","last_became_active":"2024-04-27T03:30:13.378368+0000","last_became_peered":"2024-04-27T03:30:13.378368+0000","last_unstale":"2024-04-27T03:34:24.695215+0000","last_undegraded":"2024-04-27T03:34:24.695215+0000","last_fullsized":"2024-04-27T03:34:24.695215+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:24.695112+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:24.695112+0000","last_clean_scrub_stamp":"2024-04-27T03:34:24.695112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:47.159176+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"25'2","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.669200+0000","last_change":"2024-04-27T03:34:05.669200+0000","last_active":"2024-04-27T03:34:05.669200+0000","last_peered":"2024-04-27T03:34:05.669200+0000","last_clean":"2024-04-27T03:34:05.669200+0000","last_became_active":"2024-04-27T03:30:11.371553+0000","last_became_peered":"2024-04-27T03:30:11.371553+0000","last_unstale":"2024-04-27T03:34:05.669200+0000","last_undegraded":"2024-04-27T03:34:05.669200+0000","last_fullsized":"2024-04-27T03:34:05.669200+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:05.669082+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:05.669082+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.669082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:46.383877+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"27'10","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:32.090474+0000","last_change":"2024-04-27T03:34:32.090502+0000","last_active":"2024-04-27T03:34:32.090474+0000","last_peered":"2024-04-27T03:34:32.090474+0000","last_clean":"2024-04-27T03:34:32.090474+0000","last_became_active":"2024-04-27T03:31:01.440576+0000","last_became_peered":"2024-04-27T03:31:01.440576+0000","last_unstale":"2024-04-27T03:34:32.090474+0000","last_undegraded":"2024-04-27T03:34:32.090474+0000","last_fullsized":"2024-04-27T03:34:32.090474+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:34:32.090375+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:34:32.090375+0000","last_clean_scrub_stamp":"2024-04-27T03:34:32.090375+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:40.211937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"27'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:12.120768+0000","last_change":"2024-04-27T03:34:12.120768+0000","last_active":"2024-04-27T03:34:12.120768+0000","last_peered":"2024-04-27T03:34:12.120768+0000","last_clean":"2024-04-27T03:34:12.120768+0000","last_became_active":"2024-04-27T03:30:11.369517+0000","last_became_peered":"2024-04-27T03:30:11.369517+0000","last_unstale":"2024-04-27T03:34:12.120768+0000","last_undegraded":"2024-04-27T03:34:12.120768+0000","last_fullsized":"2024-04-27T03:34:12.120768+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:12.120650+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:12.120650+0000","last_clean_scrub_stamp":"2024-04-27T03:34:12.120650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:30.112060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"27'4","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:10.688469+0000","last_change":"2024-04-27T03:34:10.688469+0000","last_active":"2024-04-27T03:34:10.688469+0000","last_peered":"2024-04-27T03:34:10.688469+0000","last_clean":"2024-04-27T03:34:10.688469+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:34:10.688469+0000","last_undegraded":"2024-04-27T03:34:10.688469+0000","last_fullsized":"2024-04-27T03:34:10.688469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:10.688326+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:10.688326+0000","last_clean_scrub_stamp":"2024-04-27T03:34:10.688326+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:44.126012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:22.881026+0000","last_change":"2024-04-27T03:34:22.881026+0000","last_active":"2024-04-27T03:34:22.881026+0000","last_peered":"2024-04-27T03:34:22.881026+0000","last_clean":"2024-04-27T03:34:22.881026+0000","last_became_active":"2024-04-27T03:31:01.441159+0000","last_became_peered":"2024-04-27T03:31:01.441159+0000","last_unstale":"2024-04-27T03:34:22.881026+0000","last_undegraded":"2024-04-27T03:34:22.881026+0000","last_fullsized":"2024-04-27T03:34:22.881026+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:22.880861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:22.880861+0000","last_clean_scrub_stamp":"2024-04-27T03:34:22.880861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:40.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"27'4","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.181941+0000","last_change":"2024-04-27T03:34:07.181941+0000","last_active":"2024-04-27T03:34:07.181941+0000","last_peered":"2024-04-27T03:34:07.181941+0000","last_clean":"2024-04-27T03:34:07.181941+0000","last_became_active":"2024-04-27T03:30:13.377821+0000","last_became_peered":"2024-04-27T03:30:13.377821+0000","last_unstale":"2024-04-27T03:34:07.181941+0000","last_undegraded":"2024-04-27T03:34:07.181941+0000","last_fullsized":"2024-04-27T03:34:07.181941+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:07.181834+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:07.181834+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.181834+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:07.189714+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.697117+0000","last_change":"2024-04-27T03:34:03.697117+0000","last_active":"2024-04-27T03:34:03.697117+0000","last_peered":"2024-04-27T03:34:03.697117+0000","last_clean":"2024-04-27T03:34:03.697117+0000","last_became_active":"2024-04-27T03:30:11.371604+0000","last_became_peered":"2024-04-27T03:30:11.371604+0000","last_unstale":"2024-04-27T03:34:03.697117+0000","last_undegraded":"2024-04-27T03:34:03.697117+0000","last_fullsized":"2024-04-27T03:34:03.697117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:03.697000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:03.697000+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.697000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:34:44.936107+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"27'8","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.956162+0000","last_change":"2024-04-27T03:33:51.956191+0000","last_active":"2024-04-27T03:33:51.956162+0000","last_peered":"2024-04-27T03:33:51.956162+0000","last_clean":"2024-04-27T03:33:51.956162+0000","last_became_active":"2024-04-27T03:30:13.388421+0000","last_became_peered":"2024-04-27T03:30:13.388421+0000","last_unstale":"2024-04-27T03:33:51.956162+0000","last_undegraded":"2024-04-27T03:33:51.956162+0000","last_fullsized":"2024-04-27T03:33:51.956162+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-04-27T03:33:51.956075+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-04-27T03:33:51.956075+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.956075+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:21:03.313222+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"27'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:48.353112+0000","last_change":"2024-04-27T03:34:48.353139+0000","last_active":"2024-04-27T03:34:48.353112+0000","last_peered":"2024-04-27T03:34:48.353112+0000","last_clean":"2024-04-27T03:34:48.353112+0000","last_became_active":"2024-04-27T03:30:11.355183+0000","last_became_peered":"2024-04-27T03:30:11.355183+0000","last_unstale":"2024-04-27T03:34:48.353112+0000","last_undegraded":"2024-04-27T03:34:48.353112+0000","last_fullsized":"2024-04-27T03:34:48.353112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:48.353023+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:48.353023+0000","last_clean_scrub_stamp":"2024-04-27T03:34:48.353023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:55.651548+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:17.237435+0000","last_change":"2024-04-27T03:34:17.237459+0000","last_active":"2024-04-27T03:34:17.237435+0000","last_peered":"2024-04-27T03:34:17.237435+0000","last_clean":"2024-04-27T03:34:17.237435+0000","last_became_active":"2024-04-27T03:30:13.376875+0000","last_became_peered":"2024-04-27T03:30:13.376875+0000","last_unstale":"2024-04-27T03:34:17.237435+0000","last_undegraded":"2024-04-27T03:34:17.237435+0000","last_fullsized":"2024-04-27T03:34:17.237435+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:17.237351+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:17.237351+0000","last_clean_scrub_stamp":"2024-04-27T03:34:17.237351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:05.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.648432+0000","last_change":"2024-04-27T03:34:07.648432+0000","last_active":"2024-04-27T03:34:07.648432+0000","last_peered":"2024-04-27T03:34:07.648432+0000","last_clean":"2024-04-27T03:34:07.648432+0000","last_became_active":"2024-04-27T03:30:11.360299+0000","last_became_peered":"2024-04-27T03:30:11.360299+0000","last_unstale":"2024-04-27T03:34:07.648432+0000","last_undegraded":"2024-04-27T03:34:07.648432+0000","last_fullsized":"2024-04-27T03:34:07.648432+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:07.648243+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:07.648243+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.648243+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:22:16.432642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"27'10","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:14.664018+0000","last_change":"2024-04-27T03:34:14.664018+0000","last_active":"2024-04-27T03:34:14.664018+0000","last_peered":"2024-04-27T03:34:14.664018+0000","last_clean":"2024-04-27T03:34:14.664018+0000","last_became_active":"2024-04-27T03:30:13.376087+0000","last_became_peered":"2024-04-27T03:30:13.376087+0000","last_unstale":"2024-04-27T03:34:14.664018+0000","last_undegraded":"2024-04-27T03:34:14.664018+0000","last_fullsized":"2024-04-27T03:34:14.664018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:34:14.663832+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:34:14.663832+0000","last_clean_scrub_stamp":"2024-04-27T03:34:14.663832+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:08.919647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:10.162522+0000","last_change":"2024-04-27T03:34:10.162522+0000","last_active":"2024-04-27T03:34:10.162522+0000","last_peered":"2024-04-27T03:34:10.162522+0000","last_clean":"2024-04-27T03:34:10.162522+0000","last_became_active":"2024-04-27T03:30:11.370837+0000","last_became_peered":"2024-04-27T03:30:11.370837+0000","last_unstale":"2024-04-27T03:34:10.162522+0000","last_undegraded":"2024-04-27T03:34:10.162522+0000","last_fullsized":"2024-04-27T03:34:10.162522+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:10.162390+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:10.162390+0000","last_clean_scrub_stamp":"2024-04-27T03:34:10.162390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:13:28.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"27'6","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.179323+0000","last_change":"2024-04-27T03:34:05.179323+0000","last_active":"2024-04-27T03:34:05.179323+0000","last_peered":"2024-04-27T03:34:05.179323+0000","last_clean":"2024-04-27T03:34:05.179323+0000","last_became_active":"2024-04-27T03:30:13.373511+0000","last_became_peered":"2024-04-27T03:30:13.373511+0000","last_unstale":"2024-04-27T03:34:05.179323+0000","last_undegraded":"2024-04-27T03:34:05.179323+0000","last_fullsized":"2024-04-27T03:34:05.179323+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:05.179213+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:05.179213+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.179213+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:49.583854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:14.167281+0000","last_change":"2024-04-27T03:34:14.167281+0000","last_active":"2024-04-27T03:34:14.167281+0000","last_peered":"2024-04-27T03:34:14.167281+0000","last_clean":"2024-04-27T03:34:14.167281+0000","last_became_active":"2024-04-27T03:30:11.370283+0000","last_became_peered":"2024-04-27T03:30:11.370283+0000","last_unstale":"2024-04-27T03:34:14.167281+0000","last_undegraded":"2024-04-27T03:34:14.167281+0000","last_fullsized":"2024-04-27T03:34:14.167281+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:14.167157+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:14.167157+0000","last_clean_scrub_stamp":"2024-04-27T03:34:14.167157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:24.417880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"27'12","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:17.133790+0000","last_change":"2024-04-27T03:34:17.133790+0000","last_active":"2024-04-27T03:34:17.133790+0000","last_peered":"2024-04-27T03:34:17.133790+0000","last_clean":"2024-04-27T03:34:17.133790+0000","last_became_active":"2024-04-27T03:30:13.380404+0000","last_became_peered":"2024-04-27T03:30:13.380404+0000","last_unstale":"2024-04-27T03:34:17.133790+0000","last_undegraded":"2024-04-27T03:34:17.133790+0000","last_fullsized":"2024-04-27T03:34:17.133790+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-04-27T03:34:17.133674+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-04-27T03:34:17.133674+0000","last_clean_scrub_stamp":"2024-04-27T03:34:17.133674+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:57:39.000345+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:24.348395+0000","last_change":"2024-04-27T03:34:24.348395+0000","last_active":"2024-04-27T03:34:24.348395+0000","last_peered":"2024-04-27T03:34:24.348395+0000","last_clean":"2024-04-27T03:34:24.348395+0000","last_became_active":"2024-04-27T03:30:11.373026+0000","last_became_peered":"2024-04-27T03:30:11.373026+0000","last_unstale":"2024-04-27T03:34:24.348395+0000","last_undegraded":"2024-04-27T03:34:24.348395+0000","last_fullsized":"2024-04-27T03:34:24.348395+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:24.348277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:24.348277+0000","last_clean_scrub_stamp":"2024-04-27T03:34:24.348277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:23.175329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"27'2","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.031559+0000","last_change":"2024-04-27T03:33:55.031559+0000","last_active":"2024-04-27T03:33:55.031559+0000","last_peered":"2024-04-27T03:33:55.031559+0000","last_clean":"2024-04-27T03:33:55.031559+0000","last_became_active":"2024-04-27T03:30:13.385791+0000","last_became_peered":"2024-04-27T03:30:13.385791+0000","last_unstale":"2024-04-27T03:33:55.031559+0000","last_undegraded":"2024-04-27T03:33:55.031559+0000","last_fullsized":"2024-04-27T03:33:55.031559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:55.031445+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:55.031445+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.031445+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:01.039513+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:36.354134+0000","last_change":"2024-04-27T03:34:36.354134+0000","last_active":"2024-04-27T03:34:36.354134+0000","last_peered":"2024-04-27T03:34:36.354134+0000","last_clean":"2024-04-27T03:34:36.354134+0000","last_became_active":"2024-04-27T03:30:11.357898+0000","last_became_peered":"2024-04-27T03:30:11.357898+0000","last_unstale":"2024-04-27T03:34:36.354134+0000","last_undegraded":"2024-04-27T03:34:36.354134+0000","last_fullsized":"2024-04-27T03:34:36.354134+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:36.354013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:36.354013+0000","last_clean_scrub_stamp":"2024-04-27T03:34:36.354013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:36.076803+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"27'6","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:24.943886+0000","last_change":"2024-04-27T03:34:24.943920+0000","last_active":"2024-04-27T03:34:24.943886+0000","last_peered":"2024-04-27T03:34:24.943886+0000","last_clean":"2024-04-27T03:34:24.943886+0000","last_became_active":"2024-04-27T03:31:01.440820+0000","last_became_peered":"2024-04-27T03:31:01.440820+0000","last_unstale":"2024-04-27T03:34:24.943886+0000","last_undegraded":"2024-04-27T03:34:24.943886+0000","last_fullsized":"2024-04-27T03:34:24.943886+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:24.943789+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:24.943789+0000","last_clean_scrub_stamp":"2024-04-27T03:34:24.943789+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:19.199603+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.846496+0000","last_change":"2024-04-27T03:34:05.846517+0000","last_active":"2024-04-27T03:34:05.846496+0000","last_peered":"2024-04-27T03:34:05.846496+0000","last_clean":"2024-04-27T03:34:05.846496+0000","last_became_active":"2024-04-27T03:30:11.362788+0000","last_became_peered":"2024-04-27T03:30:11.362788+0000","last_unstale":"2024-04-27T03:34:05.846496+0000","last_undegraded":"2024-04-27T03:34:05.846496+0000","last_fullsized":"2024-04-27T03:34:05.846496+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:05.846403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:05.846403+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.846403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:30.818661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"25'2","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.209205+0000","last_change":"2024-04-27T03:34:09.209205+0000","last_active":"2024-04-27T03:34:09.209205+0000","last_peered":"2024-04-27T03:34:09.209205+0000","last_clean":"2024-04-27T03:34:09.209205+0000","last_became_active":"2024-04-27T03:30:13.377845+0000","last_became_peered":"2024-04-27T03:30:13.377845+0000","last_unstale":"2024-04-27T03:34:09.209205+0000","last_undegraded":"2024-04-27T03:34:09.209205+0000","last_fullsized":"2024-04-27T03:34:09.209205+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:09.209073+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:09.209073+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.209073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:40.804195+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:25.348774+0000","last_change":"2024-04-27T03:34:25.348820+0000","last_active":"2024-04-27T03:34:25.348774+0000","last_peered":"2024-04-27T03:34:25.348774+0000","last_clean":"2024-04-27T03:34:25.348774+0000","last_became_active":"2024-04-27T03:30:11.362559+0000","last_became_peered":"2024-04-27T03:30:11.362559+0000","last_unstale":"2024-04-27T03:34:25.348774+0000","last_undegraded":"2024-04-27T03:34:25.348774+0000","last_fullsized":"2024-04-27T03:34:25.348774+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:25.348662+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:25.348662+0000","last_clean_scrub_stamp":"2024-04-27T03:34:25.348662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:25.577657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:29.307800+0000","last_change":"2024-04-27T03:34:29.307826+0000","last_active":"2024-04-27T03:34:29.307800+0000","last_peered":"2024-04-27T03:34:29.307800+0000","last_clean":"2024-04-27T03:34:29.307800+0000","last_became_active":"2024-04-27T03:30:11.363340+0000","last_became_peered":"2024-04-27T03:30:11.363340+0000","last_unstale":"2024-04-27T03:34:29.307800+0000","last_undegraded":"2024-04-27T03:34:29.307800+0000","last_fullsized":"2024-04-27T03:34:29.307800+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:29.307702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:29.307702+0000","last_clean_scrub_stamp":"2024-04-27T03:34:29.307702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:45.266144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"27'4","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.668243+0000","last_change":"2024-04-27T03:33:56.668296+0000","last_active":"2024-04-27T03:33:56.668243+0000","last_peered":"2024-04-27T03:33:56.668243+0000","last_clean":"2024-04-27T03:33:56.668243+0000","last_became_active":"2024-04-27T03:30:13.385705+0000","last_became_peered":"2024-04-27T03:30:13.385705+0000","last_unstale":"2024-04-27T03:33:56.668243+0000","last_undegraded":"2024-04-27T03:33:56.668243+0000","last_fullsized":"2024-04-27T03:33:56.668243+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:56.668140+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:56.668140+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.668140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:00.786770+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"25'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:23.731875+0000","last_change":"2024-04-27T03:34:23.731902+0000","last_active":"2024-04-27T03:34:23.731875+0000","last_peered":"2024-04-27T03:34:23.731875+0000","last_clean":"2024-04-27T03:34:23.731875+0000","last_became_active":"2024-04-27T03:30:11.360667+0000","last_became_peered":"2024-04-27T03:30:11.360667+0000","last_unstale":"2024-04-27T03:34:23.731875+0000","last_undegraded":"2024-04-27T03:34:23.731875+0000","last_fullsized":"2024-04-27T03:34:23.731875+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:23.731763+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:23.731763+0000","last_clean_scrub_stamp":"2024-04-27T03:34:23.731763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:18:06.049050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"27'14","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:15.127191+0000","last_change":"2024-04-27T03:34:15.127239+0000","last_active":"2024-04-27T03:34:15.127191+0000","last_peered":"2024-04-27T03:34:15.127191+0000","last_clean":"2024-04-27T03:34:15.127191+0000","last_became_active":"2024-04-27T03:30:13.380347+0000","last_became_peered":"2024-04-27T03:30:13.380347+0000","last_unstale":"2024-04-27T03:34:15.127191+0000","last_undegraded":"2024-04-27T03:34:15.127191+0000","last_fullsized":"2024-04-27T03:34:15.127191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:34:15.127095+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:34:15.127095+0000","last_clean_scrub_stamp":"2024-04-27T03:34:15.127095+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:03.309204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":274,"ondisk_log_size":274,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2281,"ondisk_log_size":2281,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":1888,"num_read_kb":1653,"num_write":789,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2692,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1751,"ondisk_log_size":1751,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":482,"ondisk_log_size":482,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":82,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705723,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39904,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331936,"statfs":{"total":96636764160,"available":96595902464,"internally_reserved":0,"allocated":417792,"data_stored":120948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705724,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40416,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331424,"statfs":{"total":96636764160,"available":96595378176,"internally_reserved":0,"allocated":942080,"data_stored":708075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705723,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335556,"statfs":{"total":96636764160,"available":96599609344,"internally_reserved":0,"allocated":970752,"data_stored":708832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705725,"num_pgs":134,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39868,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331972,"statfs":{"total":96636764160,"available":96595939328,"internally_reserved":0,"allocated":380928,"data_stored":119531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T03:34:53.531 INFO:tasks.ceph:pgid 3.39 last_scrub_stamp 2024-04-27T03:30:10.339743+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=10, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:53.532 INFO:tasks.ceph:pgid 8.0 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:53.533 INFO:tasks.ceph:pgid 8.b last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:53.534 INFO:tasks.ceph:pgid 8.1c last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:53.534 INFO:tasks.ceph:pgid 8.11 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:53.534 INFO:tasks.ceph:pgid 8.10 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:53.535 INFO:tasks.ceph:pgid 8.16 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:53.535 INFO:tasks.ceph:pgid 8.15 last_scrub_stamp 2024-04-27T03:30:20.391893+0000 time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=30, tm_sec=20, tm_wday=5, tm_yday=118, tm_isdst=-1) <= time.struct_time(tm_year=2024, tm_mon=4, tm_mday=27, tm_hour=3, tm_min=33, tm_sec=29, tm_wday=5, tm_yday=118, tm_isdst=0) 2024-04-27T03:34:53.535 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2024-04-27T03:35:13.536 DEBUG:teuthology.orchestra.run.smithi089:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-27T03:35:13.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.638+0000 7f030a40f640 1 Processor -- start 2024-04-27T03:35:13.644 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.638+0000 7f030a40f640 1 -- start start 2024-04-27T03:35:13.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.638+0000 7f030a40f640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0304127ce0 0x7f03041280c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:35:13.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.638+0000 7f030a40f640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f0304127b00 con 0x7f0304070100 2024-04-27T03:35:13.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.638+0000 7f030a40f640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f03040708e0 con 0x7f0304127ce0 2024-04-27T03:35:13.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.638+0000 7f0303fff640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f0304127ce0 0x7f03041280c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:35:13.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.638+0000 7f03037fe640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f0304070100 0x7f030412b000 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48600/0 (socket says 172.21.15.89:48600) 2024-04-27T03:35:13.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.638+0000 7f03037fe640 1 -- 172.21.15.89:0/2483663234 learned_addr learned my addr 172.21.15.89:0/2483663234 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:35:13.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.639+0000 7f0303fff640 1 -- 172.21.15.89:0/2483663234 >> v1:172.21.15.89:6789/0 conn(0x7f0304070100 legacy=0x7f030412b000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:35:13.645 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.639+0000 7f0303fff640 1 -- 172.21.15.89:0/2483663234 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0304098800 con 0x7f0304127ce0 2024-04-27T03:35:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.639+0000 7f0303fff640 1 --2- 172.21.15.89:0/2483663234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0304127ce0 0x7f03041280c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009720 tx=0x7f02f802b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8bcbdd89bb8087a server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:35:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.640+0000 7f0302ffd640 1 -- 172.21.15.89:0/2483663234 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02f80316d0 con 0x7f0304127ce0 2024-04-27T03:35:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.640+0000 7f0302ffd640 1 -- 172.21.15.89:0/2483663234 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f8037070 con 0x7f0304127ce0 2024-04-27T03:35:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.640+0000 7f0302ffd640 1 -- 172.21.15.89:0/2483663234 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02f80319c0 con 0x7f0304127ce0 2024-04-27T03:35:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.640+0000 7f030a40f640 1 -- 172.21.15.89:0/2483663234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0304127ce0 msgr2=0x7f03041280c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:35:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.640+0000 7f030a40f640 1 --2- 172.21.15.89:0/2483663234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0304127ce0 0x7f03041280c0 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009720 tx=0x7f02f802b840 comp rx=0 tx=0).stop 2024-04-27T03:35:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.640+0000 7f030a40f640 1 -- 172.21.15.89:0/2483663234 shutdown_connections 2024-04-27T03:35:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.640+0000 7f030a40f640 1 --2- 172.21.15.89:0/2483663234 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0304127ce0 0x7f03041280c0 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:35:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.640+0000 7f030a40f640 1 -- 172.21.15.89:0/2483663234 >> 172.21.15.89:0/2483663234 conn(0x7f0304099680 msgr2=0x7f0304099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:35:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.640+0000 7f030a40f640 1 -- 172.21.15.89:0/2483663234 shutdown_connections 2024-04-27T03:35:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.640+0000 7f030a40f640 1 -- 172.21.15.89:0/2483663234 wait complete. 2024-04-27T03:35:13.647 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.641+0000 7f030a40f640 1 Processor -- start 2024-04-27T03:35:13.648 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.641+0000 7f030a40f640 1 -- start start 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.642+0000 7f030a40f640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0304070100 0x7f03041bd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.642+0000 7f030a40f640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f0304128780 con 0x7f0304070100 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.642+0000 7f0303fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0304070100 0x7f03041bd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.642+0000 7f0303fff640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0304070100 0x7f03041bd8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41930/0 (socket says 172.21.15.89:41930) 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.642+0000 7f0303fff640 1 -- 172.21.15.89:0/3997091885 learned_addr learned my addr 172.21.15.89:0/3997091885 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.642+0000 7f0303fff640 1 -- 172.21.15.89:0/3997091885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f030408a580 con 0x7f0304070100 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.642+0000 7f0303fff640 1 --2- 172.21.15.89:0/3997091885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0304070100 0x7f03041bd8d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009720 tx=0x7f02f80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.642+0000 7f0300ff9640 1 -- 172.21.15.89:0/3997091885 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02f8032770 con 0x7f0304070100 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.642+0000 7f0300ff9640 1 -- 172.21.15.89:0/3997091885 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f02f8032910 con 0x7f0304070100 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.642+0000 7f030a40f640 1 -- 172.21.15.89:0/3997091885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0304124b30 con 0x7f0304070100 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.642+0000 7f0300ff9640 1 -- 172.21.15.89:0/3997091885 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f02f8032bf0 con 0x7f0304070100 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.642+0000 7f030a40f640 1 -- 172.21.15.89:0/3997091885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f03041257c0 con 0x7f0304070100 2024-04-27T03:35:13.650 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.643+0000 7f0300ff9640 1 -- 172.21.15.89:0/3997091885 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f02f804d3f0 con 0x7f0304070100 2024-04-27T03:35:13.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.643+0000 7f0300ff9640 1 --2- 172.21.15.89:0/3997091885 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f02f0038060 0x7f02f003a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:35:13.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.643+0000 7f0300ff9640 1 -- 172.21.15.89:0/3997091885 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f02f802f050 con 0x7f0304070100 2024-04-27T03:35:13.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.643+0000 7f030a40f640 1 -- 172.21.15.89:0/3997091885 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0304062da0 con 0x7f0304070100 2024-04-27T03:35:13.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.644+0000 7f03037fe640 1 --2- 172.21.15.89:0/3997091885 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f02f0038060 0x7f02f003a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:35:13.651 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.644+0000 7f03037fe640 1 --2- 172.21.15.89:0/3997091885 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f02f0038060 0x7f02f003a520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f02f4007fd0 tx=0x7f02f4007ac0 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:35:13.653 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.648+0000 7f0300ff9640 1 -- 172.21.15.89:0/3997091885 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f0304124b30 con 0x7f0304070100 2024-04-27T03:35:13.825 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.820+0000 7f030a40f640 1 -- 172.21.15.89:0/3997091885 --> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f030409ba00 con 0x7f02f0038060 2024-04-27T03:35:13.841 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.835+0000 7f0300ff9640 1 -- 172.21.15.89:0/3997091885 <== mgr.4102 v2:172.21.15.165:6824/3121580967 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+665809 (secure 0 0 0) 0x7f030409ba00 con 0x7f02f0038060 2024-04-27T03:35:13.849 INFO:teuthology.orchestra.run.smithi089.stdout: 2024-04-27T03:35:13.854 INFO:teuthology.orchestra.run.smithi089.stderr:dumped all 2024-04-27T03:35:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.840+0000 7f030a40f640 1 -- 172.21.15.89:0/3997091885 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f02f0038060 msgr2=0x7f02f003a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:35:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.840+0000 7f030a40f640 1 --2- 172.21.15.89:0/3997091885 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f02f0038060 0x7f02f003a520 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f02f4007fd0 tx=0x7f02f4007ac0 comp rx=0 tx=0).stop 2024-04-27T03:35:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.840+0000 7f030a40f640 1 -- 172.21.15.89:0/3997091885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0304070100 msgr2=0x7f03041bd8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:35:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.840+0000 7f030a40f640 1 --2- 172.21.15.89:0/3997091885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0304070100 0x7f03041bd8d0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f02f8009720 tx=0x7f02f80046c0 comp rx=0 tx=0).stop 2024-04-27T03:35:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.840+0000 7f030a40f640 1 -- 172.21.15.89:0/3997091885 shutdown_connections 2024-04-27T03:35:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.840+0000 7f030a40f640 1 --2- 172.21.15.89:0/3997091885 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f02f0038060 0x7f02f003a520 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:35:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.840+0000 7f030a40f640 1 --2- 172.21.15.89:0/3997091885 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f0304070100 0x7f03041bd8d0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:35:13.857 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.840+0000 7f030a40f640 1 -- 172.21.15.89:0/3997091885 >> 172.21.15.89:0/3997091885 conn(0x7f0304099680 msgr2=0x7f0304070ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:35:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.840+0000 7f030a40f640 1 -- 172.21.15.89:0/3997091885 shutdown_connections 2024-04-27T03:35:13.858 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.840+0000 7f030a40f640 1 -- 172.21.15.89:0/3997091885 wait complete. 2024-04-27T03:35:13.858 INFO:teuthology.orchestra.run.smithi089.stdout:{"pg_ready":true,"pg_map":{"version":178,"stamp":"2024-04-27T03:35:12.304809+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":605383,"num_objects":252,"num_object_clones":0,"num_object_copies":504,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":252,"num_whiteouts":0,"num_read":5378,"num_read_kb":4932,"num_write":2369,"num_write_kb":61373,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":18,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":3041,"num_omap_keys":16,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":5010,"ondisk_log_size":5010,"up":530,"acting":530,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":530,"num_osds":4,"num_per_pool_osds":4,"num_per_pool_omap_osds":4,"kb":377487360,"kb_used":156472,"kb_used_data":2648,"kb_used_omap":6,"kb_used_meta":153785,"kb_avail":377330888,"statfs":{"total":386547056640,"available":386386829312,"internally_reserved":0,"allocated":2711552,"data_stored":1657386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":6378,"internal_metadata":157476630},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.004174"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.834248+0000","last_change":"2024-04-27T03:34:07.834248+0000","last_active":"2024-04-27T03:34:07.834248+0000","last_peered":"2024-04-27T03:34:07.834248+0000","last_clean":"2024-04-27T03:34:07.834248+0000","last_became_active":"2024-04-27T03:30:11.362574+0000","last_became_peered":"2024-04-27T03:30:11.362574+0000","last_unstale":"2024-04-27T03:34:07.834248+0000","last_undegraded":"2024-04-27T03:34:07.834248+0000","last_fullsized":"2024-04-27T03:34:07.834248+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:07.834117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:07.834117+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.834117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:00:15.180175+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.38","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.077803+0000","last_change":"2024-04-27T03:33:56.077828+0000","last_active":"2024-04-27T03:33:56.077803+0000","last_peered":"2024-04-27T03:33:56.077803+0000","last_clean":"2024-04-27T03:33:56.077803+0000","last_became_active":"2024-04-27T03:30:13.384580+0000","last_became_peered":"2024-04-27T03:30:13.384580+0000","last_unstale":"2024-04-27T03:33:56.077803+0000","last_undegraded":"2024-04-27T03:33:56.077803+0000","last_fullsized":"2024-04-27T03:33:56.077803+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:56.077710+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:56.077710+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.077710+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:21.049969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3e","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.703599+0000","last_change":"2024-04-27T03:34:09.703599+0000","last_active":"2024-04-27T03:34:09.703599+0000","last_peered":"2024-04-27T03:34:09.703599+0000","last_clean":"2024-04-27T03:34:09.703599+0000","last_became_active":"2024-04-27T03:30:11.360857+0000","last_became_peered":"2024-04-27T03:30:11.360857+0000","last_unstale":"2024-04-27T03:34:09.703599+0000","last_undegraded":"2024-04-27T03:34:09.703599+0000","last_fullsized":"2024-04-27T03:34:09.703599+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:09.703487+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:09.703487+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.703487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:08:10.048372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":5,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.39","version":"27'12","reported_seq":88,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:21.258937+0000","last_change":"2024-04-27T03:34:21.258982+0000","last_active":"2024-04-27T03:34:21.258937+0000","last_peered":"2024-04-27T03:34:21.258937+0000","last_clean":"2024-04-27T03:34:21.258937+0000","last_became_active":"2024-04-27T03:30:13.376221+0000","last_became_peered":"2024-04-27T03:30:13.376221+0000","last_unstale":"2024-04-27T03:34:21.258937+0000","last_undegraded":"2024-04-27T03:34:21.258937+0000","last_fullsized":"2024-04-27T03:34:21.258937+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-04-27T03:34:21.258835+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-04-27T03:34:21.258835+0000","last_clean_scrub_stamp":"2024-04-27T03:34:21.258835+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:22.651830+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":52,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:27.334011+0000","last_change":"2024-04-27T03:34:27.334036+0000","last_active":"2024-04-27T03:34:27.334011+0000","last_peered":"2024-04-27T03:34:27.334011+0000","last_clean":"2024-04-27T03:34:27.334011+0000","last_became_active":"2024-04-27T03:30:11.362739+0000","last_became_peered":"2024-04-27T03:30:11.362739+0000","last_unstale":"2024-04-27T03:34:27.334011+0000","last_undegraded":"2024-04-27T03:34:27.334011+0000","last_fullsized":"2024-04-27T03:34:27.334011+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:27.333877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:27.333877+0000","last_clean_scrub_stamp":"2024-04-27T03:34:27.333877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:14:08.432542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3a","version":"27'8","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:18.221789+0000","last_change":"2024-04-27T03:34:18.221821+0000","last_active":"2024-04-27T03:34:18.221789+0000","last_peered":"2024-04-27T03:34:18.221789+0000","last_clean":"2024-04-27T03:34:18.221789+0000","last_became_active":"2024-04-27T03:30:13.376524+0000","last_became_peered":"2024-04-27T03:30:13.376524+0000","last_unstale":"2024-04-27T03:34:18.221789+0000","last_undegraded":"2024-04-27T03:34:18.221789+0000","last_fullsized":"2024-04-27T03:34:18.221789+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-04-27T03:34:18.221705+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-04-27T03:34:18.221705+0000","last_clean_scrub_stamp":"2024-04-27T03:34:18.221705+0000","objects_scrubbed":1,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:20.194608+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":7,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3c","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:15.859998+0000","last_change":"2024-04-27T03:34:15.860023+0000","last_active":"2024-04-27T03:34:15.859998+0000","last_peered":"2024-04-27T03:34:15.859998+0000","last_clean":"2024-04-27T03:34:15.859998+0000","last_became_active":"2024-04-27T03:30:11.360188+0000","last_became_peered":"2024-04-27T03:30:11.360188+0000","last_unstale":"2024-04-27T03:34:15.859998+0000","last_undegraded":"2024-04-27T03:34:15.859998+0000","last_fullsized":"2024-04-27T03:34:15.859998+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:15.859866+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:15.859866+0000","last_clean_scrub_stamp":"2024-04-27T03:34:15.859866+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:16.509376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3b","version":"27'16","reported_seq":64,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:20.246700+0000","last_change":"2024-04-27T03:34:20.246757+0000","last_active":"2024-04-27T03:34:20.246700+0000","last_peered":"2024-04-27T03:34:20.246700+0000","last_clean":"2024-04-27T03:34:20.246700+0000","last_became_active":"2024-04-27T03:30:13.376288+0000","last_became_peered":"2024-04-27T03:30:13.376288+0000","last_unstale":"2024-04-27T03:34:20.246700+0000","last_undegraded":"2024-04-27T03:34:20.246700+0000","last_fullsized":"2024-04-27T03:34:20.246700+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:20.246599+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:20.246599+0000","last_clean_scrub_stamp":"2024-04-27T03:34:20.246599+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:48:03.552732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":52,"num_read_kb":40,"num_write":22,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:47.331517+0000","last_change":"2024-04-27T03:34:47.331559+0000","last_active":"2024-04-27T03:34:47.331517+0000","last_peered":"2024-04-27T03:34:47.331517+0000","last_clean":"2024-04-27T03:34:47.331517+0000","last_became_active":"2024-04-27T03:30:11.354989+0000","last_became_peered":"2024-04-27T03:30:11.354989+0000","last_unstale":"2024-04-27T03:34:47.331517+0000","last_undegraded":"2024-04-27T03:34:47.331517+0000","last_fullsized":"2024-04-27T03:34:47.331517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:47.331390+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:47.331390+0000","last_clean_scrub_stamp":"2024-04-27T03:34:47.331390+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:36:56.995689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3c","version":"27'6","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.200104+0000","last_change":"2024-04-27T03:34:04.200104+0000","last_active":"2024-04-27T03:34:04.200104+0000","last_peered":"2024-04-27T03:34:04.200104+0000","last_clean":"2024-04-27T03:34:04.200104+0000","last_became_active":"2024-04-27T03:30:13.373424+0000","last_became_peered":"2024-04-27T03:30:13.373424+0000","last_unstale":"2024-04-27T03:34:04.200104+0000","last_undegraded":"2024-04-27T03:34:04.200104+0000","last_fullsized":"2024-04-27T03:34:04.200104+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:04.199924+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:04.199924+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.199924+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:59.695293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":8,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3a","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:14.842885+0000","last_change":"2024-04-27T03:34:14.842915+0000","last_active":"2024-04-27T03:34:14.842885+0000","last_peered":"2024-04-27T03:34:14.842885+0000","last_clean":"2024-04-27T03:34:14.842885+0000","last_became_active":"2024-04-27T03:30:11.360219+0000","last_became_peered":"2024-04-27T03:30:11.360219+0000","last_unstale":"2024-04-27T03:34:14.842885+0000","last_undegraded":"2024-04-27T03:34:14.842885+0000","last_fullsized":"2024-04-27T03:34:14.842885+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:14.842794+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:14.842794+0000","last_clean_scrub_stamp":"2024-04-27T03:34:14.842794+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:59.247435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3d","version":"25'8","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.275565+0000","last_change":"2024-04-27T03:34:05.275565+0000","last_active":"2024-04-27T03:34:05.275565+0000","last_peered":"2024-04-27T03:34:05.275565+0000","last_clean":"2024-04-27T03:34:05.275565+0000","last_became_active":"2024-04-27T03:30:13.385665+0000","last_became_peered":"2024-04-27T03:30:13.385665+0000","last_unstale":"2024-04-27T03:34:05.275565+0000","last_undegraded":"2024-04-27T03:34:05.275565+0000","last_fullsized":"2024-04-27T03:34:05.275565+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'8","last_scrub_stamp":"2024-04-27T03:34:05.275413+0000","last_deep_scrub":"25'8","last_deep_scrub_stamp":"2024-04-27T03:34:05.275413+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.275413+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:20:59.991475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":34,"num_read_kb":30,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:49.333805+0000","last_change":"2024-04-27T03:34:49.333830+0000","last_active":"2024-04-27T03:34:49.333805+0000","last_peered":"2024-04-27T03:34:49.333805+0000","last_clean":"2024-04-27T03:34:49.333805+0000","last_became_active":"2024-04-27T03:30:11.372704+0000","last_became_peered":"2024-04-27T03:30:11.372704+0000","last_unstale":"2024-04-27T03:34:49.333805+0000","last_undegraded":"2024-04-27T03:34:49.333805+0000","last_fullsized":"2024-04-27T03:34:49.333805+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:49.333690+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:49.333690+0000","last_clean_scrub_stamp":"2024-04-27T03:34:49.333690+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:13:17.426476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.3e","version":"27'4","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.771780+0000","last_change":"2024-04-27T03:33:56.771780+0000","last_active":"2024-04-27T03:33:56.771780+0000","last_peered":"2024-04-27T03:33:56.771780+0000","last_clean":"2024-04-27T03:33:56.771780+0000","last_became_active":"2024-04-27T03:30:13.374199+0000","last_became_peered":"2024-04-27T03:30:13.374199+0000","last_unstale":"2024-04-27T03:33:56.771780+0000","last_undegraded":"2024-04-27T03:33:56.771780+0000","last_fullsized":"2024-04-27T03:33:56.771780+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:56.771668+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:56.771668+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.771668+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:18.489287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.38","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:19.635377+0000","last_change":"2024-04-27T03:34:19.635403+0000","last_active":"2024-04-27T03:34:19.635377+0000","last_peered":"2024-04-27T03:34:19.635377+0000","last_clean":"2024-04-27T03:34:19.635377+0000","last_became_active":"2024-04-27T03:30:11.358181+0000","last_became_peered":"2024-04-27T03:30:11.358181+0000","last_unstale":"2024-04-27T03:34:19.635377+0000","last_undegraded":"2024-04-27T03:34:19.635377+0000","last_fullsized":"2024-04-27T03:34:19.635377+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:19.635237+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:19.635237+0000","last_clean_scrub_stamp":"2024-04-27T03:34:19.635237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:22.761966+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3f","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.810889+0000","last_change":"2024-04-27T03:34:00.810889+0000","last_active":"2024-04-27T03:34:00.810889+0000","last_peered":"2024-04-27T03:34:00.810889+0000","last_clean":"2024-04-27T03:34:00.810889+0000","last_became_active":"2024-04-27T03:30:13.379059+0000","last_became_peered":"2024-04-27T03:30:13.379059+0000","last_unstale":"2024-04-27T03:34:00.810889+0000","last_undegraded":"2024-04-27T03:34:00.810889+0000","last_fullsized":"2024-04-27T03:34:00.810889+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:00.810774+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:00.810774+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.810774+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:01.151920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.30","version":"27'6","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.269111+0000","last_change":"2024-04-27T03:34:06.269111+0000","last_active":"2024-04-27T03:34:06.269111+0000","last_peered":"2024-04-27T03:34:06.269111+0000","last_clean":"2024-04-27T03:34:06.269111+0000","last_became_active":"2024-04-27T03:30:13.383694+0000","last_became_peered":"2024-04-27T03:30:13.383694+0000","last_unstale":"2024-04-27T03:34:06.269111+0000","last_undegraded":"2024-04-27T03:34:06.269111+0000","last_fullsized":"2024-04-27T03:34:06.269111+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:06.268965+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:06.268965+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.268965+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:24:49.525413+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:24.057220+0000","last_change":"2024-04-27T03:34:24.057220+0000","last_active":"2024-04-27T03:34:24.057220+0000","last_peered":"2024-04-27T03:34:24.057220+0000","last_clean":"2024-04-27T03:34:24.057220+0000","last_became_active":"2024-04-27T03:30:11.370157+0000","last_became_peered":"2024-04-27T03:30:11.370157+0000","last_unstale":"2024-04-27T03:34:24.057220+0000","last_undegraded":"2024-04-27T03:34:24.057220+0000","last_fullsized":"2024-04-27T03:34:24.057220+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:24.057056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:24.057056+0000","last_clean_scrub_stamp":"2024-04-27T03:34:24.057056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:46.521120+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.31","version":"27'6","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.201590+0000","last_change":"2024-04-27T03:34:03.201590+0000","last_active":"2024-04-27T03:34:03.201590+0000","last_peered":"2024-04-27T03:34:03.201590+0000","last_clean":"2024-04-27T03:34:03.201590+0000","last_became_active":"2024-04-27T03:30:13.372973+0000","last_became_peered":"2024-04-27T03:30:13.372973+0000","last_unstale":"2024-04-27T03:34:03.201590+0000","last_undegraded":"2024-04-27T03:34:03.201590+0000","last_fullsized":"2024-04-27T03:34:03.201590+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:03.201445+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:03.201445+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.201445+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:20:57.765391+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.817515+0000","last_change":"2024-04-27T03:34:03.817515+0000","last_active":"2024-04-27T03:34:03.817515+0000","last_peered":"2024-04-27T03:34:03.817515+0000","last_clean":"2024-04-27T03:34:03.817515+0000","last_became_active":"2024-04-27T03:30:11.363109+0000","last_became_peered":"2024-04-27T03:30:11.363109+0000","last_unstale":"2024-04-27T03:34:03.817515+0000","last_undegraded":"2024-04-27T03:34:03.817515+0000","last_fullsized":"2024-04-27T03:34:03.817515+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:03.817361+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:03.817361+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.817361+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:30.353138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"27'10","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.121867+0000","last_change":"2024-04-27T03:33:58.121867+0000","last_active":"2024-04-27T03:33:58.121867+0000","last_peered":"2024-04-27T03:33:58.121867+0000","last_clean":"2024-04-27T03:33:58.121867+0000","last_became_active":"2024-04-27T03:30:13.383742+0000","last_became_peered":"2024-04-27T03:30:13.383742+0000","last_unstale":"2024-04-27T03:33:58.121867+0000","last_undegraded":"2024-04-27T03:33:58.121867+0000","last_fullsized":"2024-04-27T03:33:58.121867+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:33:58.121739+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:33:58.121739+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.121739+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:37.923650+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":19,"num_read_kb":15,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.35","version":"25'4","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:26.075209+0000","last_change":"2024-04-27T03:34:26.075209+0000","last_active":"2024-04-27T03:34:26.075209+0000","last_peered":"2024-04-27T03:34:26.075209+0000","last_clean":"2024-04-27T03:34:26.075209+0000","last_became_active":"2024-04-27T03:30:11.370380+0000","last_became_peered":"2024-04-27T03:30:11.370380+0000","last_unstale":"2024-04-27T03:34:26.075209+0000","last_undegraded":"2024-04-27T03:34:26.075209+0000","last_fullsized":"2024-04-27T03:34:26.075209+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'4","last_scrub_stamp":"2024-04-27T03:34:26.075076+0000","last_deep_scrub":"25'4","last_deep_scrub_stamp":"2024-04-27T03:34:26.075076+0000","last_clean_scrub_stamp":"2024-04-27T03:34:26.075076+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:58.085501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":17,"num_read_kb":10,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.33","version":"27'6","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:31.721798+0000","last_change":"2024-04-27T03:34:31.721798+0000","last_active":"2024-04-27T03:34:31.721798+0000","last_peered":"2024-04-27T03:34:31.721798+0000","last_clean":"2024-04-27T03:34:31.721798+0000","last_became_active":"2024-04-27T03:31:01.440762+0000","last_became_peered":"2024-04-27T03:31:01.440762+0000","last_unstale":"2024-04-27T03:34:31.721798+0000","last_undegraded":"2024-04-27T03:34:31.721798+0000","last_fullsized":"2024-04-27T03:34:31.721798+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:31.721686+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:31.721686+0000","last_clean_scrub_stamp":"2024-04-27T03:34:31.721686+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:53:14.945006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.34","version":"27'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:21.120644+0000","last_change":"2024-04-27T03:34:21.120644+0000","last_active":"2024-04-27T03:34:21.120644+0000","last_peered":"2024-04-27T03:34:21.120644+0000","last_clean":"2024-04-27T03:34:21.120644+0000","last_became_active":"2024-04-27T03:30:11.369551+0000","last_became_peered":"2024-04-27T03:30:11.369551+0000","last_unstale":"2024-04-27T03:34:21.120644+0000","last_undegraded":"2024-04-27T03:34:21.120644+0000","last_fullsized":"2024-04-27T03:34:21.120644+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:21.120533+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:21.120533+0000","last_clean_scrub_stamp":"2024-04-27T03:34:21.120533+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:45:54.803174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.34","version":"27'2","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.850412+0000","last_change":"2024-04-27T03:34:04.850412+0000","last_active":"2024-04-27T03:34:04.850412+0000","last_peered":"2024-04-27T03:34:04.850412+0000","last_clean":"2024-04-27T03:34:04.850412+0000","last_became_active":"2024-04-27T03:30:13.381901+0000","last_became_peered":"2024-04-27T03:30:13.381901+0000","last_unstale":"2024-04-27T03:34:04.850412+0000","last_undegraded":"2024-04-27T03:34:04.850412+0000","last_fullsized":"2024-04-27T03:34:04.850412+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:04.850293+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:04.850293+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.850293+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:21:29.194843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.33","version":"25'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:15.639038+0000","last_change":"2024-04-27T03:34:15.639038+0000","last_active":"2024-04-27T03:34:15.639038+0000","last_peered":"2024-04-27T03:34:15.639038+0000","last_clean":"2024-04-27T03:34:15.639038+0000","last_became_active":"2024-04-27T03:30:11.360515+0000","last_became_peered":"2024-04-27T03:30:11.360515+0000","last_unstale":"2024-04-27T03:34:15.639038+0000","last_undegraded":"2024-04-27T03:34:15.639038+0000","last_fullsized":"2024-04-27T03:34:15.639038+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:15.638909+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:15.638909+0000","last_clean_scrub_stamp":"2024-04-27T03:34:15.638909+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:04.458649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.35","version":"27'4","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.155166+0000","last_change":"2024-04-27T03:33:59.155166+0000","last_active":"2024-04-27T03:33:59.155166+0000","last_peered":"2024-04-27T03:33:59.155166+0000","last_clean":"2024-04-27T03:33:59.155166+0000","last_became_active":"2024-04-27T03:30:13.378932+0000","last_became_peered":"2024-04-27T03:30:13.378932+0000","last_unstale":"2024-04-27T03:33:59.155166+0000","last_undegraded":"2024-04-27T03:33:59.155166+0000","last_fullsized":"2024-04-27T03:33:59.155166+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:59.155047+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:59.155047+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.155047+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:36.999661+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:37.333059+0000","last_change":"2024-04-27T03:34:37.333059+0000","last_active":"2024-04-27T03:34:37.333059+0000","last_peered":"2024-04-27T03:34:37.333059+0000","last_clean":"2024-04-27T03:34:37.333059+0000","last_became_active":"2024-04-27T03:30:11.372673+0000","last_became_peered":"2024-04-27T03:30:11.372673+0000","last_unstale":"2024-04-27T03:34:37.333059+0000","last_undegraded":"2024-04-27T03:34:37.333059+0000","last_fullsized":"2024-04-27T03:34:37.333059+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:37.332923+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:37.332923+0000","last_clean_scrub_stamp":"2024-04-27T03:34:37.332923+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:44:48.977095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"27'36","reported_seq":109,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:30.101093+0000","last_change":"2024-04-27T03:34:30.101126+0000","last_active":"2024-04-27T03:34:30.101093+0000","last_peered":"2024-04-27T03:34:30.101093+0000","last_clean":"2024-04-27T03:34:30.101093+0000","last_became_active":"2024-04-27T03:30:21.413255+0000","last_became_peered":"2024-04-27T03:30:21.413255+0000","last_unstale":"2024-04-27T03:34:30.101093+0000","last_undegraded":"2024-04-27T03:34:30.101093+0000","last_fullsized":"2024-04-27T03:34:30.101093+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-04-27T03:34:30.100998+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-04-27T03:34:30.100998+0000","last_clean_scrub_stamp":"2024-04-27T03:34:30.100998+0000","objects_scrubbed":1,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:07:40.938805+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":7,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":170,"num_read_kb":170,"num_write":97,"num_write_kb":32,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.14","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:16.259291+0000","last_change":"2024-04-27T03:34:16.259291+0000","last_active":"2024-04-27T03:34:16.259291+0000","last_peered":"2024-04-27T03:34:16.259291+0000","last_clean":"2024-04-27T03:34:16.259291+0000","last_became_active":"2024-04-27T03:30:13.377110+0000","last_became_peered":"2024-04-27T03:30:13.377110+0000","last_unstale":"2024-04-27T03:34:16.259291+0000","last_undegraded":"2024-04-27T03:34:16.259291+0000","last_fullsized":"2024-04-27T03:34:16.259291+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:16.259165+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:16.259165+0000","last_clean_scrub_stamp":"2024-04-27T03:34:16.259165+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:04:04.414839+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:30.301717+0000","last_change":"2024-04-27T03:34:30.301717+0000","last_active":"2024-04-27T03:34:30.301717+0000","last_peered":"2024-04-27T03:34:30.301717+0000","last_clean":"2024-04-27T03:34:30.301717+0000","last_became_active":"2024-04-27T03:30:11.359892+0000","last_became_peered":"2024-04-27T03:30:11.359892+0000","last_unstale":"2024-04-27T03:34:30.301717+0000","last_undegraded":"2024-04-27T03:34:30.301717+0000","last_fullsized":"2024-04-27T03:34:30.301717+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:30.301573+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:30.301573+0000","last_clean_scrub_stamp":"2024-04-27T03:34:30.301573+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:05:34.290404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.148173+0000","last_change":"2024-04-27T03:33:38.148173+0000","last_active":"2024-04-27T03:33:38.148173+0000","last_peered":"2024-04-27T03:33:38.148173+0000","last_clean":"2024-04-27T03:33:38.148173+0000","last_became_active":"2024-04-27T03:30:15.380864+0000","last_became_peered":"2024-04-27T03:30:15.380864+0000","last_unstale":"2024-04-27T03:33:38.148173+0000","last_undegraded":"2024-04-27T03:33:38.148173+0000","last_fullsized":"2024-04-27T03:33:38.148173+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:38.148061+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:38.148061+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.148061+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:12:53.865374+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.16","version":"27'50","reported_seq":118,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.088242+0000","last_change":"2024-04-27T03:33:43.088242+0000","last_active":"2024-04-27T03:33:43.088242+0000","last_peered":"2024-04-27T03:33:43.088242+0000","last_clean":"2024-04-27T03:33:43.088242+0000","last_became_active":"2024-04-27T03:30:17.387391+0000","last_became_peered":"2024-04-27T03:30:17.387391+0000","last_unstale":"2024-04-27T03:33:43.088242+0000","last_undegraded":"2024-04-27T03:33:43.088242+0000","last_fullsized":"2024-04-27T03:33:43.088242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'50","last_scrub_stamp":"2024-04-27T03:33:43.088100+0000","last_deep_scrub":"27'50","last_deep_scrub_stamp":"2024-04-27T03:33:43.088100+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.088100+0000","objects_scrubbed":6,"log_size":50,"log_dups_size":0,"ondisk_log_size":50,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:13:27.492179+0000","scrub_duration":7,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":45,"num_write":26,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.971352+0000","last_change":"2024-04-27T03:33:44.971352+0000","last_active":"2024-04-27T03:33:44.971352+0000","last_peered":"2024-04-27T03:33:44.971352+0000","last_clean":"2024-04-27T03:33:44.971352+0000","last_became_active":"2024-04-27T03:30:19.407637+0000","last_became_peered":"2024-04-27T03:30:19.407637+0000","last_unstale":"2024-04-27T03:33:44.971352+0000","last_undegraded":"2024-04-27T03:33:44.971352+0000","last_fullsized":"2024-04-27T03:33:44.971352+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:44.971239+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:44.971239+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.971239+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:22.448440+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.19","version":"27'9","reported_seq":36,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:31.139111+0000","last_change":"2024-04-27T03:34:31.139137+0000","last_active":"2024-04-27T03:34:31.139111+0000","last_peered":"2024-04-27T03:34:31.139111+0000","last_clean":"2024-04-27T03:34:31.139111+0000","last_became_active":"2024-04-27T03:30:21.413940+0000","last_became_peered":"2024-04-27T03:30:21.413940+0000","last_unstale":"2024-04-27T03:34:31.139111+0000","last_undegraded":"2024-04-27T03:34:31.139111+0000","last_fullsized":"2024-04-27T03:34:31.139111+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-04-27T03:34:31.139021+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-04-27T03:34:31.139021+0000","last_clean_scrub_stamp":"2024-04-27T03:34:31.139021+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:52.945498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":16,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.15","version":"27'16","reported_seq":77,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.266003+0000","last_change":"2024-04-27T03:34:09.266003+0000","last_active":"2024-04-27T03:34:09.266003+0000","last_peered":"2024-04-27T03:34:09.266003+0000","last_clean":"2024-04-27T03:34:09.266003+0000","last_became_active":"2024-04-27T03:30:13.381487+0000","last_became_peered":"2024-04-27T03:30:13.381487+0000","last_unstale":"2024-04-27T03:34:09.266003+0000","last_undegraded":"2024-04-27T03:34:09.266003+0000","last_fullsized":"2024-04-27T03:34:09.266003+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:09.265817+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:09.265817+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.265817+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:49:46.218428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":50,"num_read_kb":44,"num_write":19,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:13.813035+0000","last_change":"2024-04-27T03:34:13.813035+0000","last_active":"2024-04-27T03:34:13.813035+0000","last_peered":"2024-04-27T03:34:13.813035+0000","last_clean":"2024-04-27T03:34:13.813035+0000","last_became_active":"2024-04-27T03:30:11.359446+0000","last_became_peered":"2024-04-27T03:30:11.359446+0000","last_unstale":"2024-04-27T03:34:13.813035+0000","last_undegraded":"2024-04-27T03:34:13.813035+0000","last_fullsized":"2024-04-27T03:34:13.813035+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:13.812877+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:13.812877+0000","last_clean_scrub_stamp":"2024-04-27T03:34:13.812877+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:26:46.859807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.671603+0000","last_change":"2024-04-27T03:33:55.671603+0000","last_active":"2024-04-27T03:33:55.671603+0000","last_peered":"2024-04-27T03:33:55.671603+0000","last_clean":"2024-04-27T03:33:55.671603+0000","last_became_active":"2024-04-27T03:30:15.380313+0000","last_became_peered":"2024-04-27T03:30:15.380313+0000","last_unstale":"2024-04-27T03:33:55.671603+0000","last_undegraded":"2024-04-27T03:33:55.671603+0000","last_fullsized":"2024-04-27T03:33:55.671603+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:55.671471+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:55.671471+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.671471+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:20.643729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.17","version":"27'16","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.841840+0000","last_change":"2024-04-27T03:33:37.841840+0000","last_active":"2024-04-27T03:33:37.841840+0000","last_peered":"2024-04-27T03:33:37.841840+0000","last_clean":"2024-04-27T03:33:37.841840+0000","last_became_active":"2024-04-27T03:30:17.389126+0000","last_became_peered":"2024-04-27T03:30:17.389126+0000","last_unstale":"2024-04-27T03:33:37.841840+0000","last_undegraded":"2024-04-27T03:33:37.841840+0000","last_fullsized":"2024-04-27T03:33:37.841840+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:33:37.841643+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:33:37.841643+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.841643+0000","objects_scrubbed":4,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:42.665962+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":18,"num_read_kb":14,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.837497+0000","last_change":"2024-04-27T03:33:36.837497+0000","last_active":"2024-04-27T03:33:36.837497+0000","last_peered":"2024-04-27T03:33:36.837497+0000","last_clean":"2024-04-27T03:33:36.837497+0000","last_became_active":"2024-04-27T03:30:19.405170+0000","last_became_peered":"2024-04-27T03:30:19.405170+0000","last_unstale":"2024-04-27T03:33:36.837497+0000","last_undegraded":"2024-04-27T03:33:36.837497+0000","last_fullsized":"2024-04-27T03:33:36.837497+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.837388+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.837388+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.837388+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:14.962289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1a","version":"27'6","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.661566+0000","last_change":"2024-04-27T03:33:58.661623+0000","last_active":"2024-04-27T03:33:58.661566+0000","last_peered":"2024-04-27T03:33:58.661566+0000","last_clean":"2024-04-27T03:33:58.661566+0000","last_became_active":"2024-04-27T03:30:21.407917+0000","last_became_peered":"2024-04-27T03:30:21.407917+0000","last_unstale":"2024-04-27T03:33:58.661566+0000","last_undegraded":"2024-04-27T03:33:58.661566+0000","last_fullsized":"2024-04-27T03:33:58.661566+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:58.661469+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:58.661469+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.661469+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:46:52.661163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":11,"num_write":17,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.16","version":"27'18","reported_seq":105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:12.193369+0000","last_change":"2024-04-27T03:34:12.193369+0000","last_active":"2024-04-27T03:34:12.193369+0000","last_peered":"2024-04-27T03:34:12.193369+0000","last_clean":"2024-04-27T03:34:12.193369+0000","last_became_active":"2024-04-27T03:30:13.377423+0000","last_became_peered":"2024-04-27T03:30:13.377423+0000","last_unstale":"2024-04-27T03:34:12.193369+0000","last_undegraded":"2024-04-27T03:34:12.193369+0000","last_fullsized":"2024-04-27T03:34:12.193369+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-04-27T03:34:12.193224+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-04-27T03:34:12.193224+0000","last_clean_scrub_stamp":"2024-04-27T03:34:12.193224+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:00:13.595186+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":67,"num_read_kb":67,"num_write":18,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:32.333004+0000","last_change":"2024-04-27T03:34:32.333004+0000","last_active":"2024-04-27T03:34:32.333004+0000","last_peered":"2024-04-27T03:34:32.333004+0000","last_clean":"2024-04-27T03:34:32.333004+0000","last_became_active":"2024-04-27T03:30:11.359261+0000","last_became_peered":"2024-04-27T03:30:11.359261+0000","last_unstale":"2024-04-27T03:34:32.333004+0000","last_undegraded":"2024-04-27T03:34:32.333004+0000","last_fullsized":"2024-04-27T03:34:32.333004+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:32.332847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:32.332847+0000","last_clean_scrub_stamp":"2024-04-27T03:34:32.332847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:01:05.240389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.753294+0000","last_change":"2024-04-27T03:33:47.753294+0000","last_active":"2024-04-27T03:33:47.753294+0000","last_peered":"2024-04-27T03:33:47.753294+0000","last_clean":"2024-04-27T03:33:47.753294+0000","last_became_active":"2024-04-27T03:30:15.383618+0000","last_became_peered":"2024-04-27T03:30:15.383618+0000","last_unstale":"2024-04-27T03:33:47.753294+0000","last_undegraded":"2024-04-27T03:33:47.753294+0000","last_fullsized":"2024-04-27T03:33:47.753294+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:47.753177+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:47.753177+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.753177+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:54.718147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.14","version":"27'30","reported_seq":82,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.888412+0000","last_change":"2024-04-27T03:33:37.888412+0000","last_active":"2024-04-27T03:33:37.888412+0000","last_peered":"2024-04-27T03:33:37.888412+0000","last_clean":"2024-04-27T03:33:37.888412+0000","last_became_active":"2024-04-27T03:30:17.392223+0000","last_became_peered":"2024-04-27T03:30:17.392223+0000","last_unstale":"2024-04-27T03:33:37.888412+0000","last_undegraded":"2024-04-27T03:33:37.888412+0000","last_fullsized":"2024-04-27T03:33:37.888412+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'30","last_scrub_stamp":"2024-04-27T03:33:37.888233+0000","last_deep_scrub":"27'30","last_deep_scrub_stamp":"2024-04-27T03:33:37.888233+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.888233+0000","objects_scrubbed":7,"log_size":30,"log_dups_size":0,"ondisk_log_size":30,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:27:30.955280+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":33,"num_read_kb":26,"num_write":22,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.15","version":"27'286","reported_seq":395,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.206432+0000","last_change":"2024-04-27T03:34:03.206432+0000","last_active":"2024-04-27T03:34:03.206432+0000","last_peered":"2024-04-27T03:34:03.206432+0000","last_clean":"2024-04-27T03:34:03.206432+0000","last_became_active":"2024-04-27T03:30:19.405730+0000","last_became_peered":"2024-04-27T03:30:19.405730+0000","last_unstale":"2024-04-27T03:34:03.206432+0000","last_undegraded":"2024-04-27T03:34:03.206432+0000","last_fullsized":"2024-04-27T03:34:03.206432+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'286","last_scrub_stamp":"2024-04-27T03:34:03.206276+0000","last_deep_scrub":"27'286","last_deep_scrub_stamp":"2024-04-27T03:34:03.206276+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.206276+0000","objects_scrubbed":1,"log_size":286,"log_dups_size":0,"ondisk_log_size":286,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:25.670340+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:19.898900+0000","last_change":"2024-04-27T03:34:19.898939+0000","last_active":"2024-04-27T03:34:19.898900+0000","last_peered":"2024-04-27T03:34:19.898900+0000","last_clean":"2024-04-27T03:34:19.898900+0000","last_became_active":"2024-04-27T03:30:21.409156+0000","last_became_peered":"2024-04-27T03:30:21.409156+0000","last_unstale":"2024-04-27T03:34:19.898900+0000","last_undegraded":"2024-04-27T03:34:19.898900+0000","last_fullsized":"2024-04-27T03:34:19.898900+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:19.898783+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:19.898783+0000","last_clean_scrub_stamp":"2024-04-27T03:34:19.898783+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:23:50.887010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.17","version":"27'6","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.704499+0000","last_change":"2024-04-27T03:33:59.704499+0000","last_active":"2024-04-27T03:33:59.704499+0000","last_peered":"2024-04-27T03:33:59.704499+0000","last_clean":"2024-04-27T03:33:59.704499+0000","last_became_active":"2024-04-27T03:30:13.385491+0000","last_became_peered":"2024-04-27T03:30:13.385491+0000","last_unstale":"2024-04-27T03:33:59.704499+0000","last_undegraded":"2024-04-27T03:33:59.704499+0000","last_fullsized":"2024-04-27T03:33:59.704499+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:59.704383+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:59.704383+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.704383+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:13:15.421696+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:23.050167+0000","last_change":"2024-04-27T03:34:23.050167+0000","last_active":"2024-04-27T03:34:23.050167+0000","last_peered":"2024-04-27T03:34:23.050167+0000","last_clean":"2024-04-27T03:34:23.050167+0000","last_became_active":"2024-04-27T03:30:11.369939+0000","last_became_peered":"2024-04-27T03:30:11.369939+0000","last_unstale":"2024-04-27T03:34:23.050167+0000","last_undegraded":"2024-04-27T03:34:23.050167+0000","last_fullsized":"2024-04-27T03:34:23.050167+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:23.050056+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:23.050056+0000","last_clean_scrub_stamp":"2024-04-27T03:34:23.050056+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:26:29.085590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.134999+0000","last_change":"2024-04-27T03:33:33.134999+0000","last_active":"2024-04-27T03:33:33.134999+0000","last_peered":"2024-04-27T03:33:33.134999+0000","last_clean":"2024-04-27T03:33:33.134999+0000","last_became_active":"2024-04-27T03:30:15.384114+0000","last_became_peered":"2024-04-27T03:30:15.384114+0000","last_unstale":"2024-04-27T03:33:33.134999+0000","last_undegraded":"2024-04-27T03:33:33.134999+0000","last_fullsized":"2024-04-27T03:33:33.134999+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:33.134840+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:33.134840+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.134840+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:20:57.479389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"27'43","reported_seq":105,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.859665+0000","last_change":"2024-04-27T03:33:38.859665+0000","last_active":"2024-04-27T03:33:38.859665+0000","last_peered":"2024-04-27T03:33:38.859665+0000","last_clean":"2024-04-27T03:33:38.859665+0000","last_became_active":"2024-04-27T03:30:17.391880+0000","last_became_peered":"2024-04-27T03:30:17.391880+0000","last_unstale":"2024-04-27T03:33:38.859665+0000","last_undegraded":"2024-04-27T03:33:38.859665+0000","last_fullsized":"2024-04-27T03:33:38.859665+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'43","last_scrub_stamp":"2024-04-27T03:33:38.859525+0000","last_deep_scrub":"27'43","last_deep_scrub_stamp":"2024-04-27T03:33:38.859525+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.859525+0000","objects_scrubbed":5,"log_size":43,"log_dups_size":0,"ondisk_log_size":43,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:41:06.442676+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.14","version":"27'285","reported_seq":418,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.785553+0000","last_change":"2024-04-27T03:33:35.785553+0000","last_active":"2024-04-27T03:33:35.785553+0000","last_peered":"2024-04-27T03:33:35.785553+0000","last_clean":"2024-04-27T03:33:35.785553+0000","last_became_active":"2024-04-27T03:30:19.404744+0000","last_became_peered":"2024-04-27T03:30:19.404744+0000","last_unstale":"2024-04-27T03:33:35.785553+0000","last_undegraded":"2024-04-27T03:33:35.785553+0000","last_fullsized":"2024-04-27T03:33:35.785553+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:35.785437+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:35.785437+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.785437+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:23.864435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.8","version":"27'7","reported_seq":62,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.165611+0000","last_change":"2024-04-27T03:33:49.165611+0000","last_active":"2024-04-27T03:33:49.165611+0000","last_peered":"2024-04-27T03:33:49.165611+0000","last_clean":"2024-04-27T03:33:49.165611+0000","last_became_active":"2024-04-27T03:30:13.387335+0000","last_became_peered":"2024-04-27T03:30:13.387335+0000","last_unstale":"2024-04-27T03:33:49.165611+0000","last_undegraded":"2024-04-27T03:33:49.165611+0000","last_fullsized":"2024-04-27T03:33:49.165611+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:33:49.165500+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:33:49.165500+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.165500+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:22.603187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":31,"num_read_kb":27,"num_write":9,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.841348+0000","last_change":"2024-04-27T03:34:01.841348+0000","last_active":"2024-04-27T03:34:01.841348+0000","last_peered":"2024-04-27T03:34:01.841348+0000","last_clean":"2024-04-27T03:34:01.841348+0000","last_became_active":"2024-04-27T03:30:11.371600+0000","last_became_peered":"2024-04-27T03:30:11.371600+0000","last_unstale":"2024-04-27T03:34:01.841348+0000","last_undegraded":"2024-04-27T03:34:01.841348+0000","last_fullsized":"2024-04-27T03:34:01.841348+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:01.841193+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:01.841193+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.841193+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:21:13.198341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.990760+0000","last_change":"2024-04-27T03:33:46.990760+0000","last_active":"2024-04-27T03:33:46.990760+0000","last_peered":"2024-04-27T03:33:46.990760+0000","last_clean":"2024-04-27T03:33:46.990760+0000","last_became_active":"2024-04-27T03:30:15.381359+0000","last_became_peered":"2024-04-27T03:30:15.381359+0000","last_unstale":"2024-04-27T03:33:46.990760+0000","last_undegraded":"2024-04-27T03:33:46.990760+0000","last_fullsized":"2024-04-27T03:33:46.990760+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:46.990601+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:46.990601+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.990601+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:08.708219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"27'58","reported_seq":125,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.809671+0000","last_change":"2024-04-27T03:33:41.809671+0000","last_active":"2024-04-27T03:33:41.809671+0000","last_peered":"2024-04-27T03:33:41.809671+0000","last_clean":"2024-04-27T03:33:41.809671+0000","last_became_active":"2024-04-27T03:30:17.390021+0000","last_became_peered":"2024-04-27T03:30:17.390021+0000","last_unstale":"2024-04-27T03:33:41.809671+0000","last_undegraded":"2024-04-27T03:33:41.809671+0000","last_fullsized":"2024-04-27T03:33:41.809671+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'58","last_scrub_stamp":"2024-04-27T03:33:41.809501+0000","last_deep_scrub":"27'58","last_deep_scrub_stamp":"2024-04-27T03:33:41.809501+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.809501+0000","objects_scrubbed":10,"log_size":58,"log_dups_size":0,"ondisk_log_size":58,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:46.251050+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":10,"num_object_clones":0,"num_object_copies":20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":10,"num_whiteouts":0,"num_read":59,"num_read_kb":47,"num_write":30,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.738255+0000","last_change":"2024-04-27T03:33:32.738255+0000","last_active":"2024-04-27T03:33:32.738255+0000","last_peered":"2024-04-27T03:33:32.738255+0000","last_clean":"2024-04-27T03:33:32.738255+0000","last_became_active":"2024-04-27T03:30:19.404948+0000","last_became_peered":"2024-04-27T03:30:19.404948+0000","last_unstale":"2024-04-27T03:33:32.738255+0000","last_undegraded":"2024-04-27T03:33:32.738255+0000","last_fullsized":"2024-04-27T03:33:32.738255+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:32.738165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:32.738165+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.738165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:19.681571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"27'3","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:21.929604+0000","last_change":"2024-04-27T03:34:21.929604+0000","last_active":"2024-04-27T03:34:21.929604+0000","last_peered":"2024-04-27T03:34:21.929604+0000","last_clean":"2024-04-27T03:34:21.929604+0000","last_became_active":"2024-04-27T03:30:21.412331+0000","last_became_peered":"2024-04-27T03:30:21.412331+0000","last_unstale":"2024-04-27T03:34:21.929604+0000","last_undegraded":"2024-04-27T03:34:21.929604+0000","last_fullsized":"2024-04-27T03:34:21.929604+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-27T03:34:21.929483+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-27T03:34:21.929483+0000","last_clean_scrub_stamp":"2024-04-27T03:34:21.929483+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:01:02.076280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":7,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"25'6","reported_seq":58,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.184718+0000","last_change":"2024-04-27T03:34:06.184718+0000","last_active":"2024-04-27T03:34:06.184718+0000","last_peered":"2024-04-27T03:34:06.184718+0000","last_clean":"2024-04-27T03:34:06.184718+0000","last_became_active":"2024-04-27T03:30:13.373226+0000","last_became_peered":"2024-04-27T03:30:13.373226+0000","last_unstale":"2024-04-27T03:34:06.184718+0000","last_undegraded":"2024-04-27T03:34:06.184718+0000","last_fullsized":"2024-04-27T03:34:06.184718+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'6","last_scrub_stamp":"2024-04-27T03:34:06.184601+0000","last_deep_scrub":"25'6","last_deep_scrub_stamp":"2024-04-27T03:34:06.184601+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.184601+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:06.834111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.791865+0000","last_change":"2024-04-27T03:33:59.791865+0000","last_active":"2024-04-27T03:33:59.791865+0000","last_peered":"2024-04-27T03:33:59.791865+0000","last_clean":"2024-04-27T03:33:59.791865+0000","last_became_active":"2024-04-27T03:30:11.373430+0000","last_became_peered":"2024-04-27T03:30:11.373430+0000","last_unstale":"2024-04-27T03:33:59.791865+0000","last_undegraded":"2024-04-27T03:33:59.791865+0000","last_fullsized":"2024-04-27T03:33:59.791865+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:59.791751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:59.791751+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.791751+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:23.025879+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.8","version":"27'2","reported_seq":90,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:25.988249+0000","last_change":"2024-04-27T03:34:25.988283+0000","last_active":"2024-04-27T03:34:25.988249+0000","last_peered":"2024-04-27T03:34:25.988249+0000","last_clean":"2024-04-27T03:34:25.988249+0000","last_became_active":"2024-04-27T03:31:01.443259+0000","last_became_peered":"2024-04-27T03:31:01.443259+0000","last_unstale":"2024-04-27T03:34:25.988249+0000","last_undegraded":"2024-04-27T03:34:25.988249+0000","last_fullsized":"2024-04-27T03:34:25.988249+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:25.988159+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:25.988159+0000","last_clean_scrub_stamp":"2024-04-27T03:34:25.988159+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:14.807899+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":436,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":285,"num_read_kb":285,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.b","version":"27'47","reported_seq":109,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.230642+0000","last_change":"2024-04-27T03:34:04.230642+0000","last_active":"2024-04-27T03:34:04.230642+0000","last_peered":"2024-04-27T03:34:04.230642+0000","last_clean":"2024-04-27T03:34:04.230642+0000","last_became_active":"2024-04-27T03:30:17.388994+0000","last_became_peered":"2024-04-27T03:30:17.388994+0000","last_unstale":"2024-04-27T03:34:04.230642+0000","last_undegraded":"2024-04-27T03:34:04.230642+0000","last_fullsized":"2024-04-27T03:34:04.230642+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-04-27T03:34:04.230519+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-04-27T03:34:04.230519+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.230519+0000","objects_scrubbed":5,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:43:53.086291+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":44,"num_read_kb":37,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:30.805201+0000","last_change":"2024-04-27T03:33:30.805201+0000","last_active":"2024-04-27T03:33:30.805201+0000","last_peered":"2024-04-27T03:33:30.805201+0000","last_clean":"2024-04-27T03:33:30.805201+0000","last_became_active":"2024-04-27T03:30:19.405334+0000","last_became_peered":"2024-04-27T03:30:19.405334+0000","last_unstale":"2024-04-27T03:33:30.805201+0000","last_undegraded":"2024-04-27T03:33:30.805201+0000","last_fullsized":"2024-04-27T03:33:30.805201+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:30.805019+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:30.805019+0000","last_clean_scrub_stamp":"2024-04-27T03:33:30.805019+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:46:37.451973+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.5","version":"27'4","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.099611+0000","last_change":"2024-04-27T03:33:57.099611+0000","last_active":"2024-04-27T03:33:57.099611+0000","last_peered":"2024-04-27T03:33:57.099611+0000","last_clean":"2024-04-27T03:33:57.099611+0000","last_became_active":"2024-04-27T03:30:21.409922+0000","last_became_peered":"2024-04-27T03:30:21.409922+0000","last_unstale":"2024-04-27T03:33:57.099611+0000","last_undegraded":"2024-04-27T03:33:57.099611+0000","last_fullsized":"2024-04-27T03:33:57.099611+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:57.099495+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:57.099495+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.099495+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:01.218125+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":7,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":169,"num_omap_keys":1,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.a","version":"27'6","reported_seq":61,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.819584+0000","last_change":"2024-04-27T03:34:02.819584+0000","last_active":"2024-04-27T03:34:02.819584+0000","last_peered":"2024-04-27T03:34:02.819584+0000","last_clean":"2024-04-27T03:34:02.819584+0000","last_became_active":"2024-04-27T03:30:13.381270+0000","last_became_peered":"2024-04-27T03:30:13.381270+0000","last_unstale":"2024-04-27T03:34:02.819584+0000","last_undegraded":"2024-04-27T03:34:02.819584+0000","last_fullsized":"2024-04-27T03:34:02.819584+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:02.819411+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:02.819411+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.819411+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:59.129585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":20,"num_read_kb":20,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:39.379731+0000","last_change":"2024-04-27T03:34:39.379731+0000","last_active":"2024-04-27T03:34:39.379731+0000","last_peered":"2024-04-27T03:34:39.379731+0000","last_clean":"2024-04-27T03:34:39.379731+0000","last_became_active":"2024-04-27T03:30:11.372583+0000","last_became_peered":"2024-04-27T03:30:11.372583+0000","last_unstale":"2024-04-27T03:34:39.379731+0000","last_undegraded":"2024-04-27T03:34:39.379731+0000","last_fullsized":"2024-04-27T03:34:39.379731+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:39.379616+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:39.379616+0000","last_clean_scrub_stamp":"2024-04-27T03:34:39.379616+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:03:33.468387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.709320+0000","last_change":"2024-04-27T03:33:51.709320+0000","last_active":"2024-04-27T03:33:51.709320+0000","last_peered":"2024-04-27T03:33:51.709320+0000","last_clean":"2024-04-27T03:33:51.709320+0000","last_became_active":"2024-04-27T03:30:15.382207+0000","last_became_peered":"2024-04-27T03:30:15.382207+0000","last_unstale":"2024-04-27T03:33:51.709320+0000","last_undegraded":"2024-04-27T03:33:51.709320+0000","last_fullsized":"2024-04-27T03:33:51.709320+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:51.709192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:51.709192+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.709192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:40.618465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"27'61","reported_seq":132,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.184072+0000","last_change":"2024-04-27T03:34:00.184072+0000","last_active":"2024-04-27T03:34:00.184072+0000","last_peered":"2024-04-27T03:34:00.184072+0000","last_clean":"2024-04-27T03:34:00.184072+0000","last_became_active":"2024-04-27T03:30:17.389242+0000","last_became_peered":"2024-04-27T03:30:17.389242+0000","last_unstale":"2024-04-27T03:34:00.184072+0000","last_undegraded":"2024-04-27T03:34:00.184072+0000","last_fullsized":"2024-04-27T03:34:00.184072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'61","last_scrub_stamp":"2024-04-27T03:34:00.183892+0000","last_deep_scrub":"27'61","last_deep_scrub_stamp":"2024-04-27T03:34:00.183892+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.183892+0000","objects_scrubbed":9,"log_size":61,"log_dups_size":0,"ondisk_log_size":61,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:30:17.176389+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":62,"num_read_kb":51,"num_write":32,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.9","version":"27'285","reported_seq":361,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.734598+0000","last_change":"2024-04-27T03:33:38.734598+0000","last_active":"2024-04-27T03:33:38.734598+0000","last_peered":"2024-04-27T03:33:38.734598+0000","last_clean":"2024-04-27T03:33:38.734598+0000","last_became_active":"2024-04-27T03:30:19.406364+0000","last_became_peered":"2024-04-27T03:30:19.406364+0000","last_unstale":"2024-04-27T03:33:38.734598+0000","last_undegraded":"2024-04-27T03:33:38.734598+0000","last_fullsized":"2024-04-27T03:33:38.734598+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:38.734486+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:38.734486+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.734486+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:11.608629+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:27.705542+0000","last_change":"2024-04-27T03:34:27.705542+0000","last_active":"2024-04-27T03:34:27.705542+0000","last_peered":"2024-04-27T03:34:27.705542+0000","last_clean":"2024-04-27T03:34:27.705542+0000","last_became_active":"2024-04-27T03:30:21.408168+0000","last_became_peered":"2024-04-27T03:30:21.408168+0000","last_unstale":"2024-04-27T03:34:27.705542+0000","last_undegraded":"2024-04-27T03:34:27.705542+0000","last_fullsized":"2024-04-27T03:34:27.705542+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:27.705421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:27.705421+0000","last_clean_scrub_stamp":"2024-04-27T03:34:27.705421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:25:10.961869+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.b","version":"27'14","reported_seq":111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.819144+0000","last_change":"2024-04-27T03:33:55.819144+0000","last_active":"2024-04-27T03:33:55.819144+0000","last_peered":"2024-04-27T03:33:55.819144+0000","last_clean":"2024-04-27T03:33:55.819144+0000","last_became_active":"2024-04-27T03:30:13.371503+0000","last_became_peered":"2024-04-27T03:30:13.371503+0000","last_unstale":"2024-04-27T03:33:55.819144+0000","last_undegraded":"2024-04-27T03:33:55.819144+0000","last_fullsized":"2024-04-27T03:33:55.819144+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:33:55.819026+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:33:55.819026+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.819026+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:03:53.861728+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":84,"num_read_kb":84,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.c","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:38.360071+0000","last_change":"2024-04-27T03:34:38.360071+0000","last_active":"2024-04-27T03:34:38.360071+0000","last_peered":"2024-04-27T03:34:38.360071+0000","last_clean":"2024-04-27T03:34:38.360071+0000","last_became_active":"2024-04-27T03:30:11.356739+0000","last_became_peered":"2024-04-27T03:30:11.356739+0000","last_unstale":"2024-04-27T03:34:38.360071+0000","last_undegraded":"2024-04-27T03:34:38.360071+0000","last_fullsized":"2024-04-27T03:34:38.360071+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:38.359934+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:38.359934+0000","last_clean_scrub_stamp":"2024-04-27T03:34:38.359934+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:24:19.694594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":20,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:23.927184+0000","last_change":"2024-04-27T03:34:23.927207+0000","last_active":"2024-04-27T03:34:23.927184+0000","last_peered":"2024-04-27T03:34:23.927184+0000","last_clean":"2024-04-27T03:34:23.927184+0000","last_became_active":"2024-04-27T03:31:01.443118+0000","last_became_peered":"2024-04-27T03:31:01.443118+0000","last_unstale":"2024-04-27T03:34:23.927184+0000","last_undegraded":"2024-04-27T03:34:23.927184+0000","last_fullsized":"2024-04-27T03:34:23.927184+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:23.927098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:23.927098+0000","last_clean_scrub_stamp":"2024-04-27T03:34:23.927098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:25:26.728092+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"27'79","reported_seq":165,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.115644+0000","last_change":"2024-04-27T03:33:56.115644+0000","last_active":"2024-04-27T03:33:56.115644+0000","last_peered":"2024-04-27T03:33:56.115644+0000","last_clean":"2024-04-27T03:33:56.115644+0000","last_became_active":"2024-04-27T03:30:17.387318+0000","last_became_peered":"2024-04-27T03:30:17.387318+0000","last_unstale":"2024-04-27T03:33:56.115644+0000","last_undegraded":"2024-04-27T03:33:56.115644+0000","last_fullsized":"2024-04-27T03:33:56.115644+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'79","last_scrub_stamp":"2024-04-27T03:33:56.115468+0000","last_deep_scrub":"27'79","last_deep_scrub_stamp":"2024-04-27T03:33:56.115468+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.115468+0000","objects_scrubbed":7,"log_size":79,"log_dups_size":0,"ondisk_log_size":79,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:06.434627+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":79,"num_read_kb":68,"num_write":34,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.701877+0000","last_change":"2024-04-27T03:33:37.701877+0000","last_active":"2024-04-27T03:33:37.701877+0000","last_peered":"2024-04-27T03:33:37.701877+0000","last_clean":"2024-04-27T03:33:37.701877+0000","last_became_active":"2024-04-27T03:30:19.404265+0000","last_became_peered":"2024-04-27T03:30:19.404265+0000","last_unstale":"2024-04-27T03:33:37.701877+0000","last_undegraded":"2024-04-27T03:33:37.701877+0000","last_fullsized":"2024-04-27T03:33:37.701877+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:37.701748+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:37.701748+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.701748+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:42.526067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.7","version":"27'21","reported_seq":57,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:29.689488+0000","last_change":"2024-04-27T03:34:29.689488+0000","last_active":"2024-04-27T03:34:29.689488+0000","last_peered":"2024-04-27T03:34:29.689488+0000","last_clean":"2024-04-27T03:34:29.689488+0000","last_became_active":"2024-04-27T03:30:21.413757+0000","last_became_peered":"2024-04-27T03:30:21.413757+0000","last_unstale":"2024-04-27T03:34:29.689488+0000","last_undegraded":"2024-04-27T03:34:29.689488+0000","last_fullsized":"2024-04-27T03:34:29.689488+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'21","last_scrub_stamp":"2024-04-27T03:34:29.689358+0000","last_deep_scrub":"27'21","last_deep_scrub_stamp":"2024-04-27T03:34:29.689358+0000","last_clean_scrub_stamp":"2024-04-27T03:34:29.689358+0000","objects_scrubbed":3,"log_size":21,"log_dups_size":0,"ondisk_log_size":21,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:17:30.208007+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":904,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":42,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.c","version":"27'4","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:11.168986+0000","last_change":"2024-04-27T03:34:11.168986+0000","last_active":"2024-04-27T03:34:11.168986+0000","last_peered":"2024-04-27T03:34:11.168986+0000","last_clean":"2024-04-27T03:34:11.168986+0000","last_became_active":"2024-04-27T03:30:13.379861+0000","last_became_peered":"2024-04-27T03:30:13.379861+0000","last_unstale":"2024-04-27T03:34:11.168986+0000","last_undegraded":"2024-04-27T03:34:11.168986+0000","last_fullsized":"2024-04-27T03:34:11.168986+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:11.168855+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:11.168855+0000","last_clean_scrub_stamp":"2024-04-27T03:34:11.168855+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:36.750674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:18.628832+0000","last_change":"2024-04-27T03:34:18.628832+0000","last_active":"2024-04-27T03:34:18.628832+0000","last_peered":"2024-04-27T03:34:18.628832+0000","last_clean":"2024-04-27T03:34:18.628832+0000","last_became_active":"2024-04-27T03:30:11.357176+0000","last_became_peered":"2024-04-27T03:30:11.357176+0000","last_unstale":"2024-04-27T03:34:18.628832+0000","last_undegraded":"2024-04-27T03:34:18.628832+0000","last_fullsized":"2024-04-27T03:34:18.628832+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:18.628698+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:18.628698+0000","last_clean_scrub_stamp":"2024-04-27T03:34:18.628698+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:24.899427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.754119+0000","last_change":"2024-04-27T03:33:49.754119+0000","last_active":"2024-04-27T03:33:49.754119+0000","last_peered":"2024-04-27T03:33:49.754119+0000","last_clean":"2024-04-27T03:33:49.754119+0000","last_became_active":"2024-04-27T03:30:15.382290+0000","last_became_peered":"2024-04-27T03:30:15.382290+0000","last_unstale":"2024-04-27T03:33:49.754119+0000","last_undegraded":"2024-04-27T03:33:49.754119+0000","last_fullsized":"2024-04-27T03:33:49.754119+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:49.754004+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:49.754004+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.754004+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:57.154906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.e","version":"27'46","reported_seq":109,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.971630+0000","last_change":"2024-04-27T03:33:41.971630+0000","last_active":"2024-04-27T03:33:41.971630+0000","last_peered":"2024-04-27T03:33:41.971630+0000","last_clean":"2024-04-27T03:33:41.971630+0000","last_became_active":"2024-04-27T03:30:17.385902+0000","last_became_peered":"2024-04-27T03:30:17.385902+0000","last_unstale":"2024-04-27T03:33:41.971630+0000","last_undegraded":"2024-04-27T03:33:41.971630+0000","last_fullsized":"2024-04-27T03:33:41.971630+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:41.971499+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:41.971499+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.971499+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:48.621217+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.094322+0000","last_change":"2024-04-27T03:33:55.094322+0000","last_active":"2024-04-27T03:33:55.094322+0000","last_peered":"2024-04-27T03:33:55.094322+0000","last_clean":"2024-04-27T03:33:55.094322+0000","last_became_active":"2024-04-27T03:30:19.405450+0000","last_became_peered":"2024-04-27T03:30:19.405450+0000","last_unstale":"2024-04-27T03:33:55.094322+0000","last_undegraded":"2024-04-27T03:33:55.094322+0000","last_fullsized":"2024-04-27T03:33:55.094322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:55.094207+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:55.094207+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.094207+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:33.101519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"27'10","reported_seq":37,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:50.335775+0000","last_change":"2024-04-27T03:34:50.335826+0000","last_active":"2024-04-27T03:34:50.335775+0000","last_peered":"2024-04-27T03:34:50.335775+0000","last_clean":"2024-04-27T03:34:50.335775+0000","last_became_active":"2024-04-27T03:30:21.415494+0000","last_became_peered":"2024-04-27T03:30:21.415494+0000","last_unstale":"2024-04-27T03:34:50.335775+0000","last_undegraded":"2024-04-27T03:34:50.335775+0000","last_fullsized":"2024-04-27T03:34:50.335775+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:34:50.335674+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:34:50.335674+0000","last_clean_scrub_stamp":"2024-04-27T03:34:50.335674+0000","objects_scrubbed":1,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:17:04.783004+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":199,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":25,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"27'6","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.201236+0000","last_change":"2024-04-27T03:33:48.201236+0000","last_active":"2024-04-27T03:33:48.201236+0000","last_peered":"2024-04-27T03:33:48.201236+0000","last_clean":"2024-04-27T03:33:48.201236+0000","last_became_active":"2024-04-27T03:30:13.387533+0000","last_became_peered":"2024-04-27T03:30:13.387533+0000","last_unstale":"2024-04-27T03:33:48.201236+0000","last_undegraded":"2024-04-27T03:33:48.201236+0000","last_fullsized":"2024-04-27T03:33:48.201236+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:33:48.201118+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:33:48.201118+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.201118+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:07:48.546003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:22.293400+0000","last_change":"2024-04-27T03:34:22.293400+0000","last_active":"2024-04-27T03:34:22.293400+0000","last_peered":"2024-04-27T03:34:22.293400+0000","last_clean":"2024-04-27T03:34:22.293400+0000","last_became_active":"2024-04-27T03:30:11.373440+0000","last_became_peered":"2024-04-27T03:30:11.373440+0000","last_unstale":"2024-04-27T03:34:22.293400+0000","last_undegraded":"2024-04-27T03:34:22.293400+0000","last_fullsized":"2024-04-27T03:34:22.293400+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:22.293281+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:22.293281+0000","last_clean_scrub_stamp":"2024-04-27T03:34:22.293281+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:13:00.638888+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.731024+0000","last_change":"2024-04-27T03:33:48.731024+0000","last_active":"2024-04-27T03:33:48.731024+0000","last_peered":"2024-04-27T03:33:48.731024+0000","last_clean":"2024-04-27T03:33:48.731024+0000","last_became_active":"2024-04-27T03:30:15.382913+0000","last_became_peered":"2024-04-27T03:30:15.382913+0000","last_unstale":"2024-04-27T03:33:48.731024+0000","last_undegraded":"2024-04-27T03:33:48.731024+0000","last_fullsized":"2024-04-27T03:33:48.731024+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.730891+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.730891+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.730891+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:43:49.302463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.f","version":"27'45","reported_seq":104,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.809220+0000","last_change":"2024-04-27T03:33:45.809220+0000","last_active":"2024-04-27T03:33:45.809220+0000","last_peered":"2024-04-27T03:33:45.809220+0000","last_clean":"2024-04-27T03:33:45.809220+0000","last_became_active":"2024-04-27T03:30:17.381106+0000","last_became_peered":"2024-04-27T03:30:17.381106+0000","last_unstale":"2024-04-27T03:33:45.809220+0000","last_undegraded":"2024-04-27T03:33:45.809220+0000","last_fullsized":"2024-04-27T03:33:45.809220+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'45","last_scrub_stamp":"2024-04-27T03:33:45.809061+0000","last_deep_scrub":"27'45","last_deep_scrub_stamp":"2024-04-27T03:33:45.809061+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.809061+0000","objects_scrubbed":7,"log_size":45,"log_dups_size":0,"ondisk_log_size":45,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:33.888060+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"27'285","reported_seq":416,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.864480+0000","last_change":"2024-04-27T03:33:31.864480+0000","last_active":"2024-04-27T03:33:31.864480+0000","last_peered":"2024-04-27T03:33:31.864480+0000","last_clean":"2024-04-27T03:33:31.864480+0000","last_became_active":"2024-04-27T03:30:19.403413+0000","last_became_peered":"2024-04-27T03:30:19.403413+0000","last_unstale":"2024-04-27T03:33:31.864480+0000","last_undegraded":"2024-04-27T03:33:31.864480+0000","last_fullsized":"2024-04-27T03:33:31.864480+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:31.864348+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:31.864348+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.864348+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:20:27.874206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1","version":"27'15","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.705571+0000","last_change":"2024-04-27T03:33:54.705571+0000","last_active":"2024-04-27T03:33:54.705571+0000","last_peered":"2024-04-27T03:33:54.705571+0000","last_clean":"2024-04-27T03:33:54.705571+0000","last_became_active":"2024-04-27T03:30:21.402102+0000","last_became_peered":"2024-04-27T03:30:21.402102+0000","last_unstale":"2024-04-27T03:33:54.705571+0000","last_undegraded":"2024-04-27T03:33:54.705571+0000","last_fullsized":"2024-04-27T03:33:54.705571+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-04-27T03:33:54.705410+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-04-27T03:33:54.705410+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.705410+0000","objects_scrubbed":1,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:00.713477+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":24,"num_read_kb":23,"num_write":24,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":180,"num_omap_keys":1,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"27'12","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.884586+0000","last_change":"2024-04-27T03:34:08.884586+0000","last_active":"2024-04-27T03:34:08.884586+0000","last_peered":"2024-04-27T03:34:08.884586+0000","last_clean":"2024-04-27T03:34:08.884586+0000","last_became_active":"2024-04-27T03:30:13.382899+0000","last_became_peered":"2024-04-27T03:30:13.382899+0000","last_unstale":"2024-04-27T03:34:08.884586+0000","last_undegraded":"2024-04-27T03:34:08.884586+0000","last_fullsized":"2024-04-27T03:34:08.884586+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-04-27T03:34:08.884470+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-04-27T03:34:08.884470+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.884470+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:46.729084+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.9","version":"27'4","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:34.054235+0000","last_change":"2024-04-27T03:34:34.054287+0000","last_active":"2024-04-27T03:34:34.054235+0000","last_peered":"2024-04-27T03:34:34.054235+0000","last_clean":"2024-04-27T03:34:34.054235+0000","last_became_active":"2024-04-27T03:31:01.439776+0000","last_became_peered":"2024-04-27T03:31:01.439776+0000","last_unstale":"2024-04-27T03:34:34.054235+0000","last_undegraded":"2024-04-27T03:34:34.054235+0000","last_fullsized":"2024-04-27T03:34:34.054235+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:34.054128+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:34.054128+0000","last_clean_scrub_stamp":"2024-04-27T03:34:34.054128+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:49:23.829672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"27'5","reported_seq":429,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.950380+0000","last_change":"2024-04-27T03:33:43.950380+0000","last_active":"2024-04-27T03:33:43.950380+0000","last_peered":"2024-04-27T03:33:43.950380+0000","last_clean":"2024-04-27T03:33:43.950380+0000","last_became_active":"2024-04-27T03:30:15.382502+0000","last_became_peered":"2024-04-27T03:30:15.382502+0000","last_unstale":"2024-04-27T03:33:43.950380+0000","last_undegraded":"2024-04-27T03:33:43.950380+0000","last_fullsized":"2024-04-27T03:33:43.950380+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:43.950258+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:43.950258+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.950258+0000","objects_scrubbed":2,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:48.355472+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":92,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":385,"num_read_kb":385,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.c","version":"27'15","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.197757+0000","last_change":"2024-04-27T03:33:58.197757+0000","last_active":"2024-04-27T03:33:58.197757+0000","last_peered":"2024-04-27T03:33:58.197757+0000","last_clean":"2024-04-27T03:33:58.197757+0000","last_became_active":"2024-04-27T03:30:17.387739+0000","last_became_peered":"2024-04-27T03:30:17.387739+0000","last_unstale":"2024-04-27T03:33:58.197757+0000","last_undegraded":"2024-04-27T03:33:58.197757+0000","last_fullsized":"2024-04-27T03:33:58.197757+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'15","last_scrub_stamp":"2024-04-27T03:33:58.197595+0000","last_deep_scrub":"27'15","last_deep_scrub_stamp":"2024-04-27T03:33:58.197595+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.197595+0000","objects_scrubbed":5,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:01.324323+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":15,"num_read_kb":10,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.d","version":"27'285","reported_seq":523,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.933929+0000","last_change":"2024-04-27T03:33:32.933929+0000","last_active":"2024-04-27T03:33:32.933929+0000","last_peered":"2024-04-27T03:33:32.933929+0000","last_clean":"2024-04-27T03:33:32.933929+0000","last_became_active":"2024-04-27T03:30:19.402484+0000","last_became_peered":"2024-04-27T03:30:19.402484+0000","last_unstale":"2024-04-27T03:33:32.933929+0000","last_undegraded":"2024-04-27T03:33:32.933929+0000","last_fullsized":"2024-04-27T03:33:32.933929+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:32.933800+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:32.933800+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.933800+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:19.924355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.2","version":"27'4","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.778552+0000","last_change":"2024-04-27T03:33:52.778552+0000","last_active":"2024-04-27T03:33:52.778552+0000","last_peered":"2024-04-27T03:33:52.778552+0000","last_clean":"2024-04-27T03:33:52.778552+0000","last_became_active":"2024-04-27T03:30:21.408512+0000","last_became_peered":"2024-04-27T03:30:21.408512+0000","last_unstale":"2024-04-27T03:33:52.778552+0000","last_undegraded":"2024-04-27T03:33:52.778552+0000","last_fullsized":"2024-04-27T03:33:52.778552+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:52.778420+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:52.778420+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.778420+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:29:57.183634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.f","version":"27'14","reported_seq":80,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.153985+0000","last_change":"2024-04-27T03:34:02.153985+0000","last_active":"2024-04-27T03:34:02.153985+0000","last_peered":"2024-04-27T03:34:02.153985+0000","last_clean":"2024-04-27T03:34:02.153985+0000","last_became_active":"2024-04-27T03:30:13.380540+0000","last_became_peered":"2024-04-27T03:30:13.380540+0000","last_unstale":"2024-04-27T03:34:02.153985+0000","last_undegraded":"2024-04-27T03:34:02.153985+0000","last_fullsized":"2024-04-27T03:34:02.153985+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:34:02.153863+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:34:02.153863+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.153863+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:47.534998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":37,"num_write":14,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"27'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:33.346199+0000","last_change":"2024-04-27T03:34:33.346199+0000","last_active":"2024-04-27T03:34:33.346199+0000","last_peered":"2024-04-27T03:34:33.346199+0000","last_clean":"2024-04-27T03:34:33.346199+0000","last_became_active":"2024-04-27T03:30:11.372193+0000","last_became_peered":"2024-04-27T03:30:11.372193+0000","last_unstale":"2024-04-27T03:34:33.346199+0000","last_undegraded":"2024-04-27T03:34:33.346199+0000","last_fullsized":"2024-04-27T03:34:33.346199+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:33.346065+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:33.346065+0000","last_clean_scrub_stamp":"2024-04-27T03:34:33.346065+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:54.581545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.115491+0000","last_change":"2024-04-27T03:33:35.115491+0000","last_active":"2024-04-27T03:33:35.115491+0000","last_peered":"2024-04-27T03:33:35.115491+0000","last_clean":"2024-04-27T03:33:35.115491+0000","last_became_active":"2024-04-27T03:30:15.383924+0000","last_became_peered":"2024-04-27T03:30:15.383924+0000","last_unstale":"2024-04-27T03:33:35.115491+0000","last_undegraded":"2024-04-27T03:33:35.115491+0000","last_fullsized":"2024-04-27T03:33:35.115491+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:35.115383+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:35.115383+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.115383+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:19:37.564842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.d","version":"27'48","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:38.883223+0000","last_change":"2024-04-27T03:33:38.883223+0000","last_active":"2024-04-27T03:33:38.883223+0000","last_peered":"2024-04-27T03:33:38.883223+0000","last_clean":"2024-04-27T03:33:38.883223+0000","last_became_active":"2024-04-27T03:30:17.387578+0000","last_became_peered":"2024-04-27T03:30:17.387578+0000","last_unstale":"2024-04-27T03:33:38.883223+0000","last_undegraded":"2024-04-27T03:33:38.883223+0000","last_fullsized":"2024-04-27T03:33:38.883223+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-04-27T03:33:38.883110+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-04-27T03:33:38.883110+0000","last_clean_scrub_stamp":"2024-04-27T03:33:38.883110+0000","objects_scrubbed":8,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:11.757254+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":47,"num_read_kb":37,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.856137+0000","last_change":"2024-04-27T03:33:32.856137+0000","last_active":"2024-04-27T03:33:32.856137+0000","last_peered":"2024-04-27T03:33:32.856137+0000","last_clean":"2024-04-27T03:33:32.856137+0000","last_became_active":"2024-04-27T03:30:19.405512+0000","last_became_peered":"2024-04-27T03:30:19.405512+0000","last_unstale":"2024-04-27T03:33:32.856137+0000","last_undegraded":"2024-04-27T03:33:32.856137+0000","last_fullsized":"2024-04-27T03:33:32.856137+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:32.856020+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:32.856020+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.856020+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:34.194673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.3","version":"27'14","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.990573+0000","last_change":"2024-04-27T03:33:48.990573+0000","last_active":"2024-04-27T03:33:48.990573+0000","last_peered":"2024-04-27T03:33:48.990573+0000","last_clean":"2024-04-27T03:33:48.990573+0000","last_became_active":"2024-04-27T03:30:21.406883+0000","last_became_peered":"2024-04-27T03:30:21.406883+0000","last_unstale":"2024-04-27T03:33:48.990573+0000","last_undegraded":"2024-04-27T03:33:48.990573+0000","last_fullsized":"2024-04-27T03:33:48.990573+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:33:48.990468+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:33:48.990468+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.990468+0000","objects_scrubbed":1,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:14:34.099716+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":18,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":78,"num_read_kb":59,"num_write":34,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.5","version":"27'20","reported_seq":89,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.148613+0000","last_change":"2024-04-27T03:33:52.148613+0000","last_active":"2024-04-27T03:33:52.148613+0000","last_peered":"2024-04-27T03:33:52.148613+0000","last_clean":"2024-04-27T03:33:52.148613+0000","last_became_active":"2024-04-27T03:30:13.387180+0000","last_became_peered":"2024-04-27T03:30:13.387180+0000","last_unstale":"2024-04-27T03:33:52.148613+0000","last_undegraded":"2024-04-27T03:33:52.148613+0000","last_fullsized":"2024-04-27T03:33:52.148613+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-04-27T03:33:52.148491+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-04-27T03:33:52.148491+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.148491+0000","objects_scrubbed":0,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:24:54.949485+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":26,"num_write_kb":10,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.0","version":"27'119","reported_seq":205,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:19.121523+0000","last_change":"2024-04-27T03:34:19.121563+0000","last_active":"2024-04-27T03:34:19.121523+0000","last_peered":"2024-04-27T03:34:19.121523+0000","last_clean":"2024-04-27T03:34:19.121523+0000","last_became_active":"2024-04-27T03:30:06.102893+0000","last_became_peered":"2024-04-27T03:30:06.102893+0000","last_unstale":"2024-04-27T03:34:19.121523+0000","last_undegraded":"2024-04-27T03:34:19.121523+0000","last_fullsized":"2024-04-27T03:34:19.121523+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'119","last_scrub_stamp":"2024-04-27T03:34:19.121425+0000","last_deep_scrub":"27'119","last_deep_scrub_stamp":"2024-04-27T03:34:19.121425+0000","last_clean_scrub_stamp":"2024-04-27T03:34:19.121425+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:50:22.248153+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:42.415967+0000","last_change":"2024-04-27T03:34:42.416007+0000","last_active":"2024-04-27T03:34:42.415967+0000","last_peered":"2024-04-27T03:34:42.415967+0000","last_clean":"2024-04-27T03:34:42.415967+0000","last_became_active":"2024-04-27T03:30:06.106511+0000","last_became_peered":"2024-04-27T03:30:06.106511+0000","last_unstale":"2024-04-27T03:34:42.415967+0000","last_undegraded":"2024-04-27T03:34:42.415967+0000","last_fullsized":"2024-04-27T03:34:42.415967+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:42.415847+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:42.415847+0000","last_clean_scrub_stamp":"2024-04-27T03:34:42.415847+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:15:33.081995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:28.327417+0000","last_change":"2024-04-27T03:34:28.327451+0000","last_active":"2024-04-27T03:34:28.327417+0000","last_peered":"2024-04-27T03:34:28.327417+0000","last_clean":"2024-04-27T03:34:28.327417+0000","last_became_active":"2024-04-27T03:30:11.362445+0000","last_became_peered":"2024-04-27T03:30:11.362445+0000","last_unstale":"2024-04-27T03:34:28.327417+0000","last_undegraded":"2024-04-27T03:34:28.327417+0000","last_fullsized":"2024-04-27T03:34:28.327417+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:28.327321+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:28.327321+0000","last_clean_scrub_stamp":"2024-04-27T03:34:28.327321+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:37.622290+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:33.076789+0000","last_change":"2024-04-27T03:34:33.076831+0000","last_active":"2024-04-27T03:34:33.076789+0000","last_peered":"2024-04-27T03:34:33.076789+0000","last_clean":"2024-04-27T03:34:33.076789+0000","last_became_active":"2024-04-27T03:31:01.440678+0000","last_became_peered":"2024-04-27T03:31:01.440678+0000","last_unstale":"2024-04-27T03:34:33.076789+0000","last_undegraded":"2024-04-27T03:34:33.076789+0000","last_fullsized":"2024-04-27T03:34:33.076789+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:33.076696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:33.076696+0000","last_clean_scrub_stamp":"2024-04-27T03:34:33.076696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:00:34.478658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.7","version":"27'95","reported_seq":192,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.176369+0000","last_change":"2024-04-27T03:33:45.176369+0000","last_active":"2024-04-27T03:33:45.176369+0000","last_peered":"2024-04-27T03:33:45.176369+0000","last_clean":"2024-04-27T03:33:45.176369+0000","last_became_active":"2024-04-27T03:30:17.387707+0000","last_became_peered":"2024-04-27T03:30:17.387707+0000","last_unstale":"2024-04-27T03:33:45.176369+0000","last_undegraded":"2024-04-27T03:33:45.176369+0000","last_fullsized":"2024-04-27T03:33:45.176369+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'95","last_scrub_stamp":"2024-04-27T03:33:45.176195+0000","last_deep_scrub":"27'95","last_deep_scrub_stamp":"2024-04-27T03:33:45.176195+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.176195+0000","objects_scrubbed":6,"log_size":95,"log_dups_size":0,"ondisk_log_size":95,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:18:45.184076+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":93,"num_read_kb":81,"num_write":34,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.779891+0000","last_change":"2024-04-27T03:33:31.779891+0000","last_active":"2024-04-27T03:33:31.779891+0000","last_peered":"2024-04-27T03:33:31.779891+0000","last_clean":"2024-04-27T03:33:31.779891+0000","last_became_active":"2024-04-27T03:30:19.405359+0000","last_became_peered":"2024-04-27T03:30:19.405359+0000","last_unstale":"2024-04-27T03:33:31.779891+0000","last_undegraded":"2024-04-27T03:33:31.779891+0000","last_fullsized":"2024-04-27T03:33:31.779891+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:31.779786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:31.779786+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.779786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:52:45.852820+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.9","version":"27'81","reported_seq":172,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.838907+0000","last_change":"2024-04-27T03:33:44.838907+0000","last_active":"2024-04-27T03:33:44.838907+0000","last_peered":"2024-04-27T03:33:44.838907+0000","last_clean":"2024-04-27T03:33:44.838907+0000","last_became_active":"2024-04-27T03:30:21.404300+0000","last_became_peered":"2024-04-27T03:30:21.404300+0000","last_unstale":"2024-04-27T03:33:44.838907+0000","last_undegraded":"2024-04-27T03:33:44.838907+0000","last_fullsized":"2024-04-27T03:33:44.838907+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'81","last_scrub_stamp":"2024-04-27T03:33:44.838760+0000","last_deep_scrub":"27'81","last_deep_scrub_stamp":"2024-04-27T03:33:44.838760+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.838760+0000","objects_scrubbed":0,"log_size":81,"log_dups_size":0,"ondisk_log_size":81,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:18.504969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":213,"num_read_kb":211,"num_write":155,"num_write_kb":51,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.6","version":"25'2","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:10.245276+0000","last_change":"2024-04-27T03:34:10.245276+0000","last_active":"2024-04-27T03:34:10.245276+0000","last_peered":"2024-04-27T03:34:10.245276+0000","last_clean":"2024-04-27T03:34:10.245276+0000","last_became_active":"2024-04-27T03:30:13.377713+0000","last_became_peered":"2024-04-27T03:30:13.377713+0000","last_unstale":"2024-04-27T03:34:10.245276+0000","last_undegraded":"2024-04-27T03:34:10.245276+0000","last_fullsized":"2024-04-27T03:34:10.245276+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:10.245096+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:10.245096+0000","last_clean_scrub_stamp":"2024-04-27T03:34:10.245096+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:32.051573+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:43.412993+0000","last_change":"2024-04-27T03:34:43.413018+0000","last_active":"2024-04-27T03:34:43.412993+0000","last_peered":"2024-04-27T03:34:43.412993+0000","last_clean":"2024-04-27T03:34:43.412993+0000","last_became_active":"2024-04-27T03:30:06.106474+0000","last_became_peered":"2024-04-27T03:30:06.106474+0000","last_unstale":"2024-04-27T03:34:43.412993+0000","last_undegraded":"2024-04-27T03:34:43.412993+0000","last_fullsized":"2024-04-27T03:34:43.412993+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:43.412878+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:43.412878+0000","last_clean_scrub_stamp":"2024-04-27T03:34:43.412878+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:59:27.850579+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1","version":"27'4","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:20.655637+0000","last_change":"2024-04-27T03:34:20.655637+0000","last_active":"2024-04-27T03:34:20.655637+0000","last_peered":"2024-04-27T03:34:20.655637+0000","last_clean":"2024-04-27T03:34:20.655637+0000","last_became_active":"2024-04-27T03:30:11.360244+0000","last_became_peered":"2024-04-27T03:30:11.360244+0000","last_unstale":"2024-04-27T03:34:20.655637+0000","last_undegraded":"2024-04-27T03:34:20.655637+0000","last_fullsized":"2024-04-27T03:34:20.655637+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:20.655487+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:20.655487+0000","last_clean_scrub_stamp":"2024-04-27T03:34:20.655487+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:53:31.032767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":7,"num_read_kb":4,"num_write":12,"num_write_kb":4097,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.098651+0000","last_change":"2024-04-27T03:33:42.098651+0000","last_active":"2024-04-27T03:33:42.098651+0000","last_peered":"2024-04-27T03:33:42.098651+0000","last_clean":"2024-04-27T03:33:42.098651+0000","last_became_active":"2024-04-27T03:30:15.379448+0000","last_became_peered":"2024-04-27T03:30:15.379448+0000","last_unstale":"2024-04-27T03:33:42.098651+0000","last_undegraded":"2024-04-27T03:33:42.098651+0000","last_fullsized":"2024-04-27T03:33:42.098651+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.098519+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.098519+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.098519+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:34:23.337627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.4","version":"27'297","reported_seq":529,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.877345+0000","last_change":"2024-04-27T03:33:35.877345+0000","last_active":"2024-04-27T03:33:35.877345+0000","last_peered":"2024-04-27T03:33:35.877345+0000","last_clean":"2024-04-27T03:33:35.877345+0000","last_became_active":"2024-04-27T03:30:17.390884+0000","last_became_peered":"2024-04-27T03:30:17.390884+0000","last_unstale":"2024-04-27T03:33:35.877345+0000","last_undegraded":"2024-04-27T03:33:35.877345+0000","last_fullsized":"2024-04-27T03:33:35.877345+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'297","last_scrub_stamp":"2024-04-27T03:33:35.877210+0000","last_deep_scrub":"27'297","last_deep_scrub_stamp":"2024-04-27T03:33:35.877210+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.877210+0000","objects_scrubbed":11,"log_size":297,"log_dups_size":0,"ondisk_log_size":297,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:27:23.956209+0000","scrub_duration":8,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":358,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":387,"num_read_kb":371,"num_write":44,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:11.241230+0000","last_change":"2024-04-27T03:34:11.241230+0000","last_active":"2024-04-27T03:34:11.241230+0000","last_peered":"2024-04-27T03:34:11.241230+0000","last_clean":"2024-04-27T03:34:11.241230+0000","last_became_active":"2024-04-27T03:30:19.405777+0000","last_became_peered":"2024-04-27T03:30:19.405777+0000","last_unstale":"2024-04-27T03:34:11.241230+0000","last_undegraded":"2024-04-27T03:34:11.241230+0000","last_fullsized":"2024-04-27T03:34:11.241230+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:11.241087+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:11.241087+0000","last_clean_scrub_stamp":"2024-04-27T03:34:11.241087+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:19.362648+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.a","version":"27'7","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.973335+0000","last_change":"2024-04-27T03:33:52.973335+0000","last_active":"2024-04-27T03:33:52.973335+0000","last_peered":"2024-04-27T03:33:52.973335+0000","last_clean":"2024-04-27T03:33:52.973335+0000","last_became_active":"2024-04-27T03:30:21.409359+0000","last_became_peered":"2024-04-27T03:30:21.409359+0000","last_unstale":"2024-04-27T03:33:52.973335+0000","last_undegraded":"2024-04-27T03:33:52.973335+0000","last_fullsized":"2024-04-27T03:33:52.973335+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:33:52.973192+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:33:52.973192+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.973192+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:49.283366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":14,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"25'10","reported_seq":82,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.632293+0000","last_change":"2024-04-27T03:33:57.632293+0000","last_active":"2024-04-27T03:33:57.632293+0000","last_peered":"2024-04-27T03:33:57.632293+0000","last_clean":"2024-04-27T03:33:57.632293+0000","last_became_active":"2024-04-27T03:30:13.385521+0000","last_became_peered":"2024-04-27T03:30:13.385521+0000","last_unstale":"2024-04-27T03:33:57.632293+0000","last_undegraded":"2024-04-27T03:33:57.632293+0000","last_fullsized":"2024-04-27T03:33:57.632293+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'10","last_scrub_stamp":"2024-04-27T03:33:57.632133+0000","last_deep_scrub":"25'10","last_deep_scrub_stamp":"2024-04-27T03:33:57.632133+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.632133+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:26:37.434045+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":45,"num_read_kb":45,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:10.860062+0000","last_change":"2024-04-27T03:34:10.860062+0000","last_active":"2024-04-27T03:34:10.860062+0000","last_peered":"2024-04-27T03:34:10.860062+0000","last_clean":"2024-04-27T03:34:10.860062+0000","last_became_active":"2024-04-27T03:30:06.103441+0000","last_became_peered":"2024-04-27T03:30:06.103441+0000","last_unstale":"2024-04-27T03:34:10.860062+0000","last_undegraded":"2024-04-27T03:34:10.860062+0000","last_fullsized":"2024-04-27T03:34:10.860062+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:10.859905+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:10.859905+0000","last_clean_scrub_stamp":"2024-04-27T03:34:10.859905+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:58.011918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:21.646692+0000","last_change":"2024-04-27T03:34:21.646749+0000","last_active":"2024-04-27T03:34:21.646692+0000","last_peered":"2024-04-27T03:34:21.646692+0000","last_clean":"2024-04-27T03:34:21.646692+0000","last_became_active":"2024-04-27T03:30:11.360443+0000","last_became_peered":"2024-04-27T03:30:11.360443+0000","last_unstale":"2024-04-27T03:34:21.646692+0000","last_undegraded":"2024-04-27T03:34:21.646692+0000","last_fullsized":"2024-04-27T03:34:21.646692+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:21.646559+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:21.646559+0000","last_clean_scrub_stamp":"2024-04-27T03:34:21.646559+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:30:38.639056+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.6","version":"27'2","reported_seq":41,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.772935+0000","last_change":"2024-04-27T03:33:46.772935+0000","last_active":"2024-04-27T03:33:46.772935+0000","last_peered":"2024-04-27T03:33:46.772935+0000","last_clean":"2024-04-27T03:33:46.772935+0000","last_became_active":"2024-04-27T03:30:15.383213+0000","last_became_peered":"2024-04-27T03:30:15.383213+0000","last_unstale":"2024-04-27T03:33:46.772935+0000","last_undegraded":"2024-04-27T03:33:46.772935+0000","last_fullsized":"2024-04-27T03:33:46.772935+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:46.772765+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:46.772765+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.772765+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:24.897665+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.5","version":"27'23","reported_seq":101,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.859533+0000","last_change":"2024-04-27T03:33:42.859533+0000","last_active":"2024-04-27T03:33:42.859533+0000","last_peered":"2024-04-27T03:33:42.859533+0000","last_clean":"2024-04-27T03:33:42.859533+0000","last_became_active":"2024-04-27T03:30:17.388588+0000","last_became_peered":"2024-04-27T03:30:17.388588+0000","last_unstale":"2024-04-27T03:33:42.859533+0000","last_undegraded":"2024-04-27T03:33:42.859533+0000","last_fullsized":"2024-04-27T03:33:42.859533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'23","last_scrub_stamp":"2024-04-27T03:33:42.859384+0000","last_deep_scrub":"27'23","last_deep_scrub_stamp":"2024-04-27T03:33:42.859384+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.859384+0000","objects_scrubbed":8,"log_size":23,"log_dups_size":0,"ondisk_log_size":23,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:54:29.849938+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":154,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":21,"num_read_kb":13,"num_write":13,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.729982+0000","last_change":"2024-04-27T03:33:46.729982+0000","last_active":"2024-04-27T03:33:46.729982+0000","last_peered":"2024-04-27T03:33:46.729982+0000","last_clean":"2024-04-27T03:33:46.729982+0000","last_became_active":"2024-04-27T03:30:19.407004+0000","last_became_peered":"2024-04-27T03:30:19.407004+0000","last_unstale":"2024-04-27T03:33:46.729982+0000","last_undegraded":"2024-04-27T03:33:46.729982+0000","last_fullsized":"2024-04-27T03:33:46.729982+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:46.729851+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:46.729851+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.729851+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:51:36.900685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.b","version":"25'2","reported_seq":29,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:55.357901+0000","last_change":"2024-04-27T03:34:55.357928+0000","last_active":"2024-04-27T03:34:55.357901+0000","last_peered":"2024-04-27T03:34:55.357901+0000","last_clean":"2024-04-27T03:34:55.357901+0000","last_became_active":"2024-04-27T03:30:21.416389+0000","last_became_peered":"2024-04-27T03:30:21.416389+0000","last_unstale":"2024-04-27T03:34:55.357901+0000","last_undegraded":"2024-04-27T03:34:55.357901+0000","last_fullsized":"2024-04-27T03:34:55.357901+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:55.357806+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:55.357806+0000","last_clean_scrub_stamp":"2024-04-27T03:34:55.357806+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:46:17.167754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.4","version":"27'8","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.675570+0000","last_change":"2024-04-27T03:34:00.675570+0000","last_active":"2024-04-27T03:34:00.675570+0000","last_peered":"2024-04-27T03:34:00.675570+0000","last_clean":"2024-04-27T03:34:00.675570+0000","last_became_active":"2024-04-27T03:30:13.369638+0000","last_became_peered":"2024-04-27T03:30:13.369638+0000","last_unstale":"2024-04-27T03:34:00.675570+0000","last_undegraded":"2024-04-27T03:34:00.675570+0000","last_fullsized":"2024-04-27T03:34:00.675570+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-04-27T03:34:00.675392+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-04-27T03:34:00.675392+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.675392+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:47.827405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":37,"num_read_kb":33,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:44.424819+0000","last_change":"2024-04-27T03:34:44.424842+0000","last_active":"2024-04-27T03:34:44.424819+0000","last_peered":"2024-04-27T03:34:44.424819+0000","last_clean":"2024-04-27T03:34:44.424819+0000","last_became_active":"2024-04-27T03:30:06.107356+0000","last_became_peered":"2024-04-27T03:30:06.107356+0000","last_unstale":"2024-04-27T03:34:44.424819+0000","last_undegraded":"2024-04-27T03:34:44.424819+0000","last_fullsized":"2024-04-27T03:34:44.424819+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:44.424721+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:44.424721+0000","last_clean_scrub_stamp":"2024-04-27T03:34:44.424721+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:19:05.627341+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:11.852227+0000","last_change":"2024-04-27T03:34:11.852227+0000","last_active":"2024-04-27T03:34:11.852227+0000","last_peered":"2024-04-27T03:34:11.852227+0000","last_clean":"2024-04-27T03:34:11.852227+0000","last_became_active":"2024-04-27T03:30:11.356685+0000","last_became_peered":"2024-04-27T03:30:11.356685+0000","last_unstale":"2024-04-27T03:34:11.852227+0000","last_undegraded":"2024-04-27T03:34:11.852227+0000","last_fullsized":"2024-04-27T03:34:11.852227+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:11.852112+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:11.852112+0000","last_clean_scrub_stamp":"2024-04-27T03:34:11.852112+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:45:57.233248+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.706693+0000","last_change":"2024-04-27T03:33:49.706693+0000","last_active":"2024-04-27T03:33:49.706693+0000","last_peered":"2024-04-27T03:33:49.706693+0000","last_clean":"2024-04-27T03:33:49.706693+0000","last_became_active":"2024-04-27T03:30:15.382326+0000","last_became_peered":"2024-04-27T03:30:15.382326+0000","last_unstale":"2024-04-27T03:33:49.706693+0000","last_undegraded":"2024-04-27T03:33:49.706693+0000","last_fullsized":"2024-04-27T03:33:49.706693+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:49.706574+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:49.706574+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.706574+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:45.223380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.6","version":"27'20","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.220511+0000","last_change":"2024-04-27T03:34:01.220511+0000","last_active":"2024-04-27T03:34:01.220511+0000","last_peered":"2024-04-27T03:34:01.220511+0000","last_clean":"2024-04-27T03:34:01.220511+0000","last_became_active":"2024-04-27T03:30:17.389373+0000","last_became_peered":"2024-04-27T03:30:17.389373+0000","last_unstale":"2024-04-27T03:34:01.220511+0000","last_undegraded":"2024-04-27T03:34:01.220511+0000","last_fullsized":"2024-04-27T03:34:01.220511+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'20","last_scrub_stamp":"2024-04-27T03:34:01.220376+0000","last_deep_scrub":"27'20","last_deep_scrub_stamp":"2024-04-27T03:34:01.220376+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.220376+0000","objects_scrubbed":7,"log_size":20,"log_dups_size":0,"ondisk_log_size":20,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:46:49.020148+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":21,"num_read_kb":15,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":350,"num_omap_keys":2,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.297531+0000","last_change":"2024-04-27T03:34:07.297531+0000","last_active":"2024-04-27T03:34:07.297531+0000","last_peered":"2024-04-27T03:34:07.297531+0000","last_clean":"2024-04-27T03:34:07.297531+0000","last_became_active":"2024-04-27T03:30:19.405953+0000","last_became_peered":"2024-04-27T03:30:19.405953+0000","last_unstale":"2024-04-27T03:34:07.297531+0000","last_undegraded":"2024-04-27T03:34:07.297531+0000","last_fullsized":"2024-04-27T03:34:07.297531+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:07.297411+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:07.297411+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.297411+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:27:38.735968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.8","version":"27'7","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:28.029576+0000","last_change":"2024-04-27T03:34:28.029576+0000","last_active":"2024-04-27T03:34:28.029576+0000","last_peered":"2024-04-27T03:34:28.029576+0000","last_clean":"2024-04-27T03:34:28.029576+0000","last_became_active":"2024-04-27T03:30:21.412790+0000","last_became_peered":"2024-04-27T03:30:21.412790+0000","last_unstale":"2024-04-27T03:34:28.029576+0000","last_undegraded":"2024-04-27T03:34:28.029576+0000","last_fullsized":"2024-04-27T03:34:28.029576+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:34:28.029410+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:34:28.029410+0000","last_clean_scrub_stamp":"2024-04-27T03:34:28.029410+0000","objects_scrubbed":1,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:10:15.508139+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":413,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":11,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2","version":"27'18","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:00.201621+0000","last_change":"2024-04-27T03:34:00.201621+0000","last_active":"2024-04-27T03:34:00.201621+0000","last_peered":"2024-04-27T03:34:00.201621+0000","last_clean":"2024-04-27T03:34:00.201621+0000","last_became_active":"2024-04-27T03:30:13.380888+0000","last_became_peered":"2024-04-27T03:30:13.380888+0000","last_unstale":"2024-04-27T03:34:00.201621+0000","last_undegraded":"2024-04-27T03:34:00.201621+0000","last_fullsized":"2024-04-27T03:34:00.201621+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'18","last_scrub_stamp":"2024-04-27T03:34:00.201503+0000","last_deep_scrub":"27'18","last_deep_scrub_stamp":"2024-04-27T03:34:00.201503+0000","last_clean_scrub_stamp":"2024-04-27T03:34:00.201503+0000","objects_scrubbed":0,"log_size":18,"log_dups_size":0,"ondisk_log_size":18,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:13:15.918816+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":81,"num_read_kb":73,"num_write":22,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.878323+0000","last_change":"2024-04-27T03:34:09.878323+0000","last_active":"2024-04-27T03:34:09.878323+0000","last_peered":"2024-04-27T03:34:09.878323+0000","last_clean":"2024-04-27T03:34:09.878323+0000","last_became_active":"2024-04-27T03:30:11.361786+0000","last_became_peered":"2024-04-27T03:30:11.361786+0000","last_unstale":"2024-04-27T03:34:09.878323+0000","last_undegraded":"2024-04-27T03:34:09.878323+0000","last_fullsized":"2024-04-27T03:34:09.878323+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:09.878163+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:09.878163+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.878163+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:13:25.595476+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:17.617492+0000","last_change":"2024-04-27T03:34:17.617492+0000","last_active":"2024-04-27T03:34:17.617492+0000","last_peered":"2024-04-27T03:34:17.617492+0000","last_clean":"2024-04-27T03:34:17.617492+0000","last_became_active":"2024-04-27T03:30:06.107594+0000","last_became_peered":"2024-04-27T03:30:06.107594+0000","last_unstale":"2024-04-27T03:34:17.617492+0000","last_undegraded":"2024-04-27T03:34:17.617492+0000","last_fullsized":"2024-04-27T03:34:17.617492+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:17.617335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:17.617335+0000","last_clean_scrub_stamp":"2024-04-27T03:34:17.617335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:24:27.936494+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"27'2","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:53.688679+0000","last_change":"2024-04-27T03:33:53.688679+0000","last_active":"2024-04-27T03:33:53.688679+0000","last_peered":"2024-04-27T03:33:53.688679+0000","last_clean":"2024-04-27T03:33:53.688679+0000","last_became_active":"2024-04-27T03:30:15.381392+0000","last_became_peered":"2024-04-27T03:30:15.381392+0000","last_unstale":"2024-04-27T03:33:53.688679+0000","last_undegraded":"2024-04-27T03:33:53.688679+0000","last_fullsized":"2024-04-27T03:33:53.688679+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:53.688558+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:53.688558+0000","last_clean_scrub_stamp":"2024-04-27T03:33:53.688558+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:08:53.046929+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":187,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.0","version":"27'87","reported_seq":176,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.739991+0000","last_change":"2024-04-27T03:33:39.739991+0000","last_active":"2024-04-27T03:33:39.739991+0000","last_peered":"2024-04-27T03:33:39.739991+0000","last_clean":"2024-04-27T03:33:39.739991+0000","last_became_active":"2024-04-27T03:30:17.390409+0000","last_became_peered":"2024-04-27T03:30:17.390409+0000","last_unstale":"2024-04-27T03:33:39.739991+0000","last_undegraded":"2024-04-27T03:33:39.739991+0000","last_fullsized":"2024-04-27T03:33:39.739991+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'87","last_scrub_stamp":"2024-04-27T03:33:39.739797+0000","last_deep_scrub":"27'87","last_deep_scrub_stamp":"2024-04-27T03:33:39.739797+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.739797+0000","objects_scrubbed":11,"log_size":87,"log_dups_size":0,"ondisk_log_size":87,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:27:02.606163+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":89,"num_read_kb":75,"num_write":42,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":292,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.814624+0000","last_change":"2024-04-27T03:33:33.814624+0000","last_active":"2024-04-27T03:33:33.814624+0000","last_peered":"2024-04-27T03:33:33.814624+0000","last_clean":"2024-04-27T03:33:33.814624+0000","last_became_active":"2024-04-27T03:30:19.402977+0000","last_became_peered":"2024-04-27T03:30:19.402977+0000","last_unstale":"2024-04-27T03:33:33.814624+0000","last_undegraded":"2024-04-27T03:33:33.814624+0000","last_fullsized":"2024-04-27T03:33:33.814624+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:33.814493+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:33.814493+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.814493+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:14:40.755714+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.e","version":"27'11","reported_seq":44,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:28.666146+0000","last_change":"2024-04-27T03:34:28.666146+0000","last_active":"2024-04-27T03:34:28.666146+0000","last_peered":"2024-04-27T03:34:28.666146+0000","last_clean":"2024-04-27T03:34:28.666146+0000","last_became_active":"2024-04-27T03:30:21.410511+0000","last_became_peered":"2024-04-27T03:30:21.410511+0000","last_unstale":"2024-04-27T03:34:28.666146+0000","last_undegraded":"2024-04-27T03:34:28.666146+0000","last_fullsized":"2024-04-27T03:34:28.666146+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'11","last_scrub_stamp":"2024-04-27T03:34:28.665996+0000","last_deep_scrub":"27'11","last_deep_scrub_stamp":"2024-04-27T03:34:28.665996+0000","last_clean_scrub_stamp":"2024-04-27T03:34:28.665996+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:00.104554+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":23,"num_read_kb":23,"num_write":20,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3","version":"27'4","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:13.162744+0000","last_change":"2024-04-27T03:34:13.162744+0000","last_active":"2024-04-27T03:34:13.162744+0000","last_peered":"2024-04-27T03:34:13.162744+0000","last_clean":"2024-04-27T03:34:13.162744+0000","last_became_active":"2024-04-27T03:30:13.379985+0000","last_became_peered":"2024-04-27T03:30:13.379985+0000","last_unstale":"2024-04-27T03:34:13.162744+0000","last_undegraded":"2024-04-27T03:34:13.162744+0000","last_fullsized":"2024-04-27T03:34:13.162744+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:13.162618+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:13.162618+0000","last_clean_scrub_stamp":"2024-04-27T03:34:13.162618+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:01:56.317292+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.4","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:13.657044+0000","last_change":"2024-04-27T03:34:13.657044+0000","last_active":"2024-04-27T03:34:13.657044+0000","last_peered":"2024-04-27T03:34:13.657044+0000","last_clean":"2024-04-27T03:34:13.657044+0000","last_became_active":"2024-04-27T03:30:11.360352+0000","last_became_peered":"2024-04-27T03:30:11.360352+0000","last_unstale":"2024-04-27T03:34:13.657044+0000","last_undegraded":"2024-04-27T03:34:13.657044+0000","last_fullsized":"2024-04-27T03:34:13.657044+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:13.656944+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:13.656944+0000","last_clean_scrub_stamp":"2024-04-27T03:34:13.656944+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:25:16.457938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:41.401887+0000","last_change":"2024-04-27T03:34:41.401910+0000","last_active":"2024-04-27T03:34:41.401887+0000","last_peered":"2024-04-27T03:34:41.401887+0000","last_clean":"2024-04-27T03:34:41.401887+0000","last_became_active":"2024-04-27T03:30:06.106475+0000","last_became_peered":"2024-04-27T03:30:06.106475+0000","last_unstale":"2024-04-27T03:34:41.401887+0000","last_undegraded":"2024-04-27T03:34:41.401887+0000","last_fullsized":"2024-04-27T03:34:41.401887+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:41.401790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:41.401790+0000","last_clean_scrub_stamp":"2024-04-27T03:34:41.401790+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:15.329806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.089706+0000","last_change":"2024-04-27T03:33:41.089706+0000","last_active":"2024-04-27T03:33:41.089706+0000","last_peered":"2024-04-27T03:33:41.089706+0000","last_clean":"2024-04-27T03:33:41.089706+0000","last_became_active":"2024-04-27T03:30:15.380030+0000","last_became_peered":"2024-04-27T03:30:15.380030+0000","last_unstale":"2024-04-27T03:33:41.089706+0000","last_undegraded":"2024-04-27T03:33:41.089706+0000","last_fullsized":"2024-04-27T03:33:41.089706+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:41.089578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:41.089578+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.089578+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:36.584947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.1","version":"27'28","reported_seq":76,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.880920+0000","last_change":"2024-04-27T03:33:34.880920+0000","last_active":"2024-04-27T03:33:34.880920+0000","last_peered":"2024-04-27T03:33:34.880920+0000","last_clean":"2024-04-27T03:33:34.880920+0000","last_became_active":"2024-04-27T03:30:17.393253+0000","last_became_peered":"2024-04-27T03:30:17.393253+0000","last_unstale":"2024-04-27T03:33:34.880920+0000","last_undegraded":"2024-04-27T03:33:34.880920+0000","last_fullsized":"2024-04-27T03:33:34.880920+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'28","last_scrub_stamp":"2024-04-27T03:33:34.880792+0000","last_deep_scrub":"27'28","last_deep_scrub_stamp":"2024-04-27T03:33:34.880792+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.880792+0000","objects_scrubbed":8,"log_size":28,"log_dups_size":0,"ondisk_log_size":28,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:44:08.547001+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":30,"num_read_kb":22,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.0","version":"27'285","reported_seq":399,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.921440+0000","last_change":"2024-04-27T03:33:33.921440+0000","last_active":"2024-04-27T03:33:33.921440+0000","last_peered":"2024-04-27T03:33:33.921440+0000","last_clean":"2024-04-27T03:33:33.921440+0000","last_became_active":"2024-04-27T03:30:19.402319+0000","last_became_peered":"2024-04-27T03:30:19.402319+0000","last_unstale":"2024-04-27T03:33:33.921440+0000","last_undegraded":"2024-04-27T03:33:33.921440+0000","last_fullsized":"2024-04-27T03:33:33.921440+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:33.921301+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:33.921301+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.921301+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:42.843568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.f","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:15.236528+0000","last_change":"2024-04-27T03:34:15.236528+0000","last_active":"2024-04-27T03:34:15.236528+0000","last_peered":"2024-04-27T03:34:15.236528+0000","last_clean":"2024-04-27T03:34:15.236528+0000","last_became_active":"2024-04-27T03:30:21.414109+0000","last_became_peered":"2024-04-27T03:30:21.414109+0000","last_unstale":"2024-04-27T03:34:15.236528+0000","last_undegraded":"2024-04-27T03:34:15.236528+0000","last_fullsized":"2024-04-27T03:34:15.236528+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:15.236366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:15.236366+0000","last_clean_scrub_stamp":"2024-04-27T03:34:15.236366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:53:40.927977+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.0","version":"25'2","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:46.192314+0000","last_change":"2024-04-27T03:33:46.192314+0000","last_active":"2024-04-27T03:33:46.192314+0000","last_peered":"2024-04-27T03:33:46.192314+0000","last_clean":"2024-04-27T03:33:46.192314+0000","last_became_active":"2024-04-27T03:30:13.387764+0000","last_became_peered":"2024-04-27T03:30:13.387764+0000","last_unstale":"2024-04-27T03:33:46.192314+0000","last_undegraded":"2024-04-27T03:33:46.192314+0000","last_fullsized":"2024-04-27T03:33:46.192314+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:33:46.192183+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:33:46.192183+0000","last_clean_scrub_stamp":"2024-04-27T03:33:46.192183+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:21:54.976582+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.7","version":"27'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:40.401033+0000","last_change":"2024-04-27T03:34:40.401060+0000","last_active":"2024-04-27T03:34:40.401033+0000","last_peered":"2024-04-27T03:34:40.401033+0000","last_clean":"2024-04-27T03:34:40.401033+0000","last_became_active":"2024-04-27T03:30:11.355704+0000","last_became_peered":"2024-04-27T03:30:11.355704+0000","last_unstale":"2024-04-27T03:34:40.401033+0000","last_undegraded":"2024-04-27T03:34:40.401033+0000","last_fullsized":"2024-04-27T03:34:40.401033+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:40.400920+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:40.400920+0000","last_clean_scrub_stamp":"2024-04-27T03:34:40.400920+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:11:24.872425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:16.597669+0000","last_change":"2024-04-27T03:34:16.597669+0000","last_active":"2024-04-27T03:34:16.597669+0000","last_peered":"2024-04-27T03:34:16.597669+0000","last_clean":"2024-04-27T03:34:16.597669+0000","last_became_active":"2024-04-27T03:30:06.103759+0000","last_became_peered":"2024-04-27T03:30:06.103759+0000","last_unstale":"2024-04-27T03:34:16.597669+0000","last_undegraded":"2024-04-27T03:34:16.597669+0000","last_fullsized":"2024-04-27T03:34:16.597669+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:16.597501+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:16.597501+0000","last_clean_scrub_stamp":"2024-04-27T03:34:16.597501+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:29:54.604812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1","version":"27'5","reported_seq":239,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:32.180895+0000","last_change":"2024-04-27T03:33:32.180895+0000","last_active":"2024-04-27T03:33:32.180895+0000","last_peered":"2024-04-27T03:33:32.180895+0000","last_clean":"2024-04-27T03:33:32.180895+0000","last_became_active":"2024-04-27T03:30:15.384372+0000","last_became_peered":"2024-04-27T03:30:15.384372+0000","last_unstale":"2024-04-27T03:33:32.180895+0000","last_undegraded":"2024-04-27T03:33:32.180895+0000","last_fullsized":"2024-04-27T03:33:32.180895+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:32.180778+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:32.180778+0000","last_clean_scrub_stamp":"2024-04-27T03:33:32.180778+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:01:22.337645+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":993,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":301,"num_read_kb":301,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"27'26","reported_seq":72,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.104813+0000","last_change":"2024-04-27T03:33:51.104813+0000","last_active":"2024-04-27T03:33:51.104813+0000","last_peered":"2024-04-27T03:33:51.104813+0000","last_clean":"2024-04-27T03:33:51.104813+0000","last_became_active":"2024-04-27T03:30:17.389154+0000","last_became_peered":"2024-04-27T03:30:17.389154+0000","last_unstale":"2024-04-27T03:33:51.104813+0000","last_undegraded":"2024-04-27T03:33:51.104813+0000","last_fullsized":"2024-04-27T03:33:51.104813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'26","last_scrub_stamp":"2024-04-27T03:33:51.104672+0000","last_deep_scrub":"27'26","last_deep_scrub_stamp":"2024-04-27T03:33:51.104672+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.104672+0000","objects_scrubbed":6,"log_size":26,"log_dups_size":0,"ondisk_log_size":26,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:09.096082+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.3","version":"27'285","reported_seq":411,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:33.780327+0000","last_change":"2024-04-27T03:33:33.780327+0000","last_active":"2024-04-27T03:33:33.780327+0000","last_peered":"2024-04-27T03:33:33.780327+0000","last_clean":"2024-04-27T03:33:33.780327+0000","last_became_active":"2024-04-27T03:30:19.404867+0000","last_became_peered":"2024-04-27T03:30:19.404867+0000","last_unstale":"2024-04-27T03:33:33.780327+0000","last_undegraded":"2024-04-27T03:33:33.780327+0000","last_fullsized":"2024-04-27T03:33:33.780327+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:33.780200+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:33.780200+0000","last_clean_scrub_stamp":"2024-04-27T03:33:33.780200+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:42.702468+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.c","version":"25'3","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:29.070899+0000","last_change":"2024-04-27T03:34:29.070899+0000","last_active":"2024-04-27T03:34:29.070899+0000","last_peered":"2024-04-27T03:34:29.070899+0000","last_clean":"2024-04-27T03:34:29.070899+0000","last_became_active":"2024-04-27T03:30:21.413264+0000","last_became_peered":"2024-04-27T03:30:21.413264+0000","last_unstale":"2024-04-27T03:34:29.070899+0000","last_undegraded":"2024-04-27T03:34:29.070899+0000","last_fullsized":"2024-04-27T03:34:29.070899+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'3","last_scrub_stamp":"2024-04-27T03:34:29.070741+0000","last_deep_scrub":"25'3","last_deep_scrub_stamp":"2024-04-27T03:34:29.070741+0000","last_clean_scrub_stamp":"2024-04-27T03:34:29.070741+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:05:54.556761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1","version":"27'4","reported_seq":65,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:18.852257+0000","last_change":"2024-04-27T03:34:18.852257+0000","last_active":"2024-04-27T03:34:18.852257+0000","last_peered":"2024-04-27T03:34:18.852257+0000","last_clean":"2024-04-27T03:34:18.852257+0000","last_became_active":"2024-04-27T03:30:13.383208+0000","last_became_peered":"2024-04-27T03:30:13.383208+0000","last_unstale":"2024-04-27T03:34:18.852257+0000","last_undegraded":"2024-04-27T03:34:18.852257+0000","last_fullsized":"2024-04-27T03:34:18.852257+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:18.852125+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:18.852125+0000","last_clean_scrub_stamp":"2024-04-27T03:34:18.852125+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:50.447247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:18.128080+0000","last_change":"2024-04-27T03:34:18.128080+0000","last_active":"2024-04-27T03:34:18.128080+0000","last_peered":"2024-04-27T03:34:18.128080+0000","last_clean":"2024-04-27T03:34:18.128080+0000","last_became_active":"2024-04-27T03:30:11.369568+0000","last_became_peered":"2024-04-27T03:30:11.369568+0000","last_unstale":"2024-04-27T03:34:18.128080+0000","last_undegraded":"2024-04-27T03:34:18.128080+0000","last_fullsized":"2024-04-27T03:34:18.128080+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:18.127955+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:18.127955+0000","last_clean_scrub_stamp":"2024-04-27T03:34:18.127955+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:53:40.539908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":53,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:45.394350+0000","last_change":"2024-04-27T03:34:45.394377+0000","last_active":"2024-04-27T03:34:45.394350+0000","last_peered":"2024-04-27T03:34:45.394350+0000","last_clean":"2024-04-27T03:34:45.394350+0000","last_became_active":"2024-04-27T03:30:06.107600+0000","last_became_peered":"2024-04-27T03:30:06.107600+0000","last_unstale":"2024-04-27T03:34:45.394350+0000","last_undegraded":"2024-04-27T03:34:45.394350+0000","last_fullsized":"2024-04-27T03:34:45.394350+0000","mapping_epoch":10,"log_start":"0'0","ondisk_log_start":"0'0","created":10,"last_epoch_clean":11,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:45.394262+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:45.394262+0000","last_clean_scrub_stamp":"2024-04-27T03:34:45.394262+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:07:26.023255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.124794+0000","last_change":"2024-04-27T03:33:40.124794+0000","last_active":"2024-04-27T03:33:40.124794+0000","last_peered":"2024-04-27T03:33:40.124794+0000","last_clean":"2024-04-27T03:33:40.124794+0000","last_became_active":"2024-04-27T03:30:15.380598+0000","last_became_peered":"2024-04-27T03:30:15.380598+0000","last_unstale":"2024-04-27T03:33:40.124794+0000","last_undegraded":"2024-04-27T03:33:40.124794+0000","last_fullsized":"2024-04-27T03:33:40.124794+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:40.124684+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:40.124684+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.124684+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:12.385924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"27'104","reported_seq":204,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.834973+0000","last_change":"2024-04-27T03:33:39.834973+0000","last_active":"2024-04-27T03:33:39.834973+0000","last_peered":"2024-04-27T03:33:39.834973+0000","last_clean":"2024-04-27T03:33:39.834973+0000","last_became_active":"2024-04-27T03:30:17.391191+0000","last_became_peered":"2024-04-27T03:30:17.391191+0000","last_unstale":"2024-04-27T03:33:39.834973+0000","last_undegraded":"2024-04-27T03:33:39.834973+0000","last_fullsized":"2024-04-27T03:33:39.834973+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'104","last_scrub_stamp":"2024-04-27T03:33:39.834804+0000","last_deep_scrub":"27'104","last_deep_scrub_stamp":"2024-04-27T03:33:39.834804+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.834804+0000","objects_scrubbed":9,"log_size":104,"log_dups_size":0,"ondisk_log_size":104,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:48:46.431245+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":330,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":102,"num_read_kb":87,"num_write":40,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.744784+0000","last_change":"2024-04-27T03:33:36.744784+0000","last_active":"2024-04-27T03:33:36.744784+0000","last_peered":"2024-04-27T03:33:36.744784+0000","last_clean":"2024-04-27T03:33:36.744784+0000","last_became_active":"2024-04-27T03:30:19.404187+0000","last_became_peered":"2024-04-27T03:30:19.404187+0000","last_unstale":"2024-04-27T03:33:36.744784+0000","last_undegraded":"2024-04-27T03:33:36.744784+0000","last_fullsized":"2024-04-27T03:33:36.744784+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.744626+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.744626+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.744626+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:08:14.869527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.769648+0000","last_change":"2024-04-27T03:33:48.769648+0000","last_active":"2024-04-27T03:33:48.769648+0000","last_peered":"2024-04-27T03:33:48.769648+0000","last_clean":"2024-04-27T03:33:48.769648+0000","last_became_active":"2024-04-27T03:30:21.407857+0000","last_became_peered":"2024-04-27T03:30:21.407857+0000","last_unstale":"2024-04-27T03:33:48.769648+0000","last_undegraded":"2024-04-27T03:33:48.769648+0000","last_fullsized":"2024-04-27T03:33:48.769648+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.769538+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.769538+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.769538+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:15:21.643681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1f","version":"25'2","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:20.938358+0000","last_change":"2024-04-27T03:34:20.938358+0000","last_active":"2024-04-27T03:34:20.938358+0000","last_peered":"2024-04-27T03:34:20.938358+0000","last_clean":"2024-04-27T03:34:20.938358+0000","last_became_active":"2024-04-27T03:30:21.411054+0000","last_became_peered":"2024-04-27T03:30:21.411054+0000","last_unstale":"2024-04-27T03:34:20.938358+0000","last_undegraded":"2024-04-27T03:34:20.938358+0000","last_fullsized":"2024-04-27T03:34:20.938358+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:20.938237+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:20.938237+0000","last_clean_scrub_stamp":"2024-04-27T03:34:20.938237+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:13:39.173999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":11,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.13","version":"27'6","reported_seq":59,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.813056+0000","last_change":"2024-04-27T03:34:06.813056+0000","last_active":"2024-04-27T03:34:06.813056+0000","last_peered":"2024-04-27T03:34:06.813056+0000","last_clean":"2024-04-27T03:34:06.813056+0000","last_became_active":"2024-04-27T03:30:13.382675+0000","last_became_peered":"2024-04-27T03:30:13.382675+0000","last_unstale":"2024-04-27T03:34:06.813056+0000","last_undegraded":"2024-04-27T03:34:06.813056+0000","last_fullsized":"2024-04-27T03:34:06.813056+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:06.812933+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:06.812933+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.812933+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:10.931563+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":16,"num_read_kb":16,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:34.339362+0000","last_change":"2024-04-27T03:34:34.339362+0000","last_active":"2024-04-27T03:34:34.339362+0000","last_peered":"2024-04-27T03:34:34.339362+0000","last_clean":"2024-04-27T03:34:34.339362+0000","last_became_active":"2024-04-27T03:30:11.359195+0000","last_became_peered":"2024-04-27T03:30:11.359195+0000","last_unstale":"2024-04-27T03:34:34.339362+0000","last_undegraded":"2024-04-27T03:34:34.339362+0000","last_fullsized":"2024-04-27T03:34:34.339362+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:34.339184+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:34.339184+0000","last_clean_scrub_stamp":"2024-04-27T03:34:34.339184+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:08:46.068821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.994275+0000","last_change":"2024-04-27T03:33:50.994275+0000","last_active":"2024-04-27T03:33:50.994275+0000","last_peered":"2024-04-27T03:33:50.994275+0000","last_clean":"2024-04-27T03:33:50.994275+0000","last_became_active":"2024-04-27T03:30:15.379541+0000","last_became_peered":"2024-04-27T03:30:15.379541+0000","last_unstale":"2024-04-27T03:33:50.994275+0000","last_undegraded":"2024-04-27T03:33:50.994275+0000","last_fullsized":"2024-04-27T03:33:50.994275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:50.994123+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:50.994123+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.994123+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:47:51.335268+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.11","version":"27'46","reported_seq":109,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:37.177105+0000","last_change":"2024-04-27T03:33:37.177105+0000","last_active":"2024-04-27T03:33:37.177105+0000","last_peered":"2024-04-27T03:33:37.177105+0000","last_clean":"2024-04-27T03:33:37.177105+0000","last_became_active":"2024-04-27T03:30:17.386496+0000","last_became_peered":"2024-04-27T03:30:17.386496+0000","last_unstale":"2024-04-27T03:33:37.177105+0000","last_undegraded":"2024-04-27T03:33:37.177105+0000","last_fullsized":"2024-04-27T03:33:37.177105+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:37.176970+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:37.176970+0000","last_clean_scrub_stamp":"2024-04-27T03:33:37.176970+0000","objects_scrubbed":6,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:56:15.021583+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":47,"num_read_kb":39,"num_write":22,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:53.149659+0000","last_change":"2024-04-27T03:33:53.149659+0000","last_active":"2024-04-27T03:33:53.149659+0000","last_peered":"2024-04-27T03:33:53.149659+0000","last_clean":"2024-04-27T03:33:53.149659+0000","last_became_active":"2024-04-27T03:30:19.405189+0000","last_became_peered":"2024-04-27T03:30:19.405189+0000","last_unstale":"2024-04-27T03:33:53.149659+0000","last_undegraded":"2024-04-27T03:33:53.149659+0000","last_fullsized":"2024-04-27T03:33:53.149659+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:53.149540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:53.149540+0000","last_clean_scrub_stamp":"2024-04-27T03:33:53.149540+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:17:47.405355+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1e","version":"27'3","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:30.679029+0000","last_change":"2024-04-27T03:34:30.679053+0000","last_active":"2024-04-27T03:34:30.679029+0000","last_peered":"2024-04-27T03:34:30.679029+0000","last_clean":"2024-04-27T03:34:30.679029+0000","last_became_active":"2024-04-27T03:30:21.413920+0000","last_became_peered":"2024-04-27T03:30:21.413920+0000","last_unstale":"2024-04-27T03:34:30.679029+0000","last_undegraded":"2024-04-27T03:34:30.679029+0000","last_fullsized":"2024-04-27T03:34:30.679029+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-27T03:34:30.678930+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-27T03:34:30.678930+0000","last_clean_scrub_stamp":"2024-04-27T03:34:30.678930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:50:07.631541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.12","version":"27'16","reported_seq":66,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:27.037018+0000","last_change":"2024-04-27T03:34:27.037018+0000","last_active":"2024-04-27T03:34:27.037018+0000","last_peered":"2024-04-27T03:34:27.037018+0000","last_clean":"2024-04-27T03:34:27.037018+0000","last_became_active":"2024-04-27T03:30:13.380240+0000","last_became_peered":"2024-04-27T03:30:13.380240+0000","last_unstale":"2024-04-27T03:34:27.037018+0000","last_undegraded":"2024-04-27T03:34:27.037018+0000","last_fullsized":"2024-04-27T03:34:27.037018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:27.036880+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:27.036880+0000","last_clean_scrub_stamp":"2024-04-27T03:34:27.036880+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:55:53.303121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":39,"num_read_kb":31,"num_write":20,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.15","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:16.846162+0000","last_change":"2024-04-27T03:34:16.846162+0000","last_active":"2024-04-27T03:34:16.846162+0000","last_peered":"2024-04-27T03:34:16.846162+0000","last_clean":"2024-04-27T03:34:16.846162+0000","last_became_active":"2024-04-27T03:30:11.360217+0000","last_became_peered":"2024-04-27T03:30:11.360217+0000","last_unstale":"2024-04-27T03:34:16.846162+0000","last_undegraded":"2024-04-27T03:34:16.846162+0000","last_fullsized":"2024-04-27T03:34:16.846162+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:16.846006+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:16.846006+0000","last_clean_scrub_stamp":"2024-04-27T03:34:16.846006+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:54:35.473558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.125275+0000","last_change":"2024-04-27T03:33:36.125275+0000","last_active":"2024-04-27T03:33:36.125275+0000","last_peered":"2024-04-27T03:33:36.125275+0000","last_clean":"2024-04-27T03:33:36.125275+0000","last_became_active":"2024-04-27T03:30:15.381507+0000","last_became_peered":"2024-04-27T03:30:15.381507+0000","last_unstale":"2024-04-27T03:33:36.125275+0000","last_undegraded":"2024-04-27T03:33:36.125275+0000","last_fullsized":"2024-04-27T03:33:36.125275+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.125165+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.125165+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.125165+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:59:43.471223+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"27'9","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.703001+0000","last_change":"2024-04-27T03:33:44.703001+0000","last_active":"2024-04-27T03:33:44.703001+0000","last_peered":"2024-04-27T03:33:44.703001+0000","last_clean":"2024-04-27T03:33:44.703001+0000","last_became_active":"2024-04-27T03:30:17.385318+0000","last_became_peered":"2024-04-27T03:30:17.385318+0000","last_unstale":"2024-04-27T03:33:44.703001+0000","last_undegraded":"2024-04-27T03:33:44.703001+0000","last_fullsized":"2024-04-27T03:33:44.703001+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'9","last_scrub_stamp":"2024-04-27T03:33:44.702827+0000","last_deep_scrub":"27'9","last_deep_scrub_stamp":"2024-04-27T03:33:44.702827+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.702827+0000","objects_scrubbed":3,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:42:55.754975+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":11,"num_read_kb":9,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":250,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:36.873142+0000","last_change":"2024-04-27T03:33:36.873142+0000","last_active":"2024-04-27T03:33:36.873142+0000","last_peered":"2024-04-27T03:33:36.873142+0000","last_clean":"2024-04-27T03:33:36.873142+0000","last_became_active":"2024-04-27T03:30:19.406736+0000","last_became_peered":"2024-04-27T03:30:19.406736+0000","last_unstale":"2024-04-27T03:33:36.873142+0000","last_undegraded":"2024-04-27T03:33:36.873142+0000","last_fullsized":"2024-04-27T03:33:36.873142+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:36.873031+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:36.873031+0000","last_clean_scrub_stamp":"2024-04-27T03:33:36.873031+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:03:57.751682+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.1d","version":"27'4","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:26.716914+0000","last_change":"2024-04-27T03:34:26.716984+0000","last_active":"2024-04-27T03:34:26.716914+0000","last_peered":"2024-04-27T03:34:26.716914+0000","last_clean":"2024-04-27T03:34:26.716914+0000","last_became_active":"2024-04-27T03:30:21.408079+0000","last_became_peered":"2024-04-27T03:30:21.408079+0000","last_unstale":"2024-04-27T03:34:26.716914+0000","last_undegraded":"2024-04-27T03:34:26.716914+0000","last_fullsized":"2024-04-27T03:34:26.716914+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:26.716795+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:26.716795+0000","last_clean_scrub_stamp":"2024-04-27T03:34:26.716795+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:21:21.432858+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.11","version":"27'4","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:12.698757+0000","last_change":"2024-04-27T03:34:12.698757+0000","last_active":"2024-04-27T03:34:12.698757+0000","last_peered":"2024-04-27T03:34:12.698757+0000","last_clean":"2024-04-27T03:34:12.698757+0000","last_became_active":"2024-04-27T03:30:13.375350+0000","last_became_peered":"2024-04-27T03:30:13.375350+0000","last_unstale":"2024-04-27T03:34:12.698757+0000","last_undegraded":"2024-04-27T03:34:12.698757+0000","last_fullsized":"2024-04-27T03:34:12.698757+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:12.698671+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:12.698671+0000","last_clean_scrub_stamp":"2024-04-27T03:34:12.698671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:30.690081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":14,"num_read_kb":14,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:17.813663+0000","last_change":"2024-04-27T03:34:17.813663+0000","last_active":"2024-04-27T03:34:17.813663+0000","last_peered":"2024-04-27T03:34:17.813663+0000","last_clean":"2024-04-27T03:34:17.813663+0000","last_became_active":"2024-04-27T03:30:11.360938+0000","last_became_peered":"2024-04-27T03:30:11.360938+0000","last_unstale":"2024-04-27T03:34:17.813663+0000","last_undegraded":"2024-04-27T03:34:17.813663+0000","last_fullsized":"2024-04-27T03:34:17.813663+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:17.813510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:17.813510+0000","last_clean_scrub_stamp":"2024-04-27T03:34:17.813510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:37:25.064062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.767359+0000","last_change":"2024-04-27T03:33:47.767359+0000","last_active":"2024-04-27T03:33:47.767359+0000","last_peered":"2024-04-27T03:33:47.767359+0000","last_clean":"2024-04-27T03:33:47.767359+0000","last_became_active":"2024-04-27T03:30:15.382287+0000","last_became_peered":"2024-04-27T03:30:15.382287+0000","last_unstale":"2024-04-27T03:33:47.767359+0000","last_undegraded":"2024-04-27T03:33:47.767359+0000","last_fullsized":"2024-04-27T03:33:47.767359+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:47.767216+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:47.767216+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.767216+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:12:52.591536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.13","version":"27'47","reported_seq":120,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:41.768233+0000","last_change":"2024-04-27T03:33:41.768233+0000","last_active":"2024-04-27T03:33:41.768233+0000","last_peered":"2024-04-27T03:33:41.768233+0000","last_clean":"2024-04-27T03:33:41.768233+0000","last_became_active":"2024-04-27T03:30:17.388260+0000","last_became_peered":"2024-04-27T03:30:17.388260+0000","last_unstale":"2024-04-27T03:33:41.768233+0000","last_undegraded":"2024-04-27T03:33:41.768233+0000","last_fullsized":"2024-04-27T03:33:41.768233+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'47","last_scrub_stamp":"2024-04-27T03:33:41.768105+0000","last_deep_scrub":"27'47","last_deep_scrub_stamp":"2024-04-27T03:33:41.768105+0000","last_clean_scrub_stamp":"2024-04-27T03:33:41.768105+0000","objects_scrubbed":6,"log_size":47,"log_dups_size":0,"ondisk_log_size":47,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:24:48.417823+0000","scrub_duration":4,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":67,"num_read_kb":70,"num_write":48,"num_write_kb":19,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":662,"num_omap_keys":2,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:31.927916+0000","last_change":"2024-04-27T03:33:31.927916+0000","last_active":"2024-04-27T03:33:31.927916+0000","last_peered":"2024-04-27T03:33:31.927916+0000","last_clean":"2024-04-27T03:33:31.927916+0000","last_became_active":"2024-04-27T03:30:19.402430+0000","last_became_peered":"2024-04-27T03:30:19.402430+0000","last_unstale":"2024-04-27T03:33:31.927916+0000","last_undegraded":"2024-04-27T03:33:31.927916+0000","last_fullsized":"2024-04-27T03:33:31.927916+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:31.927798+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:31.927798+0000","last_clean_scrub_stamp":"2024-04-27T03:33:31.927798+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:56:36.369347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:52.328225+0000","last_change":"2024-04-27T03:34:52.328297+0000","last_active":"2024-04-27T03:34:52.328225+0000","last_peered":"2024-04-27T03:34:52.328225+0000","last_clean":"2024-04-27T03:34:52.328225+0000","last_became_active":"2024-04-27T03:30:21.415984+0000","last_became_peered":"2024-04-27T03:30:21.415984+0000","last_unstale":"2024-04-27T03:34:52.328225+0000","last_undegraded":"2024-04-27T03:34:52.328225+0000","last_fullsized":"2024-04-27T03:34:52.328225+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:52.328120+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:52.328120+0000","last_clean_scrub_stamp":"2024-04-27T03:34:52.328120+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:57:42.680701+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.10","version":"27'5","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.144460+0000","last_change":"2024-04-27T03:33:50.144460+0000","last_active":"2024-04-27T03:33:50.144460+0000","last_peered":"2024-04-27T03:33:50.144460+0000","last_clean":"2024-04-27T03:33:50.144460+0000","last_became_active":"2024-04-27T03:30:13.386985+0000","last_became_peered":"2024-04-27T03:30:13.386985+0000","last_unstale":"2024-04-27T03:33:50.144460+0000","last_undegraded":"2024-04-27T03:33:50.144460+0000","last_fullsized":"2024-04-27T03:33:50.144460+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'5","last_scrub_stamp":"2024-04-27T03:33:50.144340+0000","last_deep_scrub":"27'5","last_deep_scrub_stamp":"2024-04-27T03:33:50.144340+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.144340+0000","objects_scrubbed":1,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:15.726159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:22.087002+0000","last_change":"2024-04-27T03:34:22.087002+0000","last_active":"2024-04-27T03:34:22.087002+0000","last_peered":"2024-04-27T03:34:22.087002+0000","last_clean":"2024-04-27T03:34:22.087002+0000","last_became_active":"2024-04-27T03:30:11.369824+0000","last_became_peered":"2024-04-27T03:30:11.369824+0000","last_unstale":"2024-04-27T03:34:22.087002+0000","last_undegraded":"2024-04-27T03:34:22.087002+0000","last_fullsized":"2024-04-27T03:34:22.087002+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:22.086864+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:22.086864+0000","last_clean_scrub_stamp":"2024-04-27T03:34:22.086864+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:09.886636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.155037+0000","last_change":"2024-04-27T03:33:34.155037+0000","last_active":"2024-04-27T03:33:34.155037+0000","last_peered":"2024-04-27T03:33:34.155037+0000","last_clean":"2024-04-27T03:33:34.155037+0000","last_became_active":"2024-04-27T03:30:15.383771+0000","last_became_peered":"2024-04-27T03:30:15.383771+0000","last_unstale":"2024-04-27T03:33:34.155037+0000","last_undegraded":"2024-04-27T03:33:34.155037+0000","last_fullsized":"2024-04-27T03:33:34.155037+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.154879+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.154879+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.154879+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:11:59.127137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.12","version":"27'36","reported_seq":92,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.840113+0000","last_change":"2024-04-27T03:33:43.840113+0000","last_active":"2024-04-27T03:33:43.840113+0000","last_peered":"2024-04-27T03:33:43.840113+0000","last_clean":"2024-04-27T03:33:43.840113+0000","last_became_active":"2024-04-27T03:30:17.385488+0000","last_became_peered":"2024-04-27T03:30:17.385488+0000","last_unstale":"2024-04-27T03:33:43.840113+0000","last_undegraded":"2024-04-27T03:33:43.840113+0000","last_fullsized":"2024-04-27T03:33:43.840113+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'36","last_scrub_stamp":"2024-04-27T03:33:43.839983+0000","last_deep_scrub":"27'36","last_deep_scrub_stamp":"2024-04-27T03:33:43.839983+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.839983+0000","objects_scrubbed":4,"log_size":36,"log_dups_size":0,"ondisk_log_size":36,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:42:52.762250+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":35,"num_read_kb":29,"num_write":14,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:35.844680+0000","last_change":"2024-04-27T03:33:35.844680+0000","last_active":"2024-04-27T03:33:35.844680+0000","last_peered":"2024-04-27T03:33:35.844680+0000","last_clean":"2024-04-27T03:33:35.844680+0000","last_became_active":"2024-04-27T03:30:19.398484+0000","last_became_peered":"2024-04-27T03:30:19.398484+0000","last_unstale":"2024-04-27T03:33:35.844680+0000","last_undegraded":"2024-04-27T03:33:35.844680+0000","last_fullsized":"2024-04-27T03:33:35.844680+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:35.844545+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:35.844545+0000","last_clean_scrub_stamp":"2024-04-27T03:33:35.844545+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:06:17.570301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.1f","version":"25'2","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:14.212832+0000","last_change":"2024-04-27T03:34:14.212832+0000","last_active":"2024-04-27T03:34:14.212832+0000","last_peered":"2024-04-27T03:34:14.212832+0000","last_clean":"2024-04-27T03:34:14.212832+0000","last_became_active":"2024-04-27T03:30:13.377217+0000","last_became_peered":"2024-04-27T03:30:13.377217+0000","last_unstale":"2024-04-27T03:34:14.212832+0000","last_undegraded":"2024-04-27T03:34:14.212832+0000","last_fullsized":"2024-04-27T03:34:14.212832+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:14.212665+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:14.212665+0000","last_clean_scrub_stamp":"2024-04-27T03:34:14.212665+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:40:43.816639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"25'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:31.330660+0000","last_change":"2024-04-27T03:34:31.330660+0000","last_active":"2024-04-27T03:34:31.330660+0000","last_peered":"2024-04-27T03:34:31.330660+0000","last_clean":"2024-04-27T03:34:31.330660+0000","last_became_active":"2024-04-27T03:30:11.370547+0000","last_became_peered":"2024-04-27T03:30:11.370547+0000","last_unstale":"2024-04-27T03:34:31.330660+0000","last_undegraded":"2024-04-27T03:34:31.330660+0000","last_fullsized":"2024-04-27T03:34:31.330660+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:31.330465+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:31.330465+0000","last_clean_scrub_stamp":"2024-04-27T03:34:31.330465+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:51:57.298339+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:52.700581+0000","last_change":"2024-04-27T03:33:52.700581+0000","last_active":"2024-04-27T03:33:52.700581+0000","last_peered":"2024-04-27T03:33:52.700581+0000","last_clean":"2024-04-27T03:33:52.700581+0000","last_became_active":"2024-04-27T03:30:15.381956+0000","last_became_peered":"2024-04-27T03:30:15.381956+0000","last_unstale":"2024-04-27T03:33:52.700581+0000","last_undegraded":"2024-04-27T03:33:52.700581+0000","last_fullsized":"2024-04-27T03:33:52.700581+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:52.700486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:52.700486+0000","last_clean_scrub_stamp":"2024-04-27T03:33:52.700486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:06:49.010661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1d","version":"27'70","reported_seq":149,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.743749+0000","last_change":"2024-04-27T03:33:45.743793+0000","last_active":"2024-04-27T03:33:45.743749+0000","last_peered":"2024-04-27T03:33:45.743749+0000","last_clean":"2024-04-27T03:33:45.743749+0000","last_became_active":"2024-04-27T03:30:17.383178+0000","last_became_peered":"2024-04-27T03:30:17.383178+0000","last_unstale":"2024-04-27T03:33:45.743749+0000","last_undegraded":"2024-04-27T03:33:45.743749+0000","last_fullsized":"2024-04-27T03:33:45.743749+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'70","last_scrub_stamp":"2024-04-27T03:33:45.743668+0000","last_deep_scrub":"27'70","last_deep_scrub_stamp":"2024-04-27T03:33:45.743668+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.743668+0000","objects_scrubbed":6,"log_size":70,"log_dups_size":0,"ondisk_log_size":70,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:37.293903+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":220,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":70,"num_read_kb":60,"num_write":28,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.812908+0000","last_change":"2024-04-27T03:33:34.812933+0000","last_active":"2024-04-27T03:33:34.812908+0000","last_peered":"2024-04-27T03:33:34.812908+0000","last_clean":"2024-04-27T03:33:34.812908+0000","last_became_active":"2024-04-27T03:30:19.402781+0000","last_became_peered":"2024-04-27T03:30:19.402781+0000","last_unstale":"2024-04-27T03:33:34.812908+0000","last_undegraded":"2024-04-27T03:33:34.812908+0000","last_fullsized":"2024-04-27T03:33:34.812908+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.812814+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.812814+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.812814+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:40:12.666738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.13","version":"27'16","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:25.726878+0000","last_change":"2024-04-27T03:34:25.726878+0000","last_active":"2024-04-27T03:34:25.726878+0000","last_peered":"2024-04-27T03:34:25.726878+0000","last_clean":"2024-04-27T03:34:25.726878+0000","last_became_active":"2024-04-27T03:30:21.408049+0000","last_became_peered":"2024-04-27T03:30:21.408049+0000","last_unstale":"2024-04-27T03:34:25.726878+0000","last_undegraded":"2024-04-27T03:34:25.726878+0000","last_fullsized":"2024-04-27T03:34:25.726878+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'16","last_scrub_stamp":"2024-04-27T03:34:25.726721+0000","last_deep_scrub":"27'16","last_deep_scrub_stamp":"2024-04-27T03:34:25.726721+0000","last_clean_scrub_stamp":"2024-04-27T03:34:25.726721+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:44:14.582492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":12,"num_write":21,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.693523+0000","last_change":"2024-04-27T03:34:08.693523+0000","last_active":"2024-04-27T03:34:08.693523+0000","last_peered":"2024-04-27T03:34:08.693523+0000","last_clean":"2024-04-27T03:34:08.693523+0000","last_became_active":"2024-04-27T03:30:13.373257+0000","last_became_peered":"2024-04-27T03:30:13.373257+0000","last_unstale":"2024-04-27T03:34:08.693523+0000","last_undegraded":"2024-04-27T03:34:08.693523+0000","last_fullsized":"2024-04-27T03:34:08.693523+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:08.693374+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:08.693374+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.693374+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:22.810525+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.222565+0000","last_change":"2024-04-27T03:34:08.222565+0000","last_active":"2024-04-27T03:34:08.222565+0000","last_peered":"2024-04-27T03:34:08.222565+0000","last_clean":"2024-04-27T03:34:08.222565+0000","last_became_active":"2024-04-27T03:30:11.370091+0000","last_became_peered":"2024-04-27T03:30:11.370091+0000","last_unstale":"2024-04-27T03:34:08.222565+0000","last_undegraded":"2024-04-27T03:34:08.222565+0000","last_fullsized":"2024-04-27T03:34:08.222565+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:08.222454+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:08.222454+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.222454+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:22:17.006853+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.756557+0000","last_change":"2024-04-27T03:33:51.756557+0000","last_active":"2024-04-27T03:33:51.756557+0000","last_peered":"2024-04-27T03:33:51.756557+0000","last_clean":"2024-04-27T03:33:51.756557+0000","last_became_active":"2024-04-27T03:30:15.379800+0000","last_became_peered":"2024-04-27T03:30:15.379800+0000","last_unstale":"2024-04-27T03:33:51.756557+0000","last_undegraded":"2024-04-27T03:33:51.756557+0000","last_fullsized":"2024-04-27T03:33:51.756557+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:51.756438+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:51.756438+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.756438+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:14:49.565231+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1c","version":"27'46","reported_seq":104,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.217186+0000","last_change":"2024-04-27T03:34:02.217229+0000","last_active":"2024-04-27T03:34:02.217186+0000","last_peered":"2024-04-27T03:34:02.217186+0000","last_clean":"2024-04-27T03:34:02.217186+0000","last_became_active":"2024-04-27T03:30:17.388246+0000","last_became_peered":"2024-04-27T03:30:17.388246+0000","last_unstale":"2024-04-27T03:34:02.217186+0000","last_undegraded":"2024-04-27T03:34:02.217186+0000","last_fullsized":"2024-04-27T03:34:02.217186+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:34:02.217099+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:34:02.217099+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.217099+0000","objects_scrubbed":7,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:17:44.534386+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":44,"num_read_kb":35,"num_write":20,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.948322+0000","last_change":"2024-04-27T03:33:40.948348+0000","last_active":"2024-04-27T03:33:40.948322+0000","last_peered":"2024-04-27T03:33:40.948322+0000","last_clean":"2024-04-27T03:33:40.948322+0000","last_became_active":"2024-04-27T03:30:19.406707+0000","last_became_peered":"2024-04-27T03:30:19.406707+0000","last_unstale":"2024-04-27T03:33:40.948322+0000","last_undegraded":"2024-04-27T03:33:40.948322+0000","last_fullsized":"2024-04-27T03:33:40.948322+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:40.948229+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:40.948229+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.948229+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:11.173852+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:23.317834+0000","last_change":"2024-04-27T03:34:23.317834+0000","last_active":"2024-04-27T03:34:23.317834+0000","last_peered":"2024-04-27T03:34:23.317834+0000","last_clean":"2024-04-27T03:34:23.317834+0000","last_became_active":"2024-04-27T03:30:21.415441+0000","last_became_peered":"2024-04-27T03:30:21.415441+0000","last_unstale":"2024-04-27T03:34:23.317834+0000","last_undegraded":"2024-04-27T03:34:23.317834+0000","last_fullsized":"2024-04-27T03:34:23.317834+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:23.317720+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:23.317720+0000","last_clean_scrub_stamp":"2024-04-27T03:34:23.317720+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:41:31.120111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1d","version":"27'2","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:44.127154+0000","last_change":"2024-04-27T03:33:44.127210+0000","last_active":"2024-04-27T03:33:44.127154+0000","last_peered":"2024-04-27T03:33:44.127154+0000","last_clean":"2024-04-27T03:33:44.127154+0000","last_became_active":"2024-04-27T03:30:13.387397+0000","last_became_peered":"2024-04-27T03:30:13.387397+0000","last_unstale":"2024-04-27T03:33:44.127154+0000","last_undegraded":"2024-04-27T03:33:44.127154+0000","last_fullsized":"2024-04-27T03:33:44.127154+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:44.127050+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:44.127050+0000","last_clean_scrub_stamp":"2024-04-27T03:33:44.127050+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:24.841845+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:16.127749+0000","last_change":"2024-04-27T03:34:16.127771+0000","last_active":"2024-04-27T03:34:16.127749+0000","last_peered":"2024-04-27T03:34:16.127749+0000","last_clean":"2024-04-27T03:34:16.127749+0000","last_became_active":"2024-04-27T03:30:11.369799+0000","last_became_peered":"2024-04-27T03:30:11.369799+0000","last_unstale":"2024-04-27T03:34:16.127749+0000","last_undegraded":"2024-04-27T03:34:16.127749+0000","last_fullsized":"2024-04-27T03:34:16.127749+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:16.127666+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:16.127666+0000","last_clean_scrub_stamp":"2024-04-27T03:34:16.127666+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:05:04.947407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:53.786343+0000","last_change":"2024-04-27T03:33:53.786369+0000","last_active":"2024-04-27T03:33:53.786343+0000","last_peered":"2024-04-27T03:33:53.786343+0000","last_clean":"2024-04-27T03:33:53.786343+0000","last_became_active":"2024-04-27T03:30:15.379873+0000","last_became_peered":"2024-04-27T03:30:15.379873+0000","last_unstale":"2024-04-27T03:33:53.786343+0000","last_undegraded":"2024-04-27T03:33:53.786343+0000","last_fullsized":"2024-04-27T03:33:53.786343+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:53.786220+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:53.786220+0000","last_clean_scrub_stamp":"2024-04-27T03:33:53.786220+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:41:31.263421+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.1f","version":"27'48","reported_seq":111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:43.722022+0000","last_change":"2024-04-27T03:33:43.722022+0000","last_active":"2024-04-27T03:33:43.722022+0000","last_peered":"2024-04-27T03:33:43.722022+0000","last_clean":"2024-04-27T03:33:43.722022+0000","last_became_active":"2024-04-27T03:30:17.387205+0000","last_became_peered":"2024-04-27T03:30:17.387205+0000","last_unstale":"2024-04-27T03:33:43.722022+0000","last_undegraded":"2024-04-27T03:33:43.722022+0000","last_fullsized":"2024-04-27T03:33:43.722022+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'48","last_scrub_stamp":"2024-04-27T03:33:43.721859+0000","last_deep_scrub":"27'48","last_deep_scrub_stamp":"2024-04-27T03:33:43.721859+0000","last_clean_scrub_stamp":"2024-04-27T03:33:43.721859+0000","objects_scrubbed":6,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:58:42.507737+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":53,"num_read_kb":63,"num_write":26,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:34.822276+0000","last_change":"2024-04-27T03:33:34.822276+0000","last_active":"2024-04-27T03:33:34.822276+0000","last_peered":"2024-04-27T03:33:34.822276+0000","last_clean":"2024-04-27T03:33:34.822276+0000","last_became_active":"2024-04-27T03:30:19.405462+0000","last_became_peered":"2024-04-27T03:30:19.405462+0000","last_unstale":"2024-04-27T03:33:34.822276+0000","last_undegraded":"2024-04-27T03:33:34.822276+0000","last_fullsized":"2024-04-27T03:33:34.822276+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:34.822124+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:34.822124+0000","last_clean_scrub_stamp":"2024-04-27T03:33:34.822124+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:04:28.428726+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.11","version":"27'7","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:54.357406+0000","last_change":"2024-04-27T03:34:54.357406+0000","last_active":"2024-04-27T03:34:54.357406+0000","last_peered":"2024-04-27T03:34:54.357406+0000","last_clean":"2024-04-27T03:34:54.357406+0000","last_became_active":"2024-04-27T03:30:21.416592+0000","last_became_peered":"2024-04-27T03:30:21.416592+0000","last_unstale":"2024-04-27T03:34:54.357406+0000","last_undegraded":"2024-04-27T03:34:54.357406+0000","last_fullsized":"2024-04-27T03:34:54.357406+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'7","last_scrub_stamp":"2024-04-27T03:34:54.357267+0000","last_deep_scrub":"27'7","last_deep_scrub_stamp":"2024-04-27T03:34:54.357267+0000","last_clean_scrub_stamp":"2024-04-27T03:34:54.357267+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:32:14.911867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":14,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1c","version":"27'6","reported_seq":63,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:04.718193+0000","last_change":"2024-04-27T03:34:04.718246+0000","last_active":"2024-04-27T03:34:04.718193+0000","last_peered":"2024-04-27T03:34:04.718193+0000","last_clean":"2024-04-27T03:34:04.718193+0000","last_became_active":"2024-04-27T03:30:13.369643+0000","last_became_peered":"2024-04-27T03:30:13.369643+0000","last_unstale":"2024-04-27T03:34:04.718193+0000","last_undegraded":"2024-04-27T03:34:04.718193+0000","last_fullsized":"2024-04-27T03:34:04.718193+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:04.718068+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:04.718068+0000","last_clean_scrub_stamp":"2024-04-27T03:34:04.718068+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:13:49.122146+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":25,"num_read_kb":25,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.1b","version":"27'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:35.349265+0000","last_change":"2024-04-27T03:34:35.349290+0000","last_active":"2024-04-27T03:34:35.349265+0000","last_peered":"2024-04-27T03:34:35.349265+0000","last_clean":"2024-04-27T03:34:35.349265+0000","last_became_active":"2024-04-27T03:30:11.372220+0000","last_became_peered":"2024-04-27T03:30:11.372220+0000","last_unstale":"2024-04-27T03:34:35.349265+0000","last_undegraded":"2024-04-27T03:34:35.349265+0000","last_fullsized":"2024-04-27T03:34:35.349265+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:35.349174+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:35.349174+0000","last_clean_scrub_stamp":"2024-04-27T03:34:35.349174+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:02:22.579508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":625,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:49.972519+0000","last_change":"2024-04-27T03:33:49.972545+0000","last_active":"2024-04-27T03:33:49.972519+0000","last_peered":"2024-04-27T03:33:49.972519+0000","last_clean":"2024-04-27T03:33:49.972519+0000","last_became_active":"2024-04-27T03:30:15.380573+0000","last_became_peered":"2024-04-27T03:30:15.380573+0000","last_unstale":"2024-04-27T03:33:49.972519+0000","last_undegraded":"2024-04-27T03:33:49.972519+0000","last_fullsized":"2024-04-27T03:33:49.972519+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:49.972421+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:49.972421+0000","last_clean_scrub_stamp":"2024-04-27T03:33:49.972421+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:49:13.206549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1e","version":"27'22","reported_seq":68,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:47.170926+0000","last_change":"2024-04-27T03:33:47.170926+0000","last_active":"2024-04-27T03:33:47.170926+0000","last_peered":"2024-04-27T03:33:47.170926+0000","last_clean":"2024-04-27T03:33:47.170926+0000","last_became_active":"2024-04-27T03:30:17.388289+0000","last_became_peered":"2024-04-27T03:30:17.388289+0000","last_unstale":"2024-04-27T03:33:47.170926+0000","last_undegraded":"2024-04-27T03:33:47.170926+0000","last_fullsized":"2024-04-27T03:33:47.170926+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'22","last_scrub_stamp":"2024-04-27T03:33:47.170743+0000","last_deep_scrub":"27'22","last_deep_scrub_stamp":"2024-04-27T03:33:47.170743+0000","last_clean_scrub_stamp":"2024-04-27T03:33:47.170743+0000","objects_scrubbed":6,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:35:01.287895+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":24,"num_read_kb":18,"num_write":16,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"27'285","reported_seq":385,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.829103+0000","last_change":"2024-04-27T03:33:40.829103+0000","last_active":"2024-04-27T03:33:40.829103+0000","last_peered":"2024-04-27T03:33:40.829103+0000","last_clean":"2024-04-27T03:33:40.829103+0000","last_became_active":"2024-04-27T03:30:19.406854+0000","last_became_peered":"2024-04-27T03:30:19.406854+0000","last_unstale":"2024-04-27T03:33:40.829103+0000","last_undegraded":"2024-04-27T03:33:40.829103+0000","last_fullsized":"2024-04-27T03:33:40.829103+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'285","last_scrub_stamp":"2024-04-27T03:33:40.828943+0000","last_deep_scrub":"27'285","last_deep_scrub_stamp":"2024-04-27T03:33:40.828943+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.828943+0000","objects_scrubbed":1,"log_size":285,"log_dups_size":0,"ondisk_log_size":285,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:28:59.341953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":27,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:51.324016+0000","last_change":"2024-04-27T03:34:51.324016+0000","last_active":"2024-04-27T03:34:51.324016+0000","last_peered":"2024-04-27T03:34:51.324016+0000","last_clean":"2024-04-27T03:34:51.324016+0000","last_became_active":"2024-04-27T03:30:21.415534+0000","last_became_peered":"2024-04-27T03:30:21.415534+0000","last_unstale":"2024-04-27T03:34:51.324016+0000","last_undegraded":"2024-04-27T03:34:51.324016+0000","last_fullsized":"2024-04-27T03:34:51.324016+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:51.323902+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:51.323902+0000","last_clean_scrub_stamp":"2024-04-27T03:34:51.323902+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:55:21.963088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1b","version":"27'24","reported_seq":111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.819891+0000","last_change":"2024-04-27T03:33:57.819934+0000","last_active":"2024-04-27T03:33:57.819891+0000","last_peered":"2024-04-27T03:33:57.819891+0000","last_clean":"2024-04-27T03:33:57.819891+0000","last_became_active":"2024-04-27T03:30:13.371158+0000","last_became_peered":"2024-04-27T03:30:13.371158+0000","last_unstale":"2024-04-27T03:33:57.819891+0000","last_undegraded":"2024-04-27T03:33:57.819891+0000","last_fullsized":"2024-04-27T03:33:57.819891+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-04-27T03:33:57.819758+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-04-27T03:33:57.819758+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.819758+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:10:53.203585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":88,"num_write":28,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":46,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:20.150149+0000","last_change":"2024-04-27T03:34:20.150172+0000","last_active":"2024-04-27T03:34:20.150149+0000","last_peered":"2024-04-27T03:34:20.150149+0000","last_clean":"2024-04-27T03:34:20.150149+0000","last_became_active":"2024-04-27T03:30:11.358893+0000","last_became_peered":"2024-04-27T03:30:11.358893+0000","last_unstale":"2024-04-27T03:34:20.150149+0000","last_undegraded":"2024-04-27T03:34:20.150149+0000","last_fullsized":"2024-04-27T03:34:20.150149+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:20.150063+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:20.150063+0000","last_clean_scrub_stamp":"2024-04-27T03:34:20.150063+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:19:31.126632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:48.016872+0000","last_change":"2024-04-27T03:33:48.016893+0000","last_active":"2024-04-27T03:33:48.016872+0000","last_peered":"2024-04-27T03:33:48.016872+0000","last_clean":"2024-04-27T03:33:48.016872+0000","last_became_active":"2024-04-27T03:30:15.381175+0000","last_became_peered":"2024-04-27T03:30:15.381175+0000","last_unstale":"2024-04-27T03:33:48.016872+0000","last_undegraded":"2024-04-27T03:33:48.016872+0000","last_fullsized":"2024-04-27T03:33:48.016872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:48.016786+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:48.016786+0000","last_clean_scrub_stamp":"2024-04-27T03:33:48.016786+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:39:37.651908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.19","version":"27'73","reported_seq":156,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.916480+0000","last_change":"2024-04-27T03:33:39.916480+0000","last_active":"2024-04-27T03:33:39.916480+0000","last_peered":"2024-04-27T03:33:39.916480+0000","last_clean":"2024-04-27T03:33:39.916480+0000","last_became_active":"2024-04-27T03:30:17.386673+0000","last_became_peered":"2024-04-27T03:30:17.386673+0000","last_unstale":"2024-04-27T03:33:39.916480+0000","last_undegraded":"2024-04-27T03:33:39.916480+0000","last_fullsized":"2024-04-27T03:33:39.916480+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'73","last_scrub_stamp":"2024-04-27T03:33:39.916339+0000","last_deep_scrub":"27'73","last_deep_scrub_stamp":"2024-04-27T03:33:39.916339+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.916339+0000","objects_scrubbed":11,"log_size":73,"log_dups_size":0,"ondisk_log_size":73,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:28:47.317240+0000","scrub_duration":6,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":80,"num_read_kb":67,"num_write":44,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.711878+0000","last_change":"2024-04-27T03:33:50.711878+0000","last_active":"2024-04-27T03:33:50.711878+0000","last_peered":"2024-04-27T03:33:50.711878+0000","last_clean":"2024-04-27T03:33:50.711878+0000","last_became_active":"2024-04-27T03:30:19.406990+0000","last_became_peered":"2024-04-27T03:30:19.406990+0000","last_unstale":"2024-04-27T03:33:50.711878+0000","last_undegraded":"2024-04-27T03:33:50.711878+0000","last_fullsized":"2024-04-27T03:33:50.711878+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:50.711713+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:50.711713+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.711713+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:39:29.194718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.179312+0000","last_change":"2024-04-27T03:34:01.179339+0000","last_active":"2024-04-27T03:34:01.179312+0000","last_peered":"2024-04-27T03:34:01.179312+0000","last_clean":"2024-04-27T03:34:01.179312+0000","last_became_active":"2024-04-27T03:30:21.410060+0000","last_became_peered":"2024-04-27T03:30:21.410060+0000","last_unstale":"2024-04-27T03:34:01.179312+0000","last_undegraded":"2024-04-27T03:34:01.179312+0000","last_fullsized":"2024-04-27T03:34:01.179312+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:01.179224+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:01.179224+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.179224+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:48.331237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"27'10","reported_seq":75,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.805577+0000","last_change":"2024-04-27T03:33:54.805600+0000","last_active":"2024-04-27T03:33:54.805577+0000","last_peered":"2024-04-27T03:33:54.805577+0000","last_clean":"2024-04-27T03:33:54.805577+0000","last_became_active":"2024-04-27T03:30:13.371866+0000","last_became_peered":"2024-04-27T03:30:13.371866+0000","last_unstale":"2024-04-27T03:33:54.805577+0000","last_undegraded":"2024-04-27T03:33:54.805577+0000","last_fullsized":"2024-04-27T03:33:54.805577+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:33:54.805478+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:33:54.805478+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.805478+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:21:58.032925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":49,"num_read_kb":45,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1d","version":"27'4","reported_seq":55,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:26.311334+0000","last_change":"2024-04-27T03:34:26.311360+0000","last_active":"2024-04-27T03:34:26.311334+0000","last_peered":"2024-04-27T03:34:26.311334+0000","last_clean":"2024-04-27T03:34:26.311334+0000","last_became_active":"2024-04-27T03:30:11.362420+0000","last_became_peered":"2024-04-27T03:30:11.362420+0000","last_unstale":"2024-04-27T03:34:26.311334+0000","last_undegraded":"2024-04-27T03:34:26.311334+0000","last_fullsized":"2024-04-27T03:34:26.311334+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:26.311245+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:26.311245+0000","last_clean_scrub_stamp":"2024-04-27T03:34:26.311245+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:11:05.338841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":13,"num_read_kb":8,"num_write":28,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":38,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:50.769662+0000","last_change":"2024-04-27T03:33:50.769691+0000","last_active":"2024-04-27T03:33:50.769662+0000","last_peered":"2024-04-27T03:33:50.769662+0000","last_clean":"2024-04-27T03:33:50.769662+0000","last_became_active":"2024-04-27T03:30:15.382305+0000","last_became_peered":"2024-04-27T03:30:15.382305+0000","last_unstale":"2024-04-27T03:33:50.769662+0000","last_undegraded":"2024-04-27T03:33:50.769662+0000","last_fullsized":"2024-04-27T03:33:50.769662+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:50.769565+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:50.769565+0000","last_clean_scrub_stamp":"2024-04-27T03:33:50.769565+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:59:30.717513+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.18","version":"27'46","reported_seq":108,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:40.785057+0000","last_change":"2024-04-27T03:33:40.785057+0000","last_active":"2024-04-27T03:33:40.785057+0000","last_peered":"2024-04-27T03:33:40.785057+0000","last_clean":"2024-04-27T03:33:40.785057+0000","last_became_active":"2024-04-27T03:30:17.390281+0000","last_became_peered":"2024-04-27T03:30:17.390281+0000","last_unstale":"2024-04-27T03:33:40.785057+0000","last_undegraded":"2024-04-27T03:33:40.785057+0000","last_fullsized":"2024-04-27T03:33:40.785057+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'46","last_scrub_stamp":"2024-04-27T03:33:40.784918+0000","last_deep_scrub":"27'46","last_deep_scrub_stamp":"2024-04-27T03:33:40.784918+0000","last_clean_scrub_stamp":"2024-04-27T03:33:40.784918+0000","objects_scrubbed":5,"log_size":46,"log_dups_size":0,"ondisk_log_size":46,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:07:11.010541+0000","scrub_duration":3,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4096,"num_objects":5,"num_object_clones":0,"num_object_copies":10,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":49,"num_read_kb":50,"num_write":24,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":31,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:59.154184+0000","last_change":"2024-04-27T03:33:59.154184+0000","last_active":"2024-04-27T03:33:59.154184+0000","last_peered":"2024-04-27T03:33:59.154184+0000","last_clean":"2024-04-27T03:33:59.154184+0000","last_became_active":"2024-04-27T03:30:19.405353+0000","last_became_peered":"2024-04-27T03:30:19.405353+0000","last_unstale":"2024-04-27T03:33:59.154184+0000","last_undegraded":"2024-04-27T03:33:59.154184+0000","last_fullsized":"2024-04-27T03:33:59.154184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:59.154022+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:59.154022+0000","last_clean_scrub_stamp":"2024-04-27T03:33:59.154022+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:53:09.531303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":26,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:53.364879+0000","last_change":"2024-04-27T03:34:53.364924+0000","last_active":"2024-04-27T03:34:53.364879+0000","last_peered":"2024-04-27T03:34:53.364879+0000","last_clean":"2024-04-27T03:34:53.364879+0000","last_became_active":"2024-04-27T03:30:21.416276+0000","last_became_peered":"2024-04-27T03:30:21.416276+0000","last_unstale":"2024-04-27T03:34:53.364879+0000","last_undegraded":"2024-04-27T03:34:53.364879+0000","last_fullsized":"2024-04-27T03:34:53.364879+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:53.364768+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:53.364768+0000","last_clean_scrub_stamp":"2024-04-27T03:34:53.364768+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:06:04.438786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"27'24","reported_seq":111,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:13.170839+0000","last_change":"2024-04-27T03:34:13.170839+0000","last_active":"2024-04-27T03:34:13.170839+0000","last_peered":"2024-04-27T03:34:13.170839+0000","last_clean":"2024-04-27T03:34:13.170839+0000","last_became_active":"2024-04-27T03:30:13.376415+0000","last_became_peered":"2024-04-27T03:30:13.376415+0000","last_unstale":"2024-04-27T03:34:13.170839+0000","last_undegraded":"2024-04-27T03:34:13.170839+0000","last_fullsized":"2024-04-27T03:34:13.170839+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'24","last_scrub_stamp":"2024-04-27T03:34:13.170716+0000","last_deep_scrub":"27'24","last_deep_scrub_stamp":"2024-04-27T03:34:13.170716+0000","last_clean_scrub_stamp":"2024-04-27T03:34:13.170716+0000","objects_scrubbed":0,"log_size":24,"log_dups_size":0,"ondisk_log_size":24,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:49:02.946260+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":109,"num_read_kb":97,"num_write":30,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:11.675077+0000","last_change":"2024-04-27T03:34:11.675077+0000","last_active":"2024-04-27T03:34:11.675077+0000","last_peered":"2024-04-27T03:34:11.675077+0000","last_clean":"2024-04-27T03:34:11.675077+0000","last_became_active":"2024-04-27T03:30:11.360590+0000","last_became_peered":"2024-04-27T03:30:11.360590+0000","last_unstale":"2024-04-27T03:34:11.675077+0000","last_undegraded":"2024-04-27T03:34:11.675077+0000","last_fullsized":"2024-04-27T03:34:11.675077+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:11.674894+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:11.674894+0000","last_clean_scrub_stamp":"2024-04-27T03:34:11.674894+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:39:37.256713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":39,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.996361+0000","last_change":"2024-04-27T03:33:42.996361+0000","last_active":"2024-04-27T03:33:42.996361+0000","last_peered":"2024-04-27T03:33:42.996361+0000","last_clean":"2024-04-27T03:33:42.996361+0000","last_became_active":"2024-04-27T03:30:15.385614+0000","last_became_peered":"2024-04-27T03:30:15.385614+0000","last_unstale":"2024-04-27T03:33:42.996361+0000","last_undegraded":"2024-04-27T03:33:42.996361+0000","last_fullsized":"2024-04-27T03:33:42.996361+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.996219+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.996219+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.996219+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:58.172124+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"27'37","reported_seq":96,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:39.164208+0000","last_change":"2024-04-27T03:33:39.164242+0000","last_active":"2024-04-27T03:33:39.164208+0000","last_peered":"2024-04-27T03:33:39.164208+0000","last_clean":"2024-04-27T03:33:39.164208+0000","last_became_active":"2024-04-27T03:30:17.386983+0000","last_became_peered":"2024-04-27T03:30:17.386983+0000","last_unstale":"2024-04-27T03:33:39.164208+0000","last_undegraded":"2024-04-27T03:33:39.164208+0000","last_fullsized":"2024-04-27T03:33:39.164208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'37","last_scrub_stamp":"2024-04-27T03:33:39.164121+0000","last_deep_scrub":"27'37","last_deep_scrub_stamp":"2024-04-27T03:33:39.164121+0000","last_clean_scrub_stamp":"2024-04-27T03:33:39.164121+0000","objects_scrubbed":3,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:23:26.316134+0000","scrub_duration":2,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":3,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":38,"num_read_kb":33,"num_write":16,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.113482+0000","last_change":"2024-04-27T03:33:54.113533+0000","last_active":"2024-04-27T03:33:54.113482+0000","last_peered":"2024-04-27T03:33:54.113482+0000","last_clean":"2024-04-27T03:33:54.113482+0000","last_became_active":"2024-04-27T03:30:19.402652+0000","last_became_peered":"2024-04-27T03:30:19.402652+0000","last_unstale":"2024-04-27T03:33:54.113482+0000","last_undegraded":"2024-04-27T03:33:54.113482+0000","last_fullsized":"2024-04-27T03:33:54.113482+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:54.113385+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:54.113385+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.113385+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:04:42.933126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.15","version":"27'6","reported_seq":32,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:56.370851+0000","last_change":"2024-04-27T03:34:56.370878+0000","last_active":"2024-04-27T03:34:56.370851+0000","last_peered":"2024-04-27T03:34:56.370851+0000","last_clean":"2024-04-27T03:34:56.370851+0000","last_became_active":"2024-04-27T03:30:21.420725+0000","last_became_peered":"2024-04-27T03:30:21.420725+0000","last_unstale":"2024-04-27T03:34:56.370851+0000","last_undegraded":"2024-04-27T03:34:56.370851+0000","last_fullsized":"2024-04-27T03:34:56.370851+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:56.370762+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:56.370762+0000","last_clean_scrub_stamp":"2024-04-27T03:34:56.370762+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:04:04.146837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":3,"num_write":11,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":43,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:58.788151+0000","last_change":"2024-04-27T03:33:58.788151+0000","last_active":"2024-04-27T03:33:58.788151+0000","last_peered":"2024-04-27T03:33:58.788151+0000","last_clean":"2024-04-27T03:33:58.788151+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:33:58.788151+0000","last_undegraded":"2024-04-27T03:33:58.788151+0000","last_fullsized":"2024-04-27T03:33:58.788151+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:58.788034+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:58.788034+0000","last_clean_scrub_stamp":"2024-04-27T03:33:58.788034+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:14:43.897282+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:25.076472+0000","last_change":"2024-04-27T03:34:25.076512+0000","last_active":"2024-04-27T03:34:25.076472+0000","last_peered":"2024-04-27T03:34:25.076472+0000","last_clean":"2024-04-27T03:34:25.076472+0000","last_became_active":"2024-04-27T03:30:11.369877+0000","last_became_peered":"2024-04-27T03:30:11.369877+0000","last_unstale":"2024-04-27T03:34:25.076472+0000","last_undegraded":"2024-04-27T03:34:25.076472+0000","last_fullsized":"2024-04-27T03:34:25.076472+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:25.076378+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:25.076378+0000","last_clean_scrub_stamp":"2024-04-27T03:34:25.076378+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:44:13.932150+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.19","version":"27'3","reported_seq":42,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:45.974632+0000","last_change":"2024-04-27T03:33:45.974632+0000","last_active":"2024-04-27T03:33:45.974632+0000","last_peered":"2024-04-27T03:33:45.974632+0000","last_clean":"2024-04-27T03:33:45.974632+0000","last_became_active":"2024-04-27T03:30:15.381574+0000","last_became_peered":"2024-04-27T03:30:15.381574+0000","last_unstale":"2024-04-27T03:33:45.974632+0000","last_undegraded":"2024-04-27T03:33:45.974632+0000","last_fullsized":"2024-04-27T03:33:45.974632+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-27T03:33:45.974523+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-27T03:33:45.974523+0000","last_clean_scrub_stamp":"2024-04-27T03:33:45.974523+0000","objects_scrubbed":1,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:00:37.524758+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":46,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1a","version":"27'53","reported_seq":122,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:57.159896+0000","last_change":"2024-04-27T03:33:57.159964+0000","last_active":"2024-04-27T03:33:57.159896+0000","last_peered":"2024-04-27T03:33:57.159896+0000","last_clean":"2024-04-27T03:33:57.159896+0000","last_became_active":"2024-04-27T03:30:17.388277+0000","last_became_peered":"2024-04-27T03:30:17.388277+0000","last_unstale":"2024-04-27T03:33:57.159896+0000","last_undegraded":"2024-04-27T03:33:57.159896+0000","last_fullsized":"2024-04-27T03:33:57.159896+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'53","last_scrub_stamp":"2024-04-27T03:33:57.159788+0000","last_deep_scrub":"27'53","last_deep_scrub_stamp":"2024-04-27T03:33:57.159788+0000","last_clean_scrub_stamp":"2024-04-27T03:33:57.159788+0000","objects_scrubbed":7,"log_size":53,"log_dups_size":0,"ondisk_log_size":53,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:37:03.430517+0000","scrub_duration":5,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":110,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":58,"num_read_kb":52,"num_write":32,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":1138,"num_omap_keys":6,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":30,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:42.728511+0000","last_change":"2024-04-27T03:33:42.728536+0000","last_active":"2024-04-27T03:33:42.728511+0000","last_peered":"2024-04-27T03:33:42.728511+0000","last_clean":"2024-04-27T03:33:42.728511+0000","last_became_active":"2024-04-27T03:30:19.406565+0000","last_became_peered":"2024-04-27T03:30:19.406565+0000","last_unstale":"2024-04-27T03:33:42.728511+0000","last_undegraded":"2024-04-27T03:33:42.728511+0000","last_fullsized":"2024-04-27T03:33:42.728511+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:33:42.728420+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:33:42.728420+0000","last_clean_scrub_stamp":"2024-04-27T03:33:42.728420+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:18:57.904325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.14","version":"27'4","reported_seq":33,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:19.270233+0000","last_change":"2024-04-27T03:34:19.270233+0000","last_active":"2024-04-27T03:34:19.270233+0000","last_peered":"2024-04-27T03:34:19.270233+0000","last_clean":"2024-04-27T03:34:19.270233+0000","last_became_active":"2024-04-27T03:30:21.414510+0000","last_became_peered":"2024-04-27T03:30:21.414510+0000","last_unstale":"2024-04-27T03:34:19.270233+0000","last_undegraded":"2024-04-27T03:34:19.270233+0000","last_fullsized":"2024-04-27T03:34:19.270233+0000","mapping_epoch":23,"log_start":"0'0","ondisk_log_start":"0'0","created":23,"last_epoch_clean":24,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:19.270117+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:19.270117+0000","last_clean_scrub_stamp":"2024-04-27T03:34:19.270117+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:43:26.209785+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":10,"num_read_kb":8,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.27","version":"25'8","reported_seq":60,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:54.006570+0000","last_change":"2024-04-27T03:33:54.006570+0000","last_active":"2024-04-27T03:33:54.006570+0000","last_peered":"2024-04-27T03:33:54.006570+0000","last_clean":"2024-04-27T03:33:54.006570+0000","last_became_active":"2024-04-27T03:30:13.387149+0000","last_became_peered":"2024-04-27T03:30:13.387149+0000","last_unstale":"2024-04-27T03:33:54.006570+0000","last_undegraded":"2024-04-27T03:33:54.006570+0000","last_fullsized":"2024-04-27T03:33:54.006570+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'8","last_scrub_stamp":"2024-04-27T03:33:54.006451+0000","last_deep_scrub":"25'8","last_deep_scrub_stamp":"2024-04-27T03:33:54.006451+0000","last_clean_scrub_stamp":"2024-04-27T03:33:54.006451+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:40:20.542228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":29,"num_read_kb":25,"num_write":10,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.20","version":"25'6","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:12.836026+0000","last_change":"2024-04-27T03:34:12.836026+0000","last_active":"2024-04-27T03:34:12.836026+0000","last_peered":"2024-04-27T03:34:12.836026+0000","last_clean":"2024-04-27T03:34:12.836026+0000","last_became_active":"2024-04-27T03:30:11.357878+0000","last_became_peered":"2024-04-27T03:30:11.357878+0000","last_unstale":"2024-04-27T03:34:12.836026+0000","last_undegraded":"2024-04-27T03:34:12.836026+0000","last_fullsized":"2024-04-27T03:34:12.836026+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'6","last_scrub_stamp":"2024-04-27T03:34:12.835816+0000","last_deep_scrub":"25'6","last_deep_scrub_stamp":"2024-04-27T03:34:12.835816+0000","last_clean_scrub_stamp":"2024-04-27T03:34:12.835816+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:20:53.733778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":3,"num_write":14,"num_write_kb":4722,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.26","version":"27'4","reported_seq":56,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:02.707653+0000","last_change":"2024-04-27T03:34:02.707653+0000","last_active":"2024-04-27T03:34:02.707653+0000","last_peered":"2024-04-27T03:34:02.707653+0000","last_clean":"2024-04-27T03:34:02.707653+0000","last_became_active":"2024-04-27T03:30:13.369681+0000","last_became_peered":"2024-04-27T03:30:13.369681+0000","last_unstale":"2024-04-27T03:34:02.707653+0000","last_undegraded":"2024-04-27T03:34:02.707653+0000","last_fullsized":"2024-04-27T03:34:02.707653+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:02.707538+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:02.707538+0000","last_clean_scrub_stamp":"2024-04-27T03:34:02.707538+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:58.202906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":15,"num_read_kb":15,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.21","version":"25'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:01.696795+0000","last_change":"2024-04-27T03:34:01.696795+0000","last_active":"2024-04-27T03:34:01.696795+0000","last_peered":"2024-04-27T03:34:01.696795+0000","last_clean":"2024-04-27T03:34:01.696795+0000","last_became_active":"2024-04-27T03:30:11.373186+0000","last_became_peered":"2024-04-27T03:30:11.373186+0000","last_unstale":"2024-04-27T03:34:01.696795+0000","last_undegraded":"2024-04-27T03:34:01.696795+0000","last_fullsized":"2024-04-27T03:34:01.696795+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:01.696661+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:01.696661+0000","last_clean_scrub_stamp":"2024-04-27T03:34:01.696661+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:04:33.957902+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.25","version":"27'2","reported_seq":48,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:08.249759+0000","last_change":"2024-04-27T03:34:08.249759+0000","last_active":"2024-04-27T03:34:08.249759+0000","last_peered":"2024-04-27T03:34:08.249759+0000","last_clean":"2024-04-27T03:34:08.249759+0000","last_became_active":"2024-04-27T03:30:13.382069+0000","last_became_peered":"2024-04-27T03:30:13.382069+0000","last_unstale":"2024-04-27T03:34:08.249759+0000","last_undegraded":"2024-04-27T03:34:08.249759+0000","last_fullsized":"2024-04-27T03:34:08.249759+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:08.249643+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:08.249643+0000","last_clean_scrub_stamp":"2024-04-27T03:34:08.249643+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:17:08.768291+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.22","version":"27'3","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:46.362974+0000","last_change":"2024-04-27T03:34:46.363030+0000","last_active":"2024-04-27T03:34:46.362974+0000","last_peered":"2024-04-27T03:34:46.362974+0000","last_clean":"2024-04-27T03:34:46.362974+0000","last_became_active":"2024-04-27T03:30:11.350295+0000","last_became_peered":"2024-04-27T03:30:11.350295+0000","last_unstale":"2024-04-27T03:34:46.362974+0000","last_undegraded":"2024-04-27T03:34:46.362974+0000","last_fullsized":"2024-04-27T03:34:46.362974+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'3","last_scrub_stamp":"2024-04-27T03:34:46.362862+0000","last_deep_scrub":"27'3","last_deep_scrub_stamp":"2024-04-27T03:34:46.362862+0000","last_clean_scrub_stamp":"2024-04-27T03:34:46.362862+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:28:34.791681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":12,"num_read_kb":8,"num_write":12,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"27'4","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:06.676435+0000","last_change":"2024-04-27T03:34:06.676435+0000","last_active":"2024-04-27T03:34:06.676435+0000","last_peered":"2024-04-27T03:34:06.676435+0000","last_clean":"2024-04-27T03:34:06.676435+0000","last_became_active":"2024-04-27T03:30:13.370066+0000","last_became_peered":"2024-04-27T03:30:13.370066+0000","last_unstale":"2024-04-27T03:34:06.676435+0000","last_undegraded":"2024-04-27T03:34:06.676435+0000","last_fullsized":"2024-04-27T03:34:06.676435+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:06.676262+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:06.676262+0000","last_clean_scrub_stamp":"2024-04-27T03:34:06.676262+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:06.684143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.23","version":"27'4","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:22.693546+0000","last_change":"2024-04-27T03:34:22.693546+0000","last_active":"2024-04-27T03:34:22.693546+0000","last_peered":"2024-04-27T03:34:22.693546+0000","last_clean":"2024-04-27T03:34:22.693546+0000","last_became_active":"2024-04-27T03:30:11.360376+0000","last_became_peered":"2024-04-27T03:30:11.360376+0000","last_unstale":"2024-04-27T03:34:22.693546+0000","last_undegraded":"2024-04-27T03:34:22.693546+0000","last_fullsized":"2024-04-27T03:34:22.693546+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:22.693417+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:22.693417+0000","last_clean_scrub_stamp":"2024-04-27T03:34:22.693417+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:47:10.493189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":2,"num_read_kb":0,"num_write":4,"num_write_kb":8192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.23","version":"27'4","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:24.695215+0000","last_change":"2024-04-27T03:34:24.695265+0000","last_active":"2024-04-27T03:34:24.695215+0000","last_peered":"2024-04-27T03:34:24.695215+0000","last_clean":"2024-04-27T03:34:24.695215+0000","last_became_active":"2024-04-27T03:30:13.378368+0000","last_became_peered":"2024-04-27T03:30:13.378368+0000","last_unstale":"2024-04-27T03:34:24.695215+0000","last_undegraded":"2024-04-27T03:34:24.695215+0000","last_fullsized":"2024-04-27T03:34:24.695215+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:24.695112+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:24.695112+0000","last_clean_scrub_stamp":"2024-04-27T03:34:24.695112+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:47.159176+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"25'2","reported_seq":51,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.669200+0000","last_change":"2024-04-27T03:34:05.669200+0000","last_active":"2024-04-27T03:34:05.669200+0000","last_peered":"2024-04-27T03:34:05.669200+0000","last_clean":"2024-04-27T03:34:05.669200+0000","last_became_active":"2024-04-27T03:30:11.371553+0000","last_became_peered":"2024-04-27T03:30:11.371553+0000","last_unstale":"2024-04-27T03:34:05.669200+0000","last_undegraded":"2024-04-27T03:34:05.669200+0000","last_fullsized":"2024-04-27T03:34:05.669200+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:05.669082+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:05.669082+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.669082+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:30:46.383877+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":6,"num_read_kb":4,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.22","version":"27'10","reported_seq":73,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:32.090474+0000","last_change":"2024-04-27T03:34:32.090502+0000","last_active":"2024-04-27T03:34:32.090474+0000","last_peered":"2024-04-27T03:34:32.090474+0000","last_clean":"2024-04-27T03:34:32.090474+0000","last_became_active":"2024-04-27T03:31:01.440576+0000","last_became_peered":"2024-04-27T03:31:01.440576+0000","last_unstale":"2024-04-27T03:34:32.090474+0000","last_undegraded":"2024-04-27T03:34:32.090474+0000","last_fullsized":"2024-04-27T03:34:32.090474+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:34:32.090375+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:34:32.090375+0000","last_clean_scrub_stamp":"2024-04-27T03:34:32.090375+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:16:40.211937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":36,"num_read_kb":32,"num_write":12,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.25","version":"27'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:12.120768+0000","last_change":"2024-04-27T03:34:12.120768+0000","last_active":"2024-04-27T03:34:12.120768+0000","last_peered":"2024-04-27T03:34:12.120768+0000","last_clean":"2024-04-27T03:34:12.120768+0000","last_became_active":"2024-04-27T03:30:11.369517+0000","last_became_peered":"2024-04-27T03:30:11.369517+0000","last_unstale":"2024-04-27T03:34:12.120768+0000","last_undegraded":"2024-04-27T03:34:12.120768+0000","last_fullsized":"2024-04-27T03:34:12.120768+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:12.120650+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:12.120650+0000","last_clean_scrub_stamp":"2024-04-27T03:34:12.120650+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:30.112060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.21","version":"27'4","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:10.688469+0000","last_change":"2024-04-27T03:34:10.688469+0000","last_active":"2024-04-27T03:34:10.688469+0000","last_peered":"2024-04-27T03:34:10.688469+0000","last_clean":"2024-04-27T03:34:10.688469+0000","last_became_active":"2024-04-27T03:30:13.373770+0000","last_became_peered":"2024-04-27T03:30:13.373770+0000","last_unstale":"2024-04-27T03:34:10.688469+0000","last_undegraded":"2024-04-27T03:34:10.688469+0000","last_fullsized":"2024-04-27T03:34:10.688469+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:10.688326+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:10.688326+0000","last_clean_scrub_stamp":"2024-04-27T03:34:10.688326+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T15:09:44.126012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":52,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:22.881026+0000","last_change":"2024-04-27T03:34:22.881026+0000","last_active":"2024-04-27T03:34:22.881026+0000","last_peered":"2024-04-27T03:34:22.881026+0000","last_clean":"2024-04-27T03:34:22.881026+0000","last_became_active":"2024-04-27T03:31:01.441159+0000","last_became_peered":"2024-04-27T03:31:01.441159+0000","last_unstale":"2024-04-27T03:34:22.881026+0000","last_undegraded":"2024-04-27T03:34:22.881026+0000","last_fullsized":"2024-04-27T03:34:22.881026+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:22.880861+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:22.880861+0000","last_clean_scrub_stamp":"2024-04-27T03:34:22.880861+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:23:40.872271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.20","version":"27'4","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.181941+0000","last_change":"2024-04-27T03:34:07.181941+0000","last_active":"2024-04-27T03:34:07.181941+0000","last_peered":"2024-04-27T03:34:07.181941+0000","last_clean":"2024-04-27T03:34:07.181941+0000","last_became_active":"2024-04-27T03:30:13.377821+0000","last_became_peered":"2024-04-27T03:30:13.377821+0000","last_unstale":"2024-04-27T03:34:07.181941+0000","last_undegraded":"2024-04-27T03:34:07.181941+0000","last_fullsized":"2024-04-27T03:34:07.181941+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:07.181834+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:07.181834+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.181834+0000","objects_scrubbed":1,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T08:19:07.189714+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:03.697117+0000","last_change":"2024-04-27T03:34:03.697117+0000","last_active":"2024-04-27T03:34:03.697117+0000","last_peered":"2024-04-27T03:34:03.697117+0000","last_clean":"2024-04-27T03:34:03.697117+0000","last_became_active":"2024-04-27T03:30:11.371604+0000","last_became_peered":"2024-04-27T03:30:11.371604+0000","last_unstale":"2024-04-27T03:34:03.697117+0000","last_undegraded":"2024-04-27T03:34:03.697117+0000","last_fullsized":"2024-04-27T03:34:03.697117+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:03.697000+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:03.697000+0000","last_clean_scrub_stamp":"2024-04-27T03:34:03.697000+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:34:44.936107+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2f","version":"27'8","reported_seq":74,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:51.956191+0000","last_change":"2024-04-27T03:33:51.956191+0000","last_active":"2024-04-27T03:33:51.956191+0000","last_peered":"2024-04-27T03:33:51.956191+0000","last_clean":"2024-04-27T03:33:51.956191+0000","last_became_active":"2024-04-27T03:30:13.388421+0000","last_became_peered":"2024-04-27T03:30:13.388421+0000","last_unstale":"2024-04-27T03:33:51.956191+0000","last_undegraded":"2024-04-27T03:33:51.956191+0000","last_fullsized":"2024-04-27T03:33:51.956191+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'8","last_scrub_stamp":"2024-04-27T03:33:51.956075+0000","last_deep_scrub":"27'8","last_deep_scrub_stamp":"2024-04-27T03:33:51.956075+0000","last_clean_scrub_stamp":"2024-04-27T03:33:51.956075+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T14:21:03.313222+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":35,"num_read_kb":35,"num_write":8,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.28","version":"27'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:48.353112+0000","last_change":"2024-04-27T03:34:48.353139+0000","last_active":"2024-04-27T03:34:48.353112+0000","last_peered":"2024-04-27T03:34:48.353112+0000","last_clean":"2024-04-27T03:34:48.353112+0000","last_became_active":"2024-04-27T03:30:11.355183+0000","last_became_peered":"2024-04-27T03:30:11.355183+0000","last_unstale":"2024-04-27T03:34:48.353112+0000","last_undegraded":"2024-04-27T03:34:48.353112+0000","last_fullsized":"2024-04-27T03:34:48.353112+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:48.353023+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:48.353023+0000","last_clean_scrub_stamp":"2024-04-27T03:34:48.353023+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:44:55.651548+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2e","version":"27'4","reported_seq":54,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:17.237459+0000","last_change":"2024-04-27T03:34:17.237459+0000","last_active":"2024-04-27T03:34:17.237459+0000","last_peered":"2024-04-27T03:34:17.237459+0000","last_clean":"2024-04-27T03:34:17.237459+0000","last_became_active":"2024-04-27T03:30:13.376875+0000","last_became_peered":"2024-04-27T03:30:13.376875+0000","last_unstale":"2024-04-27T03:34:17.237459+0000","last_undegraded":"2024-04-27T03:34:17.237459+0000","last_fullsized":"2024-04-27T03:34:17.237459+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:34:17.237351+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:34:17.237351+0000","last_clean_scrub_stamp":"2024-04-27T03:34:17.237351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:06:05.267721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":11,"num_read_kb":11,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.29","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:07.648432+0000","last_change":"2024-04-27T03:34:07.648432+0000","last_active":"2024-04-27T03:34:07.648432+0000","last_peered":"2024-04-27T03:34:07.648432+0000","last_clean":"2024-04-27T03:34:07.648432+0000","last_became_active":"2024-04-27T03:30:11.360299+0000","last_became_peered":"2024-04-27T03:30:11.360299+0000","last_unstale":"2024-04-27T03:34:07.648432+0000","last_undegraded":"2024-04-27T03:34:07.648432+0000","last_fullsized":"2024-04-27T03:34:07.648432+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:07.648243+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:07.648243+0000","last_clean_scrub_stamp":"2024-04-27T03:34:07.648243+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:22:16.432642+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2d","version":"27'10","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:14.664018+0000","last_change":"2024-04-27T03:34:14.664018+0000","last_active":"2024-04-27T03:34:14.664018+0000","last_peered":"2024-04-27T03:34:14.664018+0000","last_clean":"2024-04-27T03:34:14.664018+0000","last_became_active":"2024-04-27T03:30:13.376087+0000","last_became_peered":"2024-04-27T03:30:13.376087+0000","last_unstale":"2024-04-27T03:34:14.664018+0000","last_undegraded":"2024-04-27T03:34:14.664018+0000","last_fullsized":"2024-04-27T03:34:14.664018+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'10","last_scrub_stamp":"2024-04-27T03:34:14.663832+0000","last_deep_scrub":"27'10","last_deep_scrub_stamp":"2024-04-27T03:34:14.663832+0000","last_clean_scrub_stamp":"2024-04-27T03:34:14.663832+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:08.919647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":10,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:10.162522+0000","last_change":"2024-04-27T03:34:10.162522+0000","last_active":"2024-04-27T03:34:10.162522+0000","last_peered":"2024-04-27T03:34:10.162522+0000","last_clean":"2024-04-27T03:34:10.162522+0000","last_became_active":"2024-04-27T03:30:11.370837+0000","last_became_peered":"2024-04-27T03:30:11.370837+0000","last_unstale":"2024-04-27T03:34:10.162522+0000","last_undegraded":"2024-04-27T03:34:10.162522+0000","last_fullsized":"2024-04-27T03:34:10.162522+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:10.162390+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:10.162390+0000","last_clean_scrub_stamp":"2024-04-27T03:34:10.162390+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:13:28.398152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":4,"num_read_kb":3,"num_write":10,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2c","version":"27'6","reported_seq":67,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.179323+0000","last_change":"2024-04-27T03:34:05.179323+0000","last_active":"2024-04-27T03:34:05.179323+0000","last_peered":"2024-04-27T03:34:05.179323+0000","last_clean":"2024-04-27T03:34:05.179323+0000","last_became_active":"2024-04-27T03:30:13.373511+0000","last_became_peered":"2024-04-27T03:30:13.373511+0000","last_unstale":"2024-04-27T03:34:05.179323+0000","last_undegraded":"2024-04-27T03:34:05.179323+0000","last_fullsized":"2024-04-27T03:34:05.179323+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:05.179213+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:05.179213+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.179213+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:57:49.583854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":26,"num_read_kb":26,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"27'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:14.167281+0000","last_change":"2024-04-27T03:34:14.167281+0000","last_active":"2024-04-27T03:34:14.167281+0000","last_peered":"2024-04-27T03:34:14.167281+0000","last_clean":"2024-04-27T03:34:14.167281+0000","last_became_active":"2024-04-27T03:30:11.370283+0000","last_became_peered":"2024-04-27T03:30:11.370283+0000","last_unstale":"2024-04-27T03:34:14.167281+0000","last_undegraded":"2024-04-27T03:34:14.167281+0000","last_fullsized":"2024-04-27T03:34:14.167281+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:14.167157+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:14.167157+0000","last_clean_scrub_stamp":"2024-04-27T03:34:14.167157+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T09:13:24.417880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.2b","version":"27'12","reported_seq":70,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:17.133790+0000","last_change":"2024-04-27T03:34:17.133790+0000","last_active":"2024-04-27T03:34:17.133790+0000","last_peered":"2024-04-27T03:34:17.133790+0000","last_clean":"2024-04-27T03:34:17.133790+0000","last_became_active":"2024-04-27T03:30:13.380404+0000","last_became_peered":"2024-04-27T03:30:13.380404+0000","last_unstale":"2024-04-27T03:34:17.133790+0000","last_undegraded":"2024-04-27T03:34:17.133790+0000","last_fullsized":"2024-04-27T03:34:17.133790+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'12","last_scrub_stamp":"2024-04-27T03:34:17.133674+0000","last_deep_scrub":"27'12","last_deep_scrub_stamp":"2024-04-27T03:34:17.133674+0000","last_clean_scrub_stamp":"2024-04-27T03:34:17.133674+0000","objects_scrubbed":1,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:57:39.000345+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":39,"num_read_kb":35,"num_write":13,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:24.348395+0000","last_change":"2024-04-27T03:34:24.348395+0000","last_active":"2024-04-27T03:34:24.348395+0000","last_peered":"2024-04-27T03:34:24.348395+0000","last_clean":"2024-04-27T03:34:24.348395+0000","last_became_active":"2024-04-27T03:30:11.373026+0000","last_became_peered":"2024-04-27T03:30:11.373026+0000","last_unstale":"2024-04-27T03:34:24.348395+0000","last_undegraded":"2024-04-27T03:34:24.348395+0000","last_fullsized":"2024-04-27T03:34:24.348395+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:24.348277+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:24.348277+0000","last_clean_scrub_stamp":"2024-04-27T03:34:24.348277+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T10:48:23.175329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.2a","version":"27'2","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:55.031559+0000","last_change":"2024-04-27T03:33:55.031559+0000","last_active":"2024-04-27T03:33:55.031559+0000","last_peered":"2024-04-27T03:33:55.031559+0000","last_clean":"2024-04-27T03:33:55.031559+0000","last_became_active":"2024-04-27T03:30:13.385791+0000","last_became_peered":"2024-04-27T03:30:13.385791+0000","last_unstale":"2024-04-27T03:33:55.031559+0000","last_undegraded":"2024-04-27T03:33:55.031559+0000","last_fullsized":"2024-04-27T03:33:55.031559+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:33:55.031445+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:33:55.031445+0000","last_clean_scrub_stamp":"2024-04-27T03:33:55.031445+0000","objects_scrubbed":1,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T07:00:01.039513+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:36.354134+0000","last_change":"2024-04-27T03:34:36.354134+0000","last_active":"2024-04-27T03:34:36.354134+0000","last_peered":"2024-04-27T03:34:36.354134+0000","last_clean":"2024-04-27T03:34:36.354134+0000","last_became_active":"2024-04-27T03:30:11.357898+0000","last_became_peered":"2024-04-27T03:30:11.357898+0000","last_unstale":"2024-04-27T03:34:36.354134+0000","last_undegraded":"2024-04-27T03:34:36.354134+0000","last_fullsized":"2024-04-27T03:34:36.354134+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:36.354013+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:36.354013+0000","last_clean_scrub_stamp":"2024-04-27T03:34:36.354013+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T04:09:36.076803+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.29","version":"27'6","reported_seq":34,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:24.943886+0000","last_change":"2024-04-27T03:34:24.943920+0000","last_active":"2024-04-27T03:34:24.943886+0000","last_peered":"2024-04-27T03:34:24.943886+0000","last_clean":"2024-04-27T03:34:24.943886+0000","last_became_active":"2024-04-27T03:31:01.440820+0000","last_became_peered":"2024-04-27T03:31:01.440820+0000","last_unstale":"2024-04-27T03:34:24.943886+0000","last_undegraded":"2024-04-27T03:34:24.943886+0000","last_fullsized":"2024-04-27T03:34:24.943886+0000","mapping_epoch":26,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":27,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'6","last_scrub_stamp":"2024-04-27T03:34:24.943789+0000","last_deep_scrub":"27'6","last_deep_scrub_stamp":"2024-04-27T03:34:24.943789+0000","last_clean_scrub_stamp":"2024-04-27T03:34:24.943789+0000","objects_scrubbed":1,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T13:18:19.199603+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":10,"num_read_kb":10,"num_write":5,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:05.846517+0000","last_change":"2024-04-27T03:34:05.846517+0000","last_active":"2024-04-27T03:34:05.846517+0000","last_peered":"2024-04-27T03:34:05.846517+0000","last_clean":"2024-04-27T03:34:05.846517+0000","last_became_active":"2024-04-27T03:30:11.362788+0000","last_became_peered":"2024-04-27T03:30:11.362788+0000","last_unstale":"2024-04-27T03:34:05.846517+0000","last_undegraded":"2024-04-27T03:34:05.846517+0000","last_fullsized":"2024-04-27T03:34:05.846517+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:05.846403+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:05.846403+0000","last_clean_scrub_stamp":"2024-04-27T03:34:05.846403+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:12:30.818661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.28","version":"25'2","reported_seq":45,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:09.209205+0000","last_change":"2024-04-27T03:34:09.209205+0000","last_active":"2024-04-27T03:34:09.209205+0000","last_peered":"2024-04-27T03:34:09.209205+0000","last_clean":"2024-04-27T03:34:09.209205+0000","last_became_active":"2024-04-27T03:30:13.377845+0000","last_became_peered":"2024-04-27T03:30:13.377845+0000","last_unstale":"2024-04-27T03:34:09.209205+0000","last_undegraded":"2024-04-27T03:34:09.209205+0000","last_fullsized":"2024-04-27T03:34:09.209205+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:09.209073+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:09.209073+0000","last_clean_scrub_stamp":"2024-04-27T03:34:09.209073+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:24:40.804195+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"27'2","reported_seq":49,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:25.348820+0000","last_change":"2024-04-27T03:34:25.348820+0000","last_active":"2024-04-27T03:34:25.348820+0000","last_peered":"2024-04-27T03:34:25.348820+0000","last_clean":"2024-04-27T03:34:25.348820+0000","last_became_active":"2024-04-27T03:30:11.362559+0000","last_became_peered":"2024-04-27T03:30:11.362559+0000","last_unstale":"2024-04-27T03:34:25.348820+0000","last_undegraded":"2024-04-27T03:34:25.348820+0000","last_fullsized":"2024-04-27T03:34:25.348820+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'2","last_scrub_stamp":"2024-04-27T03:34:25.348662+0000","last_deep_scrub":"27'2","last_deep_scrub_stamp":"2024-04-27T03:34:25.348662+0000","last_clean_scrub_stamp":"2024-04-27T03:34:25.348662+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T05:14:25.577657+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":2,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":47,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:29.307826+0000","last_change":"2024-04-27T03:34:29.307826+0000","last_active":"2024-04-27T03:34:29.307826+0000","last_peered":"2024-04-27T03:34:29.307826+0000","last_clean":"2024-04-27T03:34:29.307826+0000","last_became_active":"2024-04-27T03:30:11.363340+0000","last_became_peered":"2024-04-27T03:30:11.363340+0000","last_unstale":"2024-04-27T03:34:29.307826+0000","last_undegraded":"2024-04-27T03:34:29.307826+0000","last_fullsized":"2024-04-27T03:34:29.307826+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-27T03:34:29.307702+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-27T03:34:29.307702+0000","last_clean_scrub_stamp":"2024-04-27T03:34:29.307702+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T06:33:45.266144+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.37","version":"27'4","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:33:56.668296+0000","last_change":"2024-04-27T03:33:56.668296+0000","last_active":"2024-04-27T03:33:56.668296+0000","last_peered":"2024-04-27T03:33:56.668296+0000","last_clean":"2024-04-27T03:33:56.668296+0000","last_became_active":"2024-04-27T03:30:13.385705+0000","last_became_peered":"2024-04-27T03:30:13.385705+0000","last_unstale":"2024-04-27T03:33:56.668296+0000","last_undegraded":"2024-04-27T03:33:56.668296+0000","last_fullsized":"2024-04-27T03:33:56.668296+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'4","last_scrub_stamp":"2024-04-27T03:33:56.668140+0000","last_deep_scrub":"27'4","last_deep_scrub_stamp":"2024-04-27T03:33:56.668140+0000","last_clean_scrub_stamp":"2024-04-27T03:33:56.668140+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T11:12:00.786770+0000","scrub_duration":1,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":5,"num_read_kb":5,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"25'2","reported_seq":50,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:23.731902+0000","last_change":"2024-04-27T03:34:23.731902+0000","last_active":"2024-04-27T03:34:23.731902+0000","last_peered":"2024-04-27T03:34:23.731902+0000","last_clean":"2024-04-27T03:34:23.731902+0000","last_became_active":"2024-04-27T03:30:11.360667+0000","last_became_peered":"2024-04-27T03:30:11.360667+0000","last_unstale":"2024-04-27T03:34:23.731902+0000","last_undegraded":"2024-04-27T03:34:23.731902+0000","last_fullsized":"2024-04-27T03:34:23.731902+0000","mapping_epoch":13,"log_start":"0'0","ondisk_log_start":"0'0","created":13,"last_epoch_clean":14,"parent":"0.0","parent_split_bits":0,"last_scrub":"25'2","last_scrub_stamp":"2024-04-27T03:34:23.731763+0000","last_deep_scrub":"25'2","last_deep_scrub_stamp":"2024-04-27T03:34:23.731763+0000","last_clean_scrub_stamp":"2024-04-27T03:34:23.731763+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T12:18:06.049050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":3,"num_read_kb":2,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"27'14","reported_seq":78,"reported_epoch":27,"state":"active+clean","last_fresh":"2024-04-27T03:34:15.127239+0000","last_change":"2024-04-27T03:34:15.127239+0000","last_active":"2024-04-27T03:34:15.127239+0000","last_peered":"2024-04-27T03:34:15.127239+0000","last_clean":"2024-04-27T03:34:15.127239+0000","last_became_active":"2024-04-27T03:30:13.380347+0000","last_became_peered":"2024-04-27T03:30:13.380347+0000","last_unstale":"2024-04-27T03:34:15.127239+0000","last_undegraded":"2024-04-27T03:34:15.127239+0000","last_fullsized":"2024-04-27T03:34:15.127239+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"27'14","last_scrub_stamp":"2024-04-27T03:34:15.127095+0000","last_deep_scrub":"27'14","last_deep_scrub_stamp":"2024-04-27T03:34:15.127095+0000","last_clean_scrub_stamp":"2024-04-27T03:34:15.127095+0000","objects_scrubbed":0,"log_size":14,"log_dups_size":0,"ondisk_log_size":14,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-28T03:37:03.309204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":48,"num_read_kb":44,"num_write":16,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":1541,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":655,"num_read_kb":623,"num_write":558,"num_write_kb":171,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":349,"num_omap_keys":2,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3082,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":275,"ondisk_log_size":275,"up":64,"acting":64,"num_store_stats":4},{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":8,"num_object_clones":0,"num_object_copies":16,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2281,"ondisk_log_size":2281,"up":64,"acting":64,"num_store_stats":4},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":11674,"num_objects":215,"num_object_clones":0,"num_object_copies":430,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":215,"num_whiteouts":0,"num_read":1888,"num_read_kb":1653,"num_write":789,"num_write_kb":72,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":2692,"num_omap_keys":14,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":278528,"data_stored":7404,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":1751,"ondisk_log_size":1751,"up":64,"acting":64,"num_store_stats":4},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":1800,"num_objects":7,"num_object_clones":0,"num_object_copies":14,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":972,"num_read_kb":972,"num_write":19,"num_write_kb":12,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":436,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":3600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":19,"ondisk_log_size":19,"up":64,"acting":64,"num_store_stats":4},{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":1612,"num_read_kb":1502,"num_write":528,"num_write_kb":85,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":11,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":482,"ondisk_log_size":482,"up":128,"acting":128,"num_store_stats":4},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":135,"num_read_kb":82,"num_write":316,"num_write_kb":58617,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":128,"acting":128,"num_store_stats":4},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":3,"up_from":9,"seq":38654705727,"num_pgs":152,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39904,"kb_used_data":408,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331936,"statfs":{"total":96636764160,"available":96595902464,"internally_reserved":0,"allocated":417792,"data_stored":120948,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":40434119},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705728,"num_pgs":125,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40416,"kb_used_data":920,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331424,"statfs":{"total":96636764160,"available":96595378176,"internally_reserved":0,"allocated":942080,"data_stored":708075,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705727,"num_pgs":119,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36284,"kb_used_data":948,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":94335556,"statfs":{"total":96636764160,"available":96599609344,"internally_reserved":0,"allocated":970752,"data_stored":708832,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705729,"num_pgs":134,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39868,"kb_used_data":372,"kb_used_omap":1,"kb_used_meta":39486,"kb_avail":94331972,"statfs":{"total":96636764160,"available":96595939328,"internally_reserved":0,"allocated":380928,"data_stored":119531,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":40434117},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1318,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":574,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1226,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1650,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":61440,"data_stored":1612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":57344,"data_stored":1584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":2558,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":637,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":1523,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-27T03:35:13.872 DEBUG:teuthology.orchestra.run.smithi089:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2024-04-27T03:35:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.986+0000 7f4a046a7640 1 Processor -- start 2024-04-27T03:35:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.986+0000 7f4a046a7640 1 -- start start 2024-04-27T03:35:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.986+0000 7f4a046a7640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f49fc1287a0 0x7f49fc1303c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:35:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.987+0000 7f4a046a7640 1 -- --> v1:172.21.15.89:6789/0 -- auth(proto 0 30 bytes epoch 0) -- 0x7f49fc098800 con 0x7f49fc127cb0 2024-04-27T03:35:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.987+0000 7f4a046a7640 1 -- --> v2:172.21.15.89:3300/0 -- mon_getmap magic: 0 -- 0x7f49fc127b00 con 0x7f49fc1287a0 2024-04-27T03:35:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.987+0000 7f4a01c42640 1 --2- >> v2:172.21.15.89:3300/0 conn(0x7f49fc1287a0 0x7f49fc1303c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:35:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.987+0000 7f4a02443640 1 --1- >> v1:172.21.15.89:6789/0 conn(0x7f49fc127cb0 0x7f49fc128090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.89:6789/0 says I am v1:172.21.15.89:48606/0 (socket says 172.21.15.89:48606) 2024-04-27T03:35:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.987+0000 7f4a02443640 1 -- 172.21.15.89:0/2053411491 learned_addr learned my addr 172.21.15.89:0/2053411491 (peer_addr_for_me v1:172.21.15.89:0/0) 2024-04-27T03:35:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.987+0000 7f4a01c42640 1 -- 172.21.15.89:0/2053411491 >> v1:172.21.15.89:6789/0 conn(0x7f49fc127cb0 legacy=0x7f49fc128090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:35:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.987+0000 7f4a01c42640 1 -- 172.21.15.89:0/2053411491 --> v2:172.21.15.89:3300/0 -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc06f940 con 0x7f49fc1287a0 2024-04-27T03:35:13.993 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.987+0000 7f4a01441640 1 -- 172.21.15.89:0/2053411491 <== mon.0 v1:172.21.15.89:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) ==== 33+0+0 (unknown 3848441598 0 0) 0x7f49fc098800 con 0x7f49fc127cb0 2024-04-27T03:35:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.988+0000 7f4a01c42640 1 --2- 172.21.15.89:0/2053411491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49fc1287a0 0x7f49fc1303c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009720 tx=0x7f49ec02b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbb7b9c5672f250b server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:35:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.988+0000 7f4a01441640 1 -- 172.21.15.89:0/2053411491 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49ec0316d0 con 0x7f49fc1287a0 2024-04-27T03:35:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.988+0000 7f4a01441640 1 -- 172.21.15.89:0/2053411491 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49ec037070 con 0x7f49fc1287a0 2024-04-27T03:35:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.988+0000 7f4a01441640 1 -- 172.21.15.89:0/2053411491 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49ec0319c0 con 0x7f49fc1287a0 2024-04-27T03:35:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.988+0000 7f4a046a7640 1 -- 172.21.15.89:0/2053411491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49fc1287a0 msgr2=0x7f49fc1303c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:35:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.988+0000 7f4a046a7640 1 --2- 172.21.15.89:0/2053411491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49fc1287a0 0x7f49fc1303c0 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009720 tx=0x7f49ec02b840 comp rx=0 tx=0).stop 2024-04-27T03:35:13.996 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.988+0000 7f4a046a7640 1 -- 172.21.15.89:0/2053411491 shutdown_connections 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.988+0000 7f4a046a7640 1 --2- 172.21.15.89:0/2053411491 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49fc1287a0 0x7f49fc1303c0 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.988+0000 7f4a046a7640 1 -- 172.21.15.89:0/2053411491 >> 172.21.15.89:0/2053411491 conn(0x7f49fc099680 msgr2=0x7f49fc099a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.988+0000 7f4a046a7640 1 -- 172.21.15.89:0/2053411491 shutdown_connections 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.988+0000 7f4a046a7640 1 -- 172.21.15.89:0/2053411491 wait complete. 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.989+0000 7f4a046a7640 1 Processor -- start 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.989+0000 7f4a046a7640 1 -- start start 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.989+0000 7f4a046a7640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49fc127cb0 0x7f49fc1247d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.989+0000 7f4a046a7640 1 -- --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_getmap magic: 0 -- 0x7f49fc130900 con 0x7f49fc127cb0 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.990+0000 7f4a02443640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49fc127cb0 0x7f49fc1247d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.990+0000 7f4a02443640 1 --2- >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49fc127cb0 0x7f49fc1247d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.89:3300/0 says I am v2:172.21.15.89:41946/0 (socket says 172.21.15.89:41946) 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.990+0000 7f4a02443640 1 -- 172.21.15.89:0/568951973 learned_addr learned my addr 172.21.15.89:0/568951973 (peer_addr_for_me v2:172.21.15.89:0/0) 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.990+0000 7f4a02443640 1 -- 172.21.15.89:0/568951973 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f49fc126c30 con 0x7f49fc127cb0 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.990+0000 7f4a02443640 1 --2- 172.21.15.89:0/568951973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49fc127cb0 0x7f49fc1247d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f49f800c420 tx=0x7f49f800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:35:13.997 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.990+0000 7f49f2ffd640 1 -- 172.21.15.89:0/568951973 <== mon.0 v2:172.21.15.89:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49f8018070 con 0x7f49fc127cb0 2024-04-27T03:35:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.990+0000 7f49f2ffd640 1 -- 172.21.15.89:0/568951973 <== mon.0 v2:172.21.15.89:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7f49f80074d0 con 0x7f49fc127cb0 2024-04-27T03:35:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.990+0000 7f4a046a7640 1 -- 172.21.15.89:0/568951973 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f49fc126fc0 con 0x7f49fc127cb0 2024-04-27T03:35:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.990+0000 7f49f2ffd640 1 -- 172.21.15.89:0/568951973 <== mon.0 v2:172.21.15.89:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f49f8004950 con 0x7f49fc127cb0 2024-04-27T03:35:13.998 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.990+0000 7f4a046a7640 1 -- 172.21.15.89:0/568951973 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f49fc08a4f0 con 0x7f49fc127cb0 2024-04-27T03:35:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.991+0000 7f49f2ffd640 1 -- 172.21.15.89:0/568951973 <== mon.0 v2:172.21.15.89:3300/0 4 ==== mgrmap(e 5) ==== 45563+0+0 (secure 0 0 0) 0x7f49f801b070 con 0x7f49fc127cb0 2024-04-27T03:35:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.991+0000 7f4a046a7640 1 -- 172.21.15.89:0/568951973 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f49fc127760 con 0x7f49fc127cb0 2024-04-27T03:35:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.991+0000 7f49f2ffd640 1 --2- 172.21.15.89:0/568951973 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f49e0038060 0x7f49e003a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-27T03:35:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.991+0000 7f49f2ffd640 1 -- 172.21.15.89:0/568951973 <== mon.0 v2:172.21.15.89:3300/0 5 ==== osd_map(27..27 src has 1..27) ==== 5938+0+0 (secure 0 0 0) 0x7f49f804ac40 con 0x7f49fc127cb0 2024-04-27T03:35:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.992+0000 7f4a01c42640 1 --2- 172.21.15.89:0/568951973 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f49e0038060 0x7f49e003a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-27T03:35:13.999 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.992+0000 7f4a01c42640 1 --2- 172.21.15.89:0/568951973 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f49e0038060 0x7f49e003a520 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009720 tx=0x7f49ec008030 comp rx=0 tx=0).ready entity=mgr.4102 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-27T03:35:14.001 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:13.996+0000 7f49f2ffd640 1 -- 172.21.15.89:0/568951973 <== mon.0 v2:172.21.15.89:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+185023 (secure 0 0 0) 0x7f49fc127760 con 0x7f49fc127cb0 2024-04-27T03:35:14.172 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.166+0000 7f4a046a7640 1 -- 172.21.15.89:0/568951973 --> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) -- 0x7f49fc062da0 con 0x7f49fc127cb0 2024-04-27T03:35:14.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.169+0000 7f49f2ffd640 1 -- 172.21.15.89:0/568951973 <== mon.0 v2:172.21.15.89:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v6)=0 v6) ==== 123+0+0 (secure 0 0 0) 0x7f49f8011960 con 0x7f49fc127cb0 2024-04-27T03:35:14.175 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.170+0000 7f49f2ffd640 1 -- 172.21.15.89:0/568951973 <== mon.0 v2:172.21.15.89:3300/0 8 ==== config(1 keys) ==== 35+0+0 (secure 0 0 0) 0x7f49f8011bc0 con 0x7f49fc127cb0 2024-04-27T03:35:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.171+0000 7f4a046a7640 1 -- 172.21.15.89:0/568951973 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f49e0038060 msgr2=0x7f49e003a520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:35:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.171+0000 7f4a046a7640 1 --2- 172.21.15.89:0/568951973 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f49e0038060 0x7f49e003a520 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f49ec009720 tx=0x7f49ec008030 comp rx=0 tx=0).stop 2024-04-27T03:35:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.171+0000 7f4a046a7640 1 -- 172.21.15.89:0/568951973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49fc127cb0 msgr2=0x7f49fc1247d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-27T03:35:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.171+0000 7f4a046a7640 1 --2- 172.21.15.89:0/568951973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49fc127cb0 0x7f49fc1247d0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f49f800c420 tx=0x7f49f800c8e0 comp rx=0 tx=0).stop 2024-04-27T03:35:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.171+0000 7f4a046a7640 1 -- 172.21.15.89:0/568951973 shutdown_connections 2024-04-27T03:35:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.171+0000 7f4a046a7640 1 --2- 172.21.15.89:0/568951973 >> [v2:172.21.15.165:6824/3121580967,v1:172.21.15.165:6825/3121580967] conn(0x7f49e0038060 0x7f49e003a520 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:35:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.171+0000 7f4a046a7640 1 --2- 172.21.15.89:0/568951973 >> [v2:172.21.15.89:3300/0,v1:172.21.15.89:6789/0] conn(0x7f49fc127cb0 0x7f49fc1247d0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-27T03:35:14.178 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.171+0000 7f4a046a7640 1 -- 172.21.15.89:0/568951973 >> 172.21.15.89:0/568951973 conn(0x7f49fc099680 msgr2=0x7f49fc12c150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-27T03:35:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.171+0000 7f4a046a7640 1 -- 172.21.15.89:0/568951973 shutdown_connections 2024-04-27T03:35:14.179 INFO:teuthology.orchestra.run.smithi089.stderr:2024-04-27T03:35:14.171+0000 7f4a046a7640 1 -- 172.21.15.89:0/568951973 wait complete. 2024-04-27T03:35:14.187 INFO:teuthology.misc:Shutting down mds daemons... 2024-04-27T03:35:14.188 INFO:teuthology.misc:Shutting down osd daemons... 2024-04-27T03:35:14.188 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2024-04-27T03:35:14.188 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T03:35:14.234 INFO:tasks.ceph.osd.0:Stopped 2024-04-27T03:35:14.235 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2024-04-27T03:35:14.235 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T03:35:14.267 INFO:tasks.ceph.osd.1:Stopped 2024-04-27T03:35:14.267 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2024-04-27T03:35:14.267 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T03:35:14.340 INFO:tasks.ceph.osd.2:Stopped 2024-04-27T03:35:14.340 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2024-04-27T03:35:14.341 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T03:35:14.408 INFO:tasks.ceph.osd.3:Stopped 2024-04-27T03:35:14.408 INFO:teuthology.misc:Shutting down mgr daemons... 2024-04-27T03:35:14.409 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2024-04-27T03:35:14.409 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T03:35:14.477 INFO:tasks.ceph.mgr.x:Stopped 2024-04-27T03:35:14.478 INFO:teuthology.misc:Shutting down mon daemons... 2024-04-27T03:35:14.478 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2024-04-27T03:35:14.478 INFO:teuthology.orchestra.run:waiting for 300 2024-04-27T03:35:14.489 INFO:tasks.ceph.mon.a:Stopped 2024-04-27T03:35:14.489 INFO:tasks.ceph:Checking cluster log for badness... 2024-04-27T03:35:14.490 DEBUG:teuthology.orchestra.run.smithi089:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(PG_AVAILABILITY\)' | egrep -v '\(PG_DEGRADED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'not have an application enabled' | head -n 1 2024-04-27T03:35:14.562 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi089.front.sepia.ceph.com 2024-04-27T03:35:14.562 DEBUG:teuthology.orchestra.run.smithi089:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2024-04-27T03:35:14.730 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-27T03:35:14.730 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2024-04-27T03:35:14.820 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-27T03:35:14.821 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2024-04-27T03:35:14.883 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi165.front.sepia.ceph.com 2024-04-27T03:35:14.884 DEBUG:teuthology.orchestra.run.smithi165:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2024-04-27T03:35:14.934 INFO:tasks.ceph:Archiving mon data... 2024-04-27T03:35:14.935 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672/data/mon.a.tgz 2024-04-27T03:35:14.935 DEBUG:teuthology.orchestra.run.smithi089:> mktemp 2024-04-27T03:35:14.951 INFO:teuthology.orchestra.run.smithi089.stdout:/tmp/tmp.ozzYf13dBR 2024-04-27T03:35:14.951 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.ozzYf13dBR 2024-04-27T03:35:15.076 DEBUG:teuthology.orchestra.run.smithi089:> sudo chmod 0666 /tmp/tmp.ozzYf13dBR 2024-04-27T03:35:15.158 DEBUG:teuthology.orchestra.remote:smithi089:/tmp/tmp.ozzYf13dBR is 257KB 2024-04-27T03:35:15.256 DEBUG:teuthology.orchestra.run.smithi089:> rm -fr /tmp/tmp.ozzYf13dBR 2024-04-27T03:35:15.270 INFO:tasks.ceph:Cleaning ceph cluster... 2024-04-27T03:35:15.270 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-27T03:35:15.312 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2024-04-27T03:35:15.387 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-27T03:35:15.405 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2024-04-27T03:35:15.405 INFO:tasks.ceph:Archiving crash dumps... 2024-04-27T03:35:15.407 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672/remote/smithi089/crash 2024-04-27T03:35:15.407 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-27T03:35:15.450 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/lib/ceph/crash to /home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672/remote/smithi165/crash 2024-04-27T03:35:15.451 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2024-04-27T03:35:15.488 INFO:tasks.ceph:Compressing logs... 2024-04-27T03:35:15.488 DEBUG:teuthology.orchestra.run.smithi089:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-27T03:35:15.495 DEBUG:teuthology.orchestra.run.smithi165:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-27T03:35:15.515 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29773.log 2024-04-27T03:35:15.515 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29873.log 2024-04-27T03:35:15.515 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29773.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31003.log 2024-04-27T03:35:15.515 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29873.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31172.log 2024-04-27T03:35:15.515 INFO:teuthology.orchestra.run.smithi089.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.29773.log.gz 2024-04-27T03:35:15.531 INFO:teuthology.orchestra.run.smithi089.stderr: 87.5% -- replaced with /var/log/ceph/ceph-client.admin.29873.log.gz 2024-04-27T03:35:15.531 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5/var/log/ceph/ceph-client.admin.31003.log: --verbose -- /var/log/ceph/ceph-client.admin.29874.log 2024-04-27T03:35:15.531 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31172.log: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.31003.log.gzgzip 2024-04-27T03:35:15.531 INFO:teuthology.orchestra.run.smithi089.stderr: -5 --verbose -- /var/log/ceph/ceph.log 2024-04-27T03:35:15.531 INFO:teuthology.orchestra.run.smithi089.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.31172.log.gz 2024-04-27T03:35:15.531 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29874.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30171.log 2024-04-27T03:35:15.531 INFO:teuthology.orchestra.run.smithi089.stderr: 87.8% -- replaced with /var/log/ceph/ceph-client.admin.29874.log.gz 2024-04-27T03:35:15.531 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30041.log 2024-04-27T03:35:15.531 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30171.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31129.log 2024-04-27T03:35:15.531 INFO:teuthology.orchestra.run.smithi089.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.30171.log.gz 2024-04-27T03:35:15.531 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30583.log 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.30041.log.gz 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31129.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30583.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27699.log 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr: 86.4% -- replaced with /var/log/ceph/ceph.log.gz 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.31129.log.gz 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.30583.log.gz 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30210.log 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.27699.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30835.log 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr: 88.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.27699.log.gz 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30210.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30457.log 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.30210.log.gz 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30835.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.25984.log 2024-04-27T03:35:15.532 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30457.log: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.30835.log.gz 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30667.log 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.30457.log.gz 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph.tmp-client.admin.25984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31087.log 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr: 0.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.25984.log.gz 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30667.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30499.log 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr: 85.6%/var/log/ceph/ceph-client.admin.31087.log: -- replaced with /var/log/ceph/ceph-client.admin.30667.log.gz 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30499.log: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.31087.log.gz 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30319.log 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.30499.log.gz 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27817.log 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29631.log 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.30319.log.gz 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr:gzip/var/log/ceph/ceph-client.admin.27817.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.29588.log 2024-04-27T03:35:15.533 INFO:teuthology.orchestra.run.smithi089.stderr: 85.0%/var/log/ceph/ceph-client.admin.29631.log: -- replaced with /var/log/ceph/ceph-client.admin.27817.log.gz 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29588.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.29631.log.gz 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29872.log 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.29588.log.gz 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30042.log 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29530.log 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30042.log: 87.3% -- replaced with /var/log/ceph/ceph-client.admin.29872.log.gz 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30361.log 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.30042.log.gz 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30541.log 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29530.log: /var/log/ceph/ceph-client.admin.30361.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30211.log 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.29530.log.gz 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30541.log: 85.4% -- replaced with /var/log/ceph/ceph-client.admin.30361.log.gz 2024-04-27T03:35:15.534 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27855.log 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr: 85.6% -- replaced with /var/log/ceph/ceph-client.admin.30541.log.gz 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30877.log 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30211.log: /var/log/ceph/ceph-client.admin.27855.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30403.log 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.30211.log.gz 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr: 84.8% -- replaced with /var/log/ceph/ceph-client.admin.27855.log.gz 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30877.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30170.log 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr: 87.2% -- replaced with /var/log/ceph/ceph-client.admin.30877.log.gz 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30919.log 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30403.log: /var/log/ceph/ceph-client.admin.30170.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29673.log 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.30403.log.gz 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.30170.log.gz 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30919.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30625.log 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.30919.log.gz 2024-04-27T03:35:15.535 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31045.log 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29673.log: /var/log/ceph/ceph-client.admin.30625.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30709.log 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr: 86.5% -- replaced with /var/log/ceph/ceph-client.admin.29673.log.gz 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.30625.log.gz 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30793.log 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr: 85.8% -- replaced with /var/log/ceph/ceph-client.admin.31045.log.gz 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30040.log 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30709.log: /var/log/ceph/ceph-client.admin.30793.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29875.log 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr: 86.8% -- replaced with /var/log/ceph/ceph-client.admin.30709.log.gz 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr: 86.9% -- replaced with /var/log/ceph/ceph-client.admin.30793.log.gz 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30040.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29723.log 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr: 86.0% -- replaced with /var/log/ceph/ceph-client.admin.30040.log.gz 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.29875.log: -- /var/log/ceph/ceph-client.admin.30961.log 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.29723.log: 87.4% -- replaced with /var/log/ceph/ceph-client.admin.29875.log.gz 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30043.log 2024-04-27T03:35:15.536 INFO:teuthology.orchestra.run.smithi089.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.29723.log.gz 2024-04-27T03:35:15.537 INFO:teuthology.orchestra.run.smithi089.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.30751.log 2024-04-27T03:35:15.537 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30961.log: /var/log/ceph/ceph-client.admin.30043.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.31212.log 2024-04-27T03:35:15.537 INFO:teuthology.orchestra.run.smithi089.stderr: 85.5% -- replaced with /var/log/ceph/ceph-client.admin.30961.log.gz 2024-04-27T03:35:15.537 INFO:teuthology.orchestra.run.smithi089.stderr: 86.2% -- replaced with /var/log/ceph/ceph-client.admin.30043.log.gz 2024-04-27T03:35:15.537 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.30751.log: 85.7% -- replaced with /var/log/ceph/ceph-client.admin.30751.log.gz 2024-04-27T03:35:15.537 INFO:teuthology.orchestra.run.smithi089.stderr:/var/log/ceph/ceph-client.admin.31212.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.31212.log.gz 2024-04-27T03:35:15.556 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29893.log 2024-04-27T03:35:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.60404.log 2024-04-27T03:35:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29893.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38025.log 2024-04-27T03:35:15.580 INFO:teuthology.orchestra.run.smithi165.stderr: 85.2% -- replaced with /var/log/ceph/ceph-client.admin.29893.log.gz 2024-04-27T03:35:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.60404.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71294.log 2024-04-27T03:35:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.38025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.94254.log 2024-04-27T03:35:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.71294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39834.log 2024-04-27T03:35:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.94254.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67664.log 2024-04-27T03:35:15.580 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.60404.log.gz 2024-04-27T03:35:15.580 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.39834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.67059.log 2024-04-27T03:35:15.581 INFO:teuthology.orchestra.run.smithi165.stderr: 91.4%gzip -- replaced with /var/log/ceph/ceph-client.0.38025.log.gz -5 2024-04-27T03:35:15.581 INFO:teuthology.orchestra.run.smithi165.stderr: --verbose -- /var/log/ceph/ceph-client.0.72504.log 2024-04-27T03:35:15.581 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.67664.log: /var/log/ceph/ceph-client.0.67059.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80369.log 2024-04-27T03:35:15.581 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.94254.log.gz 2024-04-27T03:35:15.581 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.72504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85197.log 2024-04-27T03:35:15.581 INFO:teuthology.orchestra.run.smithi165.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.39834.log.gz 2024-04-27T03:35:15.581 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.80369.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.38630.log 2024-04-27T03:35:15.581 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.85197.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51329.log 2024-04-27T03:35:15.581 INFO:teuthology.orchestra.run.smithi165.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.71294.log.gz 2024-04-27T03:35:15.581 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.38630.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.85796.log 2024-04-27T03:35:15.581 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.72504.log.gz 2024-04-27T03:35:15.582 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.67059.log.gz 2024-04-27T03:35:15.582 INFO:teuthology.orchestra.run.smithi165.stderr: 90.4% -- replaced with /var/log/ceph/ceph-client.0.85197.log.gz 2024-04-27T03:35:15.582 INFO:teuthology.orchestra.run.smithi165.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.0.38630.log.gz 2024-04-27T03:35:15.582 INFO:teuthology.orchestra.run.smithi165.stderr:gzip/var/log/ceph/ceph-client.0.51329.log: -5 --verbose -- /var/log/ceph/ceph-client.0.45884.log 2024-04-27T03:35:15.582 INFO:teuthology.orchestra.run.smithi165.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.67664.log.gz 2024-04-27T03:35:15.582 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.85796.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70084.log 2024-04-27T03:35:15.582 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.45884.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2024-04-27T03:35:15.582 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.70084.log: 90.4% -- replaced with /var/log/ceph/ceph-client.0.85796.log.gz 2024-04-27T03:35:15.582 INFO:teuthology.orchestra.run.smithi165.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.80369.log.gz 2024-04-27T03:35:15.582 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65244.log 2024-04-27T03:35:15.582 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50119.log 2024-04-27T03:35:15.582 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.51329.log.gz 2024-04-27T03:35:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.65244.log: gzip -5 91.0% --verbose -- -- replaced with /var/log/ceph/ceph-client.0.45884.log.gz /var/log/ceph/ceph-client.0.77344.log 2024-04-27T03:35:15.583 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-04-27T03:35:15.583 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.70084.log.gz 2024-04-27T03:35:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93044.log 2024-04-27T03:35:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.50119.log: /var/log/ceph/ceph-client.0.77344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54959.log 2024-04-27T03:35:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.93044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.94859.log 2024-04-27T03:35:15.583 INFO:teuthology.orchestra.run.smithi165.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.65244.log.gz 2024-04-27T03:35:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.54959.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29973.log 2024-04-27T03:35:15.583 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.94859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64639.log 91.3% -- replaced with /var/log/ceph/ceph-client.0.77344.log.gz 2024-04-27T03:35:15.583 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.50119.log.gz 2024-04-27T03:35:15.584 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-04-27T03:35:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29973.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.93044.log.gz 2024-04-27T03:35:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.75529.log 2024-04-27T03:35:15.584 INFO:teuthology.orchestra.run.smithi165.stderr: 85.3% -- replaced with /var/log/ceph/ceph-client.admin.29973.log.gz 2024-04-27T03:35:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.0.64639.log: --verbose -- /var/log/ceph/ceph-client.0.61614.log 2024-04-27T03:35:15.584 INFO:teuthology.orchestra.run.smithi165.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.54959.log.gz 2024-04-27T03:35:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.75529.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.57984.log 2024-04-27T03:35:15.584 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.94859.log.gz 2024-04-27T03:35:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.61614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.63429.log 2024-04-27T03:35:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.57984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.50724.log 2024-04-27T03:35:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.63429.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.75529.log.gz 2024-04-27T03:35:15.584 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.77949.log 2024-04-27T03:35:15.585 INFO:teuthology.orchestra.run.smithi165.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.64639.log.gz 2024-04-27T03:35:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.50724.log: 90.9% -- replaced with /var/log/ceph/ceph-client.0.61614.log.gz 2024-04-27T03:35:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2024-04-27T03:35:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.77949.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73109.log 2024-04-27T03:35:15.585 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.57984.log.gz 2024-04-27T03:35:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.54354.log 2024-04-27T03:35:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.73109.log: 91.0%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.55564.log 2024-04-27T03:35:15.585 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.0.63429.log.gz 2024-04-27T03:35:15.585 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.50724.log.gz 2024-04-27T03:35:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.54354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47699.log 2024-04-27T03:35:15.585 INFO:teuthology.orchestra.run.smithi165.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.77949.log.gz 2024-04-27T03:35:15.585 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34812.log 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.55564.log: gzip -5 --verbose -- /var/log/ceph/ops-log-ceph-client.0.log/var/log/ceph/ceph-client.0.47699.log: 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.34812.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59194.log 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.73109.log.gz 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr: 86.1% -- replaced with /var/log/ceph/ceph-client.admin.34812.log.gz 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.54354.log.gz 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ops-log-ceph-client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76739.log 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr: 91.5% -- replaced with /var/log/ceph/ceph-client.0.55564.log.gz 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.47699.log.gz 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.59194.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.46489.log 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr: 91.9% -- replaced with /var/log/ceph/ops-log-ceph-client.0.log.gz 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.76739.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79159.log 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.46489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34772.log 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.79159.log: gzip -5 --verbose -- /var/log/ceph/rgw.ceph.client.0.log 2024-04-27T03:35:15.586 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.59194.log.gz 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.34772.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.76739.log.gz 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose 86.2% -- replaced with /var/log/ceph/ceph-client.admin.34772.log.gz 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr: -- /var/log/ceph/ceph-osd.1.log 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/rgw.ceph.client.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83394.log 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.46489.log.gz 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- 91.1% /var/log/ceph/ceph-client.0.86395.log 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.0.79159.log.gz 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.83394.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74924.log 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.86395.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29933.log 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.74924.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.70689.log 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr: 90.3% -- replaced with /var/log/ceph/ceph-client.0.86395.log.gz 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.83394.log.gz 2024-04-27T03:35:15.587 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29933.log: 84.9% -- replaced with /var/log/ceph/ceph-client.admin.29933.log.gz 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.87599.log 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.74924.log.gz 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.84598.log 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.70689.log: /var/log/ceph/ceph-client.0.87599.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.89414.log 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.84598.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.40439.log 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.89414.log: 91.5% -- replaced with /var/log/ceph/ceph-client.0.84598.log.gz 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.87599.log.gz 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.70689.log.gz 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.74319.log 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.89414.log.gz 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.40439.log: /var/log/ceph/ceph-client.0.74319.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42859.log 2024-04-27T03:35:15.588 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.53749.log 2024-04-27T03:35:15.604 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.42859.log: 91.7% -- replaced with /var/log/ceph/ceph-client.0.74319.log.gz 2024-04-27T03:35:15.604 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.40439.log.gz 2024-04-27T03:35:15.604 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.39229.log 2024-04-27T03:35:15.604 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.42859.log.gz 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5/var/log/ceph/ceph-client.0.53749.log: --verbose -- /var/log/ceph/ceph-client.0.53144.log 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.39229.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.61009.log 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.53144.log: 91.9% -- replaced with /var/log/ceph/ceph-client.0.53749.log.gz 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.39229.log.gz 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr:gzip 91.4% -- replaced with /var/log/ceph/ceph-client.0.53144.log.gz 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr: -5 --verbose -- /var/log/ceph/ceph-client.0.57379.log 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.61009.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.66454.log 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.57379.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68269.log 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.66454.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.61009.log.gz 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.57379.log.gz 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.52539.log 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0%/var/log/ceph/ceph-client.0.68269.log: -- replaced with /var/log/ceph/ceph-client.0.66454.log.gz 2024-04-27T03:35:15.605 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36210.log 2024-04-27T03:35:15.606 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.52539.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.95516.log 2024-04-27T03:35:15.606 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.36210.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.52539.log.gz 2024-04-27T03:35:15.606 INFO:teuthology.orchestra.run.smithi165.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.36210.log.gz 2024-04-27T03:35:15.606 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88204.log 2024-04-27T03:35:15.606 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.95516.log: 93.4% -- replaced with /var/log/ceph/ceph-client.0.68269.log.gz 2024-04-27T03:35:15.606 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.81579.log 2024-04-27T03:35:15.606 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48304.log 2024-04-27T03:35:15.606 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.81579.log: /var/log/ceph/ceph-client.0.88204.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.81579.log.gz 2024-04-27T03:35:15.606 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.88204.log.gz 2024-04-27T03:35:15.606 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.91229.log 2024-04-27T03:35:15.607 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.48304.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.68874.log 2024-04-27T03:35:15.607 INFO:teuthology.orchestra.run.smithi165.stderr: 93.2% -- replaced with /var/log/ceph/ceph-client.0.95516.log.gz 2024-04-27T03:35:15.607 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.91229.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.48304.log.gz 2024-04-27T03:35:15.607 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.91229.log.gz 2024-04-27T03:35:15.607 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90019.log 2024-04-27T03:35:15.607 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.59799.log 2024-04-27T03:35:15.607 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.68874.log: /var/log/ceph/ceph-client.0.90019.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41649.log 2024-04-27T03:35:15.607 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.59799.log: 91.2% -- replaced with /var/log/ceph/ceph-client.0.90019.log.gz 2024-04-27T03:35:15.607 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.59799.log.gz 2024-04-27T03:35:15.608 INFO:teuthology.orchestra.run.smithi165.stderr: 92.6% -- replaced with /var/log/ceph/ceph-client.0.68874.log.gz 2024-04-27T03:35:15.608 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.73714.log 2024-04-27T03:35:15.608 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34692.log 2024-04-27T03:35:15.608 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.41649.log: /var/log/ceph/ceph-client.0.73714.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56774.log 2024-04-27T03:35:15.608 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.34692.log: 85.9% -- replaced with /var/log/ceph/ceph-client.admin.34692.log.gz 2024-04-27T03:35:15.608 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.41649.log.gz 2024-04-27T03:35:15.608 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.58589.log 2024-04-27T03:35:15.608 INFO:teuthology.orchestra.run.smithi165.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.73714.log.gz 2024-04-27T03:35:15.608 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44674.log 2024-04-27T03:35:15.608 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.56774.log: /var/log/ceph/ceph-client.0.58589.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.90624.log 2024-04-27T03:35:15.608 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.44674.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.56774.log.gz 2024-04-27T03:35:15.608 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.58589.log.gz 2024-04-27T03:35:15.609 INFO:teuthology.orchestra.run.smithi165.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.44674.log.gz 2024-04-27T03:35:15.609 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.83999.log 2024-04-27T03:35:15.609 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.91834.log 2024-04-27T03:35:15.609 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.90624.log: /var/log/ceph/ceph-client.0.83999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.45279.log 2024-04-27T03:35:15.615 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.91834.log: 90.4% -- replaced with /var/log/ceph/ceph-client.0.83999.log.gz 2024-04-27T03:35:15.615 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.90624.log.gz 2024-04-27T03:35:15.615 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.41044.log 2024-04-27T03:35:15.615 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.91834.log.gz 2024-04-27T03:35:15.616 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.65849.log 2024-04-27T03:35:15.616 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.45279.log: /var/log/ceph/ceph-client.0.41044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.35605.log 2024-04-27T03:35:15.616 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.65849.log: 91.2% 91.3% -- replaced with /var/log/ceph/ceph-client.0.45279.log.gz 2024-04-27T03:35:15.616 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.0.41044.log.gz 2024-04-27T03:35:15.616 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.71899.log 2024-04-27T03:35:15.616 INFO:teuthology.orchestra.run.smithi165.stderr: 91.7% -- replaced with /var/log/ceph/ceph-client.0.65849.log.gz 2024-04-27T03:35:15.616 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.29853.log 2024-04-27T03:35:15.616 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.35605.log: /var/log/ceph/ceph-client.0.71899.log: 91.1% -- replaced with /var/log/ceph/ceph-client.0.71899.log.gz 2024-04-27T03:35:15.616 INFO:teuthology.orchestra.run.smithi165.stderr: 92.0% -- replaced with /var/log/ceph/ceph-client.0.35605.log.gz 2024-04-27T03:35:15.616 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.51934.log 2024-04-27T03:35:15.616 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.29853.log: gzip -5 --verbose 85.5% -- /var/log/ceph/ceph-client.0.62219.log 2024-04-27T03:35:15.617 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.admin.29853.log.gz 2024-04-27T03:35:15.617 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.51934.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.64034.log 2024-04-27T03:35:15.617 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.34732.log 2024-04-27T03:35:15.617 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.62219.log: /var/log/ceph/ceph-client.0.64034.log: 91.4% -- replaced with /var/log/ceph/ceph-client.0.51934.log.gz 2024-04-27T03:35:15.617 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.49514.log 2024-04-27T03:35:15.617 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.admin.34732.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.62219.log.gz 2024-04-27T03:35:15.617 INFO:teuthology.orchestra.run.smithi165.stderr: 86.3% -- replaced with /var/log/ceph/ceph-client.admin.34732.log.gz 2024-04-27T03:35:15.617 INFO:teuthology.orchestra.run.smithi165.stderr: 91.2%gzip -5 --verbose -- /var/log/ceph/ceph-client.0.76134.log 2024-04-27T03:35:15.617 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/ceph-client.0.64034.log.gz 2024-04-27T03:35:15.620 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.49514.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.49514.log.gz 2024-04-27T03:35:15.620 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.92439.log 2024-04-27T03:35:15.620 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.79764.log 2024-04-27T03:35:15.620 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.76134.log: /var/log/ceph/ceph-client.0.92439.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.43464.log 2024-04-27T03:35:15.620 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.79764.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.76134.log.gz 2024-04-27T03:35:15.620 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.92439.log.gz 2024-04-27T03:35:15.620 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.47094.log 2024-04-27T03:35:15.620 INFO:teuthology.orchestra.run.smithi165.stderr: 91.2% -- replaced with /var/log/ceph/ceph-client.0.79764.log.gz 2024-04-27T03:35:15.620 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.69479.log 2024-04-27T03:35:15.620 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.43464.log: /var/log/ceph/ceph-client.0.47094.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.86994.log 2024-04-27T03:35:15.622 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.69479.log: 91.0% -- replaced with /var/log/ceph/ceph-client.0.43464.log.gz 2024-04-27T03:35:15.622 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.47094.log.gz 2024-04-27T03:35:15.622 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82184.log 2024-04-27T03:35:15.622 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.44069.log 2024-04-27T03:35:15.622 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.69479.log.gz 2024-04-27T03:35:15.622 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.86994.log: /var/log/ceph/ceph-client.0.82184.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.93649.log 2024-04-27T03:35:15.630 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.44069.log: 91.3% -- replaced with /var/log/ceph/ceph-client.0.82184.log.gz 2024-04-27T03:35:15.630 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.86994.log.gz 2024-04-27T03:35:15.630 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.44069.log.gz 2024-04-27T03:35:15.630 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.48909.log 2024-04-27T03:35:15.630 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.93649.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.62824.log 2024-04-27T03:35:15.630 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.48909.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.82789.log 2024-04-27T03:35:15.630 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.62824.log: 91.1% 92.2% -- replaced with /var/log/ceph/ceph-client.0.93649.log.gz 2024-04-27T03:35:15.630 INFO:teuthology.orchestra.run.smithi165.stderr: -- replaced with /var/log/ceph/rgw.ceph.client.0.log.gz 2024-04-27T03:35:15.631 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.48909.log.gz 2024-04-27T03:35:15.631 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.82789.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.37420.log 2024-04-27T03:35:15.631 INFO:teuthology.orchestra.run.smithi165.stderr: 91.3% -- replaced with /var/log/ceph/ceph-client.0.62824.log.gz 2024-04-27T03:35:15.631 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.80974.log 2024-04-27T03:35:15.631 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.37420.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.88809.log 2024-04-27T03:35:15.631 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.80974.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.36815.log 2024-04-27T03:35:15.631 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.88809.log: 91.5% -- replaced with /var/log/ceph/ceph-client.0.37420.log.gz 2024-04-27T03:35:15.631 INFO:teuthology.orchestra.run.smithi165.stderr: 91.6% -- replaced with /var/log/ceph/ceph-client.0.82789.log.gz 2024-04-27T03:35:15.631 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.42254.log 2024-04-27T03:35:15.631 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.36815.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.0.78554.log 2024-04-27T03:35:15.631 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.42254.log: 91.2% 91.6% -- replaced with /var/log/ceph/ceph-client.0.80974.log.gz -- replaced with /var/log/ceph/ceph-client.0.88809.log.gz 2024-04-27T03:35:15.632 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-04-27T03:35:15.632 INFO:teuthology.orchestra.run.smithi165.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.56169.log 2024-04-27T03:35:15.633 INFO:teuthology.orchestra.run.smithi165.stderr:/var/log/ceph/ceph-client.0.78554.log: 91.4%/var/log/ceph/ceph-client.0.56169.log: -- replaced with /var/log/ceph/ceph-client.0.36815.log.gz 2024-04-27T03:35:15.633 INFO:teuthology.orchestra.run.smithi165.stderr: 91.0% -- replaced with /var/log/ceph/ceph-client.0.42254.log.gz 2024-04-27T03:35:15.633 INFO:teuthology.orchestra.run.smithi165.stderr: 91.1% -- replaced with /var/log/ceph/ceph-client.0.78554.log.gz 2024-04-27T03:35:15.633 INFO:teuthology.orchestra.run.smithi165.stderr: 91.4% -- replaced with /var/log/ceph/ceph-client.0.56169.log.gz 2024-04-27T03:35:15.640 INFO:teuthology.orchestra.run.smithi165.stderr: 91.7% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2024-04-27T03:35:15.689 INFO:teuthology.orchestra.run.smithi089.stderr: 91.5% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2024-04-27T03:35:16.375 INFO:teuthology.orchestra.run.smithi165.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2024-04-27T03:35:16.399 INFO:teuthology.orchestra.run.smithi089.stderr: 93.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2024-04-27T03:35:16.401 INFO:teuthology.orchestra.run.smithi089.stderr: 2024-04-27T03:35:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:real 0m0.895s 2024-04-27T03:35:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:user 0m1.043s 2024-04-27T03:35:16.401 INFO:teuthology.orchestra.run.smithi089.stderr:sys 0m0.061s 2024-04-27T03:35:16.525 INFO:teuthology.orchestra.run.smithi165.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2024-04-27T03:35:16.542 INFO:teuthology.orchestra.run.smithi165.stderr: 93.8% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2024-04-27T03:35:16.544 INFO:teuthology.orchestra.run.smithi165.stderr: 2024-04-27T03:35:16.544 INFO:teuthology.orchestra.run.smithi165.stderr:real 0m0.998s 2024-04-27T03:35:16.544 INFO:teuthology.orchestra.run.smithi165.stderr:user 0m2.906s 2024-04-27T03:35:16.544 INFO:teuthology.orchestra.run.smithi165.stderr:sys 0m0.200s 2024-04-27T03:35:16.545 INFO:tasks.ceph:Archiving logs... 2024-04-27T03:35:16.545 DEBUG:teuthology.misc:Transferring archived files from smithi089:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672/remote/smithi089/log 2024-04-27T03:35:16.546 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T03:35:16.760 DEBUG:teuthology.misc:Transferring archived files from smithi165:/var/log/ceph to /home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672/remote/smithi165/log 2024-04-27T03:35:16.761 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-27T03:35:17.313 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-27T03:35:17.328 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-27T03:35:17.328 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T03:35:17.331 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-27T03:35:17.406 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-27T03:35:17.421 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-27T03:35:17.421 DEBUG:teuthology.orchestra.run.smithi089:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T03:35:17.424 DEBUG:teuthology.orchestra.run.smithi165:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-27T03:35:17.437 INFO:teuthology.orchestra.run.smithi089.stderr:bash: line 1: ntpq: command not found 2024-04-27T03:35:17.438 INFO:teuthology.orchestra.run.smithi165.stderr:bash: line 1: ntpq: command not found 2024-04-27T03:35:17.442 INFO:teuthology.orchestra.run.smithi089.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T03:35:17.442 INFO:teuthology.orchestra.run.smithi089.stdout:=============================================================================== 2024-04-27T03:35:17.442 INFO:teuthology.orchestra.run.smithi089.stdout:^* hv01.front.sepia.ceph.com 3 6 377 22 +1789us[+1790us] +/- 56ms 2024-04-27T03:35:17.443 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 -2265us[-2265us] +/- 78ms 2024-04-27T03:35:17.443 INFO:teuthology.orchestra.run.smithi089.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 23 -116us[ -113us] +/- 71ms 2024-04-27T03:35:17.443 INFO:teuthology.orchestra.run.smithi089.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:35:17.444 INFO:teuthology.orchestra.run.smithi165.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-27T03:35:17.444 INFO:teuthology.orchestra.run.smithi165.stdout:=============================================================================== 2024-04-27T03:35:17.444 INFO:teuthology.orchestra.run.smithi165.stdout:^* hv01.front.sepia.ceph.com 3 6 377 21 +1795us[+1796us] +/- 56ms 2024-04-27T03:35:17.444 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv02.front.sepia.ceph.com 3 6 377 21 -2243us[-2243us] +/- 78ms 2024-04-27T03:35:17.444 INFO:teuthology.orchestra.run.smithi165.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 21 -97us[ -97us] +/- 71ms 2024-04-27T03:35:17.444 INFO:teuthology.orchestra.run.smithi165.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2024-04-27T03:35:17.444 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-27T03:35:17.453 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-27T03:35:17.454 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-27T03:35:17.472 DEBUG:teuthology.orchestra.run.smithi089:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T03:35:17.542 DEBUG:teuthology.orchestra.run.smithi165:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-27T03:35:17.599 DEBUG:teuthology.orchestra.run.smithi089:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T03:35:17.625 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T03:35:17.626 DEBUG:teuthology.orchestra.run.smithi165:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-27T03:35:17.651 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T03:35:17.652 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-27T03:35:17.662 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-27T03:35:17.682 INFO:teuthology.task.internal:Duration was 688.232307 seconds 2024-04-27T03:35:17.683 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-27T03:35:17.716 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-27T03:35:17.716 DEBUG:teuthology.orchestra.run.smithi089:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T03:35:17.719 DEBUG:teuthology.orchestra.run.smithi165:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-27T03:35:17.754 INFO:teuthology.orchestra.run.smithi089.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T03:35:17.755 INFO:teuthology.orchestra.run.smithi165.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-27T03:35:17.937 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-27T03:35:17.938 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi089.front.sepia.ceph.com 2024-04-27T03:35:17.938 DEBUG:teuthology.orchestra.run.smithi089:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T03:35:17.969 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi165.front.sepia.ceph.com 2024-04-27T03:35:17.969 DEBUG:teuthology.orchestra.run.smithi165:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-27T03:35:18.002 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-27T03:35:18.003 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T03:35:18.012 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-27T03:35:18.067 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-27T03:35:18.068 DEBUG:teuthology.orchestra.run.smithi089:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T03:35:18.077 DEBUG:teuthology.orchestra.run.smithi165:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-27T03:35:18.197 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-27T03:35:18.209 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-27T03:35:18.210 DEBUG:teuthology.orchestra.run.smithi089:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T03:35:18.233 DEBUG:teuthology.orchestra.run.smithi165:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-27T03:35:18.256 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-27T03:35:18.267 DEBUG:teuthology.orchestra.run.smithi089:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T03:35:18.276 DEBUG:teuthology.orchestra.run.smithi165:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-27T03:35:18.295 INFO:teuthology.orchestra.run.smithi089.stdout:kernel.core_pattern = core 2024-04-27T03:35:18.318 INFO:teuthology.orchestra.run.smithi165.stdout:kernel.core_pattern = core 2024-04-27T03:35:18.340 DEBUG:teuthology.orchestra.run.smithi089:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T03:35:18.376 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T03:35:18.377 DEBUG:teuthology.orchestra.run.smithi165:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-27T03:35:18.394 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-27T03:35:18.394 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-27T03:35:18.405 INFO:teuthology.task.internal:Transferring archived files... 2024-04-27T03:35:18.406 DEBUG:teuthology.misc:Transferring archived files from smithi089:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672/remote/smithi089 2024-04-27T03:35:18.406 DEBUG:teuthology.orchestra.run.smithi089:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T03:35:18.449 DEBUG:teuthology.misc:Transferring archived files from smithi165:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/teuthology-2024-04-26_21:40:02-rgw-squid-distro-default-smithi/7675672/remote/smithi165 2024-04-27T03:35:18.449 DEBUG:teuthology.orchestra.run.smithi165:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-27T03:35:18.482 INFO:teuthology.task.internal:Removing archive directory... 2024-04-27T03:35:18.482 DEBUG:teuthology.orchestra.run.smithi089:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T03:35:18.486 DEBUG:teuthology.orchestra.run.smithi165:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-27T03:35:18.531 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-27T03:35:18.543 INFO:teuthology.task.internal:Not uploading archives. 2024-04-27T03:35:18.543 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-27T03:35:18.565 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-27T03:35:18.565 DEBUG:teuthology.orchestra.run.smithi089:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T03:35:18.568 DEBUG:teuthology.orchestra.run.smithi165:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-27T03:35:18.579 INFO:teuthology.orchestra.run.smithi089.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 03:35 /home/ubuntu/cephtest 2024-04-27T03:35:18.585 INFO:teuthology.orchestra.run.smithi165.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 27 03:35 /home/ubuntu/cephtest 2024-04-27T03:35:18.586 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-27T03:35:18.599 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-27T03:35:18.669 INFO:teuthology.run:Summary data: description: rgw/singleton/{all/radosgw-admin frontend/beast ignore-pg-availability objectstore/bluestore-bitmap overrides rgw_pool_type/replicated supported-random-distro$/{centos_latest}} duration: 688.2323067188263 flavor: default owner: scheduled_teuthology@teuthology success: true 2024-04-27T03:35:18.669 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-27T03:35:18.747 INFO:teuthology.run:pass